# AlterNet: Adaptive-Depth Mixture-of-Experts with Learned Halting for Elastic Inference

**Published:** November 9, 2026
**Author:** Alexander Kastius
**Source:** arXiv — cs.AI
**Language:** en-US
**HTML:** https://aiqazaqstan.com/en/articles/alternet-adaptive-depth-mixture-of-experts-with-learned-halting-for-elas

arXiv:2611.08741v1 Announce Type: new Abstract: Most LLM architectures have fixed compute demands determined at training time. We propose a novel architecture that combines the Mixture-of-Experts architecture with shallow experts and a novel dynamic-depth system to make the…

arXiv:2611.08741v1 Announce Type: new Abstract: Most LLM architectures have fixed compute demands determined at training time. We propose a novel architecture that combines the Mixture-of-Experts architecture with shallow experts and a novel dynamic-depth system to make the extent of a forward pass dynamically adjustable. Our approach lets the model automatically determine the compute needed for a given task, paired with a system to scale it down after training without retraining when the execution environment demands faster or cheaper inference. To achieve this, we introduce an iterative computation mechanism, sequence-level depth-based routing steered by a halting mechanism within the experts, a depth encoding that lets the model track its progress in the chain, and a regularization mechanism that incentivizes the model to use allocated compute efficiently. We present initial benchmarks and discuss future research directions.

---

*From the original source: [arXiv — cs.AI](https://arxiv.org/abs/2611.08741). Open original.*

**Tags:** #arXiv #Mixture-of-Experts #Elastic Inference #AlterNet

---

*AI Qazaqstan — non-commercial informational portal about AI in Kazakhstan. Machine translations and summaries are AI-generated.*