AlterNet: Adaptive-Depth Mixture-of-Experts with Learned Halting for Elastic Inference
arXiv:2611.08741v1 Announce Type: new Abstract: Most LLM architectures have fixed compute demands determined at training time. We propose a novel architecture that combines the Mixture-of-Experts architecture with shallow experts and a novel dynamic-depth system to make the…
arXiv — cs.AI