#Mixture-of-Experts

Thinking Machines releases Inkling 2 Small — a model that, despite its size, can compete with far larger flagships

Thinking Machines has released Inkling 2 Small — a model that, despite its size, can compete with much larger flagships. It is built on a Mixture-of-Experts architecture and contains 594 billion parameters, yet only activates a fraction of them for each token...