All published intelligence, sovereign stack research, ecosystem news, and AI foundation model reviews.
I have yet to see a use case where my customers would have needed a model of this magnitude. All that and more on today's Mixture of Experts. I'm Tim Hwang and welcome to Mixture of Experts. Each week MoE brings together panel of brilliant brains working at the frontiers of…
You're running a multi-agent application with agents taking on different tasks and passing information to one another. And something breaks. Maybe your user gets a wrong answer or no answer at all. And so you do what you've always done, and you open up your monitoring dashboard.…
So today we'll be talking about quantum machine learning. You may recall I had an episode on quantum software engineering and there I talked about the complimentary strength of quantum and classical systems and one example of uh combination of quantum and classical systems are…
you know what? I want to get a piece of work done. I set off my swarms and, you know, I'll maybe have 100 agents go off and perform a task and come back to me. And then Madison scolds me for what my token bill is going to be. But but the reality is, you're going to you're going…
The largest AI models today can have over a trillion parameters, which would take something like two terabytes of memory to store. But the largest GPU on the market holds about 288GB, which is nowhere near enough to store these huge frontier models. So how are we running these…
How much can you reliably just hand over and forget about? There are instances where it has to come to you for permission. So if it keeps coming back to you asking for permissions or asking for more clarifications, then you're not truly delegated. All that and more on this…
Last week, Bun, the all-in-one JavaScript toolkit that was acquired by Anthropic late last year, announced that they spent the equivalent of $165,000 running 64 parallel Cloud agents to refactor their entire codebase from Zig to Rust in just 11 days. Usually, a full rewrite to…
Two years ago, Mira Murati did something we've all fantasized about. She woke up, decided she'd had enough of her boss, and quit her job as CTO of OpenAI with no backup plan and nothing but a vague desire to create time and space for her own exploration. If you or I did that,…
In 2024, Apple and OpenAI got married in a shotgun wedding on stage at WWDC. Tim Cook put ChatGPT inside the iPhone while Sam Altman watched from the audience. They were the unambiguously gay duo, and together they thought they could rebrand artificial intelligence to Apple…
A few days ago, Anthropic released Claude Opus 5, its fourth frontier model in eight weeks, which means Anthropic now slops out new models faster than JavaScript developers slop out new frameworks. In May, we got Opus 4.8, in June, we got Fable, then we lost Fable, then we got…
The famous American philosopher Smashmouth once said, "The years start coming and they don't stop coming." And I've never understood those words more deeply than I did this week after what felt like years in AI bizarro land. At first, on Tuesday, Anthropic released Fable and…
Earlier this week, while I was busy losing all my money in what I thought would be the future of finance, OpenAI claimed to have solved another one of mathematics' hardest problems. This time, it was one invented back in the 1850s, when the only program on Earth was a degenerate…
Last week, Jacob Cockscomb, a 27-year-old AI research bro you've never heard of, did the unthinkable and quit his job at Anthropic before his shares vested. Usually, when somebody leaves a tech company, they humble brag about it by writing a LinkedIn post about their incredible…
Last week, everything about AI changed forever. I realize everything about AI changes forever almost every week, but this time AI changed forever more than usual because one of the OpenAI researchers behind the instruction-following work that eventually became ChatGPT just…
Last week, Sam Altman sat frustrated in his bunker as he watched Zuck unveil Muse, a free personal assistant-inspired spy that trains on everything you tell it in exchange for doing your shopping. Not to be outdone, Sam got on a stage of his own in San Francisco just a few days…
Last week a brand new uncensored model entered the AI race. Its name is Ajax and it was created by one of the world's leading experts in screaming AT VIDEO GAMES. >> [screaming] >> IT'S A MODEL THAT WILL POWER THE 90,000 star GitHub project Odysseus, a tool that can self-host…
Я занимаюсь [музыка] в области NLP, а, обработки естественного языка и работаю конкретно сейчас над [музыка] системами распознавания речи, а, которая используются в наших исследованиях, в наших продуктах. И в этом году я являюсь ментором в летней исследовательской программе от…
hello how's everyone doing today we're going over another set of news that I call the fun category the first one isn't so much fun as it is just interesting and that is a blog post by Wiz research that has compromised hugging Face's entire infrastructure so if you recall one of…
hello there today we're looking at orpo monolithic preference optimization without reference model this is a paper by researchers of kaist AI and largely deals with alignment aligning language models and instruction tuned models in some way we'll dive into what that means also I…
we have to talk about open AI there's so much been going on in the entire world of AI and today we're going to have a full plate with just open AI I know there's tons of other stuff going on but just open a has been through ups and downs and ups and downs recently and is finding…
hello everyone today we're going to look at scalable mapol free language modeling by researchers of UC Santa Cruz sucha University UC Davis and Loxy Tech this is a paper that replaces every single matrix multiplication operation inside of large language models with something…
hello there today we're looking at hallucination free assessing the reliability of leading AI legal research tools this is by a set of researchers out of Stanford and Yale as you can see here now I know on this channel we usually look at more kind of core machine learning stuff…
Hello, how's everyone doing? I hope you're having a great summer. I am back from vacation and we're ready to dive into some new papers. This paper is called privacy backdoors stealing data with corrupted pre-trained models as by Sean Long Fung and Florian Tramèr of ETH Zurich.…
hello how is everyone doing long time no see today we're going to look at this paper called scaling llm test time compute optimally can be more efficient effective than scaling model parameters this is a collaboration by Google Deep Mind and UC Berkeley and explores the question…
hello today we're going to look at where rnn's all we needed this is a collaboration of Mila and Borealis Ai and puts into question a couple of these modern takes on kind of RNN style parallela trainable models like S4 and Mamba and things like this so they they kind of question…
hello there today we're going to take a brief look at GSM symbolic understanding of the limitations of mathematical reasoning in large language models this paper is out of apple which uh continues to slowly but surely enter into the research area obviously with uh personal…
hello there today we're going to look at safety alignment should be made more than just a few tokens Deep by Anonymous authors so this is a paper that had been submitted as far as I can tell to eclair is currently on open review and has already gotten reviews and is one of the…
hello there yeah it's cold here um we cannot be dissuaded from reviewing papers today we're going to look at token forer rethinking Transformer scaling with tokenized model parameters this seems to be a collaboration of MOX plunk Institute for informatics Google and picking…
hello there today we're looking at the paper bite lat and Transformer patches scale better than tokens this paper in a sense does away with classic fixed vocabulary based tokenization and in doing so develops a new architecture called The Bite latent Transformer and in their…
Hello. Today we're taking a look at on the biology of a large language model which is a sort of paper published by anthropic on their kind of series on transformer circuits where they investigate what's going on inside of transformer language models and sort of how do they come…