All published intelligence, sovereign stack research, ecosystem news, and AI foundation model reviews.
The famous American philosopher Smashmouth once said, "The years start coming and they don't stop coming." And I've never understood those words more deeply than I did this week after what felt like years in AI bizarro land. At first, on Tuesday, Anthropic released Fable and…
Earlier this week, while I was busy losing all my money in what I thought would be the future of finance, OpenAI claimed to have solved another one of mathematics' hardest problems. This time, it was one invented back in the 1850s, when the only program on Earth was a degenerate…
Last week, Jacob Cockscomb, a 27-year-old AI research bro you've never heard of, did the unthinkable and quit his job at Anthropic before his shares vested. Usually, when somebody leaves a tech company, they humble brag about it by writing a LinkedIn post about their incredible…
Last week, everything about AI changed forever. I realize everything about AI changes forever almost every week, but this time AI changed forever more than usual because one of the OpenAI researchers behind the instruction-following work that eventually became ChatGPT just…
Last week, Sam Altman sat frustrated in his bunker as he watched Zuck unveil Muse, a free personal assistant-inspired spy that trains on everything you tell it in exchange for doing your shopping. Not to be outdone, Sam got on a stage of his own in San Francisco just a few days…
Last week a brand new uncensored model entered the AI race. Its name is Ajax and it was created by one of the world's leading experts in screaming AT VIDEO GAMES. >> [screaming] >> IT'S A MODEL THAT WILL POWER THE 90,000 star GitHub project Odysseus, a tool that can self-host…
Я занимаюсь [музыка] в области NLP, а, обработки естественного языка и работаю конкретно сейчас над [музыка] системами распознавания речи, а, которая используются в наших исследованиях, в наших продуктах. И в этом году я являюсь ментором в летней исследовательской программе от…
hello how's everyone doing today we're going over another set of news that I call the fun category the first one isn't so much fun as it is just interesting and that is a blog post by Wiz research that has compromised hugging Face's entire infrastructure so if you recall one of…
hello there today we're going to look at leave no context behind efficient infinite context Transformers with infinite attention so the specific techniques we're going to look at are called infinite attention and this is by researchers at Google so infinite attention is a…
hello there today we're looking at orpo monolithic preference optimization without reference model this is a paper by researchers of kaist AI and largely deals with alignment aligning language models and instruction tuned models in some way we'll dive into what that means also I…
we have to talk about open AI there's so much been going on in the entire world of AI and today we're going to have a full plate with just open AI I know there's tons of other stuff going on but just open a has been through ups and downs and ups and downs recently and is finding…
hello everyone today we're going to look at scalable mapol free language modeling by researchers of UC Santa Cruz sucha University UC Davis and Loxy Tech this is a paper that replaces every single matrix multiplication operation inside of large language models with something…
hello there today we're looking at hallucination free assessing the reliability of leading AI legal research tools this is by a set of researchers out of Stanford and Yale as you can see here now I know on this channel we usually look at more kind of core machine learning stuff…
Hello, how's everyone doing? I hope you're having a great summer. I am back from vacation and we're ready to dive into some new papers. This paper is called privacy backdoors stealing data with corrupted pre-trained models as by Sean Long Fung and Florian Tramèr of ETH Zurich.…
hello how is everyone doing long time no see today we're going to look at this paper called scaling llm test time compute optimally can be more efficient effective than scaling model parameters this is a collaboration by Google Deep Mind and UC Berkeley and explores the question…
hello today we're going to look at where rnn's all we needed this is a collaboration of Mila and Borealis Ai and puts into question a couple of these modern takes on kind of RNN style parallela trainable models like S4 and Mamba and things like this so they they kind of question…
hello there today we're going to take a brief look at GSM symbolic understanding of the limitations of mathematical reasoning in large language models this paper is out of apple which uh continues to slowly but surely enter into the research area obviously with uh personal…
hello there today we're going to look at safety alignment should be made more than just a few tokens Deep by Anonymous authors so this is a paper that had been submitted as far as I can tell to eclair is currently on open review and has already gotten reviews and is one of the…
hello there yeah it's cold here um we cannot be dissuaded from reviewing papers today we're going to look at token forer rethinking Transformer scaling with tokenized model parameters this seems to be a collaboration of MOX plunk Institute for informatics Google and picking…
hello there today we're looking at the paper bite lat and Transformer patches scale better than tokens this paper in a sense does away with classic fixed vocabulary based tokenization and in doing so develops a new architecture called The Bite latent Transformer and in their…
Hello. Today we're taking a look at on the biology of a large language model which is a sort of paper published by anthropic on their kind of series on transformer circuits where they investigate what's going on inside of transformer language models and sort of how do they come…
Hello and welcome back to our analysis of on the biology of a large language model by anthropic. This is a blog post that anthropic has published uh investigating what they call the biology of a large language model. This works through uh a technique called attribution graphs.…
Hello, how's it going? Today we're going to look at this paper right here. Energy- based transformers are scalable learners and thinkers. This paper combines the uh general concept of energybased models with transformers and suggests a learning paradigm for it and shows how that…
Hello, today we are looking at context rot how increasing input tokens impacts LLM performance. This paper looks at what happens if with these long context models now we are just sort of putting more and more stuff into the context of LLMs and even though all of these models do…
Hello there. Today we are looking at the free transformer from France Fur at Farret Meta. This transformer is extending the classic decoderbased transformer with uh a series of latent variables that can be used to kind of make underlying decisions about the sequence to be…
Hello there. Today we're going to look at on the theoretical limitations of embedding based retrieval. And this paper is really interesting uh because it proves something about embedding based retrieval. So think vector databases in rag and things like this. And the paper proves…
[哼哼]你好。今天我们来看看泰坦在考试时学习记忆的情况。这是 Google 研究的一篇论文，已作为其 Nurips 出版物的一部分推送。所以有一篇关于它的博客文章，人们在社交媒体上发布了关于它的文章，嗯，我不得不承认我……
大家好，今天我们来看看 Nvidia 研究人员的 TiDaR 思维扩散讲座和自回归。这是一篇非常酷的论文，因为它有效地证明了您已经观察到我们在自回归大型语言模型期间没有充分利用 GPU……
The initials GPT stand for Generative Pretrained Transformer. So that first word is straightforward enough, these are bots that generate new text. Pretrained refers to how the model went through a process of learning from a massive amount of data, and the prefix insinuates that…
In the last chapter, you and I started to step through the internal workings of a transformer. This is one of the key pieces of technology inside large language models, and a lot of other tools in the modern wave of AI. It first hit the scene in a now-famous 2017 paper called…