Smart portal about artificial intelligence and high technology in Kazakhstan
hello there today we're going to look at Transformer F feedback attention is working memory by group of researchers out of Google this paper adds what they call feedback attention to Transformers and they do so in an attempt to build a concept of working memory working memory in…
![[ML News] Chips, Robots, and Models](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FtRavLU8Ih4A%2Fhqdefault.jpg&w=3840&q=75)
hello hello uh today we're going to look at a few happenings in Industry new models new data sets and eval sets coming out and a few new tools that have been released or that could be useful and I just happen to have discovered in the last 2 weeks so first news meta is releasing…
hello there today we're looking at orpo monolithic preference optimization without reference model this is a paper by researchers of kaist AI and largely deals with alignment aligning language models and instruction tuned models in some way we'll dive into what that means also I…
![[ML News] OpenAI is in hot waters (GPT-4o, Ilya Leaving, Scarlett Johansson legal action)](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2F5bPBbQyLI7E%2Fhqdefault.jpg&w=3840&q=75)
we have to talk about open AI there's so much been going on in the entire world of AI and today we're going to have a full plate with just open AI I know there's tons of other stuff going on but just open a has been through ups and downs and ups and downs recently and is finding…
hello there today we're going to look at xlsm extended long short-term memory by maximilan Beck corbinian Pepple and a team around them especially the corresponding author here is Zep hiter who you may remember was the original or among the original inventors of lstm's long…
hello everyone today we're going to look at scalable mapol free language modeling by researchers of UC Santa Cruz sucha University UC Davis and Loxy Tech this is a paper that replaces every single matrix multiplication operation inside of large language models with something…
hello there today we're looking at hallucination free assessing the reliability of leading AI legal research tools this is by a set of researchers out of Stanford and Yale as you can see here now I know on this channel we usually look at more kind of core machine learning stuff…
Hello, how's everyone doing? I hope you're having a great summer. I am back from vacation and we're ready to dive into some new papers. This paper is called privacy backdoors stealing data with corrupted pre-trained models as by Sean Long Fung and Florian Tramèr of ETH Zurich.…
hello how is everyone doing long time no see today we're going to look at this paper called scaling llm test time compute optimally can be more efficient effective than scaling model parameters this is a collaboration by Google Deep Mind and UC Berkeley and explores the question…
hello today we're going to look at where rnn's all we needed this is a collaboration of Mila and Borealis Ai and puts into question a couple of these modern takes on kind of RNN style parallela trainable models like S4 and Mamba and things like this so they they kind of question…
hello there today we're going to take a brief look at GSM symbolic understanding of the limitations of mathematical reasoning in large language models this paper is out of apple which uh continues to slowly but surely enter into the research area obviously with uh personal…
hello there today we're going to look at safety alignment should be made more than just a few tokens Deep by Anonymous authors so this is a paper that had been submitted as far as I can tell to eclair is currently on open review and has already gotten reviews and is one of the…
hello there yeah it's cold here um we cannot be dissuaded from reviewing papers today we're going to look at token forer rethinking Transformer scaling with tokenized model parameters this seems to be a collaboration of MOX plunk Institute for informatics Google and picking…
hello there today we're looking at the paper bite lat and Transformer patches scale better than tokens this paper in a sense does away with classic fixed vocabulary based tokenization and in doing so develops a new architecture called The Bite latent Transformer and in their…
![[GRPO Explained] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FbAWV_yrqx4w%2Fhqdefault.jpg&w=3840&q=75)
Hello there. Today we're looking at DeepSeekMath, pushing the limits of mathematical reasoning in open language models. Now, this paper is a bit older. Um, it's from, as you can see over here, from the 27th of April, 2024. Nevertheless, it introduces something called GRPO, the…
Hello. Today we're taking a look at on the biology of a large language model which is a sort of paper published by anthropic on their kind of series on transformer circuits where they investigate what's going on inside of transformer language models and sort of how do they come…
Hello and welcome back to our analysis of on the biology of a large language model by anthropic. This is a blog post that anthropic has published uh investigating what they call the biology of a large language model. This works through uh a technique called attribution graphs.…
Hello, how's it going? Today we're going to look at this paper right here. Energy- based transformers are scalable learners and thinkers. This paper combines the uh general concept of energybased models with transformers and suggests a learning paradigm for it and shows how that…
Hello, today we are looking at context rot how increasing input tokens impacts LLM performance. This paper looks at what happens if with these long context models now we are just sort of putting more and more stuff into the context of LLMs and even though all of these models do…
Hello. I'm out of my usual room today. As you can see, I'm flying to the US for work, but I wanted to give some quick updates on OpenAI's new model launches the past week. So, they've released an open- source model in different variants, and they've also released a new frontier…
![[Paper Analysis] The Free Transformer (and some Variational Autoencoder stuff)](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FNao16-6l6dQ%2Fhqdefault.jpg&w=3840&q=75)
Hello there. Today we are looking at the free transformer from France Fur at Farret Meta. This transformer is extending the classic decoderbased transformer with uh a series of latent variables that can be used to kind of make underlying decisions about the sequence to be…
![[Video Response] What Cloudflare's code mode misses about MCP and tool calling](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2F0bpYCxv2qhw%2Fhqdefault.jpg&w=3840&q=75)
Hello, this is in many ways a response video to Theo or T3 GGG's video called MCP is the wrong abstraction and it is also a video about this Cloudflare article which that video is about called code mode the better way to use MCP and I want to show you this article but also point…
![[Paper Analysis] On the Theoretical Limitations of Embedding-Based Retrieval (Warning: Rant)](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FzKohTkN0Fyk%2Fhqdefault.jpg&w=3840&q=75)
Hello there. Today we're going to look at on the theoretical limitations of embedding based retrieval. And this paper is really interesting uh because it proves something about embedding based retrieval. So think vector databases in rag and things like this. And the paper proves…
[snorts] Hello. Today we're going to look at Titans learning to memorize at test time. This is a paper by Google research and has been pushed as part of their Nurips publications. So there's a blog post about it and people posted on social media about it and um I have to admit I…
Hello, today we're looking at TiDaR Thinking Diffusion Talk and Autoregression by researchers at Nvidia. And this is a really cool paper because it effectively um makes good you already observes that we don't have full GPU utilization during autoregressive large language model…
Hello, have you ever been in a situation where you're overhearing conversation and it dawns on you this person well-intended is wrong about something minuscule but important to you and duty the universe compels you to tell them why they're wrong. Watch this. I recycle plastic…
If you can add and multiply, then you can do matrix stuff! Hooray! StatQuest. Hello, I'm Josh Starmer and welcome to StatQuest. Today we're going to talk about essential matrix algebra for neural networks and it's going to be clearly explained. Start small but then build as big…
hello I'm Josh starmer and welcome to human stories in AI with stack Quest and lightning AI in this series we'll hear about the career journeys of passionate AI experts from their humble beginnings to conquered challenges will be inspired by the realworld experiences of…
hello I'm Josh starmer and welcome to human stories in AI with stack Quest and lightning AI in this series we'll hear about the career journeys of passionate AI experts from their humble beginnings to conquered challenges will be inspired by the realworld experiences of…
The initials GPT stand for Generative Pretrained Transformer. So that first word is straightforward enough, these are bots that generate new text. Pretrained refers to how the model went through a process of learning from a massive amount of data, and the prefix insinuates that…