Nonconformal Distortion and Local Orientation Reversal in the Fourier Circuit of a Modular-Addition Transformer
arXiv:2611.00414v1 Announce Type: new Abstract: We study the local differential geometry of a Fourier-based modular-addition circuit in a one-layer transformer whose algorithm has previously been reverse-engineered. For each of the five key Fourier frequencies, we restrict the…
AI Summary & Key Takeaways
- arXiv:2611.00414v1 Announce Type: new Abstract: We study the local differential geometry of a Fourier-based modular-addition circuit in a one-layer transformer whose algorithm has previously been reverse-engineered
- For each of the five key Fourier frequencies, we restrict the…
Nonconformal Distortion and Local Orientation Reversal in the Fourier Circuit of a Modular-Addition Transformer
Related stories
RUCLEAR измерила налог на токенизатор в разных языках
📌CLEAR измерила налог на токенизатор в разных языках Объединение Coalition for Language Equity in AI Research (CLEAR), опубликовало первое из своих исследований языкового равенства. Поставщики моделей берут плату за токен независимо от языка, а токенизаторы обучены в основном…
ChatGPT will soon be able to create animated GIFs.
ChatGPT will soon be able to create animated GIFs. OpenAI has already added support for generating GIF animations to Codex. The function is behind the new image_generation_animation flag. Users will be able to set the number of frames and the duration of each of them, as well as...
StateBench: Testing how language models track state changes over long documents
A Blog post by Ai2 on Hugging Face
Anthropic is still grappling with the economics of compute
At a moment when Anthropic's compute margins should be getting better, they’re getting worse.