# Measuring the entropy of English

**Published:** June 12, 2026
**Author:** 3Blue1Brown
**Source:** YouTube — Andrej Karpathy
**Language:** en-US
**HTML:** https://aiqazaqstan.com/en/articles/measuring-the-entropy-of-english

Claude Shannon figured out how the compressibility of English text depends on how predictable it is, so he made multiple attempts at understanding the probability of each new character in English. His earliest experiments involved looking at specific short sequences of…

> From the original source: **YouTube — Andrej Karpathy**
> [Read full article at YouTube — Andrej Karpathy](https://www.youtube.com/shorts/-7etvZSBxlk)

**Tags:** #Anthropic #YouTube

---

*AI Qazaqstan — non-commercial informational portal about AI in Kazakhstan. Machine translations and summaries are AI-generated.*