January
1913
Andrei Markov opened Pushkin's Eugene Onegin and counted the first 20,000 letters by hand.
Not out of literary interest, but to win an argument. A rival had claimed that the law of large numbers only works for fully independent events, and used that claim to argue for free will. Markov set out to prove him wrong with dependent data.
In his data, a vowel is followed by another vowel only 12.8% of the time.
vowel then vowel
vowel then consonant
These two numbers are a first transition probability: given the current letter, how likely is each next one. The letters were far from independent, yet the law of large numbers still held once that dependence was written down. That is the whole idea of a Markov chain, and everything ahead builds on it.
The one rule that starts everything
The next state depends only on the present one.
The entire past can be forgotten.
Whether the sun shines tomorrow depends on today's weather, not on last week's. That single rule, called memorylessness, is what makes the whole model tractable.
A whole model in one grid
| S | C | R | ||
|---|---|---|---|---|
| S | 0.60 | 0.30 | 0.10 | = 1.0 |
| C | 0.20 | 0.50 | 0.30 | = 1.0 |
| R | 0.10 | 0.30 | 0.60 | = 1.0 |
Every row sums to exactly 1.
Each number is a named, checkable probability.
Rain in two days, starting sunny? 21%.
0.6·0.1 + 0.3·0.3 + 0.1·0.6 = 0.21 — the sum over every path through an intermediate state (Chapman-Kolmogorov).
It forgets where it began
Start fully certain it is sunny.
Each step multiplies by P and spreads the probability.
It settles at 27.5 / 37.5 / 35 — the same limit from any start.
The web is one giant chain
A surfer clicks random links, forever.
Rank is just how often the walk lands on a page in the long run.
Sometimes it teleports to a random page.
That damping (d = 0.85) makes the chain irreducible, so a unique ranking is guaranteed to exist.
C wins. D hits the floor.
C: 38% · A: 37% · B: 21% · D: 3.75%. D links generously but earns nothing, because nothing points back to it.
A chain that breaks ciphers
AHOVCJQXELSZGNUBIPWDKRYFMTAHOVCJQXELSZGNUBIPWD
Guess a key. Swap two letters. Keep the guess if the text looks more like English.
“After 100 steps, a mess. After two thousand steps, it makes sense.”
The unknown normalizing constant cancels in the acceptance ratio. That is exactly why MCMC works where direct sampling fails.
Fluent word by word, lost overall
The scientist opened the box and found a folded map that pointed home glass tides humming corridors of salt birds forgetting the letters entirely
With memory of only the last word or two, the model cannot keep a sentence, let alone a paragraph, coherent.
Just remember more? The parameters explode.
Each extra word of context multiplies the model size. This is the curse of dimensionality that pushed the field toward neural models.
Not a winner, a map
Every parameter has a name. Every prediction traces back to elementary probability.
Fully interpretableBillions of weights capture long-range meaning no single number can explain.
More powerful, opaqueChoose Markov when the present holds all that matters, or when a decision must be auditable.
Choose the black box when reach beats explanation.
A poem became a tool that still proves its worth.
The same idea runs through every chapter of the paper. A system moves between states, the next step depends only on the present one, and the whole model is a matrix you can read. That is also its limit: no long-term memory. Modern language models keep the state idea but drop the strict Markov property to gain that memory. The trade is interpretability for power.