Overview of the term paper
Markov chains, made interactive.
Seven modules turn the chapters of the paper into models you can touch. Each one covers a section of the theory, from the Markov property through PageRank and a real World Cup shoot-out to the limits of the assumption. Everything runs live in your browser, and every number is computed from the model in front of you. Read a chapter, then try the matching module here.
How to read this page. A Markov chain is a system that jumps between a finite set of states. The next state depends only on the current one, not on the full history. That single rule is called the Markov property. Every module below looks at it from a different angle.
The living state graph
A small weather model with three states. A token jumps from circle to circle, following the probabilities on the arrows. Watch the next jump. It depends only on the circle the token sits in right now. The trail below has no effect on what comes next. That is what memorylessness looks like.
Currently in Sunny
From Sunny the next hop is drawn from row 1 of the matrix: Sunny 70%, Cloudy 20%, Rainy 10%.
Visited states (1)
The transition matrix editor
A Markov chain is fully described by its transition matrix P. Change any cell and the graph on the right redraws at once. A thicker arrow means a higher probability. Each row has to sum to exactly 1, because every state must go somewhere. Rows that do not are marked in red.
| → A | → B | → C | Σ | |
|---|---|---|---|---|
| A → | 1.00 | |||
| B → | 1.00 | |||
| C → | 1.00 |
Valid stochastic matrix. Every row sums to 1.
Thicker arrow = higher transition probability.
The convergence visualizer
Start fully certain that the weather is Sunny, so one bar holds 100%. Each step multiplies the distribution by P and spreads the probability across the states. Drag the slider. The bars settle into a fixed shape that stops changing. That shape is the stationary distribution π. Here it is computed live, not just claimed.
Stationary distribution π
The unique vector with π = π·P. Independent of where you start.
Keep stepping. The bars have not settled onto π yet.
Try starting from a different state in your head. This chain is irreducible and aperiodic, so the bars always drift to the same π. Forgetting where it started is what makes the long-run behaviour predictable.
The PageRank sandbox
PageRank treats a random web surfer as a Markov chain over pages, and links are the transitions. Build your own small web. Switch to Add page and click the canvas to drop pages, or stay in Add linkand click one page, then another, to connect them. The rank is the surfer’s long-run chance of visiting a page. It updates live through power iteration, and node size and shade follow it.
Ranking (5 pages, 6 links)
- 1.Page 30.468
- 2.Page 20.203
- 3.Page 10.11
- 4.Page 40.11
- 5.Page 50.11
The numbers inside the nodes are the rank in percent. A page matters when important pages link to it. One link from a strong page counts for more than many links from weak ones.
The 2022 World Cup final, kick by kick
Not every chain settles into a stationary distribution. A shoot-out instead runs until it hits one of two absorbing states: Argentina win, or France win. Give every kick the same conversion chance p, and one recursion, V(s) = p·V(goal) + (1−p)·V(miss), assigns each moment a win probability. Press play to relive the real shoot-out, or drag p to rebuild the whole curve. At p = 0.75 the numbers match the paper exactly.
Argentina win probability
matches the paperAfter 8 kicks
Kick log
- 1FRAMbappégoal43.0-7.0
- 2ARGMessigoal50.0+7.0
- 3FRAComanmiss73.1+23.1
- 4ARGDybalagoal80.8+7.7
- 5FRATchouaménimiss94.9+14.1
- 6ARGParedesgoal98.2+3.4
- 7FRAKolo Muanigoal97.7-0.6
- 8ARGMontielgoal100.0+2.3
A shared p = 0.75 comes from Vollmer, Schoch and Brandes (2024), who studied roughly 50,000 penalties and found no reliable first-kicker advantage, so both teams start even at 50%. The red crosses are France’s two missed kicks, and Coman’s miss is the single largest swing of the night, +23.1 points. These numbers isolate the structure the rules impose; they deliberately ignore the individual kicker, the goalkeeper, fatigue and pressure, which is exactly why every update stays reproducible from one stated parameter.
The Markov text generator
Here the states are words. From a body of text, the model learns how likely each next word is for a given context. Then it takes a random walk through those probabilities. Choose the order, which is how many previous words count as context. Generate a passage, then click any word to see the distribution it came from. The text reads well word by word, but drifts into nonsense overall. That is a preview of Chapter 4.
Generated text
Press “Generate text” to take a random walk through the word graph.
Where the Markov assumption breaks
Both passages continue the same prompt. The Markov chain only remembers the last word or two: each pair of neighbouring words is plausible, yet it forgets the box, wanders off, and never reaches a point. The long-context model keeps the box, the map and the grandmother connected from start to finish. Watch which of the three anchor words each passage manages to hold on to. This is why modern models such as LSTMs and Transformers drop the strict Markov property.
Prompt: “The scientist opened the ancient box and inside she found…”
The scientist opened the ancient box and inside she found the sea was wild and the keeper wrote long letters to the tower and the wind carried their cries across the water was calm on some days and the lamp turned and turned and nobody remembered the box again after that morning.
Locally fluent, globally lost. Grammar never breaks between neighbours, but with no long-term memory the thread drifts and the map and grandmother never appear.
The scientist opened the ancient box and inside she found a folded map, its ink faded but still legible. She recognised the coastline at once. It was the same shore her grandmother had described in the letters. She traced the marked route back to the box itself, and only then did she understand why it had been hidden for so long.
Dependencies at a distance. The box, map and grandmother stay linked across the whole passage — exactly the long-range memory the Markov assumption throws away.
What to take away
Seven modules, one idea. Each one shows the Markov property from a different side. Here is the short version.
One state at a time
A Markov chain only knows where it is right now. The past does not steer the next step.
A matrix is the whole model
Every rule of the chain lives in the transition matrix. Each row is a set of probabilities that sums to 1.
The long run settles
Step the chain long enough and the distribution stops moving. That fixed shape is the stationary distribution.
The same math ranks the web
PageRank is a Markov chain over pages. A page is important when important pages link to it.
Some chains are built to end
A penalty shoot-out has absorbing states. One recursion turns the live score into each team's win probability.
It writes, but only locally
Word-level chains read well from word to word, yet they lose the thread of a whole passage.
Memory is the limit
Long-range meaning needs memory the Markov property drops. That gap is why newer models replace it.
One surprising thing
A Markov chain forgets everything about its past, yet its long-run behaviour is often fully predictable. Losing the memory is exactly what makes the future easy to describe.
These modules sit next to the written paper. Read a chapter, then come back here and try the matching model for yourself.