Overview of the term paper
Markov chains, made interactive.
Six modules turn the chapters of the paper into models you can touch. Each one covers a section of the theory, from the Markov property to PageRank and the limits of the assumption. Everything runs live in your browser, and every number is computed from the model in front of you. Read a chapter, then try the matching module here.
How to read this page. A Markov chain is a system that jumps between a finite set of states. The next state depends only on the current one, not on the full history. That single rule is called the Markov property. Every module below looks at it from a different angle.
The living state graph
A small weather model with three states. A token jumps from circle to circle, following the probabilities on the arrows. Watch the next jump. It depends only on the circle the token sits in right now. The trail below has no effect on what comes next. That is what memorylessness looks like.
Currently in Sunny
From Sunny the next hop is drawn from row 1 of the matrix: Sunny 70%, Cloudy 20%, Rainy 10%.
Visited states (1)
The transition matrix editor
A Markov chain is fully described by its transition matrix P. Change any cell and the graph on the right redraws at once. A thicker arrow means a higher probability. Each row has to sum to exactly 1, because every state must go somewhere. Rows that do not are marked in red.
| → A | → B | → C | Σ | |
|---|---|---|---|---|
| A → | 1.00 | |||
| B → | 1.00 | |||
| C → | 1.00 |
Valid stochastic matrix. Every row sums to 1.
Thicker arrow = higher transition probability.
The convergence visualizer
Start fully certain that the weather is Sunny, so one bar holds 100%. Each step multiplies the distribution by P and spreads the probability across the states. Drag the slider. The bars settle into a fixed shape that stops changing. That shape is the stationary distribution π. Here it is computed live, not just claimed.
Stationary distribution π
The unique vector with π = π·P. Independent of where you start.
Keep stepping. The bars have not settled onto π yet.
Try starting from a different state in your head. This chain is irreducible and aperiodic, so the bars always drift to the same π. Forgetting where it started is what makes the long-run behaviour predictable.
The PageRank sandbox
PageRank treats a random web surfer as a Markov chain over pages, and links are the transitions. Build your own small web. Switch to Add page and click the canvas to drop pages, or stay in Add linkand click one page, then another, to connect them. The rank is the surfer’s long-run chance of visiting a page. It updates live through power iteration, and node size and shade follow it.
Ranking (5 pages, 6 links)
- 1.Page 30.468
- 2.Page 20.203
- 3.Page 10.11
- 4.Page 40.11
- 5.Page 50.11
The numbers inside the nodes are the rank in percent. A page matters when important pages link to it. One link from a strong page counts for more than many links from weak ones.
The Markov text generator
Here the states are words. From a body of text, the model learns how likely each next word is for a given context. Then it takes a random walk through those probabilities. Choose the order, which is how many previous words count as context. Generate a passage, then click any word to see the distribution it came from. The text reads well word by word, but drifts into nonsense overall. That is a preview of Chapter 4.
Generated text
Press “Generate text” to take a random walk through the word graph.
Where the Markov assumption breaks
Both passages continue the same prompt. On the left is a Markov chain that only remembers the last word or two. Each pair of neighbouring words is plausible, but the passage forgets the box, contradicts itself, and never gets to a point. On the right is a model with a long context window. It keeps the box, the map and the grandmother connected. Drag the divider to compare. This is why modern models such as LSTMs and Transformers drop the strict Markov property.
Prompt: “The scientist opened the ancient box and inside she found…”
The scientist opened the ancient box and inside she found a folded map, its ink faded but still legible. She recognised the coastline at once. It was the same shore her grandmother had described in the letters. She traced the marked route back to the box itself, and only then did she understand why it had been hidden for so long.
The scientist opened the ancient box and inside she found the sea was wild and the keeper wrote long letters to the tower and the wind carried their cries across the water was calm on some days and the lamp turned and turned and nobody remembered the box again after that morning.
What to take away
Six modules, one idea. Each one shows the Markov property from a different side. Here is the short version.
One state at a time
A Markov chain only knows where it is right now. The past does not steer the next step.
A matrix is the whole model
Every rule of the chain lives in the transition matrix. Each row is a set of probabilities that sums to 1.
The long run settles
Step the chain long enough and the distribution stops moving. That fixed shape is the stationary distribution.
The same math ranks the web
PageRank is a Markov chain over pages. A page is important when important pages link to it.
It writes, but only locally
Word-level chains read well from word to word, yet they lose the thread of a whole passage.
Memory is the limit
Long-range meaning needs memory the Markov property drops. That gap is why newer models replace it.
One surprising thing
A Markov chain forgets everything about its past, yet its long-run behaviour is often fully predictable. Losing the memory is exactly what makes the future easy to describe.
These modules sit next to the written paper. Read a chapter, then come back here and try the matching model for yourself.