05 · story
I taught a computer to watch cricket. Then it learned a few other tricks.
Thousands of video clips, and one question about each: what exactly did that ball do? I had never done this kind of work before.
1The job
The job I set it: take a clip from a TV broadcast, anything from 13 seconds to over two minutes, and in under 30 seconds, on an ordinary computer, say what a single ball did. How fast it was, where it bounced, how high it passed the stumps.
What a finished answer looks like: my system’s read of a few balls.
2Then, work out where the camera is
The camera never sits still. To turn pixels into metres you need things of a known size in the picture. A cricket pitch has them: the stumps and the painted lines are the same all over the world.

My first version found them with hand-written rules and was off by about 150 pixels. Then I let a small neural network learn from the system’s own most confident guesses. The error fell to about 3 pixels.
Where the computer thinks the pitch is
Off by about 150 pixels. Every measurement after this is wrong.Off by about 3 pixels. Now the numbers mean something.
3Side trick one: paint on the grass
By now the computer knows, for any frame from a moving TV camera, exactly where the ground is and who is standing on it. So it can paint a picture on that ground. The picture stays put while the camera moves, and players walk across it.
Watch the grass on both sides of the pitch, and the umpire’s back. Nothing in this clip was there on the day.
4Side trick two: the boards around a football pitch
The same idea on the advertising boards of a football stadium, and on the grass in front of them.
5Side trick three: who is where
And once you can find the ground and the people, you can follow every player and the ball through a football match, straight from the TV picture.
6What I took from it
None of this was the plan. It all fell out of teaching a computer to watch one ball.