h264cpu

h264cpu · standard H.264, Main profile, level 5.1

Can a video decoder execute a program?

This page compiles a small computation into a standard H.264 video stream. After a few start-up frames, the stream repeats the same 16 instruction frames, and each decoded picture supplies the state for the next step. Change the input, and watch your browser's video decoder run it.

JavaScript builds and feeds the stream. The H.264 decoder updates the running state.

1Input

starting pattern · 200 × 200 cells · edges wrap

Click or drag to draw; start a drag on a live cell to erase. Keyboard: arrow keys move (Shift for 10), Space toggles, Enter runs. A preset runs at once.

1Input

the registers at the start

    

Michael Fryers' prime program and its constants. Each word is MOV Rd, Rs: destination in the high byte, source in the low. Execution starts at R1, with one branch delay slot.

2Repeated instructions

– each. From frame – on, the decoder gets only these 16 frames, in turn, forever. They are identical except for a 4-bit frame counter, so they cannot carry the changing state.

3Output, decoded by your browser

generation 0

Writing the stream…

 

Starting…

The same stream as an ordinary MP4. Open it in a standalone player, such as VLC, QuickTime or mpv, and that player's decoder computes the same run, for as long as the file lasts.

What your decoder is fed

JavaScript writes the first frames once: the starting state, as raw samples, then copies so that every delay in the program can read it. After that it only hands your decoder the 16 compiled program frames, over and over, through WebCodecs. The page keeps no copy of the state; it lives in the decoder's reference frames.

Access unitsCoded asWhat they carryBytes

How the decoder computes a generation

H.264 builds a B-frame macroblock by copying blocks from earlier frames and mixing them with explicit weights: clip(((w0·a + w1·b + 2L) >> (L+1)) + o). Each field below is a 200 × 200 board in its own region of the frame, rewritten every frame by one such mix, and motion vectors choose the cells it reads. One generation takes 6 frames, so at 60 frames a second the board advances 10 times a second. Flat out, it advances as fast as your decoder decodes.

FieldWeighted predictionValue
p1 … p4(8·a + 8·b + 64) >> 7, L = 616 per live neighbour, two neighbours each
p5(2·a + 32) >> 6, L = 68 if the cell itself is alive
q1, q2, ca + b, then a + b − 1616 × neighbours − 16
tc + p5 (p5 read 3 frames back)16 × neighbours + 8 × alive − 16
ge24, ge48steep weights against a constant field255 if t ≥ 24, 255 if t ≥ 48
Sge24 − ge48alive next ⇔ 24 ≤ t ≤ 40 ⇔ 3 neighbours, or 2 and alive
d, Hrelu(0.8·H − S), then S + da glow: max(alive, 0.8 × last generation's glow)
r0, r1, Dtwo ramps, one three-colour mix at L = 6the coloured board, at the frame's top-left corner
T(62·a + 32) >> 6 − 1, a at a quarter-pixel offset1920 × 1080 around the board: each macroblock copies the previous frame from 3% nearer the centre, turned back 0.8°, so the decoder interpolates; aging streaks fade and drift blue

How the decoder executes an instruction

H.264 builds a B-frame macroblock by copying blocks from earlier frames and mixing them with explicit weights. With weights 2 and 2 the mix of two fields of 0s and 255s is clip(a + b), an OR; with 2 and −2 it is clip(a − b), a AND NOT b. Every field here is 16 × 64 cells: bit k of register r is cell (r, k), so the register bank is one field. The motion vectors are fixed, so the circuit cannot point at a register: it reads all 64 and keeps the one the instruction names, by masking its row and ORing the rows together with a butterfly of shifts. Rotating is reading one column over, because the board wraps round. The adder's carries run round the columns too, which gives ones' complement's end-around carry for free. The circuit is 334 fields, and its longest path is 17 frames, so one instruction takes 17 frames. The display is part of the circuit: it turns the word into decimal digits and draws their segments.

RegisterReading it givesWriting it
R00shows the word on the display
R1 … R52the registerstores; the program and its constants live here
R53, R54R54 AND NOT R53, R53 AND NOT R54stores
R55, R56, R57R56: R55 if R56 ≠ 0, else R57stores
R58, R59R58 rotated right, R59 rotated leftstores
R60, R61R61: R60 + R61, ones' complementstores
R62NOT R62stores
R63the program counterjumps, after one more instruction