AV1 vs H.264 VMAF compared

Our earlier AV1 vs VP9 vs HEVC vs H.264 post ran each codec at its own conventional CRF and said, in its own caveats, that this compares settings rather than codecs. Here is the version that does not have that problem.

Short version. Hold every codec to the same byte ceiling and score the result with VMAF, and AV1 delivers 84.8 where H.264 delivers 64.3. Twenty points of measured quality, for free, at identical size.

Same size, different quality#

The sizes are within 5% of each other. The quality scores are not close at all.

VMAF is a perceptual metric trained on human scoring, and the rough reading is that above 90 is hard to distinguish from source, 80s is good, 70s is visibly soft, and 60s is where most viewers would call it bad. So at this bitrate H.264 produces something people would complain about and AV1 produces something they would not.

That is the whole argument for a modern codec, stated in the only way that actually settles it: fix the bytes, measure the quality.

Why this comparison is different#

Codec comparisons usually fix the setting and report the size. That is easy to run and nearly meaningless, because CRF 23 does not mean the same thing to x264 and to libaom, and there is no conversion table between them.

Fixing the size instead requires a search: encode, measure, adjust, repeat until you land on the ceiling. That is what compress.target does, and it is why these rows carry a probe count. The AV1 result took 6 probe encodes to find its level. Across the full sweep the search ran 92 probe encodes.

That is also why this is rarely published. Doing it by hand means writing the search loop and running a VMAF pass on every candidate.

The part that criticises our own default#

codec: auto chose H.264, and H.264 was the worst performer in the table.

Auto is not broken. It optimises for compatibility, and H.264 plays on essentially every device made in the last fifteen years while AV1 does not. For a general-purpose default that is the right call, and it is the same reasoning behind yuv420p being the right default.

But it means auto is not the setting to use if you care about quality per byte. If your players support AV1, ask for AV1. On this evidence that request is worth 20 VMAF points at no cost in file size, and no amount of tuning H.264 will close that gap.

The floor nobody mentions#

Two cases in the sweep asked for ceilings the codec could not reach. maxBytes 100 KB and maxBytes 50 KB both returned the same file: 111.5 KB at VMAF 30.7.

The job did not pretend. It returned a low verdict and the file it could actually produce, rather than shipping something that claimed to be 50 KB.

VMAF 30.7 is well past unusable. What that row really says is that this source has a floor, and asking for half of it does not move the floor. Any compression API that returns exactly what you asked for at any target is either lying or destroying the content.

Where this stops#

One 5-second 1280x720 clip. VMAF is a model of human perception, not human perception, and it is known to be generous to some kinds of blur and harsh on some kinds of grain. A 20-point gap is far outside the range where those criticisms would reverse an ordering, but a 2-point gap would not be.

Encode cost is not in this table. AV1 is slower to encode than H.264 and SVT-AV1 presets compared puts numbers on that. If you are transcoding a library once and serving it forever, that trade is obviously worth making. If you are transcoding constantly and serving rarely, it is not.

This is one of several codec comparisons on the same source. The full set is in Video codec comparison, measured.