NVENC vs libx264 compared
Compare NVENC hardware encoding against libx264 on the same clip. The GPU job cost far less and produced a file twice the size, which is the real trade.
Abdelrahman Essawy 4 MIN READThe usual pitch for hardware encoding is that it is faster. On a short clip it was not, and the number that actually moved was the bill.
Short version. On an L4, hevc_nvenc cost $0.0008. The same libx264 command measured $0.0023 to $0.0039 across ten runs, so the GPU sits below that whole band, somewhere around three to five times cheaper. It also produced a bigger file. On speed we cannot separate it from noise, and that is covered below.
| Variant | Encode | Size | Cost |
|---|---|---|---|
| libx264 medium (CPU) | 913 ms fastest | 433.0 KB | $0.0030 |
| libx264 veryfast (CPU) | 1371 ms | 388.4 KB | $0.0031 |
| h264_nvenc p4 (GPU) | 1225 ms | 956.2 KB | $0.0018 |
| h264_nvenc p7 (GPU) | 1762 ms | 1127.4 KB | $0.0011 |
| hevc_nvenc p4 (GPU) | 1171 ms | 501.1 KB | $0.0008 |
The GPU was cheaper per job
This is the result worth sitting with, because it runs against the intuition that GPU time is expensive time.
| Cost | Size | |
|---|---|---|
| libx264 medium (CPU) | $0.0030 | 433 KB |
| h264_nvenc p4 (GPU) | $0.0018 | 956 KB |
| hevc_nvenc p4 (GPU) | $0.0008 | 501 KB |
An L4 is billed per second, and NVENC uses a dedicated encode block rather than the CPU, so the work finishes in fewer billed seconds than a CPU encode that has to think. hevc_nvenc came out cheapest of everything measured while producing a file only 16% larger than libx264 medium.
How solid is that? We ran the identical libx264 command ten times across different sweeps and it cost between $0.0023 and $0.0039, a 1.7x spread on identical work. hevc_nvenc at $0.0008 sits below that entire range, so the direction is safe even though the exact multiple is not. Call it three to five times cheaper and do not quote a decimal.
On speed, we cannot tell you
An earlier version of this post said the GPU was not faster, citing 1,225 ms against libx264’s 913 ms. That claim does not survive its own data and is retracted.
We ran that libx264 command ten times. It took between 898 ms and 1,912 ms, a 2.13x spread, while producing a byte-identical file every time. NVENC’s 1,225 ms sits comfortably inside that band, so this sweep cannot separate the two on speed at all. See FFmpeg benchmark variance for what that spread looks like and why sizes are trustworthy where timings are not.
What can still be said: hardware encoding is designed to win on throughput, many streams at once or long inputs where the encode block runs flat out. A five-second clip is the wrong test for that, and this sweep does not attempt it.
The size penalty is real and large
h264_nvenc p4 produced 956 KB where libx264 medium produced 433 KB. Even allowing for the fact that -cq 23 and -crf 23 are not the same scale, 2.2x is not a calibration artefact. NVENC’s rate control is simpler than x264’s, and it shows up as bytes.
This is the actual trade, and it is not the one the marketing describes. You are not buying speed. You are buying cheaper jobs at the cost of larger files, and whether that is a good deal depends entirely on whether you pay more for compute or for bandwidth.
Encode once and serve a million times, and 2.2x the bytes is ruinous. Transcode a huge library that is mostly never watched, and being several times cheaper per job is the only number on the page that matters.
The slower NVENC preset made a bigger file
p7 is NVENC’s slowest, highest-effort preset. It produced 1,127 KB against p4’s 956 KB, and took longer doing it.
Same mechanism as FFmpeg x264 presets compared: -cq targets quality, not size, so a preset that finds more detail is free to spend more bytes describing it. If you are choosing an NVENC preset by output size, measure it. Assuming slower means smaller will mislead you here exactly as it does on the CPU side.
About the quality settings
-cq 23 for NVENC, -crf 23 for x264, -cq 28 for HEVC. These are each encoder’s own conventional scale and they are not interchangeable numbers.
So read the table as “cost and size at settings a practitioner would actually reach for”, not as matched quality. A matched-quality comparison needs VMAF scoring on every output, which would change the size column and is a different piece of work.
What the mismatch cannot explain away is the cost gap. Billed seconds are noisy too, which is why this post gives a range rather than a figure, but a 1.7x noise band does not close a gap of that size.
Where this stops
One 5-second 1280x720 clip, audio stripped, on an L4. Short inputs are the worst case for hardware encoding. Longer inputs and concurrent streams are where NVENC is designed to win, and this sweep deliberately does not test that.
The sizes here are exact and repeatable. The timings and the costs are single samples from a distribution, which is why this post states a range for cost and refuses to state one at all for speed.
Hardware encoding is one of several codec paths measured on this source. The full set is in Video codec comparison, measured.
Frequently asked questions
Is NVENC cheaper than CPU encoding?
On an L4 it was. hevc_nvenc cost $0.0008 where the same libx264 command measured $0.0023 to $0.0039 across ten runs, so the GPU sits below that entire band.
Is NVENC faster than libx264?
This sweep cannot tell you. NVENC took 1,225 ms and libx264 ranged 898 to 1,912 ms on repeated runs, so the GPU result falls inside the CPU's own noise band on a clip this short.
Why is the NVENC file bigger?
NVENC's rate control is simpler than x264's, and it showed up as 2.2x the bytes at a nominally comparable quality setting. That is the real trade: cheaper jobs, larger files.
Does a slower NVENC preset give a smaller file?
Not necessarily. p7 produced 1,127 KB against p4's 956 KB at the same -cq, because -cq targets quality and a harder search can spend more bytes on detail it finds.