Parsing ffprobe metadata, five fields that mislead
Read the ffprobe fields that mislead a parser: rotation, still-image duration, missing channel layout, cover art as video and HDR, with real JSON for each.
Abdelrahman Essawy 5 MIN READFive ffprobe fields mislead code that reads them literally. Rotation sits in side data while width and height describe the stored frame, a still image reports a 0.04 second duration, a WAV can omit its channel layout, album art appears as a video stream, and HDR has no field of its own. Each section below shows the JSON that proves it and the rule that turns the field into something you can act on.
I learned these building the normaliser that produces the summary in Rendobar’s ffprobe job. Where a probe in Rendobar’s job history shows the problem, the JSON comes from that job. Where none does, it comes from ffprobe 8.0 on a test file, with the command that made the file so you can reproduce it.
Rotation is side data, and width and height do not change
A phone often records a portrait clip as a landscape frame plus an instruction to rotate it on playback. This makes one:
ffmpeg -display_rotation 90 -i landscape.mp4 -c copy portrait.mp4ffprobe -v error -select_streams v:0 \ -show_entries stream=width,height:stream_side_data -of json portrait.mp4"streams": [ { "width": 1920, "height": 1080, "side_data_list": [ { "side_data_type": "Display Matrix", "displaymatrix": "\n00000000: 0 -65536 0\n00000001: 65536 0 0\n00000002: 0 0 1073741824\n", "rotation": 90 } ] }]ffprobe reports 1920 by 1080, the stored frame. The Display Matrix says to rotate it 90 degrees, so it plays as a 1080 by 1920 portrait video. Read width and height alone and you treat a portrait video as landscape: the aspect ratio is inverted, a letterbox is computed for the wrong shape, and a thumbnail comes out sideways.
The rule is to find the side_data_list entry whose side_data_type is Display Matrix, normalise its rotation to 0, 90, 180 or 270 (a value of -90 is 270), and swap width and height when it is 90 or 270. Match on the type, because a stream can carry other side data. Older FFmpeg builds also copied the angle into a rotate tag. ffprobe 8.0 does not: the stream’s tags for the file above hold language, handler_name, vendor_id and encoder, and nothing else.
An earlier version of this post described the same clip as reporting 1080x1920, and its FAQ as 1920x1080. The stored frame is 1920x1080. The JSON above settles it.
A still image has a 0.04 second duration
This is the probe of a JPEG from Rendobar’s job history, the site’s own 1200 by 630 share image:
"format": { "filename": "https://cdn.rendobar.com/assets/brand/og-default.jpg", "format_name": "image2", "format_long_name": "image2 sequence", "duration": "0.040000", "size": "155613", "bit_rate": "31122600"},"streams": [ { "codec_type": "video", "codec_name": "mjpeg", "width": 1200, "height": 630, "r_frame_rate": "25/1", "duration": "0.040000" }]The image2 demuxer treats a single picture as one frame of a 25 fps sequence, so the file “lasts” one frame, 0.04 seconds. The bitrate follows from that: 155,613 bytes over 0.04 seconds is 31.1 Mbps. Neither number describes the image.
A PNG in the same history came through png_pipe and reported no duration and no bitrate at all, so the two image demuxers disagree with each other as well. The rule is to treat any file whose format_name is image2 or ends in _pipe as an image, and set duration and bitrate to null.
A WAV can leave out channel_layout
From a probe of a short WAV in the job history:
"streams": [ { "codec_type": "audio", "codec_name": "pcm_s16le", "sample_rate": "8000", "channels": 1, "duration": "1.000000", "bit_rate": "128000" }]There is a channel count and no channel_layout key. An MP3 in the same history reported "channels": 2 with "channel_layout": "stereo", so code that reads the layout string works on the MP3 and gets undefined on the WAV.
The rule is to treat channel_layout as optional. A count of 1 is mono and 2 is stereo by convention. Above that, the count alone does not say whether six channels are 5.1 or something else, so leave the layout unknown.
Album art counts as a video stream
An MP3 with an embedded cover has two streams. This makes one and probes it:
ffmpeg -i song.mp3 -i cover.jpg -map 0:a -map 1:v -c copy \ -id3v2_version 3 -disposition:v:0 attached_pic withcover.mp3ffprobe -v error -show_entries \ stream=index,codec_type,codec_name,width,height:stream_disposition=attached_pic \ -of json withcover.mp3"streams": [ { "index": 0, "codec_name": "mp3", "codec_type": "audio", "disposition": { "attached_pic": 0 } }, { "index": 1, "codec_name": "mjpeg", "codec_type": "video", "width": 300, "height": 300, "disposition": { "attached_pic": 1 } }]Count video streams to decide whether a file is a video and this song is one. The tell is disposition.attached_pic. Drop the streams where it is 1 before you classify the file, and a song with a cover goes back to being audio.
HDR is the transfer function, not the primaries
There is no "hdr": true. HDR is signalled by the transfer characteristics: color_transfer of smpte2084 is PQ, used by HDR10, and arib-std-b67 is HLG. The colour primaries do not decide it. Two 10-bit HEVC test files, both tagged with BT.2020 primaries, differ only in their transfer:
ffprobe -v error -select_streams v:0 \ -show_entries stream=pix_fmt,color_space,color_transfer,color_primaries -of json pq.mp4{ "pix_fmt": "yuv420p10le", "color_space": "bt2020nc", "color_transfer": "smpte2084", "color_primaries": "bt2020" }{ "pix_fmt": "yuv420p10le", "color_space": "bt2020nc", "color_transfer": "bt2020-10", "color_primaries": "bt2020" }The first is HDR. The second is wide-gamut SDR: same primaries, same 10-bit pixel format, and an SDR transfer. Flag it as HDR and an SDR file goes down an HDR path, tone mapping included, that it never needed. The rule is to test color_transfer for smpte2084 or arib-std-b67 and nothing else. BT.2020 primaries and a 10-bit pix_fmt are common on HDR files, but neither one proves it.
An earlier version of this post counted bt2020 primaries alone as HDR signalling. That was wrong for the reason above.
Why ffprobe does this
None of this is ffprobe being wrong. It reports what the container declares, in the container’s terms: the stored frame, the demuxer’s idea of a duration, the streams exactly as muxed. The five traps are all the gap between what the file says and what the file is.
My view after writing the normaliser: never let a raw ffprobe field reach a decision without a function in between that knows these five cases. The function is short. The bugs it prevents surface late, as a sideways thumbnail or a song classified as a video.
Rendobar’s ffprobe job runs your ffprobe command and returns the raw report next to that summary. In the summary, rotation is read from the Display Matrix and normalised to 0, 90, 180 or 270, image durations are null, and attached_pic streams are excluded when deciding kind. Its width and height are the stored frame, like ffprobe’s, so apply rotation yourself when you need display dimensions. Common ffprobe commands and what each one prints covers what the job fills in when your command leaves flags out.
import { createClient } from "@rendobar/sdk";
const rb = createClient({ apiKey: process.env.RENDOBAR_API_KEY });
const job = await rb.jobs.run({type: "ffprobe",params: { command: "https://cdn.rendobar.com/assets/brand/og-default.jpg" },});
console.log(job.output.data.format.duration); // "0.040000", the pretend frameconsole.log(job.output.data.summary.kind); // "image"console.log(job.output.data.summary.durationSec); // nullInstall with npm i @rendobar/sdk. jobs.run() submits and waits, so it returns the finished job in one call.
curl -X POST https://api.rendobar.com/jobs \-H "Authorization: Bearer $RENDOBAR_API_KEY" \-H "Content-Type: application/json" \-d '{ "type": "ffprobe", "params": { "command": "https://cdn.rendobar.com/assets/brand/og-default.jpg" }}'Returns immediately with a job id. Poll GET /jobs/{id} or register a webhook rather than blocking on the request.
These five came from MP4, MP3, WAV, JPEG and PNG files. MKV, MOV from specific cameras and HLS manifests have traps of their own that this post does not cover.