Portrait MP4 frames decoded with WebCodecs come out sideways in all four tested browsers

The author wrote, "A decoder that is fed raw samples only ever gets the picture." ^[17] The angle is in the container. In MP4 and MOV it is a 3x3 matrix in the track header box, tkhd, and mp4box returns it along with the track ^[8]. The 90-degree sample's matrix was [0, 65536, 0, -65536, 0, 0, 0, 0, 1073741824], while the upright sample with no tag returned identity ^[9]. The values are fixed-point, so 65536 is 1.0, and the first two entries are enough to recover the angle ^[9][10]:

``` const [a, b] = trak.tkhd.matrix; // 65536 = 1.0 const turn = (Math.round(Math.atan2(b, a) * 180 / Math.PI) + 360) % 360; ```

With a = 0 and b = 65536, that gives 90 degrees ^[20].

Two details tripped the author. ffmpeg prints a rotation of -90.00 degrees for the same file, so the author checks the computed turn against the painted marker blocks instead of trusting a sign ^[11]. And mp4box reports track_width and track_height as 1920x1080, the stored size, so a canvas sized from the track comes out landscape ^[12]. For a quarter turn the author swaps width and height, then rotates the context before drawing the frame ^[12]. A 180-degree clip keeps its size, so a width check never flags an upside-down frame ^[13].

The createImageBitmap(video) results show how far the browsers drift apart. Chrome 154 returned 1080x1920 with square blocks, and Chromium 149 returned a 1920x1080 bitmap that was upright but squashed, with the 160x160 blocks at about 284x90 ^[7]. Those blocks were stretched by about 1.78 across and squeezed to 0.56 down, which match 1920/1080 and 1080/1920 ^[19]. We think Chromium rotated the picture and then fitted it into the stored-size box. The post does not give a cause.

On the way out, the author tried keeping the sideways pixels and writing the tag again. ffmpeg with -noautorotate to VP9 kept the 1920x1080 sideways pixels, and the output file still carried a -90 rotation that ffmpeg could detect ^[15]. In Chromium 149 and Chrome 154 that WebM played upright at 1080x1920, while Firefox 151 and WebKit 26.5 played it lying down at 1920x1080 ^[15]. Upright pixels with no tag were upright on every path in every browser tested, so the author rotates at encode time and writes no tag ^[16]. We think that is the right call when you do not control the player.

The scope is narrow. Every tag was written by ffmpeg, and the post does not cover phone originals, HEVC, iPhone Safari or Android ^[18]. WebKit was tested on macOS ^[2]. Mirrored matrices, which the author says a front camera might write, were not tested, and the snippet ignores a flip ^[14]. For the result to transfer to your app, your clips would need a plain rotation matrix in tkhd, and your frames would have to come from VideoDecoder fed by your own demuxer.