feat(vid1): support multi-track audio in VID1 GameCube video/audio#1
Merged
Merged
Conversation
VID1 movie containers can carry multiple dubbed-language audio tracks (one AUDH header per track in HEAD, one AUDD block per track per FRAM). The parser previously only read the first AUDH header, then merged every AUDD chunk in each frame into a single packet list -- corrupting audio on any multi-track file. - Vid1AudioExtractor now parses every AUDH track and correctly routes each frame''s AUDD blocks to their matching track by position. - Probe/ProbeTracks/ConvertToWav gained per-track access; ConvertToWav converts every track by default (one WAV per track). - Vid1VideoConverter mixes every track into the exported MP4 as its own selectable audio stream. Verified against a real multi-track sample (Tony Hawk''s American Wasteland, GC): JX_Interview01.vid parses 5 audio tracks and produces 5 correctly separated WAVs / 5 muxed AAC streams in the MP4. The same VID1 container format and multi-language interview clips are also expected in Tony Hawk''s Downhill Jam (Wii).
Raw RGB24 frames were piped straight into libx264 with no output pixel format specified, so ffmpeg auto-selected the lossless-leaning High 4:4:4 Predictive profile (yuv444p) to match the RGB input. That profile decodes fine in VLC/ffplay-class software decoders, but not in restrictive decoders: Discord's embed transcoder fails to produce a preview/player for these files, and the same MP4s fail to preview inside this app's own GUI (MediaPlayerElement), since Windows Media Foundation also expects standard 4:2:0 chroma subsampling. Added -pix_fmt yuv420p (standard chroma subsampling, decodable everywhere) and -movflags +faststart (moves the moov atom to the front so playback/preview can start without downloading the whole file) to both native-decode ffmpeg pipelines: Vid1VideoConverter (VID1 GameCube) and StrConverter (PS1 STR), which shared the same raw-RGB-into-libx264 pattern. Verified via ffprobe: profile went from "High 4:4:4 Predictive" / yuv444p to standard "High" / yuv420p, with moov now ordered before mdat.
JX_Interview01.vid is from Tony Hawk's Downhill Jam (Wii), not Tony Hawk's American Wasteland (GC). RepresentativeMultiTrackSampleFile was pointing at the wrong Sample/Builds folder name.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
VID1 movie containers (THAW GameCube) can carry multiple dubbed-language audio tracks — one
AUDHheader per track inHEAD, and oneAUDDblock per track perFRAM. The parser previously only read the firstAUDHheader, then merged everyAUDDchunk in each frame into a single packet list, corrupting audio on any multi-track file (English/French/Italian/German/Spanish tracks interleaved together into one broken stream).This is directly relevant to interview/cutscene clips that ship with multiple language dubs, such as the
movies/vidinterview files in Tony Hawk's Downhill Jam (Wii).Changes
Vid1AudioExtractornow parses everyAUDHtrack inHEADand correctly routes each frame'sAUDDblocks to their matching track by position, instead of merging them.ProbeTracks/ per-trackProbe/ConvertToWavoverloads for track-level access.ConvertToWavconverts every track by default, writing one WAV per track ({stem}.wav,{stem}_track1.wav, …).Vid1VideoConverternow muxes every audio track into the exported MP4 as its own selectable AAC stream (previously only the first track was muxed in).Video preview fix
While verifying the multi-track MP4 export, found that exported MP4s failed to preview both externally (Discord's embed transcoder) and inside this app's own GUI (
MediaPlayerElement). Root cause: raw RGB24 frames were piped straight intolibx264with no output pixel format specified, so ffmpeg auto-selected theHigh 4:4:4 Predictiveprofile (yuv444p) to match the RGB input. That decodes fine in VLC/ffplay-class software decoders, but not in restrictive decoders like Discord's transcoder or Windows Media Foundation (which the GUI preview relies on) — both expect standard 4:2:0 chroma subsampling.Fixed by forcing
-pix_fmt yuv420p(standard chroma subsampling) and-movflags +faststart(movesmoovto the front so playback/preview can start without downloading the whole file) in both native-decode ffmpeg pipelines:Vid1VideoConverter(VID1 GameCube) andStrConverter(PS1 STR), which shared the same raw-RGB-into-libx264pattern. Verified viaffprobe: profile went fromHigh 4:4:4 Predictive/yuv444pto standardHigh/yuv420p, withmoovnow ordered beforemdat.Verification
Tested against a real multi-track sample (
JX_Interview01.vid, Tony Hawk's Downhill Jam Wii):Vid1AudioExtractor.ConvertToWav.Vid1VideoConverter.ConvertToMp4(verified withffprobe).Added synthetic multi-track test coverage (
Vid1TestBuilder.CreateMultiTrackVid1) plus real-sample tests that run when the sample file is present, and skip gracefully otherwise (tests/NeversoftMultitool.Tests/Core/Formats/Audio/Vid1AudioExtractorTests.cs).Full test suite: no new regressions. Two pre-existing failures remain (
Parse_AllThawGcSamples_ClassifiesLongFormAndAtvi,TryWriteDeterministicVideoStream_AllThawVidSamples_WritesNonEmptyM4v) — both expect all 8 real.vidsamples inmovies/vid/, but this environment only has 1; unrelated to this change.