Repository navigation
feat: keep 4:4:4 video sharp when importing - #4333
Open
richiemcilroy wants to merge 23 commits into
Open
richiemcilroy wants to merge 23 commits into
richiemcilroy wants to merge 23 commits into
Conversation
Hardware and platform decoders hand frames back as 4:2:0. Video with fuller chroma now decodes in software on every platform, AVAssetReader and Media Foundation decline it at init so the FFmpeg fallback takes it, and the RGBA conversion keeps every chroma sample.
A re-encode subsamples it to 4:2:0, which blurs coloured text. The import now copies the source packets into display.mp4 when every editor decoder reads them as they are, and re-encodes exactly as before otherwise.
|
Sandchest reviewed Sandchest reviews every change for vulnerabilities an attacker could use · 2 credits · Review details |
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Member
Author
|
hey @greptileai, please re-review the PR |
Member
Author
|
hey @greptileai, please re-review the PR |
…source # Conflicts: # crates/rendering/src/project_recordings.rs
The display layer reuses its texture for a frame at the same recording time, so the settled frame decoded from the kept source never replaced the one already shown. The renderer now forgets its last frame before a settled render. At the end of playback the settled frame is scheduled directly instead of first redrawing the same frame from display.mp4.
…source # Conflicts: # crates/editor/src/editor_instance.rs
Member
Author
|
hey @greptileai, please re-review the PR |
Member
Author
|
hey @greptileai, please re-review the PR |
Member
Author
|
hey @greptileai, please re-review the PR |
Member
Author
|
hey @greptileai, please re-review the PR |
Member
Author
|
hey @greptileai, please re-review the PR |
Member
Author
|
@greptileai, please re-review the PR |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Importing a 4:4:4 screen capture re-encoded it to 4:2:0, which smeared coloured text in the editor. The import now also keeps the source's H.264 4:4:4 video next to the re-encode, and the paused editor view reads it, so coloured text is sharp when you stop on a frame. Scrubbing, playback, seeking and export read the same re-encode as before, so export output and speed are exactly main's.
flowchart LR S[source] --> T[transcode_video<br/>unchanged] --> D[display.mp4] S --> P{plan_keep<br/>packet scan, no decode} P -- "H.264 4:4:4, avc1, limited range,<br/>no rotation or SAR, even size,<br/>keyframes at most 4.5 s apart, at most 1080p60" --> K[copy video packets<br/>and decode each one to check] --> DS[display-source.mp4] P -- anything else --> N[nothing kept] D --> E1[scrub, playback, seek, export] DS --> E2[paused editor frame]Import
import_video(apps/desktop-gpui/src/import.rs) runs the existingtranscode_videounchanged, thenkeep_source_videowritesdisplay-source.mp4next todisplay.mp4when the source qualifies. A source that does not qualify, or a copy that fails its check, leaves the project exactly as main writes it.avc1sample entry,yuv444plimited range, BT.601/709 tags only, progressive, even dimensions, square pixels, no display matrix, a keyframe first, strictly ordered timestamps, no discarded packets.colr) too, so both files show the same colours.display-source.mp4along withdisplay.mp4andaudio.ogg.Paused editor frame
RecordingSegmentDecodersfindsdisplay-source.mp4next to adisplay.mp4and opens a decoder for it the first time a paused frame needs it. OnlyFrameFetch::Stillreads it; playback, scrubbing and export keep readingdisplay.mp4.display.mp4as before. 150 ms after the last paused frame, and when playback stops, it renders that frame again from the kept file, both sides of a transition included. Any new request or playback start cancels it. A newForgetLastFramerenderer message makes the display layer upload that frame even though its recording time matches the one on screen.cap_video_decode::ffmpeg::chroma_is_420reads the pixel format, orchroma_format_idcfromavcCandchromaFormatfromhvcC. Hardware decoders, AVAssetReader and Media Foundation output 4:2:0, so a full-chroma file goes to FFmpeg software decode.FrameConverterconverts limited-rangeyuv444pwith a row-parallel fixed-point converter (cpu_yuv::yuv444p_to_rgba_with_matrix). Other full-chroma formats use swscaleBILINEAR, becauseFAST_BILINEARhalves chroma (35.8 dB vs 53.9 dB on the fixture). Every other format keeps its existing path.Measurements
Main at 93622e4 against this branch, on the same runners.
Paused editor frame on macOS, coloured text region against the source frame, at the window's 1248×702 preview size:
Editor latency bench on macOS, three rounds per side on two 4:4:4 imports:
Unchanged:
display.mp4, which decodes to the same frames as main's on macOS, Linux and Windows. macOS cold seek p50 / max on the text fixture: 22.7 / 27.4 ms before, 21.7 / 26.2 ms after.

Confidence Score: 5/5
The PR appears safe to merge; no new blocking issues were found.Summary
Keeps eligible 4:4:4 source video for sharper paused previews while retaining the re-encoded video for playback and export.
The earlier color, transition, cancellation, and comment concerns are addressed. No new actionable issues were found.
Reviews (14) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile