Skip to content

feat(recording): add AudioOnly capture target and recording pipeline … #1881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ManthanNimodiya wants to merge 23 commits into
base: main
Choose a base branch
from
Open

feat(recording): add AudioOnly capture target and recording pipeline … #1881

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
f8cd75a
feat(recording): add AudioOnly capture target and recording pipeline …
ManthanNimodiya Jun 1, 2026
ab6bfe3
fix(recording): return Err for audio-only in ProjectRecordingsMeta, a…
ManthanNimodiya Jun 1, 2026
07506b3
fix(recording): restore CROSS_TRACK_SNAP_SECS AV sync for screen reco…
ManthanNimodiya Jun 7, 2026
64507df
fix(recording): skip audio-only segments in needs_fragment_remux check
ManthanNimodiya Jun 7, 2026
a6f0661
Merge branch 'main' into feat/audio-only-recording
ManthanNimodiya Jun 7, 2026
146958b
Merge branch 'main' into feat/audio-only-recording
ManthanNimodiya Jun 16, 2026
b04c68d
fix(audio-only): address review comments and fix compilation errors
ManthanNimodiya Jun 18, 2026
b22097e
fix(audio-only): skip timeline generation when no display segments
ManthanNimodiya Jun 18, 2026
801e689
fix(audio-only): preserve audio_only flag in meta writes, skip camera…
ManthanNimodiya Jun 18, 2026
d7f2fb0
fix(audio-only): gate display checks on audio_only flag, fail fast on…
ManthanNimodiya Jun 18, 2026
863fe1f
fix(audio-only): add pipeline branches for AudioOnly in studio/instan…
ManthanNimodiya Jun 18, 2026
c915932
fix(audio-only): wire up recovery display gating, mic guard, and rema…
ManthanNimodiya Jun 19, 2026
66ad78a
Merge branch 'main' into feat/audio-only-recording
ManthanNimodiya Jun 20, 2026
c90d08d
fix(audio-only): fix Linux compile gap, gate recovery screenshot and …
ManthanNimodiya Jun 21, 2026
7360753
fix(audio-only): repair botched main-merge conflict resolution and up…
ManthanNimodiya Jun 21, 2026
23b11d9
fix(audio-only): fix Windows clippy exhaustiveness gaps for AudioOnly…
ManthanNimodiya Jun 22, 2026
7fe1931
fix(audio-only): return explicit error for audio-only recordings in e…
ManthanNimodiya Jun 22, 2026
0abc8d3
fix(audio-only): report clear missing-display check in CLI validator …
ManthanNimodiya Jun 23, 2026
cd870dd
fix(audio-only): avoid constructing empty display path in transport b…
ManthanNimodiya Jun 25, 2026
c396c7e
Merge branch 'main' into feat/audio-only-recording
ManthanNimodiya Jun 25, 2026
889637b
fix(audio-only): guard export preview and exporter build against audi…
ManthanNimodiya Jun 25, 2026
365a045
fix(audio-only): compute correct duration and fix unclosed if in proj…
ManthanNimodiya Aug 6, 2026
5da95f4
merge: resolve conflicts with main (debug logging, notch overlay, tim…
ManthanNimodiya Aug 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 14 additions & 8 deletions apps/cli/src/project.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,10 +123,13 @@ fn studio_checks(meta: &RecordingMeta, studio: &StudioRecordingMeta) -> Vec<File

match studio {
StudioRecordingMeta::SingleSegment { segment } => {
checks.push(required_check(
"displayVideo",
meta.path(&segment.display.path),
));
if !meta.audio_only {
let path = segment
.display
.as_ref()
.map_or_else(PathBuf::new, |d| meta.path(&d.path));

@tembo tembo Bot Jun 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PathBuf::new() here is a bit risky: an empty path often resolves to . so exists() can return true and accidentally mark a non-audio-only project as valid even when display metadata is missing.

Maybe prefer branching on the option and forcing exists: false when display is None:

Suggested change
.map_or_else(PathBuf::new, |d| meta.path(&d.path));
let path = match segment.display.as_ref() {
Some(d) => meta.path(&d.path),
None => PathBuf::from("<missing display metadata>"),
};

checks.push(required_check("displayVideo", path));

@tembo tembo Bot Jun 22, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using PathBuf::new() as a fallback here can produce an empty path, which can behave surprisingly (and makes the validation output harder to interpret). Since this branch is !meta.audio_only, I’d rather report a clear “missing display metadata” check.

Suggested change
let path = segment
.display
.as_ref()
.map_or_else(PathBuf::new, |d| meta.path(&d.path));
checks.push(required_check("displayVideo", path));
if let Some(display) = segment.display.as_ref() {
checks.push(required_check("displayVideo", meta.path(&display.path)));
} else {
checks.push(FileCheck {
role: "displayVideo",
path: PathBuf::from("<missing display metadata>"),
exists: false,
required: true,
});
}

(Same pattern for the multi-segment loop below.)

}
if let Some(camera) = &segment.camera {
checks.push(required_check("camera", meta.path(&camera.path)));
}
Expand All @@ -139,10 +142,13 @@ fn studio_checks(meta: &RecordingMeta, studio: &StudioRecordingMeta) -> Vec<File
}
StudioRecordingMeta::MultipleSegments { inner } => {
for segment in &inner.segments {
checks.push(required_check(
"displayVideo",
meta.path(&segment.display.path),
));
if !meta.audio_only {
let path = segment
.display
.as_ref()
.map_or_else(PathBuf::new, |d| meta.path(&d.path));
checks.push(required_check("displayVideo", path));
}
if let Some(camera) = &segment.camera {
checks.push(required_check("camera", meta.path(&camera.path)));
}
Expand Down
1 change: 1 addition & 0 deletions apps/cli/src/record.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -889,6 +889,7 @@ fn persist_instant_recording_meta(
sharing: None,
inner: RecordingMetaInner::Instant(meta),
upload: None,
audio_only: false,

@tembo tembo Bot Jun 25, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CompletedRecording already carries the display_source, so this can reflect audio-only instant recordings too.

Suggested change
audio_only: false,
audio_only: matches!(recording.display_source, ScreenCaptureTarget::AudioOnly),

}
.save_for_project()
.map_err(|e| format!("Failed to save instant recording meta: {e}"))?;
Expand Down
12 changes: 10 additions & 2 deletions apps/desktop/src-tauri/examples/desktop-display-transport-benchmark.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,11 @@ async fn load_recording(
if project.timeline.is_none() {
let timeline_segments = match meta.as_ref() {
StudioRecordingMeta::SingleSegment { segment } => {
let display_path = recording_meta.path(&segment.display.path);
let display_path = segment
.display
.as_ref()
.map(|d| recording_meta.path(&d.path))
.unwrap_or_default();

@tembo tembo Bot Jun 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unwrap_or_default() makes the path empty when display is missing, which can behave surprisingly. Consider computing duration directly from segment.display and defaulting to 5.0 without constructing an empty PathBuf (same in the multi-segment branch).

Suggested change
.unwrap_or_default();
let duration = segment
.display
.as_ref()
.and_then(|d| Video::new(&recording_meta.path(&d.path), 0.0).ok())
.map(|video| video.duration)
.unwrap_or(5.0);

let duration = match Video::new(&display_path, 0.0) {
Ok(video) => video.duration,
Err(_) => 5.0,
Expand All @@ -135,7 +139,11 @@ async fn load_recording(
.iter()
.enumerate()
.filter_map(|(i, segment)| {
let display_path = recording_meta.path(&segment.display.path);
let display_path = segment
.display
.as_ref()
.map(|d| recording_meta.path(&d.path))
.unwrap_or_default();
let duration = match Video::new(&display_path, 0.0) {
Ok(video) => video.duration,
Err(_) => 5.0,
Expand Down
63 changes: 43 additions & 20 deletions apps/desktop/src-tauri/src/import.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,8 @@ fn full_timeline_for_segments(
.iter()
.enumerate()
.map(|(index, segment)| {
let duration = get_video_duration_secs(&segment.display.path.to_path(project_path))?;
let display = segment.display.as_ref().ok_or("Missing display video")?;
let duration = get_video_duration_secs(&display.path.to_path(project_path))?;
Ok(TimelineSegment {
recording_clip: index as u32,
timescale: 1.0,
Expand Down Expand Up @@ -347,7 +348,10 @@ fn full_timeline_for_source_segments(
.iter()
.enumerate()
.map(|(index, segment)| {
let duration = get_source_video_duration_secs(source_meta, &segment.display)?;
let duration = get_source_video_duration_secs(
source_meta,
segment.display.as_ref().ok_or("Missing display video")?,
)?;
Ok(TimelineSegment {
recording_clip: index as u32,
timescale: 1.0,
Expand Down Expand Up @@ -593,7 +597,11 @@ fn copy_keyboard_path(
return Ok(Some(target_relative_path));
};

let Some(display_dir) = source_segment.display.path.parent() else {
let Some(display_dir) = source_segment
.display
.as_ref()
.and_then(|d| d.path.parent())
else {
return Ok(None);
};

Expand Down Expand Up @@ -870,7 +878,13 @@ fn source_timeline_segments_for_import(
let max_duration = if let Some(duration) = duration_cache.get(&source_index) {
*duration
} else {
let duration = get_source_video_duration_secs(source_meta, &source_segment.display)?;
let duration = get_source_video_duration_secs(
source_meta,
source_segment
.display
.as_ref()
.ok_or("Missing display video")?,
)?;
duration_cache.insert(source_index, duration);
duration
};
Expand Down Expand Up @@ -921,15 +935,21 @@ fn copy_source_segment(
target_relative_dir: &str,
cursor_id_map: &HashMap<String, String>,
) -> Result<MultipleSegment, String> {
let display = copy_video_meta(
&source_meta.project_path,
target_project_path,
&source_segment.display,
target_relative_dir,
"display",
true,
)?
.ok_or_else(|| "Missing display video".to_string())?;
let display = source_segment
.display
.as_ref()
.map(|d| {
copy_video_meta(
&source_meta.project_path,
target_project_path,
d,
target_relative_dir,
"display",
true,
)
})
.transpose()?
.flatten();

let camera = source_segment
.camera
Expand Down Expand Up @@ -1405,12 +1425,12 @@ pub async fn start_video_import(app: AppHandle, source_path: PathBuf) -> Result<
inner: RecordingMetaInner::Studio(Box::new(StudioRecordingMeta::MultipleSegments {
inner: MultipleSegments {
segments: vec![MultipleSegment {
display: VideoMeta {
display: Some(VideoMeta {
path: RelativePathBuf::from("content/segments/segment-0/display.mp4"),
fps: 30,
start_time: Some(0.0),
device_id: None,
},
}),
camera: None,
mic: None,
system_audio: None,
Expand All @@ -1422,6 +1442,7 @@ pub async fn start_video_import(app: AppHandle, source_path: PathBuf) -> Result<
},
})),
upload: None,
audio_only: false,
};

initial_meta
Expand Down Expand Up @@ -1498,14 +1519,14 @@ pub async fn start_video_import(app: AppHandle, source_path: PathBuf) -> Result<
StudioRecordingMeta::MultipleSegments {
inner: MultipleSegments {
segments: vec![MultipleSegment {
display: VideoMeta {
display: Some(VideoMeta {
path: RelativePathBuf::from(
"content/segments/segment-0/display.mp4",
),
fps,
start_time: Some(0.0),
device_id: None,
},
}),
camera: None,
mic: None,
system_audio,
Expand All @@ -1518,6 +1539,7 @@ pub async fn start_video_import(app: AppHandle, source_path: PathBuf) -> Result<
},
)),
upload: None,
audio_only: false,
};

if let Err(e) = meta.save_for_project() {
Expand Down Expand Up @@ -1674,12 +1696,12 @@ async fn append_mp4_to_editor_project(
};

let imported_segment = MultipleSegment {
display: VideoMeta {
display: Some(VideoMeta {
path: output_video_relative_path,
fps,
start_time: Some(0.0),
device_id: None,
},
}),
camera: None,
mic: None,
system_audio,
Expand Down Expand Up @@ -1965,7 +1987,7 @@ pub async fn start_image_import(app: AppHandle, source_path: PathBuf) -> Result<
};

let segment = SingleSegment {
display: video_meta,
display: Some(video_meta),
camera: None,
audio: None,
cursor: None,
Expand All @@ -1978,6 +2000,7 @@ pub async fn start_image_import(app: AppHandle, source_path: PathBuf) -> Result<
sharing: None,
inner: RecordingMetaInner::Studio(Box::new(StudioRecordingMeta::SingleSegment { segment })),
upload: None,
audio_only: false,
};

meta.save_for_project()
Expand Down
21 changes: 19 additions & 2 deletions apps/desktop/src-tauri/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2236,6 +2236,7 @@ enum CurrentRecordingTarget {
bounds: LogicalBounds,
},
Camera,
Audio,
}

#[derive(Serialize, Type)]
Expand Down Expand Up @@ -2288,6 +2289,7 @@ async fn get_current_recording(
bounds: *bounds,
},
ScreenCaptureTarget::CameraOnly => CurrentRecordingTarget::Camera,
ScreenCaptureTarget::AudioOnly => CurrentRecordingTarget::Audio,
};

Ok(JsonValue::new(&Some(CurrentRecording {
Expand Down Expand Up @@ -2916,12 +2918,27 @@ async fn get_video_metadata(path: PathBuf) -> Result<VideoRecordingMetadata, Str

match &**meta {
StudioRecordingMeta::SingleSegment { segment } => {
vec![recording_meta.path(&segment.display.path)]
vec![
recording_meta.path(
&segment
.display
.as_ref()
.map(|d| d.path.clone())
.unwrap_or_default(),
),
]
}
StudioRecordingMeta::MultipleSegments { inner } => inner
.segments
.iter()
.map(|s| recording_meta.path(&s.display.path))
.map(|s| {
recording_meta.path(
&s.display
.as_ref()
.map(|d| d.path.clone())
.unwrap_or_default(),
)
})
.collect(),
Comment on lines 3171 to 3194

@tembo tembo Bot Jun 25, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This unwrap_or_default() makes a missing display turn into an empty relative path (effectively the project root), so ffmpeg::format::input fails with a pretty confusing error. Probably better to explicitly error for audio_only and for corrupt meta.

Suggested change
match &**meta {
StudioRecordingMeta::SingleSegment { segment } => {
vec![recording_meta.path(&segment.display.path)]
vec![
recording_meta.path(
&segment
.display
.as_ref()
.map(|d| d.path.clone())
.unwrap_or_default(),
),
]
}
StudioRecordingMeta::MultipleSegments { inner } => inner
.segments
.iter()
.map(|s| recording_meta.path(&s.display.path))
.map(|s| {
recording_meta.path(
&s.display
.as_ref()
.map(|d| d.path.clone())
.unwrap_or_default(),
)
})
.collect(),
match &**meta {
StudioRecordingMeta::SingleSegment { segment } => {
let display = segment.display.as_ref().ok_or("Missing display video")?;
vec![recording_meta.path(&display.path)]
}
StudioRecordingMeta::MultipleSegments { inner } => {
let display_paths = inner
.segments
.iter()
.map(|s| {
s.display
.as_ref()
.ok_or("Missing display video")
.map(|d| recording_meta.path(&d.path))
})
.collect::<Result<Vec<_>, _>>()?;
if display_paths.is_empty() {
return Err("Missing display video".to_string());
}
display_paths
}
}

}
}
Expand Down
Loading
Loading