Skip to content

Stop duplicating dvc.yaml entries for nested dirs on Windows - #907

Open
Muhtasim-Munif-Fahim wants to merge 1 commit into
treeverse:mainfrom
Muhtasim-Munif-Fahim:fix/dvcyaml-windows-nested-dir
Open

Stop duplicating dvc.yaml entries for nested dirs on Windows#907
Muhtasim-Munif-Fahim wants to merge 1 commit into
treeverse:mainfrom
Muhtasim-Munif-Fahim:fix/dvcyaml-windows-nested-dir

Conversation

@Muhtasim-Munif-Fahim

Copy link
Copy Markdown

Fixes #848.

The bug

update_dvcyaml recognizes the entries it owns by testing whether a stored path starts with the dvclive directory:

dvclive_dir = os.path.relpath(live.dir, dvcyaml_dir) + "/"

but the entries themselves are written through rel_path, which normalizes to POSIX:

return str(Path(os.path.relpath(absolute_path, dvc_root_path)).as_posix())

On Windows those two disagree as soon as dir has more than one level. The prefix comes out as dvclive\subdir/ while the stored entry reads dvclive/subdir/metrics.json, so _drop_stale_dvclive_entries matches nothing, treats every existing entry as foreign, and keeps it. The new entry is then appended alongside.

A single-level dir is unaffected, because the only separator involved is the "/" that was already hardcoded — which is why #848 reports this specifically for multi-level paths.

Reproduction

Running the snippet from the issue twice on Windows:

from dvclive import Live

with Live(dir="dvclive/subdir") as live:
    live.log_metric("metric", 1)

Before:

metrics:
- dvclive/subdir/metrics.json
- dvclive/subdir/metrics.json
plots:
- dvclive/subdir/plots/metrics:
    x: step
- dvclive/subdir/plots/metrics:
    x: step

After, it matches the expected output in the issue — one entry each, replaced rather than appended.

The fix

Normalize the prefix the same way the entries are normalized:

dvclive_dir = Path(os.path.relpath(live.dir, dvcyaml_dir)).as_posix() + "/"

Tests

Adds test_make_dvcyaml_nested_dir_not_duplicated, which reruns a nested-dir Live and asserts the entries are replaced. It fails on the windows-latest leg of your matrix without this change and is a no-op on Linux and macOS, where os.path.relpath already returns POSIX separators.

Verified on Windows: the test fails before the change and passes after; tests/ excluding tests/frameworks is 232 passed.

🤖 Generated with Claude Code

`update_dvcyaml` recognizes the entries it owns by testing whether a path
starts with the dvclive directory, but it built that prefix with
`os.path.relpath`, which uses the native separator. Entries themselves are
written through `rel_path`, which normalizes to POSIX.

On Windows a nested `dir` therefore produced the prefix `dvclive\subdir/`
while the stored entry read `dvclive/subdir/metrics.json`. No existing entry
matched, so none were treated as stale and every run appended a fresh copy:

    metrics:
    - dvclive/subdir/metrics.json
    - dvclive/subdir/metrics.json

A single-level `dir` was unaffected, because the only separator involved is
the "/" that was already hardcoded.

Normalizing the prefix with `Path(...).as_posix()` makes it match what is
actually written. Adds a regression test that reruns a nested-`dir` `Live`
and asserts the entries are replaced rather than appended; it fails on the
Windows CI leg without this change.

Fixes treeverse#848

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-project-automation github-project-automation Bot moved this to Backlog in DVC Aug 13, 2026
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Repeatedly print the same live path to dvc.yaml when specifying multi-level live paths in Windows System

2 participants