[D1][Biweekly] Offline DINOv3 cache: first comparable results and mechanism analysis

[xizaoge-shuai]

D1 Progress Summary

This issue reports the first comparable experimental results and mechanism analysis for D1: frozen DINOv3 features with a LatentMixture detection head.

Related implementation PR: #239

Completed Work

• Implemented deterministic offline multi-level DINOv3 feature caching.
• The training stage does not load or update DINOv3.
• Only LatentMixture-P3/P4/P5 and the Detect head are trained.
• Added cache validation, CPU/GPU residency, parameter auditing, and student-only checkpoint validation.
• Completed comparable experiments on deterministic COCO128 and VisDrone500 subsets.
• Completed cache-residency, auxiliary-loss-weight, and backbone-size ablations.
• Completed multi-seed validation for the main P2 observations.
• D1 regression tests: 36 passed with 1 non-blocking warning.

First Comparable Results

Dataset	Method	mAP50-95	Training time	Main observation
COCO128	YOLO26n from scratch	0.007290	384.505 s	Baseline
COCO128	Offline DINOv3 + LatentMixture	0.009034	81.713 s	78.75% lower warm training cost
VisDrone500	YOLO26n from scratch	0.009780	1523.700 s	Baseline
VisDrone500	Offline DINOv3 + LatentMixture	0.009501	400.951 s	Retains 97.14% mAP50-95 and reduces cold cost by 67.07%

These results satisfy the first comparable-data milestone on two datasets. The current experiments use deterministic subsets rather than the complete COCO or VisDrone datasets.

Mechanism Analysis

Cache residency

• Stream loading: 3.096 s median epoch time.
• CPU preload: 2.845 s median epoch time.
• GPU preload: 0.533 s median epoch time.
• GPU residency reduces median epoch time by approximately 82.77% compared with stream loading, but increases GPU memory usage.

Auxiliary loss weight

• Tested weights 0, 0.01, 0.05, 0.1, and 0.2.
• Weight 0.01 was best in the initial single-seed screening.
• The gain was not reproduced consistently across three seeds, so no statistically stable improvement is claimed.

DINOv3 backbone size

• Compared ViT-S/16 and ViT-L/16 using three training seeds.
• ViT-L/16 achieved a 26.53% higher mean mAP50-95, but the paired 95% confidence interval crossed zero.
• ViT-L/16 also increased training time, peak VRAM, and cache size substantially.

Difficulties and Findings

1. Disk-streamed cache I/O can offset the benefit of frozen feature extraction. GPU residency is currently the main source of training acceleration.
2. GPU residency trades memory capacity for throughput, especially for ViT-L/16 features.
3. Single-seed hyperparameter conclusions can be misleading. Multi-seed validation changed the interpretation of both the auxiliary-loss and backbone-size results.
4. ViT-L/16 FP16 feature extraction produced NaN/Inf values in pilot runs. The current pipeline uses BF16 model forward computation and stores the validated cache in FP16.

Current Status and Next Steps

• P0 implementation and admission loop: completed.
• P1 two-dataset comparable experiments: completed.
• P2 ablations and multi-seed analysis: completed.
• Core implementation and the consolidated experimental report are available in PR [D1] Offline DINOv3 feature cache with LatentMixture detector #239.
• Next, we will address review feedback, keep the PR limited to mergeable D1 core files, and evaluate whether larger-scale datasets are needed for the next milestone.