Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

streaming import for tasks and jobs in YOLO and COCO formats #9265

Draft
wants to merge 22 commits into
base: develop
Choose a base branch
from

Conversation

Eldies
Copy link
Contributor

@Eldies Eldies commented Mar 26, 2025

Motivation and context

On dataset import, all the annotations of the dataset are loaded into RAM. It can be a problem, if annotations are large.
This PR adapts the codebase for streaming import for tasks and jobs and turns it on for YOLO and COCO formats

depends on cvat-ai/datumaro#98

How has this been tested?

Checklist

  • I submit my changes into the develop branch
  • I have created a changelog fragment
  • I have updated the documentation accordingly
  • I have added tests to cover my changes
  • I have linked related issues (see GitHub docs)

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.

@codecov-commenter
Copy link

codecov-commenter commented Mar 26, 2025

Codecov Report

Attention: Patch coverage is 94.73684% with 2 lines in your changes missing coverage. Please review.

Project coverage is 74.25%. Comparing base (630426c) to head (c6bfc4f).
Report is 2 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #9265      +/-   ##
===========================================
+ Coverage    74.20%   74.25%   +0.04%     
===========================================
  Files          429      429              
  Lines        45121    45140      +19     
  Branches      3922     3922              
===========================================
+ Hits         33484    33517      +33     
+ Misses       11637    11623      -14     
Components Coverage Δ
cvat-ui 77.23% <ø> (+0.03%) ⬆️
cvat-server 71.78% <95.12%> (+0.05%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Eldies Eldies changed the title streaming import for tasks and jobs in YOLO formats streaming import for tasks and jobs in YOLO and COCO formats Apr 1, 2025
Copy link

sonarqubecloud bot commented Apr 1, 2025

Quality Gate Failed Quality Gate failed

Failed conditions
1 Security Hotspot
3.1% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

@Eldies Eldies marked this pull request as draft April 2, 2025 10:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants