cvat_sdk 2.28.0: export_dataset does not result in a working zip-file #9086

hedrich9000 · 2025-02-10T15:14:12Z

Actions before raising this issue

I searched the existing issues and did not find anything similar.
I read/searched the docs

Steps to Reproduce

Execute following code:

task = client.tasks.retrieve(TASK_ID)

task.export_dataset(
    format_name="COCO 1.0",
    filename="example.zip",
    include_images=True,
    location=Location.LOCAL,
)

Inspect zip-file => broken
Inspect as text-file (e.g. cat)

Expected Behavior

Normally, a zip-file is expected. Currently the resulting file is a txt-file with following content:

<!DOCTYPE html>
<html>
    <body>
        <input type="hidden" id="http" name="http" value="5000">
        <input type="hidden" id="https" name="https" value="5001">
        <input type="hidden" id="prefer_https" name="prefer_https" value="true">
    </body>
    <script type="text/javascript">
        var protocol="https:";
        var port=5001;
        var URL=protocol+"//"+location.hostname+":"+port+location.pathname+location.search;
        location.replace(URL);
    </script>
</html>

Possible Solution

No response

Context

Other functions of the cvat_sdk work as expected with the on-prem CVAT-server, as far as i noticed. The CVAT-server was previously upgraded from 2.16.3 according to the upgrade procedure in the docs.

Environment

- cvat_sdk==2.28.0
- on-prem CVAT-server version: 2.28.0
- tested with following clients:
  - Windows 11 (Python 3.11)
  - Ubuntu 22.04 (Python 3.10)

The text was updated successfully, but these errors were encountered:

bsekachev · 2025-02-11T06:34:56Z

<!DOCTYPE html>
<html>
    <body>
        <input type="hidden" id="http" name="http" value="5000">
        <input type="hidden" id="https" name="https" value="5001">
        <input type="hidden" id="prefer_https" name="prefer_https" value="true">
    </body>
    <script type="text/javascript">
        var protocol="https:";
        var port=5001;
        var URL=protocol+"//"+location.hostname+":"+port+location.pathname+location.search;
        location.replace(URL);
    </script>
</html>

Hello,
This is not CVAT file. Perhaps you have deployment issues, something configured incorrectly.
Please, provide docker ps, .yaml files and shell commands you used to run CVAT.

hedrich9000 · 2025-02-11T09:37:15Z

Thanks for the reply.

Here is the docker ps:

IMAGE                                            PORTS                                                                                          NAMES
gcr.io/iguazio/alpine:3.17                                                                                                                      nuclio-local-storage-reader
cvat.pth.facebookresearch.sam.vit_h:latest-gpu   0.0.0.0:32769->8080/tcp, [::]:32769->8080/tcp                                                  nuclio-nuclio-pth-facebookresearch-sam-vit-h
cvat/ui:v2.28.0                                  80/tcp                                                                                         cvat_ui
cvat/server:v2.28.0                              8080/tcp, 0.0.0.0:9095->9095/tcp, :::9095->9095/tcp                                            cvat_worker_analytics_reports
cvat/server:v2.28.0                              8080/tcp, 0.0.0.0:9091->9091/tcp, :::9091->9091/tcp                                            cvat_worker_annotation
cvat/server:v2.28.0                              8080/tcp                                                                                       cvat_worker_chunks
cvat/server:v2.28.0                              8080/tcp, 0.0.0.0:9092->9092/tcp, :::9092->9092/tcp                                            cvat_worker_export
cvat/server:v2.28.0                              8080/tcp                                                                                       cvat_utils
cvat/server:v2.28.0                              8080/tcp, 0.0.0.0:9090->9090/tcp, :::9090->9090/tcp                                            cvat_server
cvat/server:v2.28.0                              8080/tcp, 0.0.0.0:9094->9094/tcp, :::9094->9094/tcp                                            cvat_worker_quality_reports
cvat/server:v2.28.0                              8080/tcp                                                                                       cvat_worker_webhooks
cvat/server:v2.28.0                              8080/tcp, 0.0.0.0:9093->9093/tcp, :::9093->9093/tcp                                            cvat_worker_import
timberio/vector:0.26.0-alpine                    0.0.0.0:8282->80/tcp, [::]:8282->80/tcp                                                        cvat_vector
quay.io/nuclio/dashboard:1.13.8-amd64            80/tcp, 0.0.0.0:8070->8070/tcp, :::8070->8070/tcp                                              nuclio
postgres:15-alpine                               0.0.0.0:5432->5432/tcp, :::5432->5432/tcp                                                      cvat_db
clickhouse/clickhouse-server:23.11-alpine        9000/tcp, 0.0.0.0:8123->8123/tcp, :::8123->8123/tcp, 9009/tcp                                  cvat_clickhouse
openpolicyagent/opa:0.63.0                       0.0.0.0:8181->8181/tcp, :::8181->8181/tcp                                                      cvat_opa
grafana/grafana-oss:10.1.2                       3000/tcp                                                                                       cvat_grafana
traefik:v2.10                                    0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 80/tcp, 0.0.0.0:8090->8090/tcp, :::8090->8090/tcp   traefik
redis:7.2.3-alpine                               0.0.0.0:6379->6379/tcp, :::6379->6379/tcp                                                      cvat_redis_inmem
apache/kvrocks:2.7.0                             0.0.0.0:6666->6666/tcp, :::6666->6666/tcp                                                      cvat_redis_ondisk

Here is the git diff of the yaml-files regarding the tag 2.28.0 in this repo:

diff --git a/docker-compose.yml b/docker-compose.yml
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -95,8 +95,8 @@ services:
       ADAPTIVE_AUTO_ANNOTATION: 'false'
       NUMPROCS: 2
       CVAT_ANALYTICS: 1
-      CVAT_BASE_URL:
-      ONE_RUNNING_JOB_IN_QUEUE_PER_USER:
+      SMOKESCREEN_OPTS: ${SMOKESCREEN_OPTS:-}
+      CVAT_SHARE_URL: "Mounted from XXX"
     command: init run server
     labels:
       - traefik.enable=true
@@ -108,6 +108,7 @@ services:
       - cvat_data:/home/django/data
       - cvat_keys:/home/django/keys
       - cvat_logs:/home/django/logs
+      - cvat_img_share:/home/django/share:ro
     networks:
       cvat:
         aliases:
@@ -143,6 +144,7 @@ services:
       - cvat_data:/home/django/data
       - cvat_keys:/home/django/keys
       - cvat_logs:/home/django/logs
+      - cvat_img_share:/home/django/share:ro
     networks:
       - cvat

@@ -402,13 +404,54 @@ services:
           - grafana

volumes:
-  cvat_db:
-  cvat_data:
-  cvat_keys:
-  cvat_logs:
-  cvat_inmem_db:
-  cvat_events_db:
+  cvat_db:
...

--- a/docker-compose.dev.yml
+++ b/docker-compose.dev.yml
@@ -98,6 +98,7 @@ services:
         https_proxy:
         no_proxy:
         socks_proxy:
+        CLIENT_PLUGINS: ${CLIENT_PLUGINS}
       dockerfile: Dockerfile.ui

--- /dev/null
+++ b/docker-compose.override.yml
@@ -0,0 +1,11 @@
+version: '3.3'
+
+services:
+  cvat_server:
+    environment:
+      DJANGO_SETTINGS_MODULE: settings
+      # Make CVAT_HOST accessible INSIDE the Docker container
+      CVAT_HOST: ${CVAT_HOST:-localhost}
+    volumes:
+      - ./settings.py:/home/django/settings.py:ro
+      - ./production.py:/home/django/cvat/settings/production.py:ro

--- a/components/serverless/docker-compose.serverless.yml
+++ b/components/serverless/docker-compose.serverless.yml
@@ -1,7 +1,7 @@
 services:
   nuclio:
     container_name: nuclio
-    image: quay.io/nuclio/dashboard:1.13.0-amd64
+    image: quay.io/nuclio/dashboard:1.13.8-amd64
     restart: always
     networks:
       - cvat

Additionally, we have configured LDAP.

Shell command:
docker compose -f docker-compose.yml -f docker-compose.dev.yml -f components/serverless/docker-compose.serverless.yml -f docker-compose.override.yml up -d --build

zhiltsov-max · 2025-02-11T17:07:28Z

It looks like something is trying to redirect SDK to a https connection in a bare-metal deployment.

hedrich9000 · 2025-02-12T15:00:48Z

Okay, i could get a little bit deeper into the problem. Our proxy does only allow https requests. The prepare_file request works fine. But the received url in the export_request is a http link instead of https. This can be tracked down to line 110 in downloading.py. If I adjust the resulting url to be a https, everything works. This behavior was not present with version 2.16.3.
Is there an error in our CVAT server configuration or in the sdk/api?
Thanks in advance.

nilsmelchert · 2025-02-18T13:28:14Z

This is indeed a bug as far as I can see. The download link provided by the sdk returns an http url instead of https. When manually correcting the download url to https, the code continues as expected.

hedrich9000 added the bug Something isn't working label Feb 10, 2025

bsekachev added need info Need more information to investigate the issue and removed bug Something isn't working labels Feb 11, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cvat_sdk 2.28.0: export_dataset does not result in a working zip-file #9086

cvat_sdk 2.28.0: export_dataset does not result in a working zip-file #9086

hedrich9000 commented Feb 10, 2025

bsekachev commented Feb 11, 2025 •

edited

Loading

hedrich9000 commented Feb 11, 2025 •

edited

Loading

zhiltsov-max commented Feb 11, 2025

hedrich9000 commented Feb 12, 2025

nilsmelchert commented Feb 18, 2025

cvat_sdk 2.28.0: export_dataset does not result in a working zip-file #9086

cvat_sdk 2.28.0: export_dataset does not result in a working zip-file #9086

Comments

hedrich9000 commented Feb 10, 2025

Actions before raising this issue

Steps to Reproduce

Expected Behavior

Possible Solution

Context

Environment

bsekachev commented Feb 11, 2025 • edited Loading

hedrich9000 commented Feb 11, 2025 • edited Loading

zhiltsov-max commented Feb 11, 2025

hedrich9000 commented Feb 12, 2025

nilsmelchert commented Feb 18, 2025

bsekachev commented Feb 11, 2025 •

edited

Loading

hedrich9000 commented Feb 11, 2025 •

edited

Loading