Skip to content

fix: serialize cross-service Docker port allocation under xdist #152

Description

@cofin

Problem

DockerService.run() protects each service name with its own xdist file lock, but different services can still call client.containers.run(... ports={container_port: None}) concurrently on separate workers.

On rootless Docker, one SQLSpec pytest -n 4 --dist=loadgroup run started several native pytest-databases services at once. Docker allocated adjacent host ports 33313-33322, while the MSSQL container failed on 33316:

RootlessKit PortManager.AddPort(): listen tcp4 0.0.0.0:33316:
bind: address already in use

There was no second pytest process. The concurrent starts were different service names from the same run. Because mssql_service is session-scoped, the one startup failure cascaded into every MSSQL test.

Issue #131 / PR #133 added host_port support to DockerService.run, but only the PostgreSQL-family fixture surface exposes port overrides. MSSQL, MySQL, Oracle, GizmoSQL, CockroachDB, RustFS, and other native service fixtures still use dynamic allocation and cannot opt into a pinned port.

Suggested fix

Serialize only the container-creation/port-allocation critical section across service names when xdist is active, using a global file lock under the pytest-databases temp directory. Keep the existing per-name lock for service reuse and lifecycle behavior.

Alternatively, expose an optional host-port fixture consistently for every Docker service, though a global creation lock is less configuration and protects all current and future services.

Environment

  • pytest-databases 0.19.0
  • pytest-xdist 3.8.0
  • docker-py 7.x
  • Linux rootless Docker / RootlessKit
  • Python 3.10

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions