docker-compose up -d error

[quantum-booty]

OS: Arcolinux

I have installed the prerequisites, not sure why I'm getting this error. I don't have Postgres installed, maybe it has to do with this?

➜ docker-compose up -d
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.9/http/client.py", line 1257, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1303, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1252, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1012, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 952, in send
    self.connect()
  File "/usr/lib/python3.9/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3.9/site-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3.9/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.9/http/client.py", line 1257, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1303, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1252, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1012, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 952, in send
    self.connect()
  File "/usr/lib/python3.9/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
  File "/usr/lib/python3.9/site-packages/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
  File "/usr/lib/python3.9/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/lib/python3.9/site-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.9/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 33, in <module>
    sys.exit(load_entry_point('docker-compose==1.29.2', 'console_scripts', 'docker-compose')())
  File "/usr/lib/python3.9/site-packages/compose/cli/main.py", line 81, in main
    command_func()
  File "/usr/lib/python3.9/site-packages/compose/cli/main.py", line 200, in perform_command
    project = project_from_options('.', options)
  File "/usr/lib/python3.9/site-packages/compose/cli/command.py", line 60, in project_from_options
    return get_project(
  File "/usr/lib/python3.9/site-packages/compose/cli/command.py", line 152, in get_project
    client = get_client(
  File "/usr/lib/python3.9/site-packages/compose/cli/docker_client.py", line 41, in get_client
    client = docker_client(
  File "/usr/lib/python3.9/site-packages/compose/cli/docker_client.py", line 170, in docker_client
    client = APIClient(use_ssh_client=not use_paramiko_ssh, **kwargs)
  File "/usr/lib/python3.9/site-packages/docker/api/client.py", line 197, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/lib/python3.9/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
    raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

[DriLLFreAK100]

I just cloned the repo a moment ago and ran into some errors when trying to run "docker-compose up" too. For my case, the error message was complaining about some missing mysql file towards the end..

I updated the docker image for puckel/docker-airflow to version 1.10.9 instead of 1.10.1 at the docker-compose.yml file. And also the sqlalchemy dependency to version 1.3.0.

It works after that on my machine. I'm on Windows 10 with Docker Desktop v3.5.2(66501). Hope this helps!

p.s. Also, I happened to miss out the 3rd prerequisite initially, that it to install the Apache Airflow python package. Don't miss it out.

My working version of the docker-compose.yml:

version: '3'
services:
  postgres:
    image: postgres:9.6
    environment:
      - POSTGRES_USER=airflow
      - POSTGRES_PASSWORD=airflow
      - POSTGRES_DB=airflow
    ports:
      - "5432:5432"

  webserver:
    image: puckel/docker-airflow:1.10.9
    build:
      context: https://github.com/puckel/docker-airflow.git#1.10.9
      dockerfile: Dockerfile
      args:
        AIRFLOW_DEPS: gcp_api,s3
        PYTHON_DEPS: sqlalchemy==1.3.0
    restart: always
    depends_on:
      - postgres
    environment:
      - LOAD_EX=n
      - EXECUTOR=Local
      - FERNET_KEY=jsDPRErfv8Z_eVTnGfF8ywd19j4pyqE3NpdUBA_oRTo=
    volumes:
      - ./examples/intro-example/dags:/usr/local/airflow/dags
      # Uncomment to include custom plugins
      # - ./plugins:/usr/local/airflow/plugins
    ports:
      - "8080:8080"
    command: webserver
    healthcheck:
      test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
      interval: 30s
      timeout: 30s
      retries: 3

[carlosvinter]

I applied your suggestion but I keep getting the same error with _mysql.
Docker engine v20.10.8
Docker desktop v 3.6.0

webserver_1 |
webserver_1 | Traceback (most recent call last):
webserver_1 | File "/usr/local/bin/airflow", line 21, in
webserver_1 | from airflow import configuration
webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/init.py", line 36, in
webserver_1 | from airflow import settings
webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 261, in
webserver_1 | configure_adapters()
webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 218, in configure_adapters
webserver_1 | import MySQLdb.converters
webserver_1 | File "/usr/local/lib/python3.6/site-packages/MySQLdb/init.py", line 24, in
webserver_1 | version_info, _mysql.version_info, _mysql.file
webserver_1 | NameError: name '_mysql' is not defined

[robertdefilippi]

I just cloned the repo a moment ago and ran into some errors when trying to run "docker-compose up" too. For my case, the error message was complaining about some missing mysql file towards the end..

I updated the docker image for puckel/docker-airflow to version 1.10.9 instead of 1.10.1 at the docker-compose.yml file. And also the sqlalchemy dependency to version 1.3.0.

It works after that on my machine. I'm on Windows 10 with Docker Desktop v3.5.2(66501). Hope this helps!

p.s. Also, I happened to miss out the 3rd prerequisite initially, that it to install the Apache Airflow python package. Don't miss it out.

My working version of the docker-compose.yml:

version: '3'
services:
  postgres:
    image: postgres:9.6
    environment:
      - POSTGRES_USER=airflow
      - POSTGRES_PASSWORD=airflow
      - POSTGRES_DB=airflow
    ports:
      - "5432:5432"

  webserver:
    image: puckel/docker-airflow:1.10.9
    build:
      context: https://github.com/puckel/docker-airflow.git#1.10.9
      dockerfile: Dockerfile
      args:
        AIRFLOW_DEPS: gcp_api,s3
        PYTHON_DEPS: sqlalchemy==1.3.0
    restart: always
    depends_on:
      - postgres
    environment:
      - LOAD_EX=n
      - EXECUTOR=Local
      - FERNET_KEY=jsDPRErfv8Z_eVTnGfF8ywd19j4pyqE3NpdUBA_oRTo=
    volumes:
      - ./examples/intro-example/dags:/usr/local/airflow/dags
      # Uncomment to include custom plugins
      # - ./plugins:/usr/local/airflow/plugins
    ports:
      - "8080:8080"
    command: webserver
    healthcheck:
      test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
      interval: 30s
      timeout: 30s
      retries: 3

This worked 😄

[zishen1229]

I applied your suggestion but I keep getting the same error with _mysql. Docker engine v20.10.8 Docker desktop v 3.6.0

webserver_1 | webserver_1 | Traceback (most recent call last): webserver_1 | File "/usr/local/bin/airflow", line 21, in webserver_1 | from airflow import configuration webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/init.py", line 36, in webserver_1 | from airflow import settings webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 261, in webserver_1 | configure_adapters() webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 218, in configure_adapters webserver_1 | import MySQLdb.converters webserver_1 | File "/usr/local/lib/python3.6/site-packages/MySQLdb/init.py", line 24, in webserver_1 | version_info, _mysql.version_info, _mysql.file webserver_1 | NameError: name '_mysql' is not defined

Having the same issue here. Did you find a way to fix it?

[DriLLFreAK100]

@zishen1229 Have you fulfilled the third prerequisite yet? That is to install the Python package for Apache Airflow here .

[zishen1229]

@zishen1229 Have you fulfilled the third prerequisite yet? That is to install the Python package for Apache Airflow here .

Yes, my versions are as below:
docker:
Client:
Cloud integration: 1.0.17
Version: 20.10.8
API version: 1.41
Go version: go1.16.6
Git commit: 3967b7d
Built: Fri Jul 30 19:55:20 2021
OS/Arch: darwin/amd64
Context: default
Experimental: true
docker-compose:
docker-py version: 4.4.4
CPython version: 3.8.5
OpenSSL version: OpenSSL 1.1.1h 22 Sep 2020
airflow version: 2.1.4

[tnanhd]

@zishen1229
Hi. Have you fixed the error?
After a few tries, I ran it successfully by following the guide from @DriLLFreAK100 above.
Please notice that you have to modify the context to 1.10.9 as well.
context: https://github.com/puckel/docker-airflow.git#1.10.9

[shreyanshumalviya]

For me Manual installation steps for older versions of WSL worked.

[rns4731]

This is what worked for me. I had to updated the PYTHON_DEPS

  webserver:
    image: puckel/docker-airflow:1.10.9
    build:
      context: https://github.com/puckel/docker-airflow.git#1.10.9
      dockerfile: Dockerfile
      args:
        AIRFLOW_DEPS: gcp_api,s3
        PYTHON_DEPS: sqlalchemy==1.3.0 MarkupSafe==2.0.1 wtforms<=2.3.3
    restart: always
    depends_on:
      - postgres
    environment:
      - LOAD_EX=n
      - EXECUTOR=Local
      - FERNET_KEY=jsDPRErfv8Z_eVTnGfF8ywd19j4pyqE3NpdUBA_oRTo=
    volumes:
      - ./examples/intro-example/dags:/usr/local/airflow/dags
      # Uncomment to include custom plugins
      # - ./plugins:/usr/local/airflow/plugins
    ports:
      - "8080:8080"
    command: webserver
    healthcheck:
      test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
      interval: 30s
      timeout: 30s
      retries: 3