Add documentation for setting up a mirror for Cypress binary downloads

[aarowman]

Subject

guides, examples, mirror setup

Description

The current documentation describes mirroring briefly, and there are a few different options in setting up a mirror. The docs do not give detail on how to do the actual setup of the mirror. It only references how to use a mirror that was hopefully set up correctly.

Relevant links:

• Advanced Installation > Mirroring
• Advanced Installation > Download path template

The docs reference 3 main approaches:

1. Using download.cypress.io (full site)

   If you choose to mirror the entire Cypress download site,"

   It seems that mirroring download.cypress.io, we would need to mirror everything, including the query params and redirect behavior. This is not clear how to set up correctly.

2. Using cdn.cypress.io

   To install the binary from a download mirror that matches the exact file structure of https://cdn.cypress.io (works for Cypress 9.3.0 or newer):

   export CYPRESS_DOWNLOAD_MIRROR=https://cypress-download.local
   export CYPRESS_DOWNLOAD_PATH_TEMPLATE='${endpoint}/${platform}-${arch}/cypress.zip'
   # Example of a resulting URL: https://cypress-download.local/desktop/10.11.0/linux-x64/cypress.zip
   

3. Some custom file structure

   To install the binary from a download server with a custom file structure (works for Cypress 10.6.0 or newer):

   export CYPRESS_DOWNLOAD_PATH_TEMPLATE='https://software.local/cypress/${platform}/${arch}/${version}/cypress.zip'
   # Example of a resulting URL: https://software.local/cypress/linux/x64/10.11.0/cypress.zip
   

Each of these 3 will need to be set up in different ways. Since mirrors could just copy/cache the files, or actually redirect/proxy to the source system, or both, it would be helpful to have more detailed descriptions.

Note: An important part is how to differentiate between installs (Windows, MacOS, Linux), and also how to update the files as new versions are released.

(Also see discussion at cypress-io/cypress#8471 where a mirror issue caused problems, but was not obvious why)

[aarowman]

Hello, this was labeled as triaged - is this planned to be implemented? I'm very interested for how to do this setup as we are currently having issues with setting up a mirror for both sites (download.cypress.io and cdn.cypress.io)

[AnatomicJC]

Hi!

I am interested for a documentation about how to setup a mirror. I am behind a corporate proxy and tried to setup a nginx configuration but I always get a 301 redirect to download.cypress.io.

:-(

[AnatomicJC]

Hello guys,

I implemented a home-made mirror written with python and Flask.

I can now use CYPRESS_DOWNLOAD_MIRROR="https://mymirror.local" cypress install as described in the doc.

Basically, if the cypress archive I want to download doesn't exists on my mirror server, the python script download it then serve it to the client. It will be served directly without the need to re-download it again next time.

I can use https://mymirror.local/desktop/12.14.0?platform=win32&arch=x64 to grab Windows 12.14.0 version.

I can use https://mymirror.local/desktop/11.11.0?platform=linux&arch=x64 to grab Linux 12.14.0 version.

Needed dependencies: python Flask and requests modules

My app.py:

from flask import Flask, send_from_directory, abort, request
import os
import requests

app = Flask(__name__)

DESKTOP_DIR = "/home/jc/cypress/desktop"

@app.route('/desktop/<version>')
def get_cypress_file(version):
    platform = request.args.get('platform')
    arch = request.args.get('arch')

    if not platform or not arch:
        abort(400, description="'platform' et 'arch' parameters are mandatory.")

    local_path = os.path.join(DESKTOP_DIR, version, f"{platform}-{arch}", "cypress.zip")

    if not os.path.exists(local_path):
        download_url = f"https://download.cypress.io/desktop/{version}/{platform}-{arch}/cypress.zip"
        response = requests.get(download_url, stream=True)
        if response.status_code == 200:
            os.makedirs(os.path.dirname(local_path), exist_ok=True)

            with open(local_path, 'wb') as f:
                for chunk in response.iter_content(chunk_size=8192):
                    f.write(chunk)
        else:
            abort(404)

    return send_from_directory(os.path.dirname(local_path), "cypress.zip")

if __name__ == '__main__':
    app.run(debug=True)

To quickly test it, available on localhost:5000: python app.py

To serve it in my company, I installed gunicorn and created a systemd service unit:

[Unit]
Description=Gunicorn daemon for cypress mirror
After=network.target

[Service]
User=app
WorkingDirectory=/home/app/cypress-mirror
EnvironmentFile=/etc/environment
ExecStart=/usr/bin/gunicorn --workers 8 --bind 127.0.0.1:8000 app:app --access-logfile /var/log/gunicorn/cypress-access.log --error-logfile /var/log/gunicorn/cypress-error.log
ExecReload=/bin/kill -s HUP $MAINPID

[Install]
WantedBy=multi-user.target

I use a app user, this user must have sufficient rights to write logs in /var/log/gunicorn/. I put my app.py script in /home/app/cypress-mirror folder. My /etc/environment file contains needed exports to the corporate proxy.

To enable and start systemd service file:

systemctl daemon-reload
systemctl start cypress-mirror.service

To rotate logs, here is the /etc/logrotate.d/gunicorn-cypress configuration file:

/var/log/gunicorn/*.log {
  daily
  rotate 7
  compress
  delaycompress
  missingok
  notifempty
  create 0644 app app
  postrotate
    systemctl reload cypress-mirror.service
  endscript
}

My nginx configuration to serve the gunicorn python app:

server {
  listen 80;
  server_name mymirror.local;
  return 301 https://mymirror.local$request_uri;
}

server {
  listen 443 ssl;
  server_name mymirror.local;
  include /etc/nginx/snippets.d/proxy.conf;
  include /etc/nginx/snippets.d/ssl.conf;
  location / {
    proxy_pass            http://127.0.0.1:8000;
    proxy_set_header      Host $host;
    proxy_set_header      X-Real-IP $remote_addr;
    proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header      X-Forwarded-Proto $scheme;
  }
}

Hope this help :)

[AnatomicJC]

In case you want just a webservice who serve directly file without caching:

from flask import Flask, Response, abort, request
import requests

app = Flask(__name__)

@app.route('/desktop/<version>')
def get_cypress_file(version):
    platform = request.args.get('platform')
    arch = request.args.get('arch')

    if not platform or not arch:
        abort(400, description="'platform' et 'arch' parameters are mandatory.")

    download_url = f"https://download.cypress.io/desktop/{version}/{platform}-{arch}/cypress.zip"
    response = requests.get(download_url, stream=True)

    if response.status_code != 200:
        abort(404)

    def generate():
        for chunk in response.iter_content(chunk_size=8192):
            yield chunk

    return Response(generate(), content_type="application/zip")

if __name__ == '__main__':
    app.run(debug=True)