Feature/update readme (#215)

* Update README

Author: daniele-20tab

README.md

@@ -1,59 +1,165 @@
-# Django continuous delivery
+# Talos - Django Continuous Delivery
 
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/python/black)
 
 > A [Django](https://docs.djangoproject.com) project template ready for continuous delivery.
 
-## 📝 Conventions
+## 🧩 Requirements
 
-In the following instructions:
+The Talos script can be run either using Docker or as a local shell command.
 
--   replace `projects` with your actual projects directory
--   replace `My project name` with your chosen project name
+### 🐋 Docker
 
-## Git
+In order to run Talos via Docker, a working [Docker installation](https://docs.docker.com/get-docker/) is the only requirement.
 
-To get the existing project, change directory and clone the project repository.
+### 👨‍💻 Shell command
 
+In order to run Talos as a shell command, first clone the repository in a local projects directory
 ```console
-$ cd ~/projects/
-$ git clone https://github.com/20tab/django-continuous-delivery.git
+cd ~/projects
+git clone https://github.com/20tab/django-continuous-delivery.git talos-django
 ```
+Then, install the following requirements
+| Requirements | Instructions |
+|--|--|
+|🌎 Terraform  | [Install Guide](https://learn.hashicorp.com/tutorials/terraform/install-cli)  |
+|🐍 Python Dependencies | `pip install -r talos-django/requirements/common.txt` |
 
-## 🧩 Requirements
+## 🔑 Credentials
+
+### 🦊 GitLab
+If the GitLab integration is enabled, a Personal Access Token with _api_ permission is required.<br/>
+It can be generated in the GitLab User Settings panel.
 
-A set of requirements must be installed before initializing the project.
+**Note:** the token can be generated in the Access Tokens section of the GitLab User Settings panel.<br/>
+⚠️ Beware that the token is shown only once after creation.
 
+## 🚀️ Quickstart
+
+Change to the projects directory, for example
 ```console
-$ python3 -m pip install -r django-continuous-delivery/requirements/common.txt
+cd ~/projects
 ```
 
-## 🚀️ Quickstart
+### 🐋 Docker
 
-Change directory and create a new project as in this example:
+```console
+docker run --interactive --tty --rm --volume $PWD:/data 20tab/talos-django:latest
+```
+
+### 👨‍💻 Shell command
+
+```console
+./talos-django/setup.py
+```
 
+### Example
 ```console
-$ python3 django-continuous-delivery/setup.py
-Project name: My project name
+Project name: My Project Name
 Project slug [my-project-name]:
 Service slug [backend]:
 Project dirname (backend, myprojectname) [backend]: myprojectname
 Development environment complete URL [https://dev.my-project-name.com/]:
 Staging environment complete URL [https://stage.my-project-name.com/]:
 Production environment complete URL [https://www.my-project-name.com/]:
 Media storage (local, s3-digitalocean, none) [s3-digitalocean]:
+Do you want to configure Redis? [y/N]: y
+Do you want to configure Gitlab? [Y/n]: y
+Gitlab group slug [my-project-name]:
+Make sure the Gitlab "my-project-name" group exists before proceeding. Continue? [y/N]: y
+Gitlab private token (with API scope enabled):
+Sentry DSN (leave blank if unused) []:
 Initializing the backend service:
 ...cookiecutting the service
 ...generating the .env file
 ...formatting the cookiecut python code
 ...compiling the requirements files
+	- common.txt
+	- test.txt
+	- local.txt
+	- remote.txt
+	- base.txt
 ...creating the '/static' directory
-Do you want to configure Gitlab? [Y/n]: Y
-Gitlab group slug [my-project-name]:
-Make sure the Gitlab "my-project-name" group exists before proceeding. Continue? [y/N]: y
-Gitlab private token (with API scope enabled):
-Sentry DSN (leave blank if unused) []: https://sentry.io/mydsn
-...creating the Gitlab repository and associated resources
-Project successfully initialized.
-$ cd myprojectname
+...creating the GitLab repository and associated resources
 ```
+## 🗒️ Arguments
+
+The following arguments can be appended to the Docker and shell commands
+
+#### User id
+`--uid=$UID`
+
+#### Group id
+`--gid=1000`
+
+#### Output directory
+`--output-dir="~/projects"`
+
+#### Project name
+`--project-name="My project name"`
+
+#### Project slug
+`--project-slug="my-project-name"`
+
+#### Project dirname
+`--project-dirname="myprojectname"`
+
+### 🎖️ Service
+
+#### Service slug
+`--service-slug=backend`
+
+#### Service port
+`--internal-service-port=8000`
+
+#### Project Domain
+If you don't want DigitalOcean DNS configuration the following args are required
+
+`--project-url-dev=https://dev.project-domain.com`<br/>
+`--project-url-stage=https://stage.project-domain.com`<br/>
+`--project-url-prod=https://www.project-domain.com`
+
+#### Media storage
+
+Value  | Description | Argument
+------------- | ------------- | -------------
+local  | Docker Volume are used for store media | `--media-storage=local`
+s3-digitalocean  | `--media-storage=s3-digitalocean`
+none  | Project have no media | `--media-storage=none`
+
+#### Redis
+For enable redis integration the following arguments are needed:
+
+`--use-redis`
+
+Disabled args
+`--no-redis`
+
+### 🦊 GitLab
+> **⚠️ Important:  Make sure the GitLab group exists before create.**
+> https://gitlab.com/gitlab-org/gitlab/-/issues/244345
+
+For enable gitlab integration the following arguments are needed:
+
+`--use-gitlab`<br/>
+`--gitlab-private-token={{gitlab-private-token}}`<br/>
+`--gitlab-group-slug={{gitlab-group-slug}}`
+
+Disabled args
+`--no-gitlab`
+
+Add user to repository using comma separeted arguments
+
+`--gitlab-group-owners=user1, [email protected]`<br/>
+`--gitlab-group-maintainers=user1, [email protected]`<br/>
+`--gitlab-group-developers=user1, [email protected]`
+
+#### 🪖 Sentry
+For enable sentry integration the following arguments are needed:
+
+`--sentry-dsn={{frontend-sentry-dsn}}`
+
+#### 🔇 Silent
+Is command for use default if no args are provided
+
+`--silent`

bootstrap/collector.py

@@ -171,25 +171,25 @@ def clean_use_redis(use_redis):
 
 
 def clean_use_gitlab(use_gitlab):
-    """Tell whether Gitlab should be used."""
+    """Tell whether GitLab should be used."""
     if use_gitlab is None:
-        return click.confirm(warning("Do you want to configure Gitlab?"), default=True)
+        return click.confirm(warning("Do you want to configure GitLab?"), default=True)
     return use_gitlab
 
 
 def clean_gitlab_group_data(project_slug, gitlab_group_slug, gitlab_private_token):
-    """Return Gitlab group data."""
+    """Return GitLab group data."""
     gitlab_group_slug = slugify(
-        gitlab_group_slug or click.prompt("Gitlab group slug", default=project_slug)
+        gitlab_group_slug or click.prompt("GitLab group slug", default=project_slug)
     )
     click.confirm(
         warning(
-            f'Make sure the Gitlab "{gitlab_group_slug}" group exists '
+            f'Make sure the GitLab "{gitlab_group_slug}" group exists '
             "before proceeding. Continue?"
         ),
         abort=True,
     )
     gitlab_private_token = gitlab_private_token or click.prompt(
-        "Gitlab private token (with API scope enabled)", hide_input=True
+        "GitLab private token (with API scope enabled)", hide_input=True
     )
     return gitlab_group_slug, gitlab_private_token

bootstrap/runner.py

@@ -114,7 +114,7 @@ def init_service(
             "project_url_prod": project_url_prod,
             "project_url_stage": project_url_stage,
             "service_slug": service_slug,
-            "use_redis": use_redis,
+            "use_redis": f"{use_redis}",
         },
         output_dir=output_dir,
         no_input=True,
@@ -171,8 +171,8 @@ def init_gitlab(
     terraform_dir,
     logs_dir,
 ):
-    """Initialize the Gitlab repository and associated resources."""
-    click.echo(info("...creating the Gitlab repository and associated resources"))
+    """Initialize the GitLab repository and associated resources."""
+    click.echo(info("...creating the GitLab repository and associated resources"))
     terraform_dir = Path(terraform_dir) / service_slug
     os.makedirs(terraform_dir, exist_ok=True)
     env = dict(
@@ -228,7 +228,7 @@ def init_gitlab(
             apply_stderr_path.write_text(apply_process.stderr)
             click.echo(
                 error(
-                    "Error applying Terraform Gitlab configuration "
+                    "Error applying Terraform GitLab configuration "
                     f"(check {apply_stderr_path} and {apply_log_path})"
                 )
             )

cookiecutter.json

@@ -9,6 +9,6 @@
   "project_url_stage": null,
   "project_url_prod": null,
   "media_storage": ["local", "s3-digitalocean", "none"],
-  "use_redis": false,
+  "use_redis": "False",
   "_extensions": ["cookiecutter.extensions.SlugifyExtension"]
 }

terraform/main.tf

@@ -11,7 +11,7 @@ terraform {
   required_providers {
     gitlab = {
       source  = "gitlabhq/gitlab"
-      version = "3.7.0"
+      version = "3.12.0"
     }
   }
 }
@@ -68,6 +68,7 @@ resource "null_resource" "init_repo" {
           "git checkout -b main",
           "git push -u origin main -o ci.skip",
           "git remote set-url origin %s",
+          "git checkout develop",
         ]),
         replace(
           gitlab_project.main.http_url_to_repo,
@@ -113,12 +114,6 @@ resource "gitlab_project_badge" "coverage" {
   image_url = "https://gitlab.com/%%{project_path}/badges/%%{default_branch}/coverage.svg"
 }
 
-resource "gitlab_project_badge" "pipeline" {
-  project   = gitlab_project.main.id
-  link_url  = "https://gitlab.com/%%{project_path}/pipelines"
-  image_url = "https://gitlab.com/%%{project_path}/badges/%%{default_branch}/pipeline.svg"
-}
-
 /* Group Variables */
 
 resource "gitlab_group_variable" "vars" {

terraform/variables.tf

@@ -1,22 +1,22 @@
 variable "gitlab_group_slug" {
-  description = "The slug of the Gitlab group."
+  description = "The slug of the GitLab group."
   type        = string
 }
 
 variable "gitlab_group_variables" {
-  description = "A map of Gitlab group variables to create."
+  description = "A map of GitLab group variables to create."
   type        = map(map(any))
   default     = {}
 }
 
 variable "gitlab_project_variables" {
-  description = "A map of Gitlab project variables to create."
+  description = "A map of GitLab project variables to create."
   type        = map(map(any))
   default     = {}
 }
 
 variable "gitlab_token" {
-  description = "The Gitlab token."
+  description = "The GitLab token."
   type        = string
   sensitive   = true
 }

{{cookiecutter.project_dirname}}/.gitlab-ci.yml

@@ -24,7 +24,7 @@ sentry_release:
     - docker:20-dind
   rules:
     - &pipeline-push-rule
-      if: $CI_PIPELINE_SOURCE != "push"
+      if: $CI_PIPELINE_SOURCE == "trigger"
       when: never
     - if: $SENTRY_DSN && ($CI_COMMIT_BRANCH == "develop" || $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_TAG)
   script:
@@ -49,7 +49,7 @@ test:
     - docker:20-dind
   needs: []
   rules:
-    - if: $CI_PIPELINE_SOURCE == "push"
+    - if: $CI_PIPELINE_SOURCE != "trigger" && $CI_PIPELINE_SOURCE != "merge_request_event"
   variables:
     {{ cookiecutter.service_slug|upper }}_CONTAINER_NAME: "${CI_PROJECT_PATH_SLUG}-${CI_JOB_NAME}-${CI_JOB_ID}_{{ cookiecutter.service_slug }}"
     {{ cookiecutter.service_slug|upper }}_BUILD_TARGET: "test"
@@ -267,6 +267,8 @@ build_production:
     - docker:20-dind
   variables:
     TF_ROOT: ${CI_PROJECT_DIR}/terraform
+    TERRAFORM_CMD: "gitlab-terraform"
+    TERRAFORM_IMAGE: "registry.gitlab.com/gitlab-org/terraform-images/stable:latest"
   before_script:
     - export TF_VAR_service_container_image=${CI_REGISTRY_IMAGE}:${VERSION_REF}
   script:
@@ -276,7 +278,9 @@ build_production:
       -v ${PWD}:${PWD}
       -w ${PWD}
       -e CI_JOB_TOKEN
+      -e TERRAFORM_CMD
       -e TF_ADDRESS=${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/{{ cookiecutter.project_slug }}_${ENVIRONMENT_SLUG}
+      -e TF_IN_AUTOMATION=1
       -e TF_ROOT
       -e TF_VAR_cache_url="${CACHE_URL}"
       -e TF_VAR_digitalocean_token="${DIGITALOCEAN_TOKEN}"
@@ -301,7 +305,7 @@ build_production:
       -e TF_VAR_service_replicas
       -e TF_VAR_web_concurrency="${WEB_CONCURRENCY}"
       -e TF_VAR_stack_slug="${STACK_SLUG}"
-      registry.gitlab.com/gitlab-org/terraform-images/stable:latest ./scripts/deploy.sh
+      ${TERRAFORM_IMAGE}:latest ./scripts/deploy.sh
   artifacts:
     name: plan
     paths:

{{cookiecutter.project_dirname}}/Makefile

@@ -77,7 +77,7 @@ pip: pip_update  ## Compile requirements
 
 .PHONY: pip_update
 pip_update:  ## Update requirements and dependencies
-	python3 -m pip install -q -U pip~=22.0.0 pip-tools~=6.5.0 setuptools~=60.9.0 wheel~=0.37.0
+	python3 -m pip install -q -U pip~=22.0.0 pip-tools~=6.5.0 setuptools~=60.10.0 wheel~=0.37.0
 
 .PHONY: precommit
 precommit:  ## Fix code formatting, linting and sorting imports

{{cookiecutter.project_dirname}}/README.md

@@ -20,7 +20,7 @@ A [Django](https://docs.djangoproject.com) project using [uvicorn](https://www.u
 -   [Testing](#testing)
 -   [Static files](#static-files)
 -   [Continuous Integration](#continuous-integration)
-    -   [Gitlab CI](#gitlab-ci)
+    -   [GitLab CI](#gitlab-ci)
 
 ## Conventions
 
@@ -132,6 +132,6 @@ $ make collectstatic
 
 Depending on the CI tool, you might need to configure Django environment variables.
 
-### Gitlab CI
+### GitLab CI
 
 The configuration file `.gitlab-ci.yml` should work as is, needing no further customization.

{{cookiecutter.project_dirname}}/requirements/remote.in

@@ -3,7 +3,7 @@
 {% if "s3" in cookiecutter.media_storage %}django-storages[boto3]~=1.12.0
 {% endif %}gunicorn~=20.1.0
 gunicorn~=20.1.0
-{% if cookiecutter.use_redis %}redis~=4.1.0
+{% if cookiecutter.use_redis == "True" %}redis~=4.1.0
 {% endif %}sentry-sdk~=1.5.0
 uvicorn[standard]~=0.17.0
 wait-for-it~=2.2.0