[7.x] Database stored in Docker image

This release adds highly demanded and long overdue support for using databases stored in Docker image. Using this workflow allows to build large database images in CI overnight and download databases as a fresh image in the morning to use for development and as a cache in CI.

During development, rebuilding local environments for projects with large databases will no longer be a long process (due to the time it takes to import from the database dump), but rather a matter of restarting a container. On the large databases (>1G) the time is reduced from 15-20 minutes to 15-30 seconds.

Additionally, there is now a support for several database sources: files or docker registry. As a result - there is a support for the following workflows (simplified):

• Download the database from CURL/FTP/Acquia as a dump file -> Import the database -> Develop/run tests -> Deploy
• Download the database from CURL/FTP/Acquia as a dump file -> Import into the Docker database image -> Push image to the registry -> Use the image to develop/run tests -> Deploy
• Download the database from the previous version of the image -> Use the image to develop/run tests -> Deploy

CI configuration now allows to enable using the database cache (file or Docker image) from the builds ran on the previous days. It also now supports nightly builds to cache the database to be used in builds during the day (significant builds speedup).

Lastly, the installer script was rewritten in PHP to allow more robust parameters handling and better development speed. It has own PHPUnit tests and updated Bats integration tests.

And of course, all the new functionality has automated tests.

Docker

• Updated images to Lagoon v1.4.1.
• Updated Docker Compose configuration to support custom database image taken from DATABASE_IMAGE environment variable.
• Removed quite pulling of the images. This is due to requests from developers to see which images were cached and which were downloaded during the build.

CI

• ❗❗Added support for nightly database builds + test.
• ❗Added support for per-workflow caching.
• Added support for cache fallback with a switch - using per-branch build caches from previous days.
• Increased database job timeout in CI to 30 minutes.

Scripts

• 🔥🔥🔥 Rewritten installer in PHP.
  • Better installer written in PHP + complies with Drupal coding standards.
  • Installer is interactive by default (previsously was quiet).
  • Added PHPunit tests for installer.
  • Updated existing integration Bats tests for installer.
  • Fixed several issues with automated configuration discovery for existing projects.
  • Installer now always overrides existing files during an update.
• 🔥🔥🔥 Added support for database-in-Docker-image. This allows to work with DB image with in-built data.
  • ❗❗❗ Use https://github.com/drevops/mariadb-drupal-data as a base image + DB seeding script is included.
  • ❗❗❗ Workflow switch is controlled by DATABASE_IMAGE environment variable (one variable to controll the whole workflow).
  • ❗❗ Added support for exporting image as archive. Used by CI to pass built images between jobs as cache and re-use this cache in the subsequent jobs.
  • ❗ Added support for deploying an image to a registry when exporting (if EXPORT_DB_DEPLOY_DOCKER is set).
  • Added reload-db command. Used to quickly restart database image.
  • Added ahoy pull-db command. Used to pull the latest version of the Database image.
• Added support for deployment of multiple docker images to Docker registry (according to provided service-to-image map).
• Added support for archiving and un-archiving of Docker images.
• Removed DrevOps test files from the downloadable package. This makes the downloadable archive cleaner.
• Moved DrevOps' tests and documentation under scripts/drevops directory.
• Added VERBOSE_BUILD variable to show the build progress.
• Added linting of Bats tests with Shellcheck.
• Improved bats test helpers and assertions.
• Added confirmations to ahoy build and ahoy down commands.

Back-end

• Removed Gherkin patch to run single tests with release of behat/gherkin version 4.6.1.
• Fixed cron running locally and in CI.

Front-end

No changes

Integrations

• Added support for ODE environments in Acquia (settings and hooks).
• Acquia is no longer an integration enabled by default.
• Lagoon is no longer an integration suggested by default.

Documentation

• Added more documentation in comments to .env file.
• Added more FAQs.
• Added documentation scaffolding (still WIP).

[8.x] Better CI handling 🤖 🤖 🤖

DrevOps has moved to its own GitHub organisation https://github.com/drevops! 🎉 🎉 🎉
This allows to maintain all repositories related to DrevOps in one place.

Docker

• 🔥 Updated container images to 1.2.0.
• 🔥 Updated PHP to 7.3.
• 🔥 Added Dockerfile for mariadb service with ability to provide custom configuration.
• Less output when installing FE tools.
• Added docker pull progress output suppression.

CI

• 🔥🔥🔥 Removed additional CI scripts. CI now uses ahoy scripts directly!
• 🔥🔥 Refactored CI config to use YAML anchors as variables. Also, defined section that should change per-project.
• 🔥 Move code linting is a separate job in CI.
• 🔥 Added default timezone for CI runner container.
• Updated CI builder image to use new namespace.
• Added more timeout for commands without output in CI.
• Fixed network warning in CI.
• Updated default database directory location in CI.
• Added artifacts storage in CI
• Added test results storage in CI

Scripts

• Workflow
  • 🔥🔥 Updated install-site script to support installing from database or fresh install if database dump file is not available.
  • 🔥🔥 Fixed env variables with values being overridden by values in .env. Any variable that has a value in the current environment would be overwritten by the value in the .env file.
  • Fixed ahoy clean command not removing files with readonly permissions.
  • Moved Composer validation to project build stage (but only if composer is installed).
  • Refactored all scripts to correctly work with database dump directory and filename.
  • Improved the logic of DB sanitization - now only runs if file exists.
• Tools
  • Updated lint for PHPCS to not be verbose while scanning.
  • Added logs for running unit tests.
  • Removed global unit test directory. Tests should live with modules.
• Doctor
  • Removed Composer presence check from Doctor. Composer is not required on host.
  • Fixed Doctor check for status code of the homepage to allow 200 and 403, returned depending on the installation type.
  • Updated Doctor to check the status of all containers.
  • Added DOCTOR_CHECK_MINIMAL shorthand variable to run minimal Doctor checks in CI env.
• Installation
  • Fixed install.sh not correctly discovering project settings when installed into a custom directory + tests.
  • Fixed install.sh not failing on errors.
  • Fixed install.sh to never modify DrevOps scripts when installing.
• Testing
  • Fixed some integration tests not running.
• 🔥 Added printing of debug output in DrevOps scripts using DREVOPS_DEBUG variable.
• Fixed code formatting in multiple scripts.
• Removed DOCKER_HOST temporary override in .env file.
• Removed default deployer user name from the deployment script. This should be set through env variables.

Back-end

• Core and Modules
  • Updated Drupal core version to 8.8.
  • Removed obsolete code to load hook implementations from included sub-directories.
  • Added Shield popup default title.
  • Removed custom patches. Some of the patches were breaking existing sites.
• Settings
  • Improved readability of settings.php file.
  • Updated default values in settings.local.php file.
  • Added skipping of permissions hardening locally and in CI.
• Testing
  • Renamed test helper class to a trait in your_site_core module.

Front-end

• Renamed style.scss to styles.scss.
• Fixed FE browser list for autoprefix package.
• Removed Barrio base theme and twbs/bootstrap library from theme.

Integrations

• Updated Acquia credentials in .env file to not rely on variables.
• Added NewRelic notifications for Acquia production.
• Fixed Acquia DB download script not correctly identifying it's location.
• Refactored Acquia hooks scripts to allow configuration through environment variables and comply with coding standards.

Other

• Removed obsolete Brewfile.
• Removed obsolete .nvmrc file.

[7.x] Better CI handling 🤖 🤖 🤖

DrevOps has moved to its own GitHub organisation https://github.com/drevops! 🎉 🎉 🎉
This allows to maintain all repositories related to DrevOps in one place.

Docker

• 🔥 Updated container images to 1.2.0.
• 🔥 Updated PHP to 7.3.
• 🔥 Added Dockerfile for mariadb service with ability to provide custom configuration.
• Less output when installing FE tools.
• Added docker pull progress output suppression.

CI

• 🔥🔥🔥 Removed additional CI scripts. CI now uses ahoy scripts directly!
• 🔥🔥 Refactored CI config to use YAML anchors as variables. Also, defined section that should change per-project.
• 🔥 Move code linting is a separate job in CI.
• 🔥 Added default timezone for CI runner container.
• Updated CI builder image to use new namespace.
• Added more timeout for commands without output in CI.
• Fixed network warning in CI.
• Updated default database directory location in CI.
• Added artifacts storage in CI
• Added test results storage in CI

Scripts

• Workflow
  • 🔥🔥 Updated install-site script to support installing from database or fresh install if database dump file is not available.
  • 🔥🔥 Fixed env variables with values being overridden by values in .env. Any variable that has a value in the current environment would be overwritten by the value in the .env file.
  • Fixed ahoy clean command not removing files with readonly permissions.
  • Moved Composer validation to project build stage (but only if composer is installed).
  • Refactored all scripts to correctly work with database dump directory and filename.
  • Improved the logic of DB sanitization - now only runs if file exists.
• Tools
  • Updated lint for PHPCS to not be verbose while scanning.
  • Updated PHPCS config to include more Features exclusions.
  • Added logs for running unit tests.
  • Removed global unit test directory. Tests should live with modules.
• Doctor
  • Removed Composer presence check from Doctor. Composer is not required on host.
  • Fixed Doctor check for status code of the homepage to allow 200 and 403, returned depending on the installation type.
  • Updated Doctor to check the status of all containers.
  • Added DOCTOR_CHECK_MINIMAL shorthand variable to run minimal Doctor checks in CI env.
• Installation
  • Fixed install.sh not correctly discovering project settings when installed into a custom directory + tests.
  • Fixed install.sh not failing on errors.
  • Fixed install.sh to never modify DrevOps scripts when installing.
• Testing
  • Fixed some integration tests not running.
• 🔥 Added printing of debug output in DrevOps scripts using DREVOPS_DEBUG variable.
• Fixed code formatting in multiple scripts.
• Removed DOCKER_HOST temporary override in .env file.
• Removed default deployer user name from the deployment script. This should be set through env variables.

Back-end

• Core and Modules
  • Updated Drupal core version to 7.69.
  • Removed obsolete code to load hook implementations from included sub-directories.
  • Added Shield popup default title.
  • Removed custom patches. Some of the patches were breaking existing sites.
• Settings
  • Improved readability of settings.php file.
  • Updated default values in settings.local.php file.
  • Added skipping of permissions hardening locally and in CI.
• Testing
  • Fixed phpunit bootstrap.
  • Moved Drupal tests to your_site_core module.
  • Renamed test helper class to a trait.

Front-end

• Renamed style.scss to styles.scss.
• Fixed FE browser list for autoprefix package.
• Removed Bootstrap 3 theme and library from theme.
• Fixed sass-lint config file location.

Integrations

• Updated Acquia credentials in .env file to not rely on variables.
• Added NewRelic notifications for Acquia production.
• Fixed Acquia DB download script not correctly identifying it's location.
• Refactored Acquia hooks scripts to allow configuration through environment variables and comply with coding standards.

Other

• Removed obsolete Brewfile.
• Removed obsolete .nvmrc file.

[8.x] Renamed project to DrevOps 🚀🚀🚀

It is time for this project to have a new catchy name: DrevOps

Drupal + DevOps = DrevOps

http://drevops.com

[7.x] Renamed project to DrevOps 🚀🚀🚀

It is time for this project to have a new catchy name: DrevOps

Drupal + DevOps = DrevOps

http://drevops.com

Scripts for build unification 🔥🔥🔥

This release is all about unifying workflows to work identically in all environments. It is implemented by extracting all configuration and logic from Ahoy config file into standalone scripts, which allows "cleaner" and portable logic implementation.

Added

• ❗❗❗Moved workflow logic into scripts.
• ❗❗❗Moved Gruntfile.js and other FE config to the theme.
• Deployment as webhook.
• Added config_split config for multiple environments.
• Added SCSS helpers: rem, fonts, mixins.
• Added toolbar removal mechanism to ease themeing while logged in as admin.
• Added non-www to www redirection with Acquia support to .htaccess.
• Added ahoy debug command.

Updated

• ❗❗❗Removed support for .env.local.
• ❗❗❗Removed composer.lock and package-lock.json.
• Refactored tests for easy maintenance.
• Removed local exclude support from the install script.
• Patch to support for running single Behat test.
• Updated Docker images to v1.1.2 and removed NPM workaround.
• Updated mechanism for Xdebug status retrieval.
• Updated Environment Indicator colors.
• Added tests for tests failure bypass.
• Updated dependencies.io configuration to only update PHP dependencies.
• Refactored DB caching in CI.
• Renamed ahoy test-behat to ahoy test-bdd.
• Renamed ahoy test-phpunit to ahoy test-unit.

Fixed

• Fixed demo DB not available on the first install.
• Fixed environment conditions in settings.
• Fixed Livereload not compiling JS.
• Fixed Livereload loading on the admin path and admin theme.
• Fixed testuite name in phpunit config.
• Fixed config being imported when only sanitisation is required.
• Fixed web driver not specifying explicitly to not use w3c format capabilities to the browser.

Scripts for build unification 🔥🔥🔥

This release is all about unifying workflows to work identically in all environments. It is implemented by extracting all configuration and logic from Ahoy config file into standalone scripts, which allows "cleaner" and portable logic implementation.

Added

• ❗❗❗Moved workflow logic into scripts.
• ❗❗❗Moved Gruntfile.js and other FE config to the theme.
• Deployment as webhook.
• Added SCSS helpers: rem, fonts, mixins.
• Added non-www to www redirection with Acquia support to .htaccess.
• Added ahoy debug command.

Updated

• ❗❗❗Removed support for .env.local.
• ❗❗❗Removed composer.lock and package-lock.json.
• Refactored tests for easy maintenance.
• Removed local exclude support from the install script.
• Patch to support for running single Behat test.
• Updated Docker images to v1.1.2 and removed NPM workaround.
• Updated mechanism for Xdebug status retrieval.
• Updated Environment Indicator colors.
• Added tests for tests failure bypass.
• Updated dependencies.io configuration to only update PHP dependencies.
• Refactored DB caching in CI.
• Renamed ahoy test-behat to ahoy test-bdd.
• Renamed ahoy test-phpunit to ahoy test-unit.

Fixed

• Fixed demo DB not available on the first install.
• Fixed environment conditions in settings.
• Fixed Livereload not compiling JS.
• Fixed Livereload loading on the admin path and admin theme.
• Fixed testuite name in phpunit config.
• Fixed config being imported when only sanitisation is required.
• Fixed web driver not specifying explicitly to not use w3c format capabilities to the browser.

[8.x] Stabilisation release. Upgrade now!

❗❗❗ Important note!
This release drastically changes how Drupal-Dev installs files. In previous versions, Drupal-Dev would be installing only required files and "attaching" the rest of the files in each environment by running ./drupal-dev.sh. This behaviour was (rightfully) criticised by the community, so the change was made to add all files to the project without the need to run ./drupal-dev.sh in every environment. With this change, every file of Drupal-Dev (based on your installation selection, of course) is now added to your project repository on installation, so that the project can have all required files in-place.

How to update?

Run ahoy update

Changes

Updated

• ❗❗❗Disabled "attachability" of the environments.
• Updated Drupal core and modules to the latest versions.
• Updated Docker images to the latest version.

Added

• Added Drupal-Dev version to the badge.

Fixed

• Fixed some internal files are added during installation.

[7.x] Stabilisation release. Upgrade now!

❗❗❗ Important note!
This release drastically changes how Drupal-Dev installs files. In previous versions, Drupal-Dev would be installing only required files and "attaching" the rest of the files in each environment by running ./drupal-dev.sh. This behaviour was (rightfully) criticised by the community, so the change was made to add all files to the project without the need to run ./drupal-dev.sh in every environment. With this change, every file of Drupal-Dev (based on your installation selection, of course) is now added to your project repository on installation, so that the project can have all required files in-place.

How to update?

Run ahoy update

Changes

Updated

• ❗❗❗Disabled "attachability" of the environments.
• Updated Drupal core and modules to the latest versions.
• Updated Docker images to the latest version.

Added

• Added Drupal-Dev version to the badge.stack and resolving issues easier.

Fixed

• Fixed some internal files are added during installation.

[8.x] Bug fixes and minor improvements.

Added

• ❗❗❗ Added Doctor doing pre-flight checks before, during and after build. This makes working with the stack and resolving issues easier.
• ❗❗❗ Added support for PHPUnit tests + updated to PHPUnit 6.5 (used by Drupal core).
• Added LAGOON_PROJECT variable to handle cases when project in Lagoon is named differently then git repo.
• Added default user under which the containers should run. Allows to better run in Linux.

Fixed

• Fixed default Lagoon drush stub URL.
• Fixed handling of host volumes in tests.

Updated

• Updated dependencies to the latest versions.