From 23249a803a06a960da9e065f9ea768f9fb425a64 Mon Sep 17 00:00:00 2001 From: Matt Bishop Date: Wed, 21 May 2025 17:53:27 -0400 Subject: [PATCH 1/9] Improve database management documentation --- .../database-migrations/_category_.yml | 2 + docs/contributing/database-migrations/edd.mdx | 9 +- docs/contributing/database-migrations/ef.md | 30 +++++ .../contributing/database-migrations/index.md | 104 ------------------ .../contributing/database-migrations/mssql.md | 82 ++++++++++++++ .../server/database/ef/index.mdx | 15 ++- .../server/database/mssql/index.md | 2 +- 7 files changed, 131 insertions(+), 113 deletions(-) create mode 100644 docs/contributing/database-migrations/_category_.yml create mode 100644 docs/contributing/database-migrations/ef.md delete mode 100644 docs/contributing/database-migrations/index.md create mode 100644 docs/contributing/database-migrations/mssql.md diff --git a/docs/contributing/database-migrations/_category_.yml b/docs/contributing/database-migrations/_category_.yml new file mode 100644 index 000000000..133601835 --- /dev/null +++ b/docs/contributing/database-migrations/_category_.yml @@ -0,0 +1,2 @@ +label: "Database Migrations" +position: 2 diff --git a/docs/contributing/database-migrations/edd.mdx b/docs/contributing/database-migrations/edd.mdx index 9f24b4d79..634394b41 100644 --- a/docs/contributing/database-migrations/edd.mdx +++ b/docs/contributing/database-migrations/edd.mdx @@ -1,3 +1,7 @@ +--- +sidebar_position: 3 +--- + import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; @@ -239,7 +243,7 @@ There are some important constraints to the implementation of the process: The process to support all of these constraints is a complex one. Below is an image of a state machine that will hopefully help visualize the process and what it supports. It assumes that all -database changes follow the standards that are laid out in [Migrations](./). +database changes follow the standards that are laid out in [Migrations](./mssql.md). --- @@ -265,8 +269,7 @@ will not be run until the start of the next deploy when they are moved into `DbS After this deploy, to prep for the next release, all migrations in `DbScripts_transition` are moved to `DbScripts` and then all migrations in `DbScripts_finalization` are moved to `DbScripts`, conserving their execution order for a clean install. For the current branching strategy, PRs will -be open against `main` when `rc` is cut to prep for this release. This PR automation will also -handle renaming the migration file and updating any reference of `[dbo_finalization]` to `[dbo]`. +be open against `main` when `rc` is cut to prep for this release. The next deploy will pick up the newly added migrations in `DbScripts` and set the previously repeatable _Transition_ migrations to no longer be repeatable, execute the _Finalization_ diff --git a/docs/contributing/database-migrations/ef.md b/docs/contributing/database-migrations/ef.md new file mode 100644 index 000000000..6ade94567 --- /dev/null +++ b/docs/contributing/database-migrations/ef.md @@ -0,0 +1,30 @@ +--- +sidebar_position: 2 +--- + +# Entity Framework + +:::info + +For instructions on how to apply database migrations, please refer to the +[Getting Started](../../getting-started/server/database/ef/index.mdx) documentation. + +::: + +If you alter the database schema, you must create an EF migration script to ensure that EF databases +keep pace with these changes. Developers must do this and include the migrations with their PR. + +To create these scripts, you must first update your data model in `Core/Entities` as desired. This +will be used to generate the migrations for each of our EF targets. Additionally, for table changes +it is strongly recommended to define or update an `IEntityTypeConfiguration` to accurately +represent any constraints needed on the data model. + +Once the model is updated, navigate to the `dev` directory in the `server` repo and execute the +`ef_migrate.ps1` PowerShell command. You should provide a name for the migration as the only +parameter: + +```bash +pwsh ef_migrate.ps1 [NAME_OF_MIGRATION] +``` + +This will generate the migrations, which should then be included in your PR. diff --git a/docs/contributing/database-migrations/index.md b/docs/contributing/database-migrations/index.md deleted file mode 100644 index 33ed117fb..000000000 --- a/docs/contributing/database-migrations/index.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -sidebar_position: 2 ---- - -# Database migrations - -## Applying migrations - -We use a `migrate.ps1` PowerShell script to apply migrations to the local development database. This -script handles the different database providers that we support. - -For instructions on how to use `migrate.ps1`, see the Getting Started section for -[MSSQL](../../getting-started/server/database/mssql/index.md#updating-the-database) and -[Entity Framework](../../getting-started/server/database/ef/index.mdx#migrations) - -## Creating migrations for new changes - -Any database change must be scripted as a migration for both our primary DBMS - MSSQL - as well as -for Entity Framework. Follow the instructions below for each provider. - -### MSSQL migrations - -:::tip - -We recommend reading [Evolutionary Database Design](./edd.mdx) and [T-SQL Code -Style][code-style-sql] first, since they have a major impact in how we write migrations. - -::: - -:::info - -The separate database definitions in `src/Sql/.../dbo` serve as a "master" reference for the -intended and final state of the database at that time. This is crucial because the state of database -definitions at the current moment may differ from when a migration was added in the past. These -definitions act as a lint and validation step to ensure that migrations work as expected, and the -separation helps maintain clarity and accuracy in database schema management and synchronization -processes. - -Additionally, a -[SQL database project](https://learn.microsoft.com/en-us/azure-data-studio/extensions/sql-database-project-extension-sdk-style-projects) -is in place; however, instead of using the auto-generated migrations from -[DAC](https://learn.microsoft.com/en-us/sql/relational-databases/data-tier-applications/data-tier-applications?view=sql-server-ver16), -we manually write migrations. This approach is chosen to enhance performance and prevent accidental -data loss, which is why we have both a `sqlproj` and standalone migrations. - -::: - -In accordance with the tenets of [Evolutionary Database Design](./edd.mdx), each change needs to be -considered to be split into two parts: - -1. A backwards-compatible transition migration -2. A non-backwards-compatible final migration - -It is possible that a change may not require a non-backwards-compatible end phase (i.e. all changes -may be backwards-compatible in their final form). In that case, only one phase of changes is -required. - -#### Backwards compatible migration - -1. Modify the source `.sql` files in `src/Sql/dbo`. -2. Write a migration script, and place it in `util/Migrator/DbScripts`. Each script must be prefixed - with the current date. - -#### Non-backwards compatible migration - -1. Copy the relevant `.sql` files from `src/Sql/dbo` to `src/Sql/dbo_finalization`. -2. Remove the backwards compatibility that is no longer needed. -3. Write a new Migration and place it in `src/Migrator/DbScripts_finalization`. Name it - `YYYY-0M-FinalizationMigration.sql`. - - Typically migrations are designed to be run in sequence. However since the migrations in - DbScripts_finalization can be run out of order, care must be taken to ensure they remain - compatible with the changes to DbScripts. In order to achieve this we only keep a single - migration, which executes all backwards incompatible schema changes. - -### EF migrations - -If you alter the database schema, you must create an EF migration script to ensure that EF databases -keep pace with these changes. Developers must do this and include the migrations with their PR. - -To create these scripts, you must first update your data model in `Core/Entities` as desired. This -will be used to generate the migrations for each of our EF targets. - -Once the model is updated, navigate to the `dev` directory in the `server` repo and execute the -`ef_migrate.ps1` PowerShell command. You should provide a name for the migration as the only -parameter: - -```bash -pwsh ef_migrate.ps1 [NAME_OF_MIGRATION] -``` - -This will generate the migrations, which should then be included in your PR. - -### [Not Yet Implemented] Manual MSSQL migrations - -There may be a need for a migration to be run outside of our normal update process. These types of -migrations should be saved for very exceptional purposes. One such reason could be an Index rebuild. - -1. Write a new Migration with a prefixed current date and place it in - `src/Migrator/DbScripts_manual` -2. After it has been run against our Cloud environments and we are satisfied with the outcome, - create a PR to move it to `DbScripts`. This will enable it to be run by our Migrator processes in - self-host and clean installs of both cloud and self-host environments - -[code-style-sql]: ../code-style/sql.md diff --git a/docs/contributing/database-migrations/mssql.md b/docs/contributing/database-migrations/mssql.md new file mode 100644 index 000000000..bc06e91a5 --- /dev/null +++ b/docs/contributing/database-migrations/mssql.md @@ -0,0 +1,82 @@ +--- +sidebar_position: 1 +--- + +# MSSQL + +:::info + +For instructions on how to apply database migrations, please refer to the +[Getting Started](../../getting-started/server/database/mssql/index.md) documentation. + +::: + +:::tip + +We recommend reading [T-SQL Code Style](../code-style/sql.md) since it has a major impact in how we +write migrations. + +::: + +## SQL database project + +The separate database definitions in `src/Sql/.../dbo` serve as a "master" reference for the +intended and final state of the database at that time. This is crucial because the state of database +definitions at the current moment may differ from when a migration was added in the past. These +definitions act as a lint and validation step to ensure that migrations work as expected, and the +separation helps maintain clarity and accuracy in database schema management and synchronization +processes. + +Additionally, a +[SQL database project](https://learn.microsoft.com/en-us/azure-data-studio/extensions/sql-database-project-extension-sdk-style-projects) +is in place; however, instead of using the auto-generated migrations from +[DAC](https://learn.microsoft.com/en-us/sql/relational-databases/data-tier-applications/data-tier-applications?view=sql-server-ver16), +we manually write migrations. This approach is chosen to enhance performance and prevent accidental +data loss, which is why we have both a `sqlproj` and standalone migrations. + +## Modifying the database + +In accordance with the tenets of [Evolutionary Database Design](./edd.mdx) every change must be +considered as split into two parts: + +1. A backwards-compatible transition migration +2. A non-backwards-compatible final migration + +It is likely that a change does not require a non-backwards-compatible end phase e.g. all changes +may be backwards-compatible in their final form; in that case, only one phase of changes is +required. With the use of beta testing, partial roll-outs, [feature flags](../feature-flags.md), +etc. the often-chosen path is to spread a change across several major releases with a calculated +future state that can perform a "cleanup" migration that is backwards-compatible but still +represents an overall-_incompatible_ change beyond the boundaries of what we need for individual +release safety. + +### Backwards compatible migration + +1. Modify the source `.sql` files in `src/Sql/dbo`. +2. Write a migration script, and place it in `util/Migrator/DbScripts`. Each script must be prefixed + with the current date. + +Please take care to ensure: + +- any existing stored procedure accepts the same input parameters and that new parameters have + nullable defaults +- when a column is renamed the existing stored procedures first check (coalesce) the new location + before falling back to the old location +- continued updating of the old data columns since in case of a rollback no data should be lost + +### Non-backwards compatible migration + +These changes should be written from the perspective of "all data has been migrated" and any old +stored procedures that were kept around for backwards compatibility should be removed. Any logic for +syncing old and new data should also be removed in this step. + +1. Remove the backwards compatibility that is no longer needed. +2. Write a new Migration and place it in `src/Migrator/DbScripts_finalization`. Name it + `YYYY-0M-FinalizationMigration.sql`. + - Typically migrations are designed to be run in sequence. However since the migrations in + `DbScripts_finalization` can be run out of order, care must be taken to ensure they remain + compatible with the changes to `DbScripts`. In order to achieve this we only keep a single + migration, which executes all backwards incompatible schema changes. + +Upon execution any finalization scripts will be [automatically moved](./edd.mdx#online-environments) +for proper history. diff --git a/docs/getting-started/server/database/ef/index.mdx b/docs/getting-started/server/database/ef/index.mdx index c4ebd649b..c5b3cf94b 100644 --- a/docs/getting-started/server/database/ef/index.mdx +++ b/docs/getting-started/server/database/ef/index.mdx @@ -40,9 +40,9 @@ each. Our EF implementations currently support Postgres, MySQL, and SQLite3. -## Setting Up EF Databases +## Creating the database -The workflow here is broadly the same as with the normal MSSQL implementation: set up the docker +The workflow here is broadly the same as with the normal MSSQL implementation: set up the Docker container, configure user secrets, and run migrations against their relating databases in chronological order. @@ -139,7 +139,7 @@ that the changes take effect. ::: -### Migrations +## Updating the database Date: Thu, 22 May 2025 08:40:59 -0400 Subject: [PATCH 2/9] Link to Getting Started subsection --- docs/contributing/database-migrations/ef.md | 3 ++- docs/contributing/database-migrations/mssql.md | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/contributing/database-migrations/ef.md b/docs/contributing/database-migrations/ef.md index 6ade94567..c82118391 100644 --- a/docs/contributing/database-migrations/ef.md +++ b/docs/contributing/database-migrations/ef.md @@ -7,7 +7,8 @@ sidebar_position: 2 :::info For instructions on how to apply database migrations, please refer to the -[Getting Started](../../getting-started/server/database/ef/index.mdx) documentation. +[Getting Started](../../getting-started/server/database/ef/index.mdx#updating-the-database) +documentation. ::: diff --git a/docs/contributing/database-migrations/mssql.md b/docs/contributing/database-migrations/mssql.md index bc06e91a5..f62c8afda 100644 --- a/docs/contributing/database-migrations/mssql.md +++ b/docs/contributing/database-migrations/mssql.md @@ -7,7 +7,8 @@ sidebar_position: 1 :::info For instructions on how to apply database migrations, please refer to the -[Getting Started](../../getting-started/server/database/mssql/index.md) documentation. +[Getting Started](../../getting-started/server/database/mssql/index.md#updating-the-database) +documentation. ::: From 8dc9c0b383fc02ef802f962d7533b6e83380d178 Mon Sep 17 00:00:00 2001 From: Matt Bishop Date: Thu, 22 May 2025 08:42:12 -0400 Subject: [PATCH 3/9] Untwist description of change need --- docs/contributing/database-migrations/mssql.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/docs/contributing/database-migrations/mssql.md b/docs/contributing/database-migrations/mssql.md index f62c8afda..e4aa87fc1 100644 --- a/docs/contributing/database-migrations/mssql.md +++ b/docs/contributing/database-migrations/mssql.md @@ -43,13 +43,12 @@ considered as split into two parts: 1. A backwards-compatible transition migration 2. A non-backwards-compatible final migration -It is likely that a change does not require a non-backwards-compatible end phase e.g. all changes -may be backwards-compatible in their final form; in that case, only one phase of changes is -required. With the use of beta testing, partial roll-outs, [feature flags](../feature-flags.md), -etc. the often-chosen path is to spread a change across several major releases with a calculated -future state that can perform a "cleanup" migration that is backwards-compatible but still -represents an overall-_incompatible_ change beyond the boundaries of what we need for individual -release safety. +Most changes are entirely backwards-compatible in their final form. If this is the case, only one +phase of changes is required. With the use of beta testing, partial roll-outs, +[feature flags](../feature-flags.md), etc. the often-chosen path is to spread a change across +several major releases with a calculated future state that can perform a "cleanup" migration that is +backwards-compatible but still represents an overall-_incompatible_ change beyond the boundaries of +what we need for individual release safety. ### Backwards compatible migration From aa93e62493ce56fdca43043b31e11aded4671679 Mon Sep 17 00:00:00 2001 From: Matt Bishop Date: Thu, 22 May 2025 08:44:35 -0400 Subject: [PATCH 4/9] Don't say please --- docs/contributing/database-migrations/mssql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/contributing/database-migrations/mssql.md b/docs/contributing/database-migrations/mssql.md index e4aa87fc1..7d63ba54a 100644 --- a/docs/contributing/database-migrations/mssql.md +++ b/docs/contributing/database-migrations/mssql.md @@ -56,7 +56,7 @@ what we need for individual release safety. 2. Write a migration script, and place it in `util/Migrator/DbScripts`. Each script must be prefixed with the current date. -Please take care to ensure: +Tips to ensure backwards compatibility: - any existing stored procedure accepts the same input parameters and that new parameters have nullable defaults From 03bd4c6106cd13c0720f13d884cd8209a7eee84f Mon Sep 17 00:00:00 2001 From: Matt Bishop Date: Thu, 22 May 2025 15:47:22 -0400 Subject: [PATCH 5/9] Add folder names to EDD steps --- docs/contributing/database-migrations/edd.mdx | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/contributing/database-migrations/edd.mdx b/docs/contributing/database-migrations/edd.mdx index 634394b41..affa1358f 100644 --- a/docs/contributing/database-migrations/edd.mdx +++ b/docs/contributing/database-migrations/edd.mdx @@ -134,6 +134,8 @@ actions. +Located in `util/Migrator/DbScripts`. + ```sql -- Add Column IF COL_LENGTH('[dbo].[Customer]', 'FirstName') IS NULL @@ -181,6 +183,8 @@ END +Located in `util/Migrator/DbScripts_transition`. + ```sql UPDATE [dbo].Customer SET FirstName=FName @@ -190,6 +194,8 @@ WHERE FirstName IS NULL +Located in `util/Migrator/DbScripts_finalization`. + ```sql -- Remove Column IF COL_LENGTH('[dbo].[Customer]', 'FName') IS NOT NULL From 5955f6e7376d0a416f5017a3cd4786c1654a461f Mon Sep 17 00:00:00 2001 From: Matt Bishop Date: Thu, 22 May 2025 15:51:19 -0400 Subject: [PATCH 6/9] Restore reference to finalization folders since that's been completed --- docs/contributing/database-migrations/edd.mdx | 3 ++- docs/contributing/database-migrations/mssql.md | 9 +++++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/docs/contributing/database-migrations/edd.mdx b/docs/contributing/database-migrations/edd.mdx index affa1358f..be590c612 100644 --- a/docs/contributing/database-migrations/edd.mdx +++ b/docs/contributing/database-migrations/edd.mdx @@ -275,7 +275,8 @@ will not be run until the start of the next deploy when they are moved into `DbS After this deploy, to prep for the next release, all migrations in `DbScripts_transition` are moved to `DbScripts` and then all migrations in `DbScripts_finalization` are moved to `DbScripts`, conserving their execution order for a clean install. For the current branching strategy, PRs will -be open against `main` when `rc` is cut to prep for this release. +be open against `main` when `rc` is cut to prep for this release. This PR automation will also +handle renaming the migration file and updating any reference of `[dbo_finalization]` to `[dbo]`. The next deploy will pick up the newly added migrations in `DbScripts` and set the previously repeatable _Transition_ migrations to no longer be repeatable, execute the _Finalization_ diff --git a/docs/contributing/database-migrations/mssql.md b/docs/contributing/database-migrations/mssql.md index 7d63ba54a..c4ce79ae9 100644 --- a/docs/contributing/database-migrations/mssql.md +++ b/docs/contributing/database-migrations/mssql.md @@ -70,8 +70,13 @@ These changes should be written from the perspective of "all data has been migra stored procedures that were kept around for backwards compatibility should be removed. Any logic for syncing old and new data should also be removed in this step. -1. Remove the backwards compatibility that is no longer needed. -2. Write a new Migration and place it in `src/Migrator/DbScripts_finalization`. Name it +Since the `dbo` schema represents the current state we need to introduce a "future" state that we +will call `dbo_finalization`. + +1. Copy the relevant `.sql` files from `src/Sql/dbo` to `src/Sql/dbo_finalization`. +2. Remove the backwards compatibility logic that is no longer needed e.g. dual reads and writes to + columns. +3. Write a new Migration and place it in `src/Migrator/DbScripts_finalization`. Name it `YYYY-0M-FinalizationMigration.sql`. - Typically migrations are designed to be run in sequence. However since the migrations in `DbScripts_finalization` can be run out of order, care must be taken to ensure they remain From e8fb3e4af3154c51f0b3845a8232db43f7c71e88 Mon Sep 17 00:00:00 2001 From: Matt Bishop Date: Thu, 22 May 2025 16:23:02 -0400 Subject: [PATCH 7/9] Additional content changes suggested from a prior PR --- .../contributing/database-migrations/mssql.md | 25 +++++++++++++------ 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/docs/contributing/database-migrations/mssql.md b/docs/contributing/database-migrations/mssql.md index c4ce79ae9..623408ecc 100644 --- a/docs/contributing/database-migrations/mssql.md +++ b/docs/contributing/database-migrations/mssql.md @@ -21,6 +21,24 @@ write migrations. ## SQL database project +We use a +[SDK-style SQL project](https://learn.microsoft.com/en-us/sql/azure-data-studio/extensions/sql-database-project-extension-sdk-style-projects?view=sql-server-ver16) +(`.sqlproj`) to develop the database locally -- this means we have an up-to-date representation of +the database in `src/Sql`, and any modifications need to be represented there as well. SDK-style SQL +projects are available in Visual Studio Code that provides schema comparison and more. You may also +modify the `.sql` files directly with any text editor. + +To make a database change, start by modifying the `.sql` files in `src/Sql/dbo`. These changes will +also need to be applied in a migration script. Migration scripts are located in +`util/Migrator/DbScripts`. + +You can either generate the migration scripts automatically using the _Schema Comparison_ +functionality or by manually writing them. Do note that the automatic method will only take you so +far and it will need to be manually edited to adhere to the code styles. + +For added safeguards we have automated linting and validation to ensure the SQL project is always up +to date with the migrations. + The separate database definitions in `src/Sql/.../dbo` serve as a "master" reference for the intended and final state of the database at that time. This is crucial because the state of database definitions at the current moment may differ from when a migration was added in the past. These @@ -28,13 +46,6 @@ definitions act as a lint and validation step to ensure that migrations work as separation helps maintain clarity and accuracy in database schema management and synchronization processes. -Additionally, a -[SQL database project](https://learn.microsoft.com/en-us/azure-data-studio/extensions/sql-database-project-extension-sdk-style-projects) -is in place; however, instead of using the auto-generated migrations from -[DAC](https://learn.microsoft.com/en-us/sql/relational-databases/data-tier-applications/data-tier-applications?view=sql-server-ver16), -we manually write migrations. This approach is chosen to enhance performance and prevent accidental -data loss, which is why we have both a `sqlproj` and standalone migrations. - ## Modifying the database In accordance with the tenets of [Evolutionary Database Design](./edd.mdx) every change must be From 654bbe211b5962514b30ae1eeef74492c6d767e0 Mon Sep 17 00:00:00 2001 From: Matt Bishop Date: Fri, 23 May 2025 09:42:40 -0400 Subject: [PATCH 8/9] Fix an oversight from #596 --- docs/getting-started/server/database/ef/index.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started/server/database/ef/index.mdx b/docs/getting-started/server/database/ef/index.mdx index 0dabd9a4e..d7974a41c 100644 --- a/docs/getting-started/server/database/ef/index.mdx +++ b/docs/getting-started/server/database/ef/index.mdx @@ -76,7 +76,7 @@ docker compose --profile postgres up :::note `POSTGRES_PASSWORD` only needs to be defined for database setup. It does not need to be defined to -run the mysql profile. +run the PostgreSQL profile. ::: @@ -93,7 +93,7 @@ docker compose --profile mysql up :::note `MYSQL_ROOT_PASSWORD` only needs to be defined for database setup. It does not need to be defined to -run the mysql profile. +run the MySQL profile. ::: From 77ea379c0920068fe15b5477b8852b74ee92aa66 Mon Sep 17 00:00:00 2001 From: Matt Bishop Date: Fri, 23 May 2025 11:13:52 -0400 Subject: [PATCH 9/9] Fix bungled merge from the wrong ref --- .../server/database/ef/index.mdx | 116 ++++++++++-------- 1 file changed, 63 insertions(+), 53 deletions(-) diff --git a/docs/getting-started/server/database/ef/index.mdx b/docs/getting-started/server/database/ef/index.mdx index d7974a41c..1267411e2 100644 --- a/docs/getting-started/server/database/ef/index.mdx +++ b/docs/getting-started/server/database/ef/index.mdx @@ -59,53 +59,6 @@ chronological order. You can have multiple databases configured and switch between them by changing the value of the `globalSettings:databaseProvider` user secret. You don’t have to delete your connection strings. -### Database Setup - - - - -In the `dev` folder of your server repository, run - -```bash -export POSTGRES_PASSWORD='example' -docker compose --profile postgres up -``` - -:::note - -`POSTGRES_PASSWORD` only needs to be defined for database setup. It does not need to be defined to -run the PostgreSQL profile. - -::: - - - - -In the `dev` folder of your server repository, run - -```bash -export MYSQL_ROOT_PASSWORD='example' -docker compose --profile mysql up -``` - -:::note - -`MYSQL_ROOT_PASSWORD` only needs to be defined for database setup. It does not need to be defined to -run the MySQL profile. - -::: - - - - -Select a location for your database file. You can use the `dev` folder of your server repository. -Git is configured to ignore `.db` files in this repository for this purpose. - - - - ### User secrets Add the following values to your API, Identity, and Admin user secrets. @@ -138,7 +91,8 @@ sure you update the existing values instead of creating new ones Add the following values to your API, Identity, and Admin user secrets. Note, you must set the Data -Source path. Use the file location selected in [Database Setup](#database-setup) +Source path. Git is configured to ignore `.db` files in the server repository so that the sqlite +database can be stored in `dev`. You can use any path with write permissions. ```json "globalSettings:databaseProvider": "sqlite", @@ -155,7 +109,63 @@ that the changes take effect. ::: -## Updating the database +### Updating the database + + + + +1. Confirm that `POSTGRES_PASSWORD` in `dev/.env` matches the password in `dev/secrets.json`. + +2. In the `dev` folder of your server repository, run + +```bash +docker compose --profile postgres up +``` + +:::tip[Confirm your database connection!] + +If you run into connection errors, double check that your `.env` and `secrets.json` files have +matching passwords. If they do, you may have initialized your database incorrectly. Delete the +Docker storage volume and initialize the database from scratch. + +::: + + + + +1. Confirm that `MYSQL_ROOT_PASSWORD` in `dev/.env` matches the password in `dev/secrets.json`. + +2. In the `dev` folder of your server repository, run + +```bash +docker compose --profile mysql up +``` + +:::tip[Confirm your database connection!] + +If you run into connection errors, double check that your `.env` and `secrets.json` files have +matching passwords. If they do, you may have initialized your database incorrectly. Delete the +Docker storage volume and initialize the database from scratch. + +::: + + + + +:::tip[Confirm your database path!] + +The migrator creates the database file if it doesn't exist, but it does not create folders. If you +get an error that the path doesn't exist, double check that the path exists and that the folder +containing the sqlite database has write and/or create permissions. + +::: + + + + +### Migrations @@ -179,7 +189,7 @@ In the `dev` folder run the following to update the database to the latest migra pwsh migrate.ps1 -mysql ``` -The `-mysql` flag on `migrate.ps1` will run `dotnet ef` commands to perform the migrations. +The `-mysql` flag on `migrate.ps1` runs `dotnet ef` commands to perform the migrations. @@ -190,7 +200,7 @@ In the `dev` folder run the following to update the database to the latest migra pwsh migrate.ps1 -sqlite ``` -The `-sqlite` flag on `migrate.ps1` will run `dotnet ef` commands to perform the migrations. +The `-sqlite` flag on `migrate.ps1` runs `dotnet ef` commands to perform the migrations. :::note @@ -254,7 +264,7 @@ With connection strings applied to your projects: ensure your databases are all `pwsh server/dev/migrate.ps1 --all`. Then you can run EF tests from the `test/Infrastructure.IntegrationTest` folder using `dotnet test`. -## Modifying the database +# Modifying the database The process for modifying the database is described in [Migrations](./../../../../contributing/database-migrations/ef.md).