Skip to content

Commit 18033ce

Browse files
authored
Feat: support for optional sample data and additional data migrations (#827)
2 parents b8d6cbd + 5ec56eb commit 18033ce

File tree

7 files changed

+96
-4
lines changed

7 files changed

+96
-4
lines changed

.gitignore

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
*.db
22
*.env
33
*.mo
4-
fixtures/*.json
5-
!fixtures/??_*.json
4+
benefits/core/migrations/0002_*.py
5+
!benefits/core/migrations/0002_sample_data.py
66
static/
77
!benefits/static
88
benefits/static/sha.txt

benefits/core/migrations/0002_sample_data.py

+10-2
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,16 @@
1+
"""Data migration which loads sample data.
2+
Set environment variable DJANGO_LOAD_SAMPLE_DATA to False to skip loading sample data.
3+
"""
4+
from django.conf import settings
15
from django.db import migrations
26
from django.utils.translation import gettext_lazy as _
37

48

5-
def load_initial_data(app, *args, **kwargs):
9+
def load_sample_data(app, *args, **kwargs):
10+
if not settings.LOAD_SAMPLE_DATA:
11+
print(" LOAD_SAMPLE_DATA is set to False, skipping sample data")
12+
return
13+
614
EligibilityType = app.get_model("core", "EligibilityType")
715

816
type1 = EligibilityType.objects.create(name="type1", label="Eligibility Type 1", group_id="group1")
@@ -220,5 +228,5 @@ class Migration(migrations.Migration):
220228
]
221229

222230
operations = [
223-
migrations.RunPython(load_initial_data),
231+
migrations.RunPython(load_sample_data),
224232
]
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
"""This migration simply ensures that data migrations (assumed to be files that start with 0002) will run sequentially.
2+
It does so by looking for all migration files starting with "0002" and declaring them in its dependency list.
3+
"""
4+
import os
5+
6+
from django.conf import settings
7+
from django.db import migrations
8+
9+
10+
class Migration(migrations.Migration):
11+
12+
migrations_directory = os.path.join(settings.BASE_DIR, "benefits", "core", "migrations")
13+
dependencies = [("core", file.replace(".py", "")) for file in os.listdir(migrations_directory) if file.startswith("0002")]
14+
15+
operations = []

benefits/settings.py

+2
Original file line numberDiff line numberDiff line change
@@ -172,6 +172,8 @@ def _filter_empty(ls):
172172
}
173173
}
174174

175+
LOAD_SAMPLE_DATA = os.environ.get("DJANGO_LOAD_SAMPLE_DATA", "true").lower() != "false"
176+
175177
# Password validation
176178

177179
AUTH_PASSWORD_VALIDATORS = []

bin/init.sh

+9
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,15 @@ rm -f django.db
88

99
# run database migrations
1010

11+
if [[ ${DJANGO_LOAD_SAMPLE_DATA:-true} = false ]]; then
12+
if [[ -d ${DJANGO_MIGRATIONS_DIR:-false} ]]; then
13+
echo "Copying migrations from ${DJANGO_MIGRATIONS_DIR}"
14+
cp ${DJANGO_MIGRATIONS_DIR}/0002_*.py ./benefits/core/migrations/
15+
else
16+
echo "DJANGO_MIGRATIONS_DIR is either unset or not a directory"
17+
fi
18+
fi
19+
1120
python manage.py migrate
1221

1322
# create a superuser account for backend admin access

docs/configuration/data.md

+36
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,12 @@
44

55
[`benefits/core/migrations/0002_sample_data.py`][data-sample]
66

7+
!!! example "Helper migration file to define data migration order"
8+
9+
[`benefits/core/migrations/0003_data_migration_order.py`][helper-migration]
10+
11+
Since our initial data migration files are assumed to start with `0002` and depend only on `0001_initial`, this migration file ensures the migration graph has a defined order (which is required for the migration to run). It looks in the `benefits/core/migrations` directory for migration files starting with `0002` and declares them in its dependencies list.
12+
713
!!! tldr "Django docs"
814

915
[How to provide initial data for models][django-load-initial-data]
@@ -62,8 +68,38 @@ Run these commands from within the repository root, inside the devcontainer:
6268
bin/init.sh
6369
```
6470

71+
## Loading new data for different environment
72+
73+
Django will run all migration files found in an app's `migrations` module.
74+
75+
To load new data for a different environment:
76+
77+
1. (Optional) Set an environment variable `DJANGO_LOAD_SAMPLE_DATA` to `false` if you don't want the `core` app's sample data to be loaded.
78+
1. Create a data migration file, and make sure the name is prefixed with `0002`. (The migration process for Benefits expects data migration files to named as such). The basic structure for the contents of this file is:
79+
```python
80+
from django.db import migrations
81+
82+
83+
def load_data(app, *args, **kwargs):
84+
pass
85+
86+
87+
class Migration(migrations.Migration):
88+
dependencies = [
89+
("core", "0001_initial"),
90+
]
91+
92+
operations = [
93+
migrations.RunPython(load_data),
94+
]
95+
```
96+
1. Put this file under `benefits/core/migrations`
97+
1. If `DJANGO_LOAD_SAMPLE_DATA` is `false`, you can also set [`DJANGO_MIGRATIONS_DIR`](../environment-variables/#django_migrations_dir) to a directory path, and put your data migration there.
98+
99+
65100
[core-models]: https://github.com/cal-itp/benefits/blob/dev/benefits/core/models.py
66101
[django-load-initial-data]: https://docs.djangoproject.com/en/4.0/howto/initial-data/
67102
[eligibility-server]: https://docs.calitp.org/eligibility-server
68103
[data-sample]: https://github.com/cal-itp/benefits/tree/dev/benefits/core/migrations/0002_sample_data.py
104+
[helper-migration]: https://github.com/cal-itp/benefits/tree/dev/benefits/core/migrations/0003_data_migration_order.py
69105
[init]: https://github.com/cal-itp/benefits/blob/dev/bin/init.sh

docs/configuration/environment-variables.md

+22
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,19 @@ Boolean:
7373
files are served
7474
- `False` (default): the application is launched with debug mode turned off, similar to how it runs in production
7575

76+
### `DJANGO_LOAD_SAMPLE_DATA`
77+
78+
!!! warning "Deployment configuration"
79+
80+
You may change this setting when deploying the app to a non-localhost domain
81+
82+
Boolean:
83+
84+
- `True` (default): The sample data in `benefits/core/migrations/0002_sample_data.py` is used to initialize the Django configuration database.
85+
- `False`: The sample data from `benefits/core/migrations/0002_sample_data.py` will not be loaded.
86+
87+
See [Configuration data](data.md) for more.
88+
7689
### `DJANGO_LOCAL_PORT`
7790

7891
!!! info "Local configuration"
@@ -100,6 +113,15 @@ The log level used in the application's logging configuration.
100113

101114
By default the application sends logs to `stdout`.
102115

116+
117+
### `DJANGO_MIGRATIONS_DIR`
118+
119+
!!! warning "Deployment configuration"
120+
121+
You may change this setting when deploying the app to a non-localhost domain
122+
123+
If [`DJANGO_LOAD_SAMPLE_DATA`](#django_load_sample_data) is `false`, then you can set `DJANGO_MIGRATIONS_DIR` to the path of a directory containing data migrations that you want to be run. Those data migration files need to be prefixed with `0002` so that the [helper migration file](data.md)) can find it. See [Configuration data](./data.md) for more on loading data for different environments.
124+
103125
### `DJANGO_SECRET_KEY`
104126

105127
!!! warning "Deployment configuration"

0 commit comments

Comments
 (0)