Skip to content

chore: Eliminate devstack integration #104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pwnage101 merged 1 commit into from
Apr 8, 2026
Merged

chore: Eliminate devstack integration #104

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 24 additions & 4 deletions .claude/skills/quality-tests/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
---
name: quality-tests
description: Run code quality checks (linting, style, PII annotations) in the enterprise-access Docker container. Use when the user wants to run quality checks, lint code, or verify code style compliance.
allowed-tools: Bash(docker *), Bash(make *)
allowed-tools: Bash(docker *), Bash(make *), Bash(colima *)
---

## Steps

### 1. Make sure the app container is running

Determine if the enterprise-access.app container is running:
Determine if the app container is running:

```bash
docker ps | grep enterprise-access.app
docker compose ps
```

Start the app container if not running:
Expand All @@ -23,5 +23,25 @@ make dev.up
### 2. Run quality checks

```bash
docker exec enterprise-access.app bash -c "make quality"
docker compose exec app bash -c "make quality"
```

## Troubleshooting

### ModuleNotFoundError

If quality checks fail due to missing imports, first try to install requirements:

```bash
docker compose exec app make ci_requirements
```

This is necessary at least when adding new requirements which have not yet been built into the image.

### Failed to connect to the docker API on MacOS

This likely just means colima needs to be started:

```bash
colima start
```
Comment thread
pwnage101 marked this conversation as resolved.
34 changes: 27 additions & 7 deletions .claude/skills/unit-tests/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
name: unit-tests
description: Run Django unit tests in the enterprise-access Docker container. Use when the user wants to run tests, check if tests pass, or verify test coverage.
argument-hint: "<TEST_FILES>"
allowed-tools: Bash(docker *), Bash(make *)
allowed-tools: Bash(docker *), Bash(make *), Bash(colima *)
---

## Arguments
Expand All @@ -26,10 +26,10 @@ Examples:

### 1. Make sure the app container is running

Determine if the enterprise-access.app container is running:
Determine if the app container is running:

```bash
docker ps | grep enterprise-access.app
docker compose ps
```

Start the app container if not running:
Expand All @@ -43,7 +43,7 @@ make dev.up
If `<TEST_FILES>` specify .py files or functions/classes within, run only those tests using `pytest.local.ini` to disable coverage and warnings:

```bash
docker exec enterprise-access.app bash -c "pytest -c pytest.local.ini <TEST_FILES>"
docker compose exec app bash -c "pytest -c pytest.local.ini <TEST_FILES>"
```

Example `<TEST_FILES>` which match this case:
Expand All @@ -60,10 +60,10 @@ If `<TEST_FILES>` specifies a single directory which represents a domain, such a
Convert the directory path to a Python module path for `--cov`: replace `/` with `.` and strip any trailing slash. Example: `enterprise_access/apps/bffs/` → `enterprise_access.apps.bffs`

```bash
docker exec enterprise-access.app bash -c "pytest -c pytest.local.ini <TEST_DOMAIN> --cov=<MODULE_PATH_OF_TEST_DOMAIN>"
docker compose exec app bash -c "pytest -c pytest.local.ini <TEST_DOMAIN> --cov=<MODULE_PATH_OF_TEST_DOMAIN>"

# Example: <TEST_FILES> is "enterprise_access/apps/bffs"
docker exec enterprise-access.app bash -c "pytest -c pytest.local.ini enterprise_access/apps/bffs --cov=enterprise_access.apps.bffs"
docker compose exec app bash -c "pytest -c pytest.local.ini enterprise_access/apps/bffs --cov=enterprise_access.apps.bffs"
```

Example `<TEST_FILES>` which match this case:
Expand All @@ -80,7 +80,27 @@ Whole-domain coverage will be reported in the console output. Specific line numb
If no arguments are given, assume the user wants to run the full test suite for the entire project:

```bash
docker exec enterprise-access.app bash -c "make test"
docker compose exec app bash -c "make test"
```

Whole-project coverage will be reported in the console output. Specific line numbers with missing coverage will be reported.

## Troubleshooting

### ModuleNotFoundError

If tests fail due to missing imports, first try to install requirements:

```bash
docker compose exec app make ci_requirements
```

This is necessary at least when adding new requirements which have not yet been built into the image.

### Failed to connect to the docker API on MacOS

This likely just means colima needs to be started:

```bash
colima start
```
Comment thread
pwnage101 marked this conversation as resolved.
5 changes: 2 additions & 3 deletions .ralph/PROMPT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,8 @@ Check if a reusable code pattern exists in `.ralph/specs/stdlib` before executin

### Docker Development

- Server runs on `localhost:18270`
- Uses MySQL 8.0, Memcache, Redis, and Celery worker
- Event bus via Kafka (Confluent Control Center at localhost:9021)
- Full devstack (app, worker, DB, Kafka, etc.) is managed in the devstack repository
- The `docker-compose.yml` in this repo provides a lightweight container for running tests and quality checks only

## Testing Notes

Expand Down
5 changes: 2 additions & 3 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,8 @@ directory does not exist).

### Docker Development

- Server runs on `localhost:18270`
- Uses MySQL 8.0, Memcache, Redis, and Celery worker
- Event bus via Kafka (Confluent Control Center at localhost:9021)
- Full devstack (app, worker, DB, Kafka, etc.) is managed in the devstack repository
- The `docker-compose.yml` in this repo provides a lightweight container for running tests and quality checks only

## Testing Notes

Expand Down
127 changes: 38 additions & 89 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,12 @@
test coverage isort_check isort style lint quality pii_check validate \
migrate html_coverage upgrade extract_translation dummy_translations \
compile_translations fake_translations pull_translations \
push_translations start-devstack open-devstack pkg-devstack \
detect_changed_source_translations validate_translations check_keywords

COMMON_CONSTRAINTS_TXT=requirements/common_constraints.txt
.PHONY: $(COMMON_CONSTRAINTS_TXT)
$(COMMON_CONSTRAINTS_TXT):
wget -O "$(@)" https://raw.githubusercontent.com/edx/edx-lint/master/edx_lint/files/common_constraints.txt || touch "$(@)"
push_translations \
detect_changed_source_translations validate_translations check_keywords \
dev.pull dev.up dev.down dev.stop dev.makemigrations dev.shell dev.logs \
dev.restart-container dev.attach \
app-up app-down app-stop app-makemigrations app-shell app-logs \
app-restart-container app-attach

define BROWSER_PYSCRIPT
import os, webbrowser, sys
Expand Down Expand Up @@ -103,9 +102,6 @@ migrate: ## apply database migrations
html_coverage: ## generate and view HTML coverage report
coverage html && open htmlcov/index.html

subsidy_client_local: # re-install edx-enterprise-subsidy-client from local code
pip uninstall -y edx-enterprise-subsidy-client && pip install -e /edx/src/edx-enterprise-subsidy-client/ && pip freeze | grep subsidy-client

COMMON_CONSTRAINTS_TXT=requirements/common_constraints.txt
.PHONY: $(COMMON_CONSTRAINTS_TXT)
$(COMMON_CONSTRAINTS_TXT):
Expand Down Expand Up @@ -152,12 +148,6 @@ pull_translations: ## pull translations from Transifex
push_translations: ## push source translation files (.po) from Transifex
tx push -s

start-devstack: ## run a local development copy of the server
docker-compose --x-networking up

open-devstack: ## open a shell on the server started by start-devstack
docker exec -it enterprise-access /edx/app/enterprise-access/devstack.sh open

detect_changed_source_translations: ## check if translation files are up-to-date
cd enterprise_access && i18n_tool changed

Expand All @@ -176,90 +166,49 @@ travis_docker_push: travis_docker_tag travis_docker_auth ## push to docker hub
docker push 'openedx/enterprise-access:latest-newrelic'
docker push "openedx/enterprise-access:$$TRAVIS_COMMIT-newrelic"

dev.pull:
docker image pull edxops/enterprise-access-dev

dev.provision:
bash ./provision-enterprise-access.sh

# devstack-themed shortcuts
# Starts all containers
dev.up: dev.pull dev.up.redis
docker-compose up -d

dev.up.with-events: dev.up.kafka-control-center dev.up

# Start redis via the devstack docker-compose.yml
dev.up.redis:
docker-compose -f $(DEVSTACK_WORKSPACE)/devstack/docker-compose.yml up -d redis

# Start kafka via the devstack docker-compose.yml
# https://github.com/openedx-unsupported/devstack/blob/323b475b885a2704489566b262e2895a4dca62b6/docker-compose.yml#L140
dev.up.kafka-control-center:
docker-compose -f $(DEVSTACK_WORKSPACE)/devstack/docker-compose.yml up -d kafka-control-center

# Useful for just restarting everything related to the event broker
dev.down.kafka-control-center:
docker-compose -f $(DEVSTACK_WORKSPACE)/devstack/docker-compose.yml down kafka zookeeper schema-registry kafka-control-center

# Kills containers and all of their data that isn't in volumes
dev.down:
docker-compose down

# Stops containers so they can be restarted
dev.stop:
docker-compose stop

dev.backup:
docker-compose stop app worker
docker-compose up -d mysql80
sleep 10 # let mysql process get fully warmed up
docker compose exec mysql80 mysqldump --all-databases > .dev/enterprise_access_all.sql

dev.restore:
docker-compose stop app worker
docker-compose up -d mysql80
sleep 10 # let mysql process get fully warmed up
docker compose exec -T mysql80 mysql < .dev/enterprise_access_all.sql

app-shell: # Run the app shell as root
docker exec -u 0 -it enterprise-access.app bash

db-shell-57: # Run the mysql 5.7 shell as root, enter the app's database
docker exec -u 0 -it enterprise_access.db mysql -u root enterprise_access
####################################################################
# Docker shortcuts for managing a local test/quality container. #
# For full devstack (app, worker, DB, etc.), see the devstack #
# repository which now manages enterprise-access as a first-class #
# service. #
####################################################################

db-shell-8: # Run the mysql 8 shell as root, enter the app's database
docker exec -u 0 -it enterprise_access.mysql80 mysql -u root enterprise_access
dev.pull: ## Pulls the docker image used by the app container
docker compose pull
Comment thread
pwnage101 marked this conversation as resolved.

dev.dbcopy8: ## Copy data from old mysql 5.7 container into a new 8 db
mkdir -p .dev/
docker-compose exec db bash -c "mysqldump --databases enterprise_access" > .dev/enterprise_access.sql
docker-compose exec -T mysql80 bash -c "mysql" < .dev/enterprise_access.sql
rm .dev/enterprise_access.sql
dev.up: dev.pull ## Starts the app container
docker compose up --remove-orphans -d

%-logs: # View the logs of the specified service container
docker-compose logs -f --tail=500 $*
dev.down: ## Kills the app container and all its data that isn't in volumes
docker compose down

%-restart: # Restart the specified service container
docker-compose restart $*
dev.stop: ## Stops the app container so it can be restarted
docker compose stop

app-restart-devserver: # restart just the app Django dev server
docker-compose exec app bash -c 'kill $$(ps aux | egrep "manage.py ?\w* runserver" | egrep -v "while|grep" | awk "{print \$$2}")'
dev.makemigrations: ## Create migrations via the app container
docker compose exec app python manage.py makemigrations
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dev.makemigrations runs manage.py makemigrations inside the docker-compose container, but the compose service sets DJANGO_SETTINGS_MODULE=enterprise_access.settings.test. That test settings module adds the enterprise_access.apps.workflow.tests app (with its own models/migrations), which can cause makemigrations to detect or generate migrations for test-only models. Consider overriding the settings module for this target (e.g., use the local settings module) so migration generation reflects the real app configuration.

Suggested change
docker compose exec app python manage.py makemigrations
docker compose exec -e DJANGO_SETTINGS_MODULE=enterprise_access.settings.local app python manage.py makemigrations

Copilot uses AI. Check for mistakes.

app-attach:
docker attach enterprise-access.app
dev.shell: ## Launch a shell in the app container
docker compose exec app bash

worker-attach:
docker attach enterprise_access.worker
dev.logs: ## View the logs of the app container
docker compose logs -f --tail=500 app

%-shell: # Run a shell, as root, on the specified service container
docker exec -u 0 -it enterprise_access.$* bash
dev.restart-container: ## Restart the app container
docker compose restart
Comment thread
pwnage101 marked this conversation as resolved.

dev.static:
docker-compose exec -u 0 app python3 manage.py collectstatic --noinput
dev.attach: ## Attach to the app container
docker compose attach app

dev.migrate:
docker-compose exec -u 0 app python manage.py migrate
app-up: dev.up
app-down: dev.down
app-stop: dev.stop
app-makemigrations: dev.makemigrations
app-shell: dev.shell
app-logs: dev.logs
app-restart-container: dev.restart-container
app-attach: dev.attach

github_docker_auth:
echo "$$DOCKERHUB_PASSWORD" | docker login -u "$$DOCKERHUB_USERNAME" --password-stdin
Expand Down
69 changes: 12 additions & 57 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,74 +3,29 @@ Service to manage access to content for enterprise users.
Setting up enterprise-access
--------------------------

Prerequisites
^^^^^^^^^^^^^
- Set the ``DEVSTACK_WORKSPACE`` env variable (either locally or in your shell config file: ``.bash_rc``, ``.zshrc``, or equivalent) to the folder which contains this repo and the `devstack` repo.
e.g ``export DEVSTACK_WORKSPACE=/home/<your_user>/edx``
- Set up `devstack <https://github.com/edx/devstack>`_
Full devstack setup
^^^^^^^^^^^^^^^^^^^
For running the full enterprise-access application (app, worker, database, etc.), see the
`devstack <https://github.com/edx/devstack>`_ repository, which manages enterprise-access
as a first-class service.

Quick Setup
^^^^^^^^^^^
Running tests and quality checks locally
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The ``docker-compose.yml`` in this repository provides a lightweight container for running
tests and quality checks without the full devstack infrastructure.

::

$ make docker_build
$ make dev.provision
$ make dev.up
$ make app-shell
# make requirements
# make validate # to run full test suite

The server will run on ``localhost:18270``

Running migrations
^^^^^^^^^^^^^^^^^^

::

$ make app-shell
# python ./manage.py migrate

Setting up openedx-events
Copy link
Copy Markdown
Member Author

@pwnage101 pwnage101 Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This whole section should be safe to remove because last year I spent a bunch of time ensuring edx/devstack sufficiently launches enterprise-specific consumers automatically.

^^^^^^^^^^^^^^^^^^^^^^^^^
Ensure you have installed the ``edx_event_bus_kafka`` and ``openedx_events`` requirements. Entering
a shell with ``make app-shell`` and then running ``make requirements`` should install these for you.

From your host, run ``make dev.up.with-events``, which will start a local kafka container for you.
Visit http://localhost:9021/clusters to access the local "Confluent Control Center".
Confluent is like a cloud wrapper around "vanilla" Kafka.

Your ``devstack.py`` settings should already be configured to point at this event broker,
and to configure enterprise-access as an openedx event consumer and produer.

We have a specific enterprise "ping" event and management command defined to test
that your local event bus is well-configured. Open a shell with ``make app-shell`` and run::

./manage.py consume_enterprise_ping_events

This will consume ping events from the ``dev-enterprise-core`` topic.
You may see a ``Broker: Unknown topic`` error the first time you run it. When you run your
test event production below, that error will resolve (producing the event creates the topic
if it does not exist). **Leave the consumer running.** You should see the ``enterprise-access-service``
as a registered consumer in your local confluent control center.

Now, go over to your **enterprise-subsidy** directory. Make sure requirements are installed,
specifically the ``edx_event_bus_kafka`` and ``openedx_events`` packages. Use ``make app-shell``
in this repo and we'll *produce* a ping event::

./manage.py produce_enterprise_ping_event

If this event was successfully produced, you'll see a log message that says
``Message delivered to Kafka event bus: topic=dev-events-testing``.
You should also now see the ``dev-events-testing`` topic available in your local confluent
control center, and even the test events that are being published to the topic.
$ make dev.shell
# make validate # run the full test and quality suite

A note on creating SubsidyRequestCustomerConfiguration Objects locally
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

*Important note*

In a devstack enviroment, login to the LMS and navigate to any
In a devstack environment, login to the LMS and navigate to any
MFE before creating SubsidyRequestCustomerConfiguration objects in the
enterprise-access Django admin.

Expand Down
Loading
Loading