Skip to content

Remove the old migration tool and document the new one #4447

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ crates/*/target
crates/*/node_modules
frontend/node_modules
frontend/dist
tools/syn2mas/**
docs/
.devcontainer/
.github/
Expand Down
17 changes: 0 additions & 17 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -106,20 +106,3 @@ updates:
browser-logos:
patterns:
- "@browser-logos/*"

- package-ecosystem: "npm"
directory: "/tools/syn2mas/"
labels:
- "A-Dependencies"
- "Z-Deps-Syn2Mas"
schedule:
interval: "weekly"
ignore:
# Ignore @types/node until we can upgrade to Node 20
- dependency-name: "@types/node"
update-types: ["version-update:semver-major"]
groups:
production:
dependency-type: "production"
development:
dependency-type: "development"
7 changes: 1 addition & 6 deletions .github/scripts/commit-and-tag.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,7 @@ module.exports = async ({ github, context }) => {
const parent = context.sha;
if (!version) throw new Error("VERSION is not defined");

const files = [
"Cargo.toml",
"Cargo.lock",
"tools/syn2mas/package.json",
"tools/syn2mas/package-lock.json",
];
const files = ["Cargo.toml", "Cargo.lock"];

/** @type {{path: string, mode: "100644", type: "blob", sha: string}[]} */
const tree = [];
Expand Down
63 changes: 0 additions & 63 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ env:
SCCACHE_GHA_ENABLED: "true"
RUSTC_WRAPPER: "sccache"
IMAGE: ghcr.io/element-hq/matrix-authentication-service
IMAGE_SYN2MAS: ghcr.io/element-hq/matrix-authentication-service/syn2mas
BUILDCACHE: ghcr.io/element-hq/matrix-authentication-service/buildcache
DOCKER_METADATA_ANNOTATIONS_LEVELS: manifest,index

Expand Down Expand Up @@ -253,22 +252,6 @@ jobs:
type=semver,pattern={{major}}
type=sha

- name: Docker meta (syn2mas)
id: meta-syn2mas
uses: docker/[email protected]
with:
images: "${{ env.IMAGE_SYN2MAS }}"
bake-target: docker-metadata-action-syn2mas
flavor: |
latest=auto
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
type=sha

- name: Setup Cosign
uses: sigstore/[email protected]

Expand All @@ -294,7 +277,6 @@ jobs:
./docker-bake.hcl
cwd://${{ steps.meta.outputs.bake-file }}
cwd://${{ steps.meta-debug.outputs.bake-file }}
cwd://${{ steps.meta-syn2mas.outputs.bake-file }}
set: |
base.output=type=image,push=true
base.cache-from=type=registry,ref=${{ env.BUILDCACHE }}:buildcache
Expand All @@ -318,43 +300,11 @@ jobs:
env:
REGULAR_DIGEST: ${{ steps.output.outputs.metadata && fromJSON(steps.output.outputs.metadata).regular.digest }}
DEBUG_DIGEST: ${{ steps.output.outputs.metadata && fromJSON(steps.output.outputs.metadata).debug.digest }}
SYN2MAS_DIGEST: ${{ steps.output.outputs.metadata && fromJSON(steps.output.outputs.metadata).syn2mas.digest }}

run: |-
cosign sign --yes \
"$IMAGE@$REGULAR_DIGEST" \
"$IMAGE@$DEBUG_DIGEST" \
"$IMAGE_SYN2MAS@$SYN2MAS_DIGEST"

syn2mas:
name: Release syn2mas on NPM
runs-on: ubuntu-24.04
if: github.event_name != 'pull_request'

permissions:
contents: read
id-token: write

steps:
- name: Checkout the code
uses: actions/[email protected]

- name: Install Node
uses: actions/[email protected]
with:
node-version-file: ./tools/syn2mas/.nvmrc

- name: Install Node dependencies
working-directory: ./tools/syn2mas
run: npm ci

- name: Publish
uses: JS-DevTools/npm-publish@v3
with:
package: ./tools/syn2mas
token: ${{ secrets.NPM_TOKEN }}
provenance: true
dry-run: ${{ !startsWith(github.ref, 'refs/tags/') }}

release:
name: Release
Expand All @@ -363,7 +313,6 @@ jobs:
needs:
- assemble-archives
- build-image
- syn2mas
steps:
- name: Download the artifacts from the previous job
uses: actions/download-artifact@v4
Expand Down Expand Up @@ -403,18 +352,6 @@ jobs:
') }}
```

`syn2mas` migration tool:

- Digest:
```
${{ env.IMAGE_SYN2MAS }}@${{ fromJSON(needs.build-image.outputs.metadata).syn2mas.digest }}
```
- Tags:
```
${{ join(fromJSON(needs.build-image.outputs.metadata).syn2mas.tags, '
') }}
```

files: |
artifacts/mas-cli-aarch64-linux.tar.gz
artifacts/mas-cli-x86_64-linux.tar.gz
Expand Down
29 changes: 0 additions & 29 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -311,34 +311,6 @@ jobs:
--archive-file nextest-archive.tar.zst \
--partition count:${{ matrix.partition }}/3

syn2mas:
name: Check syn2mas
runs-on: ubuntu-24.04

permissions:
contents: read

steps:
- name: Checkout the code
uses: actions/[email protected]

- name: Install Node
uses: actions/[email protected]
with:
node-version-file: ./tools/syn2mas/.nvmrc

- name: Install Node dependencies
working-directory: ./tools/syn2mas
run: npm ci

- name: Lint
working-directory: ./tools/syn2mas
run: npm run lint

- name: Build
working-directory: ./tools/syn2mas
run: npm run build

tests-done:
name: Tests done
if: ${{ always() }}
Expand All @@ -352,7 +324,6 @@ jobs:
- clippy
- check-schema
- test
- syn2mas
runs-on: ubuntu-24.04

steps:
Expand Down
4 changes: 0 additions & 4 deletions .github/workflows/tag.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,6 @@ jobs:
- name: Run `cargo metadata` to make sure the lockfile is up to date
run: cargo metadata --format-version 1

- name: Set the tools/syn2mas version
working-directory: tools/syn2mas
run: npm version "${{ inputs.version }}" --no-git-tag-version

- name: Commit and tag using the GitHub API
uses: actions/[email protected]
id: commit
Expand Down
1 change: 0 additions & 1 deletion biome.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
"frontend/.storybook/locales.ts",
"frontend/.storybook/public/mockServiceWorker.js",
"frontend/locales/*.json",
"tools/syn2mas/package.json",
"**/coverage/**",
"**/dist/**"
]
Expand Down
8 changes: 1 addition & 7 deletions docker-bake.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,11 @@
variable "VERGEN_GIT_DESCRIBE" {}

// This is what is baked by GitHub Actions
group "default" { targets = ["regular", "debug", "syn2mas"] }
group "default" { targets = ["regular", "debug"] }

// Targets filled by GitHub Actions: one for the regular tag, one for the debug tag
target "docker-metadata-action" {}
target "docker-metadata-action-debug" {}
target "docker-metadata-action-syn2mas" {}

// This sets the platforms and is further extended by GitHub Actions to set the
// output and the cache locations
Expand Down Expand Up @@ -37,8 +36,3 @@ target "debug" {
inherits = ["base", "docker-metadata-action-debug"]
target = "debug"
}

target "syn2mas" {
inherits = ["base", "docker-metadata-action-syn2mas"]
context = "./tools/syn2mas"
}
2 changes: 1 addition & 1 deletion docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@
- [Database setup](./setup/database.md)
- [Homeserver configuration](./setup/homeserver.md)
- [Configuring a reverse proxy](./setup/reverse-proxy.md)
- [Configuring .well-known](./setup/well-known.md)
- [Configure an upstream SSO provider](./setup/sso.md)
- [Running the service](./setup/running.md)
- [Migrating an existing homeserver](./setup/migration.md)
Expand All @@ -33,6 +32,7 @@
- [`database`](./reference/cli/database.md)
- [`manage`](./reference/cli/manage.md)
- [`server`](./reference/cli/server.md)
- [`syn2mas`](./reference/cli/syn2mas.md)
- [`worker`](./reference/cli/worker.md)
- [`templates`](./reference/cli/templates.md)
- [`doctor`](./reference/cli/doctor.md)
Expand Down
8 changes: 6 additions & 2 deletions docs/reference/cli/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ clients:
# ...
```

## `config generate`
## `config generate [--synapse-config <synapse-config>] [--output <output>]`

Generate a sample configuration file.
It generates random signing keys (`.secrets.keys`) and the cookie encryption secret (`.secrets.encryption`).
Expand All @@ -38,6 +38,10 @@ INFO generate:rsa: mas_config::oauth2: Done generating RSA key
INFO generate:ecdsa: mas_config::oauth2: Done generating ECDSA key
```

The `--synapse-config` option can be used to migrate over configuration options from an existing Synapse configuration.

The `--output` option can be used to specify the output file. If not specified, the output will be written to stdout.

## `config sync [--prune] [--dry-run]`

Synchronize the configuration with the database.
Expand All @@ -52,4 +56,4 @@ INFO cli.config.sync: Updating provider provider.id=01H3FDH2XZJS8ADKRGWM84PZTY
INFO cli.config.sync: Adding provider provider.id=01H3FDH2XZJS8ADKRGWM84PZTF
INFO cli.config.sync: Deleting client client.id=01GFWRB9MYE0QYK60NZP2YF905
INFO cli.config.sync: Updating client client.id=01GFWRB9MYE0QYK60NZP2YF904
```
```
29 changes: 29 additions & 0 deletions docs/reference/cli/syn2mas.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# `syn2mas`

Tool to import data from an existing Synapse homeserver into MAS.

Global options:
- `--config <config>`: Path to the MAS configuration file.
- `--help`: Print help.
- `--synapse-config <synapse-config>`: Path to the Synapse configuration file.
- `--synapse-database-uri <synapse-database-uri>`: Override the Synapse database URI.

## `syn2mas check`

Check the setup for potential problems before running a migration

```console
$ mas-cli syn2mas check --config mas_config.yaml --synapse-config homeserver.yaml
```

## `syn2mas migrate [--dry-run]`

Migrate data from the homeserver to MAS.

The `--dry-run` option will perform a dry-run of the migration, which is safe to run without stopping Synapse.
It will perform a full data migration, but then empty the MAS database at the end to roll back.


```console
$ mas-cli syn2mas migrate --config mas_config.yaml --synapse-config homeserver.yaml
```
34 changes: 1 addition & 33 deletions docs/setup/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,43 +11,11 @@ The authentication service becomes the source of truth for user accounts and acc
At time of writing, the authentication service is meant to be run on a standalone domain name (e.g. `auth.example.com`), and the homeserver on another (e.g. `matrix.example.com`).
This domain will be user-facing as part of the authentication flow.

When a client initiates an authentication flow, it will discover the authentication service through the deployment `.well-known/matrix/client` endpoint.
This file will refer to an `issuer`, which is the canonical name of the authentication service instance.
Out of that issuer, it will discover the rest of the endpoints by calling the `[issuer]/.well-known/openid-configuration` endpoint.
By default, the `issuer` will match the root domain where the service is deployed (e.g. `https://auth.example.com/`), but it can be configured to be different.

An example setup could look like this:

- The deployment domain is `example.com`, so Matrix IDs look like `@user:example.com`
- The issuer chosen is `https://auth.example.com/`
- The homeserver is deployed on `matrix.example.com`
- The authentication service is deployed on `auth.example.com`
- Calling `https://example.com/.well-known/matrix/client` returns the following JSON:

```json
{
"m.homeserver": {
"base_url": "https://matrix.example.com"
},
"org.matrix.msc2965.authentication": {
"issuer": "https://auth.example.com/",
"account": "https://auth.example.com/account"
}
}
```

- Calling `https://auth.example.com/.well-known/openid-configuration` returns a JSON document similar to the following:

```json
{
"issuer": "https://auth.example.com/",
"authorization_endpoint": "https://auth.example.com/authorize",
"token_endpoint": "https://auth.example.com/oauth2/token",
"jwks_uri": "https://auth.example.com/oauth2/keys.json",
"registration_endpoint": "https://auth.example.com/oauth2/registration",
"//": "..."
}
```
- The homeserver is deployed on `matrix.example.com`

With the installation planned, it is time to go through the installation and configuration process.
The first section focuses on [installing the service](./installation.md).
Loading
Loading