Test mode (aka "dry run")

[stevezau]

We may want to consider a feature where you can run import but in test mode, so no changes. This would be useful for when deploying to production.

It should be fairly easy to add --test param.

[ma82]

This would be very useful in CI!

With respect to the naming: --test is fine, however I think --dry-run would be more clear.

[gaui]

I can't believe that this is not possible.

[adrian-skybaker]

This is important for safe deployments - we'd like to do a dry-run of a deployment with no side effects, that logged what would be changed, then have a manual step to review before deploying the actual change.

[vallieres]

I also want to chip in as we have multiple environments (different tenants) and it happens often that we have a misconfigured .json but only for staging or production, therefore it fails during our deployment step. With a --dry-run option, we could test all of our config before ever hitting our deployment pipeline.

Please make this happen! 🙏

[TomasMorton]

This would be perfect!

[rkennedy-tpl]

--dry-run would be extremely helpful when moving from "unmanaged/by-hand configuration" to auth0-deploy-cli.

[hayashi-ay]

I can't wait for --dry-run!!!!

[SeanFeldman]

As someone who had to go through the process of trial and error, I'd really appreciate a test mode. 👍

[willvedd]

Little update on the Auth0 side – we've identified this as a high-impact feature and have it prioritized fairly high on the roadmap. Compared to some of the other initiatives we're working on, this skews towards the larger side and it's something we want to get right, so we're taking our time. However, we do see ourselves working this fairly soon!

I'd also like to take a moment to plug Auth0's Terraform Provider which provides this as a first-class feature already. If you already support Terraform workflows, you may find it to be more suitable to your needs. Both tools with exist alongside each other well into the future.

[heidemn]

It's great that Auth0 has a Terraform provider. However I'd rather wait for version 1.0 before starting to use it.
Version 0.29.0 sounds to me like "not ready for production use" or "breaking changes coming soon".

--dry-run would be extremely helpful when moving from "unmanaged/by-hand configuration" to auth0-deploy-cli.

Exactly this! 🚀

[tolgap]

Any update on this? It's been 5 months since it has moved to a "fairly high" priority 😅

[gautampachnandamat]

Any update on the prioritisation of this ticket?

[willvedd]

Despite the seeming lack of progress, this has been getting pushed forward behind the scenes, albeit slowly. It was added to the roadmap not too long ago with work slotted for next quarter. Before then though, I'll draft up a proposal similar to #451 that outlines the suggested direction as well as soliciting general feedback. Until then, if folks have any initial thoughts and suggestions, this thread is still appropriate. Being able to consider this feedback upfront will hasten the process.

Unlike other features, this request has quite a few moving pieces and will fundamentally impact the mode of interaction, so I'd like to get it right. For example, do we integrate into the import function? Do we create a new dry-run command? Both? How do we express an approval flow? Further, I'm trying to be a bit careful to not introduce any breaking changes, but adding a dry-run check before import may be a sensible default. There are several details that need to be ironed out.

In terms of a delivery date, I think late 2022 - early 2023 is a reasonable expectation. I'm hoping to hit the earlier side of this range, but again, I'd like to get this right and make sure that it suits your needs.

[tolgap]

@willvedd any updates on this :) Our Auth0 deployment logic is growing fast and not having "dry-run" functionality to verify our changes during CI is biting us hard.

[ivnnv]

friendly new year's ping on this @willvedd :)

[donratta]

Hey guys just keeping the convo alice here. The dry-run feature would make SO much sense 🙏
Happy new year!

[stabai]

As an additional note I didn't see explicitly mentioned: having a way to use this as a test to ensure the active configuration of my tenant matches the input I provided would be super useful. For example, if the "dry run" produces a diff of what changes would have been made, being able to programmatically test if that diff is empty (or just returning a non-zero exit code if it's not?) would be a useful feature for detecting unmanaged configuration skew.

So glad to see this is being worked on!

[chazmuzz]

Would love a --dry-run flag!

[tim-lar]

Still keen on this one - any news?

[fourtwentyfour]

Ditto, this would be useful.

[gerardino-smg]

Any new on this one, @willvedd? My team and I would love to see this feature come to life :)

[erdtsieck]

Will this be planned or declined?

[alex-suciu]

In terms of a delivery date, I think late 2022 - early 2023 is a reasonable expectation. I'm hoping to hit the earlier side of this range, but again, I'd like to get this right and make sure that it suits your needs.

It's now May 2025 and there's still no dry-run feature available. In my opinion this feature is critical for using this tool at all otherwise it's just hoping that the deployment to production runs fine. It would be less critical if the deployment would be transactional (rollback on failure) but this doesn't seem to be the case. I'd appreciate if someone could explain how to avoid partial deployments due to failure in production without such a feature?

[kushalshit27]

Proposal: Dry Run on Deploy CLI

Coming soon

[SeanFeldman]

In theaters near you.

[kushalshit27]

Proposal: Dry Run on Deploy CLI

Github Issue: #70

Summary

We propose the introduction of a "dry run" capability to the Auth0 Deploy CLI. This feature will allow users to simulate the import/deployment process, previewing all potential changes (creations, updates, deletions) to their Auth0 tenant resources without actually applying them.

Why Dry Run?

In managing Auth0 configurations as code, particularly when targeting production or critical tenants, the ability to foresee the impact of changes is paramount. A dry run feature offers several crucial benefits:

• Risk Mitigation: It acts as a vital safety net, allowing users to identify and rectify potential misconfigurations or unintended consequences before they affect the live tenant environment. This reduces the risk of service disruptions, security misconfigurations, or data inconsistencies.

• Configuration Validation: Users can verify the syntactical correctness of their configuration files and ensure that the proposed changes align with their intentions and Auth0's expected schemas.

• Impact Assessment: The feature will provide a clear summary of what resources (Clients, Rules, Connections, Actions, etc.) would be created, updated, or deleted. This transparency is essential for informed decision-making.

• Improved Operational Confidence: By providing a preview, dry runs empower teams to deploy changes with greater confidence, especially in complex tenants or when making substantial modifications.

Currently, users deploying configurations with auth0-deploy-cli lack a built-in mechanism to preview changes. They must either deploy to a non-production tenant or risk applying changes directly, with potential for error.

Existing Methods of Previewing Changes

Currently, the Auth0 Deploy CLI does not offer a dedicated dry run or preview functionality for the import command.

• Manual Comparison: Manually diffing their local configuration files against the current state of the tenant (obtained via export or through the Auth0 Management Dashboard). This is error-prone, time-consuming, and impractical for large configurations.

• Deployment to Test Tenants: Applying the configuration to a separate, non-production tenant. While useful, this does not guarantee identical behaviour in production due to potential environmental differences and does not prevent accidental deployment to the wrong tenant if commands are mixed up.

The absence of a direct preview mechanism aligns with the motivation behind the request in #70, which explicitly asks for a way to see planned changes without execution.

Proposed Solution

Adding a new command-line flag, --dry-run, to the import command.

Example invocation CLI:

a0deploy import -c tenant-config.json --input_file ./my-tenant-directory --dry-run

Example invocation node-module:

import { deploy } from 'auth0-deploy-cli';

deploy({
  input_file: './local/tenant.yaml',
  config: {
    AUTH0_DOMAIN: process.env.AUTH0_DOMAIN,
    AUTH0_CLIENT_ID: process.env.AUTH0_CLIENT_ID,
    AUTH0_CLIENT_SECRET: process.env.AUTH0_CLIENT_SECRET,
    AUTH0_DRY_RUN: true,
  },
})
  .then(() => {
    console.log('Auth0 configuration applied to tenant successful');
  })
  .catch((err) => {
    console.log('Error when applying configuration to Auth0 tenant:', err);
  });

Output Format:

The output should indicate the intended operations for each resource. For example:

Dry Run Mode: Simulating deployment...

The following changes would be applied:

Rules:
  [CREATE] Rule "Force Email Verification"
  [UPDATE] Rule "Add User Role on First Login"
  [DELETE] Rule "Old MFA Trigger" (if AUTH0_ALLOW_DELETE is true)

Clients:
  [CREATE] Client "New SPA Application"
  [UPDATE] Client "Existing M2M Application"

Connections:
  [UPDATE] Connection "Username-Password-Authentication"

Summary:
  Would create: 2 resources
  Would update: 3 resources
  Would delete: 1 resource (if deletions enabled)


No changes have been made to your Auth0 tenant.

Note: Please note that the output format may be subject to change in the final implementation.

Limitations:

The "live state" of Auth0 tenant may change between a --dry-run and the actual deployment, potentially leading to unexpected outcomes. Without a state file like Terraform's, we cannot lock the state or predict such changes.

We are not trying to reinvent Terraform; goal is to provide a simple, flexible tool that gives developers control over their deployments. While we will warn about potential "live state" issues, managing them remains the developer's responsibility.

🌟 Help Shape the "Dry Run on Deploy CLI" Feature! 🌟

💬 Book a quick call with us to provide feedback on this upcoming feature—your input will help make it even better.

📅 Book

Thanks for being a part of this journey! 🙌