Skip to content

Git server credentials cannot be found when resuming and the domain is aliased using a CNAME record. #6921

@DriesSchaumont

Description

@DriesSchaumont

Bug report

When resuming a workflow, Nextflow tries to fetch the appropriate git server credentials by matching the domain of the credentials to the the origin domain defined in the git configuration. However, domain stored in the git config may not match with the domain of the credentials when a DNS CNAME record is involved.

Expected behavior and actual behavior

The projectDir is already present when resuming, and it contains a git config. In this case, guessHubProviderFromGitConfig is called. In this method, the domain of the origin in the git config is matched to the domain of the provided credentials:

final result = domain ? providerConfigs.find { it -> it.domain == domain } : (ProviderConfig)null

However, the git config may contain the canonical domain when the original nextflow run used the unresolved domain instead of the canonical domain from the credentials.

Since the original workflow succeeds without a problem, I would expect the resume workflow to also be able to find the credentials.

A simple workaround is to always use the canonical domain for the git credentials.

Steps to reproduce the problem

I am unable to provide a code example because that would require me to setup a CNAME record for git server. But below are the steps. Let's say <unresolved domain> points with a CNAME record to

  1. Create a scm file with <canonical domain> as the hub
  2. Do a nextflow run <unresolved domain>. Note that this run succeeds to clone the repo and run the workflow, and puts <canonical domain> as the origin in the git config.
  3. Do a nextflow run <unresolved domain> -resume. This run fails because the credentials are resolved differently and does not match with

I've confirmed the problem by manually changing the origin in the git config, which enabled nextflow to match the credentials again and run the workflow.

Program output

Mar-13 13:29:22.859 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 24.10.6
Mar-13 13:29:22.885 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/.nextflow/plugins; core-plugins: nf-amazon@2.9.3,nf-azure@1.10.2,nf-cloudcache@0.4.2,nf-codecommit@0.2.2,nf-console@1.1.4,nf-google@1.15.4,nf-tower@1.9.3,nf-wave@1.7.5
Mar-13 13:29:22.919 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Mar-13 13:29:22.920 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Mar-13 13:29:22.924 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.12.0 in 'deployment' mode
Mar-13 13:29:22.941 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Mar-13 13:29:22.955 [main] DEBUG nextflow.scm.ProviderConfig - Detected SCM custom path: https://***/api/ephemeral/uzB1aTuulQwjhqADl-7bmg
Mar-13 13:29:23.624 [main] DEBUG nextflow.scm.ProviderConfig - Detected SCM config: [providers:[***:[server:<canonical domain>, password:****, user:***, platform:***]]]
Mar-13 13:29:23.638 [main] DEBUG nextflow.scm.RepositoryFactory - Found Git repository result: [RepositoryFactory]
Mar-13 13:29:23.667 [main] DEBUG nextflow.scm.RepositoryFactory - Git url=nextflow.scm.GitUrl(https, null, <canonical domain>, ***) -> config=ProviderConfig[name: ***, platform: ***, server: <canonical domain>]
Mar-13 13:29:23.668 [main] DEBUG nextflow.scm.AssetManager - Repository URL: <canonical domain>; Project: ***; Hub provider: ***
Mar-13 13:29:23.748 [main] DEBUG nextflow.scm.AssetManager - Git config: /***/git/config; branch: null; remote: origin; url: <unresolved domain>
Mar-13 13:29:23.755 [main] DEBUG nextflow.cli.Launcher - Operation aborted
nextflow.exception.AbortOperationException: Can't find any configured provider for git server `<unresolved domain>` -- Make sure to have specified it in your `scm` file. For details check https://www.nextflow.io/docs/latest/sharing.html#scm-configuration-file
        at nextflow.scm.AssetManager.guessHubProviderFromGitConfig(AssetManager.groovy:1128)
        at nextflow.scm.AssetManager.validateProjectDir(AssetManager.groovy:202)
        at nextflow.scm.AssetManager.build(AssetManager.groovy:133)
        at nextflow.scm.AssetManager.<init>(AssetManager.groovy:106)
        at nextflow.cli.CmdRun.getScriptFile0(CmdRun.groovy:591)
        at nextflow.cli.CmdRun.getScriptFile(CmdRun.groovy:532)
        at nextflow.cli.CmdRun.run(CmdRun.groovy:325)
        at nextflow.cli.Launcher.run(Launcher.groovy:503)
        at nextflow.cli.Launcher.main(Launcher.groovy:658)

This was tested on a pretty old version (24.10.6), but based on the code from guessHubProviderFromGitConfig I believe that this problem is also present on master.

Environment

  • Nextflow version: 24.10.6
  • Java version: 17.0.15
  • Operating system: Linux
  • Bash version: GNU Bash 5.2.15

Additional context

(Add any other context about the problem here)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions