Cargo.lock: source field does not reflect the actual source of packages

[aaronjwood]

Problem

I'm in an environment where there are two Artifactory instances, one that serves up config.json in a correct way for mirroring and another that doesn't. The instance that's correct looks like this:

{
  "dl": "https://<REDACTED>/artifactory/api/cargo/cargo-remote/v1/crates",
  "api": "https://<REDACTED>/artifactory/api/cargo/cargo-remote"
}

The other one looks like:

{
  "dl": "https://static.crates.io/crates",
  "api": "https://crates.io"
}

We're trying to migrate between these. Our CI environment has no WAN access so the registry which is serving an incorrect config.json ended up breaking things. This was wildly unexpected and it was only until config.json was scrutinized along with some verifications (more on that below) that gave away the indication of what was happening. Regardless of which environment was used the project's lock file always contained source = "registry+https://github.com/rust-lang/crates.io-index".

Since the lock file gives no indication of where things are actually coming from I had to prove this in a long-winded way. Basically setting 127.0.0.1 localhost github.com static.crates.io crates.io in /etc/hosts and running cargo build on a project showed that the packages were coming from the Artifactory instance serving the correct config.json while packages were coming from upstream when using the Artifactory instance serving the incorrect config.json.

Since source in the lock file is misleading at best and policy-breaking at worst I'm assuming this is not intended, especially since there doesn't seem to be any reasonable way for folks to know where things are actually coming from. If our CI had WAN access we'd be going out to the upstream registry to get everything and never know it (unless you assume some network monitoring team will flag it one day) which is a serious issue for us.

Steps

No response

Possible Solution(s)

Have source reflect where things are actually coming from, which is defined in the registry's index config.json.

Notes

No response

Version

cargo 1.87.0 (Homebrew)
release: 1.87.0
host: aarch64-apple-darwin
libgit2: 1.9.0 (sys:0.20.0 system)
libcurl: 8.7.1 (sys:0.4.80+curl-8.12.1 system ssl:(SecureTransport) LibreSSL/3.3.6)
os: Mac OS 15.5.0 [64-bit]

[weihanglo]

Hey. Thanks for writing up a detailed issue report.

I'm in an environment where there are two Artifactory instances,

Are you talking about JFrog Artifactory? If I remember correctly, it leverages source replcement, which is effectively for mirroring. The behavaior of source not changing is intentional, as a mirror is supposed to be the same as the orignal source, quoting:

Cargo has a core assumption about source replacement that the source code is exactly the same from both sources. Note that this also means that a replacement source is not allowed to have crates which are not present in the original source.

So nothing should be changed in your lockfile. If is intended to be use as an custom registy, use an alternative registry feature.

[aaronjwood]

Hey. Thanks for writing up a detailed issue report.

Np!

Are you talking about JFrog Artifactory?

Yup, that's right.

Artifactory is generating this for us to use:

# Makes artifactory the default registry and saves passing --registry parameter
[registry]
default = "artifactory"

[registries.artifactory]
index = "sparse+https://<REDACTED>/artifactory/api/cargo/cargo-remote/index/"

# Add these 2 sections for resolving dependencies from Artifactory
[source.artifactory-remote]
registry = "sparse+https://<REDACTED>/artifactory/api/cargo/cargo-remote/index/"

[source.crates-io]
replace-with = "artifactory-remote"

IIUC, this is both the source replacement and the alternative registry strategies together. With pointing the index to a different location shouldn't the source keys be updated to reflect that?

[weihanglo]

IIUC, this is both the source replacement and the alternative registry strategies together. With pointing the index to a different location shouldn't the source keys be updated to reflect that?

What I meant for using alternative registries is "Specifying dependencies from other registries", which you declare a dependency from an alternative one explicitly.

Besides, registry. index is kinda unused. See https://doc.rust-lang.org/cargo/reference/config.html#registryindex

[aaronjwood]

I see. For this particular scenario going through and changing all the deps to explicitly point at another registry is not very feasible, as this is largely not our own repo and more of an auxiliary component that we're including along with other things. I guess you could say you could make a patch (or a script) to change all of these but keeping it in working condition will be annoying as upstream moves along.

The only way to know if you've fetched your deps from the right place is to use this explicit declaration?

[aaronjwood]

Thinking on this more, I find the proposed solution to create a bit of a split-brain situation. If this is true

The only way to know if you've fetched your deps from the right place is to use this explicit declaration?

then that contradicts the behavior of a correctly-configured mirror. E.g. in my original example packages were correctly coming from the mirror, it was only with the incorrectly configured mirror that they were silently coming from somewhere else which was luckily caught early.

So now there are two situations where you can have packages coming from a source you expect them to: one that gives no indication of this (my original example) and one that concretely shows this (the explicit approach you mentioned.) The negation of this looks more fractured: one situation gives you no indication that packages are coming from the wrong place while the other situation gives you no context that you're missing what you need and must add something entirely new (explicit approach) to force them to come from the right place.

Is this behavior intentional?

[weihanglo]

I believe cargo --verbose shows some visual indicators of registries but I don't believe that is what you're looking for.

The behavior of source replacement is mostly intentional. As mentioned in the docs, a source replacement mirror registry is expected to hold the exact the same set of crates as what is replaced, and when removing it your cargo build should work flawlessly (of course you need to download crate source code first if in an air-gapped environment).

[weihanglo]

must add something entirely new (explicit approach) to force them to come from the right place.

You don't. Source replacement replaces the entire source (e.g., a registry) not just one package. Nothing can go beyond the replacement, if the original is already replaced. Cargo doesn't automatically fall back to the original source.

[aaronjwood]

must add something entirely new (explicit approach) to force them to come from the right place.

You don't. Source replacement replaces the entire source (e.g., a registry) not just one package. Nothing can go beyond the replacement, if the original is already replaced. Cargo doesn't automatically fall back to the original source.

Ok, for environments open to the WAN it sounds like the possible solutions are:

1. Find a way to continually force each dependency (explicitly) via Cargo.lock: source field does not reflect the actual source of packages #15663 (comment)
2. Have some tooling which enforces package fetches don't ever silently regress to the WAN
3. Have some network monitoring/enforcement that ensures package fetches don't ever silently regress to the WAN

[weihanglo]

To clarify, Cargo doesn't regress to the WAN. It respects whatever configured in config.json so I would expect the second and third approaches being done outside Cargo first.

That said, we're developing a linting system for Cargo itself, though it currently supports on Cargo.toml stuff. And I am unsure how a lint would look like for catching this kind of situation.

---

BTW do you have two source replacement sections, one for artifactory and the other one for artifactory-remote? I don't see you source-replacing one with the other. I don't know how the old one got used if it wasn't via source replacement.

[aaronjwood]

To clarify, Cargo doesn't regress to the WAN. It respects whatever configured in config.json so I would expect the second and third approaches being done outside Cargo first.

Yeah, makes sense. I was talking about this in the context of a mirror that doesn't have its config.json set perfectly. The problem I had with all of this is that it's silent and requires either some form of external detection/enforcement or someone to go dig into the endpoints, assuming they even know about the ecosystem and the presence of the config.json URI.

Thanks for the heads up regarding the WIP linting system!

BTW do you have two source replacement sections, one for artifactory and the other one for artifactory-remote? I don't see you source-replacing one with the other. I don't know how the old one got used if it wasn't via source replacement.

Sorry, not sure I'm following you here. Are you referring to this section? It should be replacing crates.io (using the magically defined source.crates-io key) with our own.

[source.artifactory-remote]
registry = "sparse+https://<REDACTED>/artifactory/api/cargo/cargo-remote/index/"

[source.crates-io]
replace-with = "artifactory-remote"