Skip to content

DerivationView: avoid copying when varying derivation inputs #16236

Description

@Ericson2314

Introduce a DerivationView (or split out sub-structs) to recover zero-copy in the input-varying operations, without giving up the type-directed structure from #16198 and friends.

#16198 and related refactors gave us a type-directed structure for derivations: DerivationT<Inputs> instantiated at FullInputs, StorePathSet, HashModuloInputs, etc., with operations that convert between instantiations (mapInputs, maskDerivation, tryResolve, unresolve). That separation of concerns is worth keeping — but it currently costs us copies we didn't pay before.

The problem is that every input-varying operation rebuilds the whole struct:

template<typename F>
DerivationT<std::invoke_result_t<F, const Inputs &>> mapInputs(F f) const
{
    return {
        .outputs = outputs,          // copied
        .inputs = f(inputs),         // the only thing that actually changes
        .platform = platform,        // copied
        .builder = builder,          // copied
        .args = args,                // copied
        .env = env,                  // copied — often the biggest field!
        .structuredAttrs = structuredAttrs, // copied
        .name = name,                // copied
    };
}

Seven fields are copied to change one. For transient values — e.g. the DerivationT<HashModuloInputs> built inside hash-modulo computation, which exists only to be unparsed and hashed — those copies are pure waste. Before the refactor this code passed the substituted inputs alongside a reference to the original derivation, which was zero-copy but structurally invisible: the "derivation with different inputs" concept existed only in the calling convention. We don't want to go back to that.

Two complementary ways to get both:

Option A: DerivationViewT<Inputs>

A non-owning mirror of DerivationT: references for the input-agnostic fields, owned (or viewed) inputs:

template<typename Inputs>
struct DerivationViewT
{
    const DerivationOutputs & outputs;
    Inputs inputs;
    std::string_view platform;
    std::string_view builder;
    const Strings & args;
    const StringPairs & env;
    const std::optional<StructuredAttrs> & structuredAttrs;
    std::string_view name;
};

Then transient input-swaps borrow instead of copy:

DerivationViewT<HashModuloInputs> masked = maskDerivation(drv, ...);
auto hash = hashString(HashAlgorithm::SHA256, unparse(masked, store));

Operations that only read a derivation (unparse, type(), invariant checks) can take the view, and an implicit conversion DerivationT<Inputs> → DerivationViewT<Inputs> lets owned derivations flow into them unchanged — the same relationship std::string/std::string_view have. The usual view caveat applies (it must not outlive the derivation it borrows from), but the intended uses are all transient locals.

Option B: split out a sub-struct

Factor the input-agnostic fields into their own struct, so a derivation is "core + inputs":

struct DerivationCore
{
    DerivationOutputs outputs;
    std::string platform;
    std::string builder;
    Strings args;
    StringPairs env;
    std::optional<StructuredAttrs> structuredAttrs;
    std::string name;
};

template<typename Inputs>
struct DerivationT
{
    DerivationCore core;
    Inputs inputs;
};

Now the view type is nearly trivial — { const DerivationCore & core; Inputs inputs; } — and functions that don't care about inputs at all can just take const DerivationCore &, making input-independence part of the signature rather than a comment. The options compose: B makes A smaller and harder to get out of sync with the owning type.

Notes

  • The hash-modulo path runs per derivation per build, so the transient-instantiation case is the natural first target.
  • Whichever shape we pick, deduplicating the field list between owner and view (or letting the sub-struct be that deduplication) would be nice — a hand-maintained parallel struct that silently drifts when a field is added would be annoying

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions