fix(gnovm): bound type-expansion fan-out before go/types validType - #5826
fix(gnovm): bound type-expansion fan-out before go/types validType#5826ltzmaxwell wants to merge 50 commits into
Conversation
🛠 PR Checks SummaryAll Automated Checks passed. ✅ Manual Checks (for Reviewers):
Read More🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers. ✅ Automated Checks (for Contributors):🟢 Maintainers must be able to edit this pull request (more info) ☑️ Contributor Actions:
☑️ Reviewer Actions:
📚 Resources:Debug
|
davd-gzl
left a comment
There was a problem hiding this comment.
The fix is correct and the guard is linear, but the validType DoS is still reachable through three type shapes the guard under-counts. Verified on 088ce87: each drop-in txtar deploys past the guard and hangs the node on the unmetered addpkg path, where the direct fan-out is rejected.
| return nil, errs | ||
| } | ||
|
|
||
| // STEP 3.5: Guard against pathological type-expansion fan-out before the |
There was a problem hiding this comment.
It's noted STEP 3, then STEP 3.5, then STEP 3 again
There was a problem hiding this comment.
Fixed in c049157 — the pre-type-check guard comments now reuse STEP 3 plainly, matching the repeated STEP 4 blocks below.
| case *ast.IndexExpr: | ||
| return cost(t.X) // generic instantiation: bound by the base type | ||
| case *ast.IndexListExpr: | ||
| return cost(t.X) |
There was a problem hiding this comment.
For W[A{n-1}] the guard counts the base W and ignores the type argument, where the doubling lives. A generic fan-out reads as a constant and passes into the validType hang. Fix: count the type arguments too, or reject the instantiation.
repro
The direct [0]T%d form at this depth is rejected in milliseconds; the generic form is not:
gh pr checkout 5826 -R gnolang/gno && git checkout 088ce87
cat > gno.land/pkg/integration/testdata/addpkg_typecheck_fanout_generic.txtar <<'EOF'
# Value-containment fan-out routed through a generic: each A_n embeds W[A_{n-1}]
# by value, W holds its type parameter twice, so validType still doubles. The
# guard drops the IndexExpr type argument, so it must ALSO reject this, not hang.
# start a new node
gnoland start
# adding the generic fan-out package must fail with a denial-of-service rejection
! gnokey maketx addpkg -pkgdir $WORK/fanout -pkgpath gno.land/r/foobar/fanout -gas-fee 350001ugnot -gas-wanted 20_000_000 -chainid=tendermint_test $test1_user_addr
stderr 'denial-of-service'
-- fanout/gnomod.toml --
module = "gno.land/r/foobar/fanout"
gno = "0.9"
-- fanout/fanout.gno --
package fanout
type W[P any] struct{ a, b [0]P }
type A0 struct{ v int }
type A1 struct{ x W[A0] }
type A2 struct{ x W[A1] }
type A3 struct{ x W[A2] }
type A4 struct{ x W[A3] }
type A5 struct{ x W[A4] }
type A6 struct{ x W[A5] }
type A7 struct{ x W[A6] }
type A8 struct{ x W[A7] }
type A9 struct{ x W[A8] }
type A10 struct{ x W[A9] }
type A11 struct{ x W[A10] }
type A12 struct{ x W[A11] }
type A13 struct{ x W[A12] }
type A14 struct{ x W[A13] }
type A15 struct{ x W[A14] }
type A16 struct{ x W[A15] }
type A17 struct{ x W[A16] }
type A18 struct{ x W[A17] }
type A19 struct{ x W[A18] }
type A20 struct{ x W[A19] }
type A21 struct{ x W[A20] }
type A22 struct{ x W[A21] }
type A23 struct{ x W[A22] }
type A24 struct{ x W[A23] }
type A25 struct{ x W[A24] }
type A26 struct{ x W[A25] }
type A27 struct{ x W[A26] }
type A28 struct{ x W[A27] }
type A29 struct{ x W[A28] }
type A30 struct{ x W[A29] }
type A31 struct{ x W[A30] }
type A32 struct{ x W[A31] }
type A33 struct{ x W[A32] }
type A34 struct{ x W[A33] }
type A35 struct{ x W[A34] }
type A36 struct{ x W[A35] }
type A37 struct{ x W[A36] }
type A38 struct{ x W[A37] }
type A39 struct{ x W[A38] }
type A40 struct{ x W[A39] }
var Sink A40
EOF
go test -count=1 -v -run TestTestdata/addpkg_typecheck_fanout_generic ./gno.land/pkg/integration/
rm gno.land/pkg/integration/testdata/addpkg_typecheck_fanout_generic.txtarObserved on 088ce87:
# adding the generic fan-out package must fail with a denial-of-service rejection (65.00s)
> ! gnokey maketx addpkg ... -pkgpath gno.land/r/foobar/fanout ...
"gnokey" error: unable to call RPC method abci_query, unable to send request, Post "http://127.0.0.1:...": context deadline exceeded
> stderr 'denial-of-service'
FAIL: testdata/addpkg_typecheck_fanout_generic.txtar:12: no match for `denial-of-service` found in stderr
There was a problem hiding this comment.
Fixed in c049157, by rejecting the instantiation rather than counting it: checkNoGenerics now rejects generics syntactically before go/types runs (Gno targets go1.17; pinning types.Config.GoVersion does not help because go/types reports the version error but still runs the validType walk). Your repro is committed as addpkg_typecheck_fanout_generic.txtar and is now rejected in milliseconds with generic type declarations are not supported (Gno targets go1.17).
| case *ast.InterfaceType: | ||
| total := uint64(1) | ||
| for _, f := range t.Methods.List { | ||
| if len(f.Names) != 0 { | ||
| continue // method: a func signature, not recursed | ||
| } | ||
| total = satAdd(total, cost(f.Type)) // embedded type / type elem | ||
| if total > typeExpansionBudget { | ||
| return total | ||
| } | ||
| } | ||
| return total |
There was a problem hiding this comment.
The guard recurses into struct fields and array elements but not into an interface type-set union. A union term [0]X | [1]X (a BinaryExpr, ~T a UnaryExpr) hits default: return 1 at line 147-148, yet validType walks both sides, so a union-doubling chain blows up unscored. Fix: recurse both sides of a | and the operand of a ~.
repro
gh pr checkout 5826 -R gnolang/gno && git checkout 088ce87
cat > gno.land/pkg/integration/testdata/addpkg_typecheck_fanout_union.txtar <<'EOF'
# Value-containment fan-out through interface type-set unions: each I_n unions
# two array types over I_{n-1}, so validType still doubles per level. The guard's
# InterfaceType arm doesn't recurse the BinaryExpr union term (default returns 1),
# so it must reject this, not hang.
# start a new node
gnoland start
# adding the union fan-out package must fail with a denial-of-service rejection
! gnokey maketx addpkg -pkgdir $WORK/fanout -pkgpath gno.land/r/foobar/fanout -gas-fee 350001ugnot -gas-wanted 20_000_000 -chainid=tendermint_test $test1_user_addr
stderr 'denial-of-service'
-- fanout/gnomod.toml --
module = "gno.land/r/foobar/fanout"
gno = "0.9"
-- fanout/fanout.gno --
package fanout
type I0 interface{ m() }
type I1 interface{ [0]I0 | [1]I0 }
type I2 interface{ [0]I1 | [1]I1 }
type I3 interface{ [0]I2 | [1]I2 }
type I4 interface{ [0]I3 | [1]I3 }
type I5 interface{ [0]I4 | [1]I4 }
type I6 interface{ [0]I5 | [1]I5 }
type I7 interface{ [0]I6 | [1]I6 }
type I8 interface{ [0]I7 | [1]I7 }
type I9 interface{ [0]I8 | [1]I8 }
type I10 interface{ [0]I9 | [1]I9 }
type I11 interface{ [0]I10 | [1]I10 }
type I12 interface{ [0]I11 | [1]I11 }
type I13 interface{ [0]I12 | [1]I12 }
type I14 interface{ [0]I13 | [1]I13 }
type I15 interface{ [0]I14 | [1]I14 }
type I16 interface{ [0]I15 | [1]I15 }
type I17 interface{ [0]I16 | [1]I16 }
type I18 interface{ [0]I17 | [1]I17 }
type I19 interface{ [0]I18 | [1]I18 }
type I20 interface{ [0]I19 | [1]I19 }
type I21 interface{ [0]I20 | [1]I20 }
type I22 interface{ [0]I21 | [1]I21 }
type I23 interface{ [0]I22 | [1]I22 }
type I24 interface{ [0]I23 | [1]I23 }
type I25 interface{ [0]I24 | [1]I24 }
type I26 interface{ [0]I25 | [1]I25 }
type I27 interface{ [0]I26 | [1]I26 }
type I28 interface{ [0]I27 | [1]I27 }
type I29 interface{ [0]I28 | [1]I28 }
type I30 interface{ [0]I29 | [1]I29 }
type I31 interface{ [0]I30 | [1]I30 }
type I32 interface{ [0]I31 | [1]I31 }
type I33 interface{ [0]I32 | [1]I32 }
type I34 interface{ [0]I33 | [1]I33 }
type I35 interface{ [0]I34 | [1]I34 }
type I36 interface{ [0]I35 | [1]I35 }
type I37 interface{ [0]I36 | [1]I36 }
type I38 interface{ [0]I37 | [1]I37 }
type I39 interface{ [0]I38 | [1]I38 }
type I40 interface{ [0]I39 | [1]I39 }
type Use struct{ x I40 }
EOF
go test -count=1 -v -run TestTestdata/addpkg_typecheck_fanout_union ./gno.land/pkg/integration/
rm gno.land/pkg/integration/testdata/addpkg_typecheck_fanout_union.txtarObserved on 088ce87:
# adding the union fan-out package must fail with a denial-of-service rejection (65.00s)
> ! gnokey maketx addpkg ... -pkgpath gno.land/r/foobar/fanout ...
"gnokey" error: unable to call RPC method abci_query, unable to send request, Post "http://127.0.0.1:...": context deadline exceeded
> stderr 'denial-of-service'
FAIL: testdata/addpkg_typecheck_fanout_union.txtar:13: no match for `denial-of-service` found in stderr
There was a problem hiding this comment.
Fixed in c049157: checkNoGenerics rejects exactly the two type-set terms cost() cannot count — union (|) and approximation (~) — before go/types runs. Bare and ;-separated type-set elements stay allowed: they are ordinary containment edges the bound already counts. Your repro is committed as addpkg_typecheck_fanout_union.txtar (rejected with interface type unions are not supported), and 9ec5b86 adds a multi-element interface fan-out case to keep that path counted.
| case *ast.SelectorExpr: | ||
| return 1 // imported type: already validated in its own package |
There was a problem hiding this comment.
The guard counts an imported type pkg.T as a flat 1. But validType re-expands imported types without caching across packages (golang/go#65711), so a doubling chain spread across deployed packages stays under the per-package guard while the walk doubles at every link until a deploy hangs the node. The guard never sees the imported cost. Fix: remember each package's worst-case expansion (e.g. in TypeCheckCache) and add it in for pkg.T.
repro (deploy chain)
p0 is a depth-16 chain (cost 2^16, under budget); p1..p5 each embed the previous package's T four times. Each passes the guard, but validType re-expands the imported chain per deploy:
gh pr checkout 5826 -R gnolang/gno && git checkout 088ce87
cat > gno.land/pkg/integration/testdata/addpkg_typecheck_fanout_imported.txtar <<'EOF'
# Value-containment fan-out split across a deploy chain. Each package passes the
# guard on its own (imported types score as a leaf: SelectorExpr returns 1), but
# validType crosses import boundaries without memoizing, so the walk doubles per
# package and the final deploy kills the node, with no package over budget.
# start a new node
gnoland start
# deploy p0
gnokey maketx addpkg -pkgdir $WORK/p0 -pkgpath gno.land/r/foobar/p0 -gas-fee 350001ugnot -gas-wanted 20_000_000 -chainid=tendermint_test $test1_user_addr
# deploy p1
gnokey maketx addpkg -pkgdir $WORK/p1 -pkgpath gno.land/r/foobar/p1 -gas-fee 350001ugnot -gas-wanted 20_000_000 -chainid=tendermint_test $test1_user_addr
# deploy p2
gnokey maketx addpkg -pkgdir $WORK/p2 -pkgpath gno.land/r/foobar/p2 -gas-fee 350001ugnot -gas-wanted 20_000_000 -chainid=tendermint_test $test1_user_addr
# deploy p3
gnokey maketx addpkg -pkgdir $WORK/p3 -pkgpath gno.land/r/foobar/p3 -gas-fee 350001ugnot -gas-wanted 20_000_000 -chainid=tendermint_test $test1_user_addr
# deploy p4
gnokey maketx addpkg -pkgdir $WORK/p4 -pkgpath gno.land/r/foobar/p4 -gas-fee 350001ugnot -gas-wanted 20_000_000 -chainid=tendermint_test $test1_user_addr
# deploy p5 (passes the guard, but its validType walk re-expands the whole imported chain)
! gnokey maketx addpkg -pkgdir $WORK/p5 -pkgpath gno.land/r/foobar/p5 -gas-fee 350001ugnot -gas-wanted 20_000_000 -chainid=tendermint_test $test1_user_addr
stderr 'denial-of-service'
-- p0/gnomod.toml --
module = "gno.land/r/foobar/p0"
gno = "0.9"
-- p0/p0.gno --
package p0
type t0 struct{ v int }
type t1 struct{ a, b [0]t0 }
type t2 struct{ a, b [0]t1 }
type t3 struct{ a, b [0]t2 }
type t4 struct{ a, b [0]t3 }
type t5 struct{ a, b [0]t4 }
type t6 struct{ a, b [0]t5 }
type t7 struct{ a, b [0]t6 }
type t8 struct{ a, b [0]t7 }
type t9 struct{ a, b [0]t8 }
type t10 struct{ a, b [0]t9 }
type t11 struct{ a, b [0]t10 }
type t12 struct{ a, b [0]t11 }
type t13 struct{ a, b [0]t12 }
type t14 struct{ a, b [0]t13 }
type t15 struct{ a, b [0]t14 }
type T struct{ a, b [0]t15 }
-- p1/gnomod.toml --
module = "gno.land/r/foobar/p1"
gno = "0.9"
-- p1/p1.gno --
package p1
import "gno.land/r/foobar/p0"
type T struct{ a, b, c, d [0]p0.T }
-- p2/gnomod.toml --
module = "gno.land/r/foobar/p2"
gno = "0.9"
-- p2/p2.gno --
package p2
import "gno.land/r/foobar/p1"
type T struct{ a, b, c, d [0]p1.T }
-- p3/gnomod.toml --
module = "gno.land/r/foobar/p3"
gno = "0.9"
-- p3/p3.gno --
package p3
import "gno.land/r/foobar/p2"
type T struct{ a, b, c, d [0]p2.T }
-- p4/gnomod.toml --
module = "gno.land/r/foobar/p4"
gno = "0.9"
-- p4/p4.gno --
package p4
import "gno.land/r/foobar/p3"
type T struct{ a, b, c, d [0]p3.T }
-- p5/gnomod.toml --
module = "gno.land/r/foobar/p5"
gno = "0.9"
-- p5/p5.gno --
package p5
import "gno.land/r/foobar/p4"
type T struct{ a, b, c, d [0]p4.T }
EOF
go test -count=1 -v -run TestTestdata/addpkg_typecheck_fanout_imported ./gno.land/pkg/integration/
rm gno.land/pkg/integration/testdata/addpkg_typecheck_fanout_imported.txtarObserved on 088ce87 (gas flat, wall-clock doubles each hop, then the node dies):
# deploy p0 (0.118s) GAS USED: 3055141
# deploy p1 (0.297s) GAS USED: 4126720
# deploy p2 (0.914s) GAS USED: 4349353
# deploy p3 (3.467s) GAS USED: 4598450
# deploy p4 (13.556s) GAS USED: 4847547
# deploy p5 (28.079s) "gnokey" error: ... Post "http://127.0.0.1:...": EOF <- node down
> stderr 'denial-of-service'
FAIL: testdata/addpkg_typecheck_fanout_imported.txtar:29: no match for `denial-of-service` found in stderr
There was a problem hiding this comment.
Fixed in c049157: the guard now follows value-containment across import boundaries. checkTypeExpansionBoundImports resolves already-deployed dependencies via the importer's getter and memoizes by (package, name), so the cross-package walk validType runs exponentially is computed linearly here, with no extra store gas for the common case. Stdlib imports remain leaves — bounded source that cannot import user packages (rationale documented in 02752d9 and the ADR). Your deploy chain is committed as addpkg_typecheck_fanout_imported.txtar: p0 (2^16, under budget) deploys, and p1 is rejected with denial-of-service at 4×2^16 > 100k.
…n-out Reject go1.18 generics/interface type-sets before go/types runs, and follow value-containment across imports (a memoizing getter keeps store gas unchanged).
…interface fan-out
|
Follow-up, not implemented here: Rationale: |
|
#5892 (per-source-byte gas) is complementary, not overlapping. It prices the linear case; a per-byte charge cannot catch an exponential one, since ~1KB of source causing ~2^40 node visits is not a function of the bytes. They compose: byte-gas gate → this PR's expansion cap → Note #5892 is stacked on #5891, which rewrites the |
|
Superseded — this comment argued against forking The corrected argument — including what a metered fork would buy, and why size and shipping risk still decide against it — is now in the PR description under Why not fork [AI-Assisted] |
|
#5921 is closed as a duplicate — this PR is the sole carrier of the guard. Placement verdict: #5921 (comment). #6059 adds the same rejection to Measured — what still passes this PR's on-chain gate:
None is a DoS vector, so the guard is complete for its purpose. But row 1 is a correctness gap on-chain, not only in dev tools: Pushed here as follow-ups: the |
|
refactoring... |
…safety
The per-package bound did not bound a transaction. Tx.Msgs is unbounded —
ValidateBasic caps gas, not the message count — and baseapp dispatches every
message to the handler, so N MsgAddPackage/MsgRun messages each paid the budget
in full. Measured at the old 1_000_000: 511 source bytes bought a package that
passed the guard and cost ~24ms of walk, i.e. 47.6us/byte against the ~1.25us/byte
PreprocessGasPerByte charges — 38x more CPU than paid for. A 1MB tx fits ~1400
such messages, so tens of seconds of unmetered walk with no prior chain state.
Rather than add a separate per-transaction counter, price the unit correctly:
gas is already per-tx and per-block bounded, so if the worst accepted package
costs no more CPU than its bytes pay for, any number of messages is bounded by
GasWanted and the block gas limit. Bytes are a poor proxy — each extra
`type tN struct{ a, b [0]tN-1 }` line is ~31 bytes and doubles the total — so the
worst shape is a doubling chain, B / (33 + 31*log2(B/14)) nodes per byte. Solving
for parity gives ~20_000; measured, the worst accepted package is now 325 bytes
at 0.81us/byte, 0.6x the priced rate.
Honest code is unaffected: the largest per-package total in real code is 181, so
20_000 keeps ~110x headroom, and TestHonestTypeExpansionUnderBudget still passes
at its 100x margin. Depth cap moves from ~1000 to ~135, still far past the
single-digit maximum measured in stdlibs and examples.
Retuned the fixtures the lower budget invalidates: linear-chain pass/reject
depths, the aggregate test's shared chain and copy counts, doublingPkgSrc depths
in the import and cache-sharing tests, and the deployed p0 chains in the
_imported and _dotimport txtars.
The transitive-dependency residual stays open and is now recorded precisely:
dependency bytes were paid for by earlier transactions, so per-message parity
does not price them. The ADR notes the fix is to charge gas for the count the
guard already computes — which is NOT the metering-inside-go/types proposal it
rejects elsewhere, a distinction the ADR previously conflated.
…er-tx gap The per-package cap did not bound a transaction. Lowering the budget to 20_000 closed the multi-message path by making per-message CPU cheaper than per-message byte-gas, but it cannot close the transitive-dependency path: one MsgAddPackage re-type-checks its whole closure, and those dependencies' source bytes were paid for by earlier transactions. Measured: a 55-byte package importing the tip of a 30-deep chain of near-budget packages pulls in 321,070 closure nodes against 68,750 gas of byte charges — 117x unpriced. The guard already computes that count exactly, linearly, and outside go/types, so there is nothing to meter inside a stdlib package: report it via TypeCheckOptions.ExpansionNodes and have the keeper charge nodes x 25 gas (1 gas ~= 1ns; the walk measures ~25ns/node). Closure cost is then tied to this tx's gas, which GasWanted and MaxBlockMaxGas already bound — no new per-tx or per-block counter, and no second cap needing its own compatibility argument. Only ACCEPTED packages are counted. Rejecting stops go/types, so a rejected total is the cost avoided; charging it prices prevented work and replaces the informative rejection with an out-of-gas. The first cut did exactly that — addpkg_typecheck_fanout.txtar reported 3.8e14 gas instead of the expected message. TestExpansionNodesZeroWhenRejected pins the corrected behaviour, and TestExpansionNodesCoversClosure pins that the count spans the closure. The cap is kept alongside the charge: the rate is calibrated in ns/node on one machine and can be wrong on another, whereas the cap cannot, so a mis-estimated rate leaves the worst package under-charged but still bounded. No gas fixture needed re-pinning — honest totals are small (largest 181, ~4.5k gas) and TestTestdata passes unchanged. The rate is a constant rather than a Params field only because Params is amino-generated and adding a field needs `go generate`, which AGENTS.md keeps out of ordinary PRs; noted in the ADR as belonging there eventually.
The previous commit charged once, after TypeCheckMemPackage returned. That prices the closure but cannot prevent it: the walk has already happened, so an out-of-gas only stops subsequent messages, never the walk it is billing for. A tiny package importing a large pre-deployed closure therefore still burned (packages x per-package cap) of CPU before being charged. Charge per package instead, before go/types walks it. gnovm hands each package's computed count to TypeCheckOptions.ChargeExpansion; the keeper's callback consumes gas there. The gas meter panics on out-of-gas and go/types re-panics anything that is not its own bailout, so the abort propagates out of cfg.Check and the remaining dependencies are never walked. A callback rather than a gas meter keeps gnovm free of any tm2 dependency; gnovm/pkg/parser's ParseFile2 uses the same pattern for per-token gas. checkTypeExpansionBoundImports now returns the count alongside the error, which also removes the accumulate-only-when-accepted subtlety: a rejected package returns zero, so it is naturally not charged. TestExpansionChargedPerPackage pins one charge per package in the closure — the entry, all dependencies, and the injected .gnobuiltins shim — and that a panicking charge stops the walk after three packages instead of all twenty-two. TestExpansionNotChargedWhenRejected pins that a rejected package is not charged.
…eper test /simplify review. Three findings, one of them a bug I had shipped. The seam was a bespoke `ChargeExpansion func(nodes uint64)` callback, justified in a comment by "gnovm keeps no dependency on tm2". That justification is false: gnolang already imports tm2's store in seven files and already carries a meter (MachineOptions.GasMeter, Allocator.SetGasMeter). The cited precedent argued the other way too — gnovm/pkg/parser needs a callback because it is a tm2-free fork of go/parser; gnolang is the layer that holds the meter. Replaced with TypeCheckOptions.GasMeter, and moved the rate constant next to the budget it prices, where tokenCostFactor and the OpCPU* tables live. The keeper side drops from 38 added lines to 2. The bug: while reshaping, my edit to AddPackage's options literal silently did not apply (I asserted on the other replacements but not that one), so MsgAddPackage passed no meter and charged nothing. Every suite still passed — the expansion cap rejects the extreme cases either way, and nothing at the keeper level asserted the charge. TestVMKeeperAddPackage_TypeExpansionGasCharged now does: it deploys a value-containment chain and a pointer chain with the same declaration count and near-identical bytes, so store, parse and preprocess costs cancel and only expansion differs. 204,905 gas apart wired, 27,855 unwired — the residual being exactly the 18-byte source difference. Verified failing without the wiring. A depth-vs-depth comparison would not have caught it, since that also changes the declaration count. Also from the review: folded nearBudgetPkgSrc into doublingPkgSrc (byte-identical apart from an optional import); the tests now use a recording wrapper over a real gas meter, so they exercise the actual out-of-gas panic rather than a stand-in; dropped a dead `nodes == 0` guard and a log-only closure; corrected the cap's rationale, which claimed the cap is machine-independent "whereas the rate is not" while both derive from the same ns/node measurement — the honest form is that the cap bounds nodes per package unconditionally, so a wrong rate under-charges but never unbounds; recorded why returning an error instead of panicking would be worse (go/types keeps resolving imports after an importer error); and trimmed the same argument from five copies down to the ADR plus pointers.
20_000 was a leftover from before per-node charging existed. It was chosen so the worst accepted package's CPU stayed under what its source BYTES paid for — using a cap to do pricing, because pricing did not exist yet. Now that each package is charged for its computed node count before being walked, the ceiling doing that job too only rejects packages a sender could afford: at 20_000 a package costs at most 500k gas, well inside a typical 20M GasWanted, so the ceiling bound where gas would happily have allowed. At 1_000_000 one package's charge (2.5e7 gas) exceeds a typical GasWanted, so gas binds first and an expensive package deploys if it is paid for. The ceiling keeps only the two jobs gas cannot do: unmetered callers (gno test, gno lint and gnodev pass no GasMeter, so nothing else stops a 2^40 walk), and surviving a mis-wired charge — which has happened, and which the ceiling is what made survivable. Kept global rather than applied only when no meter is set. Off-chain-only would leave the chain with no ceiling at all, so the AddPackage wiring slip would have been unbounded rather than merely uncharged; it would also let a package deploy on-chain that a local `gno test` rejects, and local tooling must never be stricter than the chain. Renamed typeExpansionBudget to typeExpansionCeiling: "budget" implied the primary gate, which is now the charge. Reverted the fixtures the 20_000 revision retuned (linear-chain depths, aggregate copy counts, doublingPkgSrc depths, the imported txtar's p0 chain) and dropped the byte-parity calibration from the ADR, which no longer justifies anything. Honest headroom is now ~5500x (largest real package total is 181), the depth cap returns to ~1000 from ~135, and an unmetered walk is bounded at ~25ms per package.
The fanout txtars all assert 'denial-of-service', which is correct for them — they exceed the ceiling, and the ceiling rejects before charging, so no gas is consumed and OOG never fires. But nothing exercised the other outcome, which is the mechanism the charge exists for: a closure that is affordable per package yet unaffordable in total. addpkg_typecheck_fanout_closure_gas.txtar deploys p0 (a doubling chain under the ceiling, ~11.5M gas of expansion) and then a three-line package importing it. At 5M gas-wanted the importer fails with "gas used (20821679) exceeds tx's gas wanted (5000000)" — priced for a closure whose source bytes an earlier transaction paid for — and the same package deploys once its budget covers it, which is the point: this is a pricing outcome, not a validity one. Also records in the ADR that the two rejections are distinct and both covered, and that gnokey reports the required gas-wanted, so the OOG case is actionable rather than opaque.
"Closure" was doing double duty: this PR uses it for a package's transitive dependency set, while the codebase — and this PR's own callback discussion — uses it for a function closure. keeper.go now contains both senses four lines apart, which is exactly the ambiguity to avoid. Replaced the dependency sense throughout this PR's files with "transitive dependencies", "the dependencies it pulls in", or "part-way down the import chain", per context. Renamed addpkg_typecheck_fanout_closure_gas.txtar to addpkg_typecheck_fanout_deps_gas.txtar and updated the ADR's reference. Left untouched: keeper.go:1147 and the other ~60 pre-existing uses across the repo, which correctly mean function closures.
The per-package expansion ceiling is removed. Any setting of it was either stricter than gas, refusing packages the sender paid for, or more permissive, where nothing above it is payable anyway and it only relabels an out-of-gas. Either way its scope was per-package, which no budget has: Tx.Msgs is unbounded and one message re-checks every dependency it imports. Only the per-node charge against the transaction's gas meter bounds that, so it is now the whole on-chain defence. checkTypeExpansionBoundImports therefore becomes typeExpansionCost, returning a count with no verdict, and it is not computed at all when no GasMeter is set — off-chain callers now skip resolving the dependency graph entirely. Removing the ceiling makes two things load-bearing that it had been covering: - Overflow. cost() saturates at MaxUint64, and int64(MaxUint64) is -1, so a saturated count would have charged -25 gas — a refund for the worst package a sender could submit. expansionGas() clamps at MaxInt64. - Under-counting, which is under-charging and multiplies: a leaf at the base of a depth-d doubling chain is walked 2^d times. Unresolved names were scored 1. Imported stdlib types are now scored leafExpansionBound (32, measured max over all exported stdlib types is 19), and the .gnobuiltins.gno realm/address names exactly 2 — that shim table matters because `address` is the type of much of the stdlib API, so folding it into the leaf bound moved the honest maximum from 431 to 8175 nodes. This under-count predates the ceiling's removal; the ceiling never bounded it either. Verdict-based tests become price-based: the import chain asserts each link at least doubles and that leaf-scoring under-prices the tip by >1000x, the aggregate test asserts the total scales with declaration count while the costliest single type stays under a tenth of it, and the cache test compares prices rather than verdicts. TestExpansionNotChargedWhenRejected moves onto the two syntactic guards, where "rejected, so not charged" still means something. Two txtars now assert out of gas.
25 was the validType walk measured on the development machine with the
host-calibration step omitted. This repo's gas convention is 1 gas == 1ns on
reference hardware (Intel Xeon Platinum 8168, per the OpCPU* table in machine.go),
so the rate has to be nanoseconds per node on THAT machine.
Two steps, both now recorded on the constant:
1. BenchmarkValidTypeWalk, added here, reports ns/node over a doubling chain:
30.1 / 30.4 / 34.9 at depth 18 / 20 / 22 on an Apple M5. The marginal rate
between successive depths climbs to 40.3 by depth 26 as the working set
outgrows cache, and a denial of service is the large-working-set end, so ~40
is the figure to price.
2. gnovm/cmd/calibrate ships paired benchmark output for the Xeon and Apple
silicon; over the 37 shared BenchmarkAlloc cases the Xeon is 2.96x slower
(median), 2.2-3.2x on the small allocations that most resemble validType's
pointer chasing.
40 * ~2.5 = 100. At 100 a whole block of gas (MaxBlockMaxGas = 3e9) buys 3e7
nodes, about 3s of validType on reference hardware, which is what 1 gas == 1ns is
meant to mean for a full block. At 25 the same block bought 1.2e8 nodes, ~12s — a
~4x under-charge, and since the ceiling was removed this charge is the only thing
pricing the walk, so that gap was the whole defence being off by 4x.
The calibration factor remains the dominant uncertainty (2.2-3.2x spans 88-128);
re-measuring directly on reference hardware is how to tighten it, exactly as
PreprocessGasPerByte notes for itself.
Honest code is unaffected in practice: the largest real package is 431 nodes, now
43k gas against the ~5e7 GasWanted a real deploy uses. The keeper test's
pointer-vs-value delta moves 204,905 -> 736,055 gas. Two txtars shorten their
chains by a level so their existing gas-wanted values still discriminate:
deps_gas p0 22.9M / user 11.5M, imported p0 11.5M / p1 28.7M.
… into the ADR typecheck_bound.go was 44% comment, with a 90-line block on one constant. Code comments now state what a thing is in ~3 lines and point at the ADR for why: 620 -> 444 lines, 277 -> 101 comment lines, longest block 90 -> 6, and no block over 6 lines left. Moved into the ADR rather than deleted: - The rate derivation, now a table: the walk measured by BenchmarkValidTypeWalk, and the calibration to the Xeon that 1 gas == 1ns refers to. - Why expansionGas clamps. int64(math.MaxUint64) is -1, so a saturated count would charge negative gas; the ADR had no record of this at all. - Why stdlib is estimated rather than resolved, which was only implied before: for a user dependency go/types calls GetMemPackage itself, so memoizingGetter deduplicates the guard's fetch to nothing, but ImportFrom returns stdlib from permCache before reaching the getter, so fetching it here would be a store read and a re-parse that no deploy otherwise pays — on every deploy. - Two ways to get an exact stdlib count for free, both deferred. A table precomputed during LoadStdlib, which rides on a parse that already happens there (TypeCheckMemPackage per stdlibs.InitOrder(), ASTs discarded after); or scoring from the *types.Package already in permCache, whose Scope() carries the same containment edges with no source, parse or store read. The second is cheaper but would make a consensus-visible charge a function of cache state, which is why a compile-time constant wins for now. Also corrects a claim in the rate comment: the 2.96x Xeon factor comes from rerunning cmd/calibrate's BenchmarkAlloc set locally and comparing against the shipped Xeon output, not from the two shipped files. The shipped M2 file gives 2.54x. Both figures and the resulting 88-128 range are now recorded. No code changes: the diff for typecheck_bound.go is comments only.
Two issues from the PR's lint job: - predeclared: const max in expansionGas shadows the max builtin; renamed maxNodes. - unconvert: int64(amount) in recordingGasMeter.ConsumeGas, where store.Gas is already int64.
…which fixtures show it Every test here asserted that the charge prices a pathological package; none showed the package was dangerous in the first place, so a reader could not tell the vulnerability was real rather than theoretical. TestValidTypeWalkIsExponential asserts it, in two halves on one input: - Unguarded, in a subprocess: go/types is handed a 30-level doubling chain and must still be running when the deadline fires. A subprocess because the walk cannot be cancelled — ~2^31 node visits is minutes of CPU — so it has to be killed rather than left burning a core for the life of the test binary. - Guarded, in-process: the same source through TypeCheckMemPackage with a meter returns out of gas. That this half returns AT ALL is the assertion; if the charge did not land before go/types, the test would hang. It also serves as an upstream tripwire: if the child ever finishes on its own, validType has likely been memoized (golang/go#65711), which does not make the charge wrong but does invalidate the rate derivation. Then, running the fixtures against master, two of their headers turned out to overclaim, and are corrected: - addpkg_typecheck_fanout: verified. The node never answers; after 60s gnokey fails with "unable to call RPC method abci_query ... context deadline exceeded" at broadcast.go:245 - simulate tx. Recorded in the header, including that it lands on a read-only simulate path with no fee and no tx inclusion. Same input here returns out of gas in ~2s. - addpkg_typecheck_fanout_imported: claimed "until the node hangs". It does not — p1 deploys fine on master in ~1s, since ~287k nodes is milliseconds of walk. This is structural, not a sizing slip: a cross-package fixture needs p0 legitimately deployed, and once the charge exists nothing deployable is dangerous (a block buys 3e7 nodes, ~3s). Relabelled a pricing fixture. - addpkg_typecheck_fanout_dotimport: claimed it "hangs the node rather than failing some other way". It does not — master rejects it at preprocess in ~1s, after go/types walks ~6M nodes. What the guard changes is when the rejection happens and how much unmetered CPU precedes it. Relabelled accordingly. Both corrected headers now state what they do NOT show, and point at the one fixture verified to wedge a node.
The alternatives section gave the fork one dismissive line resting on "go/types is stdlib here, not a fork". That premise is misleading: gnovm/pkg/parser is a fork of go/parser (5,460 lines) whose reason to exist is a metering hook, ParserCallback, driven by newParserCallback. Fork-and-hook is the established pattern here. The section now also concedes what the fork would buy, which was missing: metering measures where this change predicts, so it would delete cost(), the cost-guard role of both syntactic rejections, leafExpansionBound, the shim table, and the shared parse cache — along with the "cost() must never under-count" invariant entirely. The decision rests instead on size and shipping risk: go/types is 34,545 lines across 107 files, 6.3x the parser fork, and a forked type checker makes consensus-visible error text ours to keep byte-stable. It also answers the "go/types is going away, why add a layer" objection directly — this guard's lifetime is bounded by go/types' own, so disposal is symmetric and the fork is the larger layer. The corresponding PR comment carried the same weak argument and has been updated.
Four review angles over the Go diff. Applied: Reuse - The honest-code scan re-implemented typeExpansionCost's summation inline; it now calls it. A test that reimplements the formula it measures stops measuring it the moment the formula changes. - The two scan tests each had their own copy of the directory walk, the per-file parse and the "which dir is a package" rule (~55 lines); both now share gnoSrcRoots.resolver/pkgPaths. The shim was hand-parsed in two more places; both now use GoParseMemPackage, the function the production guard is fed by. - expansionGas uses this file's satMul instead of a second derived constant. Simplification - pkgDecls was a struct wrapping one map; it is now the map, dropping .byName from eleven sites. - Test source generators collapsed: doublingChain's lo was 1 at every call site; the pointer and slice table cases are chainSrc with a different element form; the union and semicolon interface cases are ifaceChainSrc with a different separator. Four near-identical inline builders became two parameterised ones. - declsFor stops descending once it has a TypeSpec — a type expression holds no further declarations. - visiting keys are deleted rather than set false, so the map no longer grows to mirror memo. Correctness found while cleaning - An unresolvable qualifier was priced at 1 while an unresolvable name was priced at leafExpansionBound. Same situation, opposite direction, so the qualifier was the one edge that under-charged. Both now route through unresolvedCost. - That change exposed a real defect in the calibration scan: it measured exported types from xxx_test packages, which no user package can name. io_test.Buffer was being scored as if it were stdlib API. The leaf scan is now prod-only, mirroring MPFProd on the deploy path — exported types measured drops 151 -> 117 and the maximum is 19 (regexp.Regexp) either way, so leafExpansionBound = 32 was never wrong, but the test was measuring a superset. - namedCost now memoizes the unresolved paths, which are a pure function of the key; previously every occurrence redid four lookups. Altitude - The two transaction-path type checks assembled TypeCheckOptions field by field, so the gas meter was opt-in and the default for any future message type was unmetered — the fail-open direction, and it already slipped once. Both now go through VMKeeper.txTypeCheckOptions. - errDotImports moved from the guard to preprocess.go, where the language rule lives: the guard exists only while go/types does, so the permanent enforcement site should not depend on the disposable one. - Recorded that gnoBuiltinShimExpansion is not keyed by gno version while makeGnoBuiltins is, that all three guards deliberately run over test files, and that memoizingGetter belongs in the transaction store. Verified: guard suite, sdk/vm gas tests, full TestTestdata, golangci-lint 0 issues on both modules, TestFiles unchanged at its 10-failure toolchain-drift baseline.
…n one tx Removing the per-package ceiling was justified by the bound holding per TRANSACTION rather than per message — Tx.Msgs is unbounded and ValidateBasic caps gas, not the message count. Nothing tested that. The existing coverage tests accumulation across a dependency GRAPH within one message (addpkg_typecheck_fanout_deps_gas.txtar) and that the charge is wired at all (TestVMKeeperAddPackage_TypeExpansionGasCharged), but not accumulation across messages, which is the property the ceiling's removal rests on. The new test asserts both halves, since only the pair means anything: four messages each fit the budget alone, and the same four sharing one budget run out partway (2 of 4 land). baseapp gives a tx one basicGasMeter(GasWanted) and then loops over msgs, so driving several AddPackage calls through one ctx and one finite meter is what a multi-message tx actually does. A per-package ceiling could not have caught this: every message here is modest on its own. The fixture holds SOURCE BYTES EQUAL between the two shapes, padding the pointer variant with a comment. That matters: the first version differed by "[0]" vs "*", making the value chain 20 bytes longer, and PreprocessGasPerByte alone carried the assertion — verified by unwiring the meter, where that version still passed. With bytes equal the gap is the expansion charge (1.43M per message, matching a depth-10 chain's ~14k nodes at 100 gas/node), and unwiring the meter now fails this test as well as the existing one.
…ng the removed ceiling
Two loose ends.
The file was named for a bound that no longer exists: nothing in the shipped design
rejects a package for being too complex, it computes a price. A reader looking for
the limit found only gas. typecheck_bound.go -> typecheck_cost.go (plus its two
test files), and parseBoundSrc/makeBoundResolver -> parseCostSrc/makeCostResolver.
leafExpansionBound keeps its name — it genuinely is an upper bound on stdlib
expansion. Git records all three as renames.
The ADR and comments also still narrated the ceiling's history — "an earlier
revision carried 1_000_000", "first per-type at 100_000". That version never
shipped, so the archaeology is noise for anyone reading the result. The ARGUMENT
for having no ceiling stays, since a reviewer needs it; only the history goes. Same
for the rate: the two-step derivation now warns that skipping calibration yields
~25 and a ~4x under-charge, without framing it as something a past revision did.
Also records the limitation this PR does not address: validType is now priced by
structure, but everything else in type-check and preprocess is priced by
PreprocessGasPerByte, which is only right for work proportional to source size —
and nobody has audited the other passes against that. The threshold is lower than
it looks: a merely QUADRATIC pass is already badly under-priced, since 1MB of
O(n^2) work is ~1e12 operations against the ~1.25e9 gas its bytes buy. Lists the
unverified candidates (constant arithmetic, method-set/interface satisfaction,
Identical on deep structural types) so the next person does not start from scratch,
and notes that the generics rejection incidentally closes the other known
super-linear behaviour in go/types.
The per-tx test's comment now also records why it cannot be an integration txtar:
every keyscli command hardcodes Msgs: []std.Msg{msg} (addpkg.go:135, call.go:133,
run.go:144), so gnokey — all a txtar can drive — cannot build a multi-message
transaction at all. The txtars cover the other accumulation, across a dependency
graph within one message.
Verified: guard suite, sdk/vm gas tests, full TestTestdata, golangci-lint 0 issues,
TestFiles unchanged at its 10-failure toolchain-drift baseline.
…after all
Retracts a claim I made in the previous commit. I wrote that this property "cannot
be an integration txtar" because every keyscli command hardcodes
Msgs: []std.Msg{msg}. That is true of maketx and irrelevant: broadcast.go:72-77
reads a JSON file into std.Tx, whose Msgs is a slice, and sign.go:140-152 signs an
arbitrary tx file. addpkg_multi_msg.txtar has been doing exactly a two-message
addpkg all along. The impossibility was asserted from one grep.
addpkg_typecheck_fanout_multi_msg.txtar now covers it over the real path — ante
handler, GasWanted, one basicGasMeter across msgs — via gnokey sign + broadcast on
a hand-written tx. Both halves share an 8,000,000 budget, so the only variable is
how many messages share it:
3 depth-10 value chains in one tx : exceeds 8,000,000, aborts mid-tx
1 identical package on its own : 4,323,324
It discriminates: with GasMeter removed from the keeper's TypeCheckOptions the
three messages cost 6,524,793 and the tx succeeds, failing the assertion. Both
figures are recorded in the header so the budget can be retuned safely.
Also corrects a number I had been quoting. I reported one such package at 1.87M
gas; end-to-end it is 4.32M. The keeper path excludes fee and signature work, so
citing its figure as the cost was misleading — and it is why the first budget I
picked sat below even a single message.
The in-process test stays, with its job stated rather than assumed: it holds source
bytes exactly equal between the two shapes, which a txtar cannot conveniently do,
so the expansion charge is provably the only difference rather than the likely one.
Each test now points at the other.
There was a problem hiding this comment.
Blocking: typeExpansionCost ranges over a map, and for a value-containment cycle the count depends on iteration order — the same package is priced at 3076 or 6136 nodes, 306,000 gas apart. Cause and fix inline.
Forks two ways: ABCIResult.Error is hashed into LastResultsHash, so at a GasWanted between the two totals one validator returns OutOfGasError and another ErrTypeCheck (58/80 vs 22/80 replaying one MsgAddPackage at GasWanted=800_000); and runTx charges GasConsumedToLimit() to the BlockGasMeter.
Only invalid recursive types are affected — I re-scored all 211 stdlib + example packages 15x, none unstable — but the charge lands before go/types rejects them, so one cheap malformed package is enough.
Repro below: six identical packages, GAS USED comes back 1636245 or 1942245 at random. 12/12 runs fail on this head, 12/12 pass with the suggestion.
go test ./gno.land/pkg/integration/ -run 'TestTestdata/addpkg_typecheck_fanout_cyclic_nondet' -count=1
addpkg_typecheck_fanout_cyclic_nondet.txtar
# The type-expansion charge is NOT deterministic for a value-containment cycle.
#
# namedCost returns 1 for a member it is already visiting and correctly does not
# memoize that. But every ANCESTOR on the cycle memoizes a value derived from that
# truncation into the shared c.memo, and which member gets truncated depends on
# which root typeExpansionCost's `range c.declsFor(entryPath)` reaches first. So
# the same source is priced at either 3076 or 6136 nodes -- 306000 gas apart.
#
# All six packages below are identical and all rejected by go/types for the same
# invalid recursive type, but the charge lands BEFORE go/types runs, so each is
# billed at random: GAS USED is 1636245 (1634985 on the last tx, which is 1260
# cheaper for unrelated reasons) or 1942245.
#
# On a real chain that is a divergent DeliverTx result between validators, and
# ABCIResult.Error is hashed into LastResultsHash.
gnoland start
! gnokey maketx addpkg -pkgdir $WORK/cyc1 -pkgpath gno.land/r/foobar/cyc1 -gas-fee 350001ugnot -gas-wanted 40_000_000 -chainid=tendermint_test $test1_user_addr
stdout 'GAS USED:\s+163'
! gnokey maketx addpkg -pkgdir $WORK/cyc2 -pkgpath gno.land/r/foobar/cyc2 -gas-fee 350001ugnot -gas-wanted 40_000_000 -chainid=tendermint_test $test1_user_addr
stdout 'GAS USED:\s+163'
! gnokey maketx addpkg -pkgdir $WORK/cyc3 -pkgpath gno.land/r/foobar/cyc3 -gas-fee 350001ugnot -gas-wanted 40_000_000 -chainid=tendermint_test $test1_user_addr
stdout 'GAS USED:\s+163'
! gnokey maketx addpkg -pkgdir $WORK/cyc4 -pkgpath gno.land/r/foobar/cyc4 -gas-fee 350001ugnot -gas-wanted 40_000_000 -chainid=tendermint_test $test1_user_addr
stdout 'GAS USED:\s+163'
! gnokey maketx addpkg -pkgdir $WORK/cyc5 -pkgpath gno.land/r/foobar/cyc5 -gas-fee 350001ugnot -gas-wanted 40_000_000 -chainid=tendermint_test $test1_user_addr
stdout 'GAS USED:\s+163'
! gnokey maketx addpkg -pkgdir $WORK/cyc6 -pkgpath gno.land/r/foobar/cyc6 -gas-fee 350001ugnot -gas-wanted 40_000_000 -chainid=tendermint_test $test1_user_addr
stdout 'GAS USED:\s+163'
-- cyc1/gnomod.toml --
module = "gno.land/r/foobar/cyc1"
gno = "0.9"
-- cyc1/cyc.gno --
package cyc1
type A struct{ x, y B }
type B struct{ x, y A }
type Z1 struct{ p, q A }
type Z2 struct{ p, q Z1 }
type Z3 struct{ p, q Z2 }
type Z4 struct{ p, q Z3 }
type Z5 struct{ p, q Z4 }
type Z6 struct{ p, q Z5 }
type Z7 struct{ p, q Z6 }
type Z8 struct{ p, q Z7 }
-- cyc2/gnomod.toml --
module = "gno.land/r/foobar/cyc2"
gno = "0.9"
-- cyc2/cyc.gno --
package cyc2
type A struct{ x, y B }
type B struct{ x, y A }
type Z1 struct{ p, q A }
type Z2 struct{ p, q Z1 }
type Z3 struct{ p, q Z2 }
type Z4 struct{ p, q Z3 }
type Z5 struct{ p, q Z4 }
type Z6 struct{ p, q Z5 }
type Z7 struct{ p, q Z6 }
type Z8 struct{ p, q Z7 }
-- cyc3/gnomod.toml --
module = "gno.land/r/foobar/cyc3"
gno = "0.9"
-- cyc3/cyc.gno --
package cyc3
type A struct{ x, y B }
type B struct{ x, y A }
type Z1 struct{ p, q A }
type Z2 struct{ p, q Z1 }
type Z3 struct{ p, q Z2 }
type Z4 struct{ p, q Z3 }
type Z5 struct{ p, q Z4 }
type Z6 struct{ p, q Z5 }
type Z7 struct{ p, q Z6 }
type Z8 struct{ p, q Z7 }
-- cyc4/gnomod.toml --
module = "gno.land/r/foobar/cyc4"
gno = "0.9"
-- cyc4/cyc.gno --
package cyc4
type A struct{ x, y B }
type B struct{ x, y A }
type Z1 struct{ p, q A }
type Z2 struct{ p, q Z1 }
type Z3 struct{ p, q Z2 }
type Z4 struct{ p, q Z3 }
type Z5 struct{ p, q Z4 }
type Z6 struct{ p, q Z5 }
type Z7 struct{ p, q Z6 }
type Z8 struct{ p, q Z7 }
-- cyc5/gnomod.toml --
module = "gno.land/r/foobar/cyc5"
gno = "0.9"
-- cyc5/cyc.gno --
package cyc5
type A struct{ x, y B }
type B struct{ x, y A }
type Z1 struct{ p, q A }
type Z2 struct{ p, q Z1 }
type Z3 struct{ p, q Z2 }
type Z4 struct{ p, q Z3 }
type Z5 struct{ p, q Z4 }
type Z6 struct{ p, q Z5 }
type Z7 struct{ p, q Z6 }
type Z8 struct{ p, q Z7 }
-- cyc6/gnomod.toml --
module = "gno.land/r/foobar/cyc6"
gno = "0.9"
-- cyc6/cyc.gno --
package cyc6
type A struct{ x, y B }
type B struct{ x, y A }
type Z1 struct{ p, q A }
type Z2 struct{ p, q Z1 }
type Z3 struct{ p, q Z2 }
type Z4 struct{ p, q Z3 }
type Z5 struct{ p, q Z4 }
type Z6 struct{ p, q Z5 }
type Z7 struct{ p, q Z6 }
type Z8 struct{ p, q Z7 }
The rest of the guard checks out: cost() never under-counts (diffed against a brute-force validType0 walk over 6,500 random single- and cross-package graphs), dot imports are closed, interface{ (A|B) } / (~A) are unreachable via go/parser, the stdlib leaf bound can't be gamed, and all chain-reachable type-check sites carry the meter.
| if c.visiting[k] { | ||
| // A value-containment cycle: an invalid recursive type that go/types | ||
| // detects and reports itself. Return a finite count so we neither loop | ||
| // nor pre-empt go/types' diagnostic. Deliberately not memoized — the | ||
| // truncated value is only valid inside this walk. | ||
| return 1 | ||
| } | ||
| c.visiting[k] = true | ||
| var best uint64 = 1 | ||
| for _, d := range specs { | ||
| if v := satAdd(1, c.cost(d.spec.Type, k.pkg, d.imports)); v > best { | ||
| best = v | ||
| } | ||
| } | ||
| delete(c.visiting, k) | ||
| c.memo[k] = best |
There was a problem hiding this comment.
Root cause. Returning 1 for a member already being visited, and not memoizing it, is right. But every ancestor on the cycle memoizes a value derived from that truncation into the shared c.memo on line 243 — and which member gets truncated depends on which root typeExpansionCost reaches first. So the charge follows Go map order.
There was a problem hiding this comment.
Confirmed, fixed in dfda73a. Exactly as you describe: the truncated 1 itself was never memoized, but every ancestor memoized a value derived from it, so the charge followed map order. Reproduced locally at exactly 3076 vs 6136 nodes (306k gas apart) before the fix.
| // validType runs once per DECLARATION, so the charge is the sum over them, not | ||
| // over names (a name declared twice is validated twice). Order-independent. | ||
| var total uint64 | ||
| for _, specs := range c.declsFor(entryPath) { | ||
| for _, d := range specs { | ||
| total = satAdd(total, satAdd(1, c.cost(d.spec.Type, entryPath, d.imports))) | ||
| } | ||
| } |
There was a problem hiding this comment.
Order-independent isn't true: the sum is, but c.memo isn't (see above).
Sorting the roots pins which member truncates. Changes no existing number — the calibration scans, -run Gas, and all seven addpkg_typecheck_* txtars still pass:
| // validType runs once per DECLARATION, so the charge is the sum over them, not | |
| // over names (a name declared twice is validated twice). Order-independent. | |
| var total uint64 | |
| for _, specs := range c.declsFor(entryPath) { | |
| for _, d := range specs { | |
| total = satAdd(total, satAdd(1, c.cost(d.spec.Type, entryPath, d.imports))) | |
| } | |
| } | |
| // validType runs once per DECLARATION, so the charge is the sum over them, not | |
| // over names (a name declared twice is validated twice). | |
| // | |
| // Sorted, not map order: namedCost truncates a containment cycle at whichever | |
| // member is reached first and every ancestor memoizes a value derived from that | |
| // truncation, so an unsorted walk prices the same package differently per run. | |
| // This count is charged as gas, so it must be identical on every node. | |
| var total uint64 | |
| decls := c.declsFor(entryPath) | |
| for _, name := range slices.Sorted(maps.Keys(decls)) { | |
| for _, d := range decls[name] { | |
| total = satAdd(total, satAdd(1, c.cost(d.spec.Type, entryPath, d.imports))) | |
| } | |
| } |
Worth a regression test pinning a cyclic package's count; nothing in the suite exercises one today.
There was a problem hiding this comment.
Applied as given in dfda73a, with two regression tests that both fail if the sort is removed: TestTypeExpansionCostCyclicIsDeterministic (300 iterations; unsorted reports the package priced 2 different ways) and your repro committed verbatim as addpkg_typecheck_fanout_cyclic_nondet.txtar. ac814c5 then moved the sort to construction: pkgDecls caches a sorted names slice next to the map and the loop iterates that, so the iteration site no longer touches a map at all — same order, no number moves. 33b0131 adds the remaining cycle topologies.
| "go/types" | ||
| "math" | ||
| "path" | ||
| "strconv" |
There was a problem hiding this comment.
Imports for the suggestion below.
| "go/types" | |
| "math" | |
| "path" | |
| "strconv" | |
| "go/types" | |
| "maps" | |
| "math" | |
| "path" | |
| "slices" | |
| "strconv" |
| continue | ||
| } | ||
| name = imp.Name.Name | ||
| } else if name = c.pkgName(impPath); name == "" { |
There was a problem hiding this comment.
Minor: declsFor runs over allgofs, so pkgName fetches imports appearing only in _test.gno files — which go/types never imports under ProdOnly. Deterministic, just an unintended store-read cost; maybe one line under "Known limits".
There was a problem hiding this comment.
Documented in dfda73a, in the ADR next to the resolver rationale ("Also unnecessary work: declsFor runs over allgofs..."). Left as a known limit rather than filtering here: deterministic and over-charge only, as you note.
| alloc := gnostore.GetAllocator() | ||
| // Per-tx preprocess allocator (see AddPackage for full rationale). | ||
| // Covers both the closure-local Machine that calls RunMemPackage and | ||
| // Covers both the dependency graph-local Machine that calls RunMemPackage and |
There was a problem hiding this comment.
Nit, unrelated to this PR: closure-local Machine became dependency graph-local Machine. Looks like a find/replace artifact — the original was correct.
There was a problem hiding this comment.
Reverted in dfda73a — a find/replace artifact from the earlier pass avoiding the word "closure"; it was the only occurrence. Good catch.
…ent cycles Blocking bug found by @omarsy in review. typeExpansionCost ranged over a map. namedCost truncates a containment cycle at whichever member it is already visiting and correctly does not memoize that 1 — but every ANCESTOR on the cycle memoizes a value DERIVED from the truncation, and which member truncates depends on which root is walked first. The same source was therefore priced at 3076 or 6136 nodes at random, 306,000 gas apart. Reproduced locally at exactly those figures. The count is charged as gas, so a node that disagrees forks: ABCIResult.Error is hashed into LastResultsHash, and runTx charges GasConsumedToLimit() to the BlockGasMeter. Only invalid recursive types diverge and go/types rejects those a moment later, but the charge lands BEFORE it does, so one cheap malformed package was enough. My own comment on that loop read "Order-independent." — I had noticed the truncated value was only valid within its walk and written that down, then failed to follow it one step to the enclosing memo entry. Fix is @omarsy's suggestion as given: walk slices.Sorted(maps.Keys(decls)). Confirmed it moves no existing number — honest maximum still 431 over 211 packages, stdlib maximum still 19, -run Gas unchanged. Two regression tests, both verified to FAIL with the sort removed: - TestTypeExpansionCostCyclicIsDeterministic: 300 iterations, one assertion. Unsorted it reports "priced 2 different ways (map[3076:88 6136:212])". - addpkg_typecheck_fanout_cyclic_nondet.txtar: the reviewer's own repro, included verbatim. Six samples over the real DeliverTx path — 6/6 runs fail unsorted, 5/5 pass sorted. Carries a maintenance note, since its assertions pin a GAS USED prefix that an unrelated gas change would shift; relaxing it to match both values is the bug it detects. Also from the same review: - keeper.go: "closure-local Machine" had become "dependency graph-local Machine", a find/replace artifact from the earlier pass avoiding the word "closure". That usage was correct — it means a Go closure. Reverted; it was the only one. - ADR records that declsFor runs over allgofs, so pkgName resolves imports appearing only in _test.gno files that ProdOnly never lets go/types import. Deterministic and only over-charges, but a store read the deploy would not otherwise make.
The three gno-security*.md docs are written for realm authors. Nothing covers changing the node itself, where the failure mode is not a bug in one contract but a chain split — so an agent or contributor touching GnoVM, the keeper or the tm2 store has no distilled rules to work from. Six bullets, each of which has already cost real debugging: - Nothing reaching consensus may depend on Go map iteration order. Called out explicitly as GO maps, because Gno's own maps iterate in insertion order (gno-data-structures.md) and intuition from writing .gno actively misleads here. - A commutative reduction is not a defence: memoizing a value derived from an early exit launders order-dependence into the total. This shipped in gnolang#5826 — a per-package gas count came out 3076 or 6136 nodes from identical source, 306k gas apart, and was caught in review rather than by a test. - Prove determinism by repetition, since Go randomizes map order per range and a single call proves nothing. - Why it is not a nit: ABCIResult.Error is hashed into LastResultsHash and runTx charges GasConsumedToLimit() to the BlockGasMeter, so two nodes that price a tx differently disagree on the block. - Charging gas before validation is fine, but then malformed input reaches the charge, so the charge must be deterministic for input that never survives. - 1 gas == 1ns is REFERENCE hardware, not the dev machine. gnovm/cmd/calibrate ships paired benchmark output for the conversion; skipping it under-charges by the machine ratio (~3x on Apple silicon). Docs-only, so no ADR. AGENTS.md invites exactly this kind of edit under "Improving This Document".
…ed map range Follow-up to the determinism fix. The charge was already deterministic — this changes how, for two measured reasons. pkgDecls now carries both a lookup map and a sorted []string of its keys, and typeExpansionCost iterates the slice. Two gains: - The iteration path no longer touches a map at all, so a future `range` over pkgDecls cannot silently reintroduce the consensus fork. The type now also has somewhere to state that contract, which a bare map did not. - Measurably faster: 1.50x at chain depth 100 (45.1 -> 30.1 us), 1.54x at 1000 (561 -> 364 us), 1.14x at 5000 (1836 -> 1612 us), medians of 3. The saving is the key-collection pass: slices.Sorted(maps.Keys(...)) walked the whole map through an iterator and grew a fresh slice on every call, whereas the names are now appended during the AST walk declsFor already performs. Where parsing dominates it makes no difference (211-package scan: 0.12s either way). Sorted, not declaration order. Declaration order is equally deterministic and needs no sort at all, but it is layout-sensitive: writing the two members of a containment cycle in the other order changes the price 2x — measured 3076 vs 6136 nodes on the same type graph. Sorting depends only on the set of names, so moving a declaration between files or reordering for readability moves nobody's gas. No number moves. Honest maximum still 431 over 211 packages / 731 types, largest exported stdlib type still 19 (regexp.Regexp), keeper pointer-vs-value delta still 736,055 gas, the cyclic fixture still 3076, and all eight addpkg_typecheck txtars pass including the one pinning a GAS USED prefix. Note this reinstates the pkgDecls struct that the earlier /simplify pass flattened for wrapping a single field. It now holds two, so the wrapper earns its keep.
Two additions to the consensus-safety section, both from reviewing how this class has actually behaved in this repo rather than from one incident. preprocess.go already sorts for the same reason twice: TestInitOrderDeterminism guards variable-initialization order against map iteration over dependency sets, and TestCircDepDeterminism exists because circular-dependency error messages came out in random order until the DFS was sorted. A map, a cycle and a consensus-visible output, in a different pass, predating gnolang#5826. So this is a documented recurrence, not an extrapolation from a single bug — which is the difference between a rule worth reading and a rule someone invented. And the part that let gnolang#5826 survive: determinism has to be tested on the REJECT path. Charging before validation means malformed input reaches the charge, and malformed input is where the order-sensitive branches are. A sweep over only valid fixtures is silent on the whole class, because nothing valid contains an invalid recursive type. Also notes that a determinism test should be checked against a reverted fix, since symmetric fixtures are stable under both orders and pass while the bug is live.
The single cyclic case only covered the shape the reviewer happened to report. Cycle truncation is the ONLY order-sensitive branch in cost() — every other branch is a pure function of its key, and satAdd/satMul saturate to a fixed point — so the topology of that cycle is the whole risk surface, and it deserves a table. Seven cases, verified against a reverted sort. Four detect the regression: 2-cycle, chain on one member 3076 / 6136 the reported one 2-cycle, chains on both members 844 / 1564 cycle across an import boundary 766 / 1510 previously uncovered 3-cycle with a chain 822 / 1578 / 3090 three-way, not binary Three cannot, and are labelled controls in the test: symmetric 2-cycle 32 equal weight either side self-reference by value 751 one type, no ordering choice acyclic control 2537 no cycle at all The cross-package case is the one worth adding on its own merits: visiting is keyed by (package, name), so the truncation can land in either package, and nothing else in the suite exercised that. The symmetric case is kept deliberately, as a documented control. It is stable under both orders, so a first attempt at this test built only from a bare A-B cycle passes while the bug is live — which is exactly what happened when I first tried to reproduce the reviewer's report and got a flat 32 over 200 runs. Both facts are recorded in the test comment, along with a note that a new case must be checked against a reverted sort: a case that cannot fail is documentation, not a test.
|
not sure if still needed given #6088. |
Charge for the unmetered, exponential
go/typesvalidTypewalk ataddpkg/MsgRunbefore it runs — a consensus DoS — and reject the syntax that makes pricing it unsound.validTypedoes not memoize visited types (commented out as a workaround for golang/go#65711), so a ~40-line "doubling" chain makes it visit ~2^41 nodes, unmetered and uninterruptible.Verified against master: that fixture wedges the node — it never answers, and after 60s
gnokeyfails withabci_query … context deadline exceededatbroadcast.go:245 - simulate tx. Note the read-only simulate path: no fee, no tx inclusion. Here the same input returnsout of gasin ~2s.Approach
typeExpansionCostcomputes the exact node countvalidTypewill visit, with the memoization it lacks (so computing it is linear), and the deploy path charges it toctx.GasMeter()at 100 gas/node — per package, before that package is walked.ConsumeGaspanics on out-of-gas andgo/typesre-panics non-bailoutvalues, so the abort lands before the remaining dependencies are walked; charging at the end would bill for CPU already spent.No ceiling. Any cap sits either below what a sender can pay, refusing packages that were paid for, or above it, where nothing is payable anyway and it only relabels an out-of-gas. It would also have to be per-package, a scope no budget has — whereas charging
ctx.GasMeter()makes the bound hold per transaction, which is what matters:Tx.Msgsis unbounded, and one message re-checks every dependency it imports, whose bytes earlier transactions paid for (measured: a 55-byte package pulls in 321,070 nodes against 68,750 gas of byte charges).Both syntactic rejections are preconditions of the pricing, not language policy: with no ceiling behind the charge, an under-counted edge is under-charging — the same DoS at a discount. So generics/type-sets and dot imports are refused rather than approximated, and unresolvable names are scored high (
leafExpansionBound) so they over-count.Calibration
1 gas == 1nsmeans reference hardware (Intel Xeon Platinum 8168, permachine.go'sOpCPU*table), so the rate needs two steps — an earlier revision shipped 25 by skipping the second, a ~4× under-charge.BenchmarkValidTypeWalkmeasures 30–40 ns/node on an Apple M5 (rising with depth as the working set outgrows cache; a DoS is the deep end), and rerunningcmd/calibrate'sBenchmarkAllocagainst its shipped Xeon output puts the Xeon 2.96× slower over 37 shared cases. 40 × ~2.5 = 100, range 88–128. The check that closes it: a whole block (MaxBlockMaxGas = 3e9) buys 3e7 nodes ≈ 3s ofvalidTypeon reference hardware.regexp.Regexp) →leafExpansionBound = 32.gnobuiltinsrealm/addressaddressis too common in the stdlib API to approximateAgainst the ~5e7
GasWanteda real deploy uses, 43k gas is not a meaningful tax. Three tests hold the wiring, and each fails if the meter is ever unwired again — verified by unwiring it. Two are in-process: one separates pointer-vs-value chains at equal declaration count by 736,055 gas, the other drives four messages through one ctx and one finite meter, where each fits the budget alone but only two land when they share it. Source bytes are held exactly equal there, so the expansion charge is provably the only difference. The third,addpkg_typecheck_fanout_multi_msg.txtar, proves the same per-transaction accumulation over the real path —gnokey sign+gnokey broadcaston a multi-message tx, sincemaketx addpkgonly builds single-msg ones: three depth-10 chains exceed an 8M budget that one identical package clears at 4.3M.Why not fork
go/typesand metervalidTypeinsideThe strongest alternative, and this repo already does it one layer up:
gnovm/pkg/parseris a fork ofgo/parser(5,460 lines) existing for itsParserCallbackmetering hook. It would also be better on soundness, since metering measures where this predicts — a metered fork deletescost(), both rejections' cost role,leafExpansionBound, the shim table, the parse cache, and the "must never under-count" invariant entirely.Rejected on size and shipping risk, not principle.
go/typesis 34,545 lines across 107 files, 6.3× the parser fork, and a type system rather than a token stream, so re-syncing per Go bump is qualitatively harder — which inverts "don't add a layer", since the fork is the larger layer. It would also make consensus-visible error text ours to keep byte-stable. On "go/typesis going away mid-term": this guard is ~450 lines whose lifetime is bounded bygo/types' own, so it is deleted rather than migrated — disposal is symmetric, and if removal slips the fork is the expensive thing to hold.Changed files
typecheck_cost.go(new) — cost model, rate, the two syntactic guards,expansionPkgCache, andexpansionGas, which clamps atMaxInt64becauseint64(math.MaxUint64)is-1and would otherwise refund gas for the worst package a sender could submit.gotypecheck.go—TypeCheckOptions.GasMeter, the three guards,memoizingGetter(store gas unchanged when guard and importer fetch the same package), sharedexpCache. Not computed at all without a meter, so off-chain callers skip resolving the dependency graph.sdk/vm/keeper.go—GasMeter: ctx.GasMeter()forAddPackageandRun.preprocess.go— both dot-import panics share the guard'serrDotImports; the two sites previously disagreed on capitalisation.go2gno.go—TODO(#6059)whereTypeParamsare dropped.AddPackagepath; andTestValidTypeWalkIsExponential, which proves the premise by running the unguarded walk in a subprocess that must be killed at its deadline while the metered path returns.gnovm/adr/pr5826_typecheck_dos_guards.md.Known limits (in the ADR, not fixed here)
gno test/gno lint/gnodev, so a pathological local package churns as long asgo builddoes. Chain-reachable code is bounded by what its deploy paid; CI jobs carrytimeout-minutes: 30.token.FileSetand on in-place AST mutations.go/typesdiagnostics, so such filetests pin two directives. Harness-only.Related
#5921 closed as duplicate; this PR is the sole carrier of
checkNoUncountableGenerics, which must sit beforego/types(verdict). #6059 is a different job: generics completeness belongs inGo2Gno, the traversal every consumer shares includinggno run, while this guard stays narrow. #5892 is complementary — it prices the linear case per source byte, which cannot catch the exponential one; it is stacked on #5891, which rewrites theGetMemPackageseam this resolver reads through, so re-verify on rebase. golang/go#65711 — ifvalidTypeis ever memoized upstream,TestValidTypeWalkIsExponentialfails and says to re-derive the rate.AI-assisted: developed with Claude Code; all measurements are reproducible from the committed tests and benchmarks.