feat(ducktyping): complete NativeAOT parity closure with strict compatibility gates#8252
Draft
tonyredondo wants to merge 67 commits intomasterfrom
Draft
feat(ducktyping): complete NativeAOT parity closure with strict compatibility gates#8252tonyredondo wants to merge 67 commits intomasterfrom
tonyredondo wants to merge 67 commits intomasterfrom
Conversation
- Add byref/reverse/generic/struct/value-type/type-conversion fixture set and parity tests - Improve emitter conversion parity for object/enum/interface cases and byref paths - Emit generated proxy metadata in compatibility artifacts - Expand trimmer descriptor roots to include bootstrap, mapped proxy/target, and generated proxy types - Add AOT engine tests for reverse registration and generated-type contract validation
- enforce one generated AOT registry assembly identity per process and throw on conflicts - add dedicated exception and coverage for cross-assembly registration attempts - add DuckTypeAotEngine.ResetForTests() and use it in AOT test suites to isolate per-test generated registries Tests: - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release -f net8.0 --filter FullyQualifiedName~DuckTypeAotEngineTests - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release -f net8.0 --filter FullyQualifiedName~DuckTypeAotProcessorsTests
- compare registry source using assembly FullName + module MVID instead of simple name - keep per-process single-registry enforcement while preventing same-name assembly collisions - add regression test for same simple-name but different assembly identity conflict Tests: - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release -f net8.0 --filter FullyQualifiedName~DuckTypeAotEngineTests - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release -f net8.0 --filter FullyQualifiedName~DuckTypeAotProcessorsTests
- add AOT runtime contract and registry metadata types - add DuckType.ValidateAotRegistryContract() and engine-side schema/runtime validation - normalize registry assembly identity comparison to canonical name/version+mvid - emit contract validation IL call in generated bootstrap before proxy registration - extend AOT engine and processor tests for contract validation and bootstrap call presence Tests: - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release -f net8.0 --filter FullyQualifiedName~DuckTypeAotEngineTests - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release -f net8.0 --filter FullyQualifiedName~DuckTypeAotProcessorsTests
- Parse and validate --generic-instantiations entries (closed generic only) - Track generic roots in mapping resolution and emit in manifest/linker descriptor - Reject open generic mappings during NativeAOT generation - Add processor tests for parsing, successful root emission, and open-generic failure Tests: - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --framework net8.0 --filter FullyQualifiedName~DuckTypeAotProcessorsTests - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --framework net8.0 --filter FullyQualifiedName~DuckTypeAotEngineTests
- Add scenarioId support in map/catalog mapping models and propagate to generated artifacts - Enforce scenario identity consistency and duplicate scenario detection during mapping resolution - Extend verify-compat with optional --mapping-catalog coverage checks and stricter matrix validation - Add deterministic mappingIdentityChecksum to compatibility matrix and manifest mappings - Expand DuckTypeAot processor tests for scenario propagation, verify-compat catalog checks, and checksum assertions Tests: - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --framework net8.0 --filter FullyQualifiedName~DuckTypeAotProcessorsTests - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --framework net8.0 --filter FullyQualifiedName~DuckTypeAotEngineTests
- Add verify-compat manifest validation (matrix/manifest key+checksum+scenario consistency) - Add optional strict assembly fingerprint validation for manifest verification - Mark explicit closed-generic mappings as unsupported with deterministic diagnostic - Fix nullable analysis warnings in AOT verify/generic parser/mapping helpers - Add netcoreapp2.1 AssemblyLoadContext shim for DuckTypeAot processor tests Tests: - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter "FullyQualifiedName~VerifyCompatProcessorShould" -m:1 - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter "FullyQualifiedName~DuckTypeAotProcessorsTests" -m:1 - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter "FullyQualifiedName~DuckTypeAot" -m:1 - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --framework net8.0 --filter "FullyQualifiedName~DuckTypeAotEngineTests"
- Add closed-generic emission path for mappings where target type is directly assignable to proxy type - Emit deterministic direct-cast activators and register closed generic Type tokens in bootstrap - Keep non-assignable closed-generic mappings explicitly unsupported with DTAOT0211 - Add closed-generic test coverage for both compatible and unsupported cases - Add shared closed-generic test types in dedicated files Tests: - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --framework net10.0 --filter "FullyQualifiedName~GenerateProcessorShouldSupportClosedGenericMappingsWhenTargetIsDirectlyAssignable" -m:1 - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter "FullyQualifiedName~GenerateProcessorShouldSupportClosedGenericMappingsWhenTargetIsDirectlyAssignable" -m:1 - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter "FullyQualifiedName~GenerateProcessorShouldReportClosedGenericMappingsAsUnsupportedWhenDuckAdaptationIsRequired" -m:1 - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter "FullyQualifiedName~DuckTypeAotProcessorsTests" -m:1 - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter "FullyQualifiedName~DuckTypeAot" -m:1 - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --framework net8.0 --filter "FullyQualifiedName~DuckTypeAotEngineTests"
- Add processor test for reverse closed-generic mappings where delegation type is directly assignable - Keep compatibility assertions aligned with closed-generic forward/reverse fast path behavior Tests: - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter "FullyQualifiedName~GenerateProcessorShouldSupportReverseClosedGenericMappingsWhenDelegationTypeIsDirectlyAssignable" -m:1 - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter "FullyQualifiedName~DuckTypeAotProcessorsTests" -m:1 - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter "FullyQualifiedName~DuckTypeAot" -m:1 - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --framework net8.0 --filter "FullyQualifiedName~DuckTypeAotEngineTests"
- add --scenario-inventory option and inventory parser/validation - enforce required scenario coverage and reject untracked matrix IDs - add verify-compat tests for matching/missing/untracked scenarios - add scaffold inventory file with A-01..E-42, FG/FS/FF/FM/RT groups, EX-01..EX-20, TX-A..TX-T Tests: dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotProcessorsTests
- add --require-mapping-catalog flag to ducktype-aot generate - fail generation when required catalog is missing or empty - cover required-catalog behavior with new processor tests Tests: dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotProcessorsTests
- add ordered remaining-work checklist under docs/development/for-ai - add initial Bible mapping-catalog artifact in DuckTyping test compatibility assets - add dynamic-vs-AOT differential parity scaffold tests (A-01, A-02, D-34) - validate checked-in catalog file parsing in DuckTypeAot processors tests Tests: - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotDifferentialParityTests - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotProcessorsTests
- extend differential dynamic-vs-AOT parity scenarios to A-03, A-04, B-16, C-28, D-35 - expand checked-in Bible mapping catalog seed entries to match new scenarios - strengthen catalog artifact validation test assertions in DuckTypeAot processors tests Tests: - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotDifferentialParityTests - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotProcessorsTests
- add differential Dynamic-vs-AOT scenarios for alias methods, nullable returns, generic methods, DuckField bindings, nullable chaining, and reverse methods - expand Bible mapping catalog seed entries to include new scenario IDs and mappings - tighten catalog validation assertions in DuckTypeAotProcessorsTests Tests: - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotDifferentialParityTests - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotProcessorsTests
- Complete explicit A-01..E-42 differential parity coverage\n- Add IL-atlas anchor scenarios (FG/FS/FF/FM/RT) with catalog wiring\n- Update runner mapping-catalog assertions and checklist checkpoint\n\nTests:\n- dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotDifferentialParityTests\n- dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotProcessorsTests
- Reintroduce RT-3 differential parity case with supported conversion-only return shape\n- Wire RT-3 mapping-catalog entry and runner catalog assertions\n- Update remaining checklist IL-atlas anchor status\n\nTests:\n- dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --framework net8.0 --filter FullyQualifiedName~DifferentialParityRT3ReturnConversionShouldMatchBetweenDynamicAndAot\n- dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotDifferentialParityTests\n- dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotProcessorsTests
Add differential parity coverage for EX-01..EX-20 and register scenarios in the mapping catalog and runner checks. Complete IL atlas scenario expansion and checklist progress for items 1/2/5. Tests: dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotBibleExamplesParityTests; dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotDifferentialParityTests; dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotProcessorsTests
Add differential parity tests for all test-adapted Bible excerpts (TX-A through TX-T), wire the scenarios into the required mapping catalog, and extend runner catalog assertions to enforce TX coverage and updated catalog size. Tests: dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotBibleExcerptsParityTests; dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotDifferentialParityTests|FullyQualifiedName~DuckTypeAotBibleExamplesParityTests|FullyQualifiedName~DuckTypeAotBibleExcerptsParityTests; dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --filter FullyQualifiedName~DuckTypeAotProcessorsTests
…overage - wire RunDuckTypeAotCompatibilityGate into managed unit-test flow and project wiring - add expected-outcomes contract file and verify-compat failure-mode/contract checks - add NativeAOT publish integration test validating generated registry execution with no dynamic code/assembly emit - update remaining checklist to mark publish integration and overall sequence complete Tests: - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --framework net8.0 --filter FullyQualifiedName~DuckTypeAot
…overage - remove runtime generic reflection activator path from DuckTypeAotEngine registration - add untyped activator fallback in DuckType.CreateTypeResult for AOT-safe instance creation - extend NativeAOT publish integration to validate forward, reverse, and DuckCopy mappings - update remaining-checklist checkpoint text for expanded publish coverage Tests: - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --framework net8.0 --filter FullyQualifiedName~DuckTypeAot - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --framework net8.0 --filter FullyQualifiedName~DuckTypeAot - dotnet run --project tracer/build/_build/_build.csproj -- RunDuckTypeAotCompatibilityGate --skip RunManagedUnitTests
…uide - Restore dynamic DuckCast(Type) error wrapping semantics while keeping AOT activator support\n- Enforce green-only full-suite parity (dynamic + aot exit success and zero failed/error outcomes)\n- Add dynamic mapping discovery recorder and test runtime bootstrap env wiring\n- Add full NativeAOT DuckTyping usage guide and link from DuckTyping.md\n- Update NativeAOT parity stabilization plan status and workflow commands
- Add end-to-end create/run guide for an AOT DuckTyping sample app\n- Include project setup, map generation, registry emission, run, and NativeAOT publish steps\n- Document expected runtime output checks
- Register and replay AOT mapping failures with dynamic-equivalent exception types - Enforce FallbackToBaseTypes semantics for private base property/field lookup in AOT emitter - Restore AOT registry bootstrap after reset-heavy tests to avoid cross-test state pollution - Add nullable value-type struct-copy regression coverage and adjust value-type field accessor expectation to dynamic semantics Validation: - DD_RUN_DUCKTYPE_AOT_FULL_SUITE_PARITY=1 parity integration passed - DuckTypeAot runner subset passed - Full Datadog.Trace.DuckTyping.Tests in AOT mode passed
- harden runtime type/assembly resolution used by AOT emission and compatibility checks - split bootstrap registration IL into chunks and avoid eager proxy type loads - update parity integration matrix harness to select compatible dotnet host per TFM and resolve runtime assemblies
- Move DuckType AOT APIs/runtime-mode logic into a new partial file - Keep only required integration hooks in DuckType.cs (AOT routing and cache/activator behavior) - Restore XML-doc changes in existing DuckTyping files and keep AOT exceptions in DuckTypeExceptions.cs Tests: dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj --filter "FullyQualifiedName~DuckTypeAotEngineTests"
- Make DuckType runtime mode atomic and AOT lookup exact-match only - Expand compatible AOT registrations at generation time and use generated failure throwers - Align verify-compat/build gates/docs with the NativeAOT contract and refresh Bible mappings Validation: - dotnet build tracer/src/Datadog.Trace/Datadog.Trace.csproj -c Release --no-restore - dotnet build tracer/src/Datadog.Trace.Tools.Runner/Datadog.Trace.Tools.Runner.csproj -c Release --no-restore - dotnet build tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --no-restore - dotnet build tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --no-restore - dotnet test tracer/test/Datadog.Trace.DuckTyping.Tests/Datadog.Trace.DuckTyping.Tests.csproj -c Release --framework net8.0 --no-build --filter FullyQualifiedName~DuckTypeAotEngineTests - dotnet test tracer/test/Datadog.Trace.Tools.Runner.Tests/Datadog.Trace.Tools.Runner.Tests.csproj -c Release --framework net8.0 --no-build --filter "FullyQualifiedName~GenerateProcessorShouldEmitAssignableAliasRegistrationsForDerivedRuntimeTargets|FullyQualifiedName~VerifyCompatProcessorShouldAllowMissingCompatReportWhenAllMappingsAreCompatible" - manual ducktype-aot discover/generate/verify-compat Bible gate against Datadog.Trace.DuckTyping.Tests net10.0
- Normalize bootstrap IL assertions to scan registration chunks instead of only Initialize - Accept Datadog.Trace in IgnoresAccessChecksTo expectations after generated failure helpers - Keep the expanded AOT runner suite green after the NativeAOT registry changes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds NativeAOT-ready DuckTyping for
Datadog.Traceby moving DuckTyping proxy generation to build time and registering those proxies at runtime with strict parity checks against the dynamic implementation.The final model is intentionally simple:
--map-file),ducktype-aot discover-mappings),ducktype-aot generate),ducktype-aot verify-compat --failure-mode strict).ducktype-aot generatenow also supports a one-step mode (--discover-mappings) that discovers compatible mappings and writes--map-filebefore generation.Why
NativeAOT support for
Datadog.Traceis being built incrementally. DuckTyping was a major blocker because dynamic runtime emit is not compatible with NativeAOT constraints.This PR closes that gap with deterministic AOT artifacts and strict compatibility validation.
End result
1) Single canonical mapping contract
ducktype-aot discover-mappingsscans type-level mapping attributes ([DuckType],[DuckCopy],[DuckReverse]) and writes canonical map output.ducktype-aot generateconsumes that canonical--map-file.ducktype-aot verify-compatvalidates compatibility against the same--map-fileand fails in strict mode on:2) Simpler CLI surface
--target-assemblyfrom both:ducktype-aot discover-mappingsducktype-aot generate--target-folderinputs.--target-filterremains available to constrain scan scope when needed.ducktype-aot generate --discover-mappings ... --map-file ... --output ...3) Runtime/emitter parity improvements
[DuckAsClass]interface proxies remain class proxies.RuntimeMethodHandle.IDuckTypeboundaries is preserved for parity.get_/set_emission).4) Strict gate and deterministic artifacts
5) Tests and docs
generate --discover-mappingsflow.Quick usage
One-step generate (discover + generate)
dotnet tracer/src/Datadog.Trace.Tools.Runner/bin/Release/Tool/net8.0/Datadog.Trace.Tools.Runner.dll \ ducktype-aot generate \ --discover-mappings \ --proxy-assembly /abs/path/ProxyContracts.dll \ --target-folder /abs/path/bin/Release/net8.0 \ --target-filter "*.dll" \ --map-file /abs/path/ducktype-aot-map.json \ --output /abs/path/Datadog.Trace.DuckType.AotRegistry.dllExplicit two-step flow
Verify strict compatibility
Reviewer focus
Please focus on:
DuckTypeAotRegistryAssemblyEmitter(proxy shape/IL + activator emission),DuckTypeAotEngine(typed activator registration/materialization),--discover-mappings) and map-file semantics,