2026-07-16 Engineering Team

[joecastiglione]

Agenda

• EET discussion followup and immediate next steps
• SANDAG Sharrow Example full scale performance test results
• SANDAG shadow price fix and results

Notes

EET (Explicit Error Terms) Follow-Up

• Continued discussion held over from Tuesday's call, where time was split with a CTPS presentation and didn't allow David to present.
• David presented results from the employment scenario (more complex than the transit scenario since accessibility changes propagate across all zones, not just an affected corridor).
• With the RNG seeding bug fixed (identified by Jan), disaggregate accessibilities for unaffected zones are now stable between base and build runs — a major improvement over the prior phase.
• Overall conclusion: EET is working as expected and is substantially production-ready, consistent with Jan's conclusion from Tuesday's presentation.
• One small, unresolved anomaly: 68 of ~3 million people show a different non-mandatory tour frequency choice in unaffected zones after controlling for known upstream causes (school escorting sort stability, shadow-price-driven school location changes). Leading hypothesis is machine-epsilon/rounding differences propagating through close-utility alternatives; David could not reproduce his own results exactly and couldn't reconcile them with Sijia's run either.
• Group agreed this level of noise is far smaller than noise introduced elsewhere (e.g., assignment iteration) and not worth further consortium resources beyond what remains in David's fixed-fee task budget.
• Jan will file GitHub issues/PRs for two related follow-ups (not blocking the current merge):
  • Sampling mechanism for desired accessibility (differing sampling utilities cause unintuitive changes across scenarios).
  • A MAZ pre-sampling fix requiring a counter-based RNG, isolated from the main choice RNG.
• Recommendation: merge the current EET pull request now (well-tested, already reviewed against detailed scenarios); treat the pre-sampling RNG fix as a separate follow-up PR.
• Need to reconcile branches: merge main into the ActivitySim explicit-error-terms branch (currently diverged from the outer-loop version). Team agreed to retarget the PR to ActivitySim main directly, which should also simplify away unrelated three-zone-model changes currently showing up in the diff.

SANDAG Shadow Price Fix Review

• Reviewed SANDAG results that reflect Sijia's shadow-pricing bug fix results: fit between choosers and target locations improved significantly (before vs. after comparison), but some anomalies remain:
  • A few zones show workers/students assigned despite little or no employment/enrollment target — possibly related to the external workplace location model not being filtered out of the summary.
  • Large aggregate mismatches for university and preschool segments (e.g., preschool target of 4.2M vs. ~228K modeled) — explained by shadow pricing normalizing supply/demand to shares rather than matching absolute totals, and by SANDAG's land-use summary not directly capturing preschool enrollment (proxied by other variables).
  • Discovered that shadow-pricing targets are not actually segmented by occupation as configuration naming suggests — all occupation segments are being combined into a single total-employment target. Sijia traced the SOC-code-to-segment mapping to a shared constants.yaml file outside the shadow pricing config.
  • Jeff flagged a possible divide-by-zero/undefined-gradient issue causing some zero-employment zones to never get corrected during shadow price iteration.
• No CI test currently exists for the other two shadow pricing methods (CT-RAMP, DaySim); only the simulation-based method has zone-reopening test coverage.
• Broader point raised: the fact that sophisticated agency users didn't catch this segmentation quirk sooner suggests a documentation/configuration clarity issue — flagged as an issue for the product team to discuss, not just engineering.

Other Open Pull Requests

• ~8 additional open PRs exist, mostly with AI-generated or minor human review comments; none are merge-ready as-is.
• Team agreed the EET PR should be prioritized for review/merge before tackling the others, since it will simplify subsequent work.

Action Items

• David: Do a final review of the EET PR before merge; check remaining budget on the fixed-fee task and email Joe with options for any leftover funds; prioritize merging main into the explicit-error-terms branch in the next couple of days; consider expanding CI test coverage to CT-RAMP and DaySim shadow pricing methods.
• Jan: Retarget the explicit-error-terms PR to ActivitySim main; open a GitHub issue on the desired-accessibility sampling mechanism; open a separate PR/issue for the pre-sampling RNG fix.
• Jeff: Continue triaging remaining open PRs; keep EET review/merge as top priority.
• Joe: Reach out to Bhargav at SANDAG regarding the shadow-pricing anomalies (especially the two zero-employment zones) and to discuss validating CT-RAMP and DaySim shadow pricing implementations.
• All: Reconvene next week.