Expand semantic syntax errors for invalid walruses

[charliermarsh]

Summary

Python rejects assignment expressions in several comprehension contexts, but we previously only reported the case in which a walrus rebound a comprehension variable.

As a result, we accepted invalid walruses in the iterable and class-body portions clause of a comprehension, and missed some rebinding cases in filter clauses and nested comprehensions. It turns out the rules are really complicated!

For example, we now report the two newly modeled syntax errors:

# error: [invalid-syntax] "assignment expression cannot be used in a comprehension iterable expression"
[x for x in (values := [1])]

class C:
    # error: [invalid-syntax] "assignment expression within a comprehension cannot be used in a class body"
    [(value := item) for item in [1]]

We also catch previously missed instances of the existing rebinding error:

# error: [invalid-syntax] "assignment expression cannot rebind comprehension variable"
[x for x in [1] if (x := 0)]

In general, I decided not to worry about emitting multiple diagnostics for a single error. These are rare, they're fatal, and it added a lot of complexity (so in some cases we may emit two errors for a walrus that is invalid for two reasons).

[astral-sh-bot]

Typing conformance results

No changes detected ✅

Current numbers

The percentage of diagnostics emitted that were expected errors held steady at 91.94%. The percentage of expected errors that received a diagnostic held steady at 87.09%. The number of fully passing files held steady at 92/134.

[astral-sh-bot]

Memory usage report

Memory usage unchanged ✅

[astral-sh-bot]

ecosystem-analyzer results

No diagnostic changes detected ✅

Full report with detailed diff (timing results)

[astral-sh-bot]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

[codspeed-hq]

Merging this PR will not alter performance

✅ 125 untouched benchmarks

---

_{Comparing charlie/walrus-syntax-errors (09158b7) with main (4a7ca06)}

[charliermarsh]

Tagging @ntBre on this one. (Not super urgent.)

[ntBre]

Looks good to me!

[ntBre]

Should we preserve this doc comment?

[ntBre]

Should we preserve this TODO somewhere too? Unless it doesn't seem useful to do it anymore.

[ntBre]

I think you could probably reuse the StoredNameFinder from the AST crate here, but the API is a little different.

ruff/crates/ruff_python_ast/src/helpers.rs

Lines 1143 to 1159 in d5e10cc

	/// A [`Visitor`] to collect all stored [`Expr::Name`] nodes in an AST.
	#[derive(Debug, Default)]
	pub struct StoredNameFinder<'a> {
	/// A map from identifier to defining expression.
	pub names: FxHashMap<&'a str, &'a ast::ExprName>,
	}
	impl<'a> Visitor<'a> for StoredNameFinder<'a> {
	fn visit_expr(&mut self, expr: &'a Expr) {
	if let Expr::Name(name) = expr {
	if name.ctx.is_store() {
	self.names.insert(&name.id, name);
	}
	}
	crate::visitor::walk_expr(self, expr);
	}
	}