Skip to content

fix: assorted operation bugs (softmax, enforce_type, merge_union_of_records, and more)#4107

Open
henryiii wants to merge 3 commits into
mainfrom
henryiii/fix-operations-batch
Open

fix: assorted operation bugs (softmax, enforce_type, merge_union_of_records, and more)#4107
henryiii wants to merge 3 commits into
mainfrom
henryiii/fix-operations-batch

Conversation

@henryiii

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

This batches up a set of small, mostly independent bug fixes in src/awkward/operations/ (plus one in _connect/rdataframe/) found during an automated review.

Correctness bugs

  • ak.softmax (ak_softmax.py): the default axis=None always raised TypeError (maybe_posaxis does axis >= 0 on None), so ak.softmax never worked without an explicit axis despite the docstring claiming None meant the innermost dimension. Default is now axis=-1, the docstring is corrected, and axis=None raises a clear ValueError.
  • ak.enforce_type (ak_enforce_type.py): the regular→regular branch of _type_is_enforceable was inverted — equal sizes reported not enforceable and unequal sizes recursed. Swapped so equal sizes recurse into the content and unequal sizes are rejected, matching _enforce_type. This drives packing decisions in the option/indexed/union paths.
  • ak.merge_union_of_records (ak_merge_union_of_records.py): in the indexed branch the outer union index must come from layout.index.data; it was self-indexing the inner content index (content.index.data[content.index.data[...]]), raising IndexError on a union containing a categorical/indexed array of records. Now uses content.index.data[layout.index.data[is_this_tag]] (indexed) and layout.index.data[is_this_tag] (non-indexed).
  • ak.argcartesian (ak_argcartesian.py): the list-input branch built ak.local_index(x), dropping the user's axis, while the dict branch correctly used ak.local_index(x, axis). Now both honour axis. (Updated tests/test_2596_named_axis.py to the now-consistent named-axis output, which matches the dict-input branch.)
  • ak.from_raggedtensor (ak_from_raggedtensor.py): _recursive_call recursed with the same count, causing RecursionError for tensors with three or more ragged dimensions. Now passes count + 1.
  • ak.to_parquet_dataset (ak_to_parquet_dataset.py): the not-a-directory error concatenated a set literal (+ {__file__}), raising TypeError on that error path. Removed.
  • ak.to_layout (ak_to_layout.py): the invalid-string_policy error interpolated {primitive_policy!r}; now reports {string_policy!r}.

Dispatch-protocol gaps

Third-party __awkward_function__ dispatch never saw array-like secondary arguments:

  • ak.unflatten (ak_unflatten.py): now yields array, counts (counts is array-like).
  • ak.nan_to_num (ak_nan_to_num.py): now also yields nan, posinf, neginf.

attrs threading

  • ak.from_rdataframe (ak_from_rdataframe.py, _connect/rdataframe/from_rdataframe.py): documented/accepted attrs=None but never forwarded it; attrs is now threaded through to the output array the same way behavior is. (ROOT is unavailable in this environment, so this was fixed by inspection.)

Cleanups

  • ak.count_nonzero (ak_count_nonzero.py): removed a duplicated HighLevelContext unwrap block.
  • ak.to_json (ak_to_json.py): removed a no-op except Exception as err: raise err from err wrapper.
  • ak.round (ak_round.py): replaced a copy-pasted "Returns the real components..." docstring with a correct one-liner.
  • ak.from_safetensors (ak_from_safetensors.py): the ImportError message said "to use ak.from_tensorflow"; now says ak.from_safetensors.

Tests

A new regression test file covers: softmax default vs. an explicit numpy softmax on a ragged array (and axis=None raising); enforce_type regular-size cases consistent with _type_is_enforceable; merge_union_of_records on a categorical/indexed record union; argcartesian list-vs-dict parity at axis=1; from_raggedtensor's _recursive_call for three ragged dimensions; and unflatten/nan_to_num dispatch via a dummy __awkward_function__ object.

AI assistance

This change was produced with AI assistance (Claude Code, via an automated multi-agent review of the operations module). Fixes were verified by running the new and directly-related existing test suites; the from_rdataframe, from_raggedtensor, and from_safetensors paths whose optional dependencies (ROOT, TensorFlow) are unavailable locally were addressed by inspection (from_raggedtensor's recursion is additionally covered by a duck-typed unit test).

🤖 Generated with Claude Code

@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.29630% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 85.38%. Comparing base (712dac0) to head (eb49445).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/awkward/operations/ak_to_parquet_dataset.py 0.00% 1 Missing ⚠️

❌ Your patch check has failed because the patch coverage (96.29%) is below the target coverage (98.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
Files with missing lines Coverage Δ
src/awkward/_connect/rdataframe/from_rdataframe.py 97.14% <ø> (ø)
src/awkward/operations/ak_argcartesian.py 83.33% <100.00%> (ø)
src/awkward/operations/ak_count_nonzero.py 96.55% <ø> (-0.23%) ⬇️
src/awkward/operations/ak_enforce_type.py 83.16% <100.00%> (+1.00%) ⬆️
src/awkward/operations/ak_from_raggedtensor.py 29.54% <ø> (+6.81%) ⬆️
src/awkward/operations/ak_from_rdataframe.py 92.00% <100.00%> (ø)
src/awkward/operations/ak_from_safetensors.py 88.88% <ø> (ø)
...rc/awkward/operations/ak_merge_union_of_records.py 90.24% <100.00%> (+8.13%) ⬆️
src/awkward/operations/ak_nan_to_num.py 98.38% <100.00%> (ø)
src/awkward/operations/ak_round.py 100.00% <ø> (ø)
... and 5 more

... and 1 file with indirect coverage changes

@henryiii henryiii force-pushed the henryiii/fix-operations-batch branch from 6e3badd to eb49445 Compare June 11, 2026 19:46
@TaiSakuma TaiSakuma added the type/fix PR title type: fix (set automatically) label Jun 12, 2026
henryiii added 3 commits July 7, 2026 13:54
…ecords, and more)

- ak.softmax: default axis was None, which always raised TypeError; default
  to axis=-1, update the docstring, and raise a clear error for axis=None.
- ak.enforce_type: fix inverted regular->regular enforceability check in
  _type_is_enforceable (equal sizes now recurse, unequal sizes are rejected),
  matching _enforce_type.
- ak.merge_union_of_records: use the outer union index (layout.index.data)
  when rewriting indexed/non-indexed union contents, fixing an IndexError on
  unions containing categorical/indexed records.
- ak.argcartesian: honour the requested axis for list inputs (was dropped),
  matching the dict-input branch.
- ak.from_raggedtensor: recurse with count + 1 to avoid RecursionError for
  tensors with three or more ragged dimensions.
- ak.to_parquet_dataset: drop set-literal concatenation in the not-a-directory
  error (was raising TypeError instead of ValueError).
- ak.to_layout: report the invalid string_policy value, not primitive_policy.
- ak.count_nonzero: remove duplicated HighLevelContext unwrap block.
- ak.to_json: remove no-op 'except Exception as err: raise err from err'.
- ak.round / ak.from_safetensors: fix copy-pasted docstring / error message.
- ak.unflatten / ak.nan_to_num: yield array-like secondary arguments to the
  __awkward_function__ dispatch protocol.
- ak.from_rdataframe: thread attrs through to the output array.

Assisted-by: ClaudeCode:claude-opus-4.8
Assisted-by: ClaudeCode:claude-opus-4.8
- Convert _Dispatcher helper class to _make_dispatch_sentinel() factory
  function using types.SimpleNamespace + closure
- Remove comments that merely restate the surrounding code
- Rebase was clean (no conflicts)

Assisted-by: ClaudeCode:claude-sonnet-4-6
@henryiii henryiii force-pushed the henryiii/fix-operations-batch branch from eb49445 to 82f6ce9 Compare July 7, 2026 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type/fix PR title type: fix (set automatically)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants