Skip to content

Conversation

@ricardoV94
Copy link
Member

@ricardoV94 ricardoV94 commented Jun 7, 2024

What is broken / not supported by Numba:

The plan now

  • Numba becomes a default dependency
  • Like the default "Mode", "FAST_RUN" is now also an abstract mode that can use either C or Numba, depending on config.linker
    • For now "config.linker" will remain C but we'll switch to Numba soon.
    • Why Numba over C?
      • Faster
      • Easier to maintain/ extend
      • Easier to install (pip install should be safe
    • This way downstream libraries can test if anything would break with the switch, and user can switch back if it breaks stuff for them
    • CI will run with both configs. To avoid runtime explosion it may be time to restrict floatX and fast_compile tests to a few files instead of being an orthogonal parametrization like it is now

Immediate action

  • I'll be cherry picking the countless fixes into their own PRs. I need help with review as both me and @jessegrabowski are pretty much at capacity
  • This PR will end up with just the test tweaks and new config flags
  • We need to implement more Ops in numba mode. We'll follow up with a PR that fails on fallback to obj mode. Help needed!

@ricardoV94 ricardoV94 force-pushed the test_on_numba branch 2 times, most recently from 2eb7fe7 to 1f093cd Compare June 7, 2024 17:24
@ricardoV94
Copy link
Member Author

Locally I ran all the tests in tests/tensor/rewriting/test_basic.py twice (to allow caching) before and after making Numba default.
Before: 6s
After: 1m34s

Numba caching is baaaaaaaaaaaaaaaaaaaaaaaaaaaaaaad

@ricardoV94

This comment was marked as outdated.

@ricardoV94 ricardoV94 changed the title Test on numba Run whole test suite on numba backend Jun 7, 2024
@aseyboldt

This comment was marked as outdated.

@ricardoV94

This comment was marked as outdated.

@ricardoV94

This comment was marked as outdated.

@ricardoV94

This comment was marked as outdated.

@ricardoV94 ricardoV94 force-pushed the test_on_numba branch 4 times, most recently from 6f6e8bb to 53d9a26 Compare June 13, 2024 14:43
@ricardoV94 ricardoV94 added this to the 3.0 release milestone Feb 14, 2025
@ricardoV94
Copy link
Member Author

After caching the test file now runs in 50s after caching vs 6s before the PR, so only 8x slower now :(

Also fix infinity identities for unsigned integers
…) advanced indexing

When default `ignore_updates=True` for inc_subtensor, and boolean indices were rewritten during specialize
Also need to re-filter warnings because pytestmark.filterwarnings overrides library filters
Mark overly specific tests as xfail
* Allow linker to change at runtime, and affect both `Mode` and `FAST_RUN` modes
* Remove float32 parametrization from test suite. Tests that care about this should cover it explicitly
@ricardoV94
Copy link
Member Author

ricardoV94 commented Dec 12, 2025

The tests are passing with all 3 modes: Numba, C and FAST_COMPILE.

I removed the floatX parametrization. It's a common source of wasted resources / contributor.confusion (assert_allclose, and test values failures) and I don't think it gives much in return.

Better for tests to be conscious and try explicitly when relevant

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Give test matrix readable names

2 participants