Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overflow happened on: -2147483648 % -1 #1412

Open
wForget opened this issue Feb 17, 2025 · 4 comments · May be fixed by #1416
Open

Overflow happened on: -2147483648 % -1 #1412

wForget opened this issue Feb 17, 2025 · 4 comments · May be fixed by #1416
Labels
bug Something isn't working

Comments

@wForget
Copy link
Member

wForget commented Feb 17, 2025

Describe the bug

-2147483648 % -1 evaluates to 0 in scala, but fails in rust.

scala:

Image

rust:

Image

Steps to reproduce

reproduce with spark sql:

  test("Overflow happened on: -2147483648 % -1") {
    withTable("t1") {
      sql("create table t1(c1 int, c2 short) using parquet")
      sql("insert into t1 values(-2147483648, -1)")
      checkSparkAnswerAndOperator("select c1 % c2 from t1")
    }
  }

Expected behavior

No response

Additional context

No response

@andygrove
Copy link
Member

Nice find. Thanks @wForget. Do you plan on working on a fix?

@wForget
Copy link
Member Author

wForget commented Feb 18, 2025

Nice find. Thanks @wForget. Do you plan on working on a fix?

I will try to fix it.

@kazuyukitanimura
Copy link
Contributor

It looks addition is working, I wonder what would be the difference

checkSparkAnswerAndOperator("select c1 + c2 from t1")

@parthchandra
Copy link
Contributor

Info on rust integer overflow: https://github.com/rust-lang/rfcs/blob/master/text/0560-integer-overflow.md

The operations +, -, *, can underflow and overflow. When checking is enabled this will panic. When checking is disabled this will two's complement wrap.
The operations /, % for the arguments INT_MIN and -1 will unconditionally panic. This is unconditional for legacy reasons.
Shift operations (<<, >>) on a value of width N can be passed a shift value >= N. It is unclear what behaviour should result from this, so the shift value is unconditionally masked to be modulo N to ensure that the argument is always in range.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
4 participants