Skip to content

fix: enable full decimal to decimal support #1385

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

himadripal
Copy link
Contributor

@himadripal himadripal commented Feb 11, 2025

Closes #375

  • enable decimal to decimal
  • remove hard coded castoptions to pass to native execution
  • fixed castTest to match arrow invalid argument error with spark's Number out of range error.

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

How are these changes tested?

@codecov-commenter
Copy link

codecov-commenter commented Feb 11, 2025

Codecov Report

Attention: Patch coverage is 50.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 58.93%. Comparing base (f09f8af) to head (fb25fd9).
Report is 67 commits behind head on main.

Files with missing lines Patch % Lines
...src/main/scala/org/apache/comet/GenerateDocs.scala 0.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1385      +/-   ##
============================================
+ Coverage     56.12%   58.93%   +2.80%     
- Complexity      976     1028      +52     
============================================
  Files           119      122       +3     
  Lines         11743    12278     +535     
  Branches       2251     2311      +60     
============================================
+ Hits           6591     7236     +645     
+ Misses         4012     3883     -129     
- Partials       1140     1159      +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kazuyukitanimura kazuyukitanimura changed the title enable full decimal to decimal support fix: enable full decimal to decimal support Feb 14, 2025
Copy link
Contributor

@kazuyukitanimura kazuyukitanimura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good, thank you @himadripal just minor comments

@andygrove
Copy link
Member

Apologies for the late review @himadripal. I checked out the branch locally and ran some additional manual testing and could not find any issues. I left a comment about a missing assertion. Could you also rebase against main since this has been pending for a while?

It would be great to get this into the 0.7.0 release next week if possible.

@himadripal himadripal force-pushed the fix_enable_full_decimal_to_decimal branch from 5cce0c4 to a4d298f Compare March 9, 2025 04:14
@himadripal
Copy link
Contributor Author

I left a comment about a missing assertion

@andygrove I did not see this comment. Although I reverted previous changes related to spark3.3 assertion and added a check for specific messages for decimal to decimal conversion.

Copy link
Contributor

@kazuyukitanimura kazuyukitanimura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (pending ci)

Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @himadripal

mbutrovich pushed a commit to mbutrovich/datafusion-comet that referenced this pull request Mar 11, 2025
@kazuyukitanimura kazuyukitanimura merged commit 15b1152 into apache:main Mar 11, 2025
78 checks passed
@kazuyukitanimura
Copy link
Contributor

Merged thanks @himadripal @andygrove @parthchandra

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement Spark-compatible cast between decimals with different precision and scale
5 participants