Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fix incorrect result from casting double to decimal #12600

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

zhli1142015
Copy link
Contributor

@zhli1142015 zhli1142015 commented Mar 11, 2025

When converting 0.03456789 to DECIMAL(38, 33), the result generated by Velox is
inconsistent with Spark and Presto. This discrepancy is caused by precision
errors due to long double multiplication. I propose fixing this issue by first
converting the result to the target type(int64_t / int128_t) and then applying rescaling.

Velox

spark.sql("select cast(0.03456789d as decimal(38, 33))").collect
res18: Array[org.apache.spark.sql.Row] = Array([0.034567889999999999999845079187456])

Spark

spark.sql("select cast(0.03456789d as decimal(38, 33))").collect
res16: Array[org.apache.spark.sql.Row] = Array([0.034567890000000000000000000000000])

Presto

presto> select cast(0.03456789 as decimal(38, 33));
                _col0
-------------------------------------
 0.034567890000000000000000000000000
(1 row)

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 11, 2025
Copy link

netlify bot commented Mar 11, 2025

Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit b64362e
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/67d0c6abc1bf760008e3c569

@zhli1142015
Copy link
Contributor Author

@rui-mo sn @mbasmanova could you help review this PR? The CI failure is unrelated.

@mbasmanova
Copy link
Contributor

@zhli1142015

The CI failure is unrelated.

Is there an issue about this failure? If not, would you create one and reference it here?

Copy link
Contributor

@mbasmanova mbasmanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Yuhta Jimmy, would you help review this fix?

@zhli1142015
Copy link
Contributor Author

@zhli1142015

The CI failure is unrelated.

Is there an issue about this failure? If not, would you create one and reference it here?

#12603
Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants