Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Unify all PPL functions in BuiltinFunctionName #1062

Merged
merged 3 commits into from
Feb 21, 2025

Conversation

qianheng-aws
Copy link
Contributor

@qianheng-aws qianheng-aws commented Feb 20, 2025

Description

Unify all PPL functions in BuiltinFunctionName. This PR includes change:

  • Add all PPL functions we have currently in BuiltinFunctionName, that place should be the single of truth to maintain all PPL built-in functions.
  • For code robust improvement, SerializableUdf::visit change to use function enumerator as its switch key instead of string.
  • Add PPL_TO_SPARK_UDF_MAPPING in BuiltinFunctionTransformer to do transformation of PPL built-in function to Spark UDF we implemented in this project.

Related Issues

Resolves: #1051

Check List

  • Updated documentation (docs/ppl-lang/README.md)
  • Implemented unit tests
  • Implemented tests for combination with other commands
  • New added source code should include a copyright header
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Heng Qian <[email protected]>
@@ -75,6 +82,14 @@
import static org.opensearch.sql.ppl.utils.DataTypeTransformer.seq;
import static scala.Option.empty;

/**
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for improving the javadoc.

Copy link
Collaborator

@penghuo penghuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. code looks good.
Could u fix the e2e IT?

Signed-off-by: Heng Qian <[email protected]>
@qianheng-aws
Copy link
Contributor Author

qianheng-aws commented Feb 21, 2025

Thanks. code looks good. Could u fix the e2e IT?

e2e IT retry successfully without any code change. Seems a flakey failure.

I don't see any failure except 1 SUITE ABORTED in the CI log for the previous run, even in the downloaded full log.

@LantaoJin LantaoJin merged commit 7c87378 into opensearch-project:main Feb 21, 2025
5 checks passed
@LantaoJin LantaoJin added the backport 0.x Backport to 0.x branch (stable branch) label Feb 21, 2025
opensearch-trigger-bot bot pushed a commit that referenced this pull request Feb 21, 2025
* [Enhancement] Unify all PPL functions in BuiltinFunctionName

Signed-off-by: Heng Qian <[email protected]>

* Fix UT

Signed-off-by: Heng Qian <[email protected]>

* Tiny change in doc

Signed-off-by: Heng Qian <[email protected]>

---------

Signed-off-by: Heng Qian <[email protected]>
(cherry picked from commit 7c87378)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
LantaoJin pushed a commit that referenced this pull request Feb 24, 2025
…1064)

* [Enhancement] Unify all PPL functions in BuiltinFunctionName



* Fix UT



* Tiny change in doc



---------


(cherry picked from commit 7c87378)

Signed-off-by: Heng Qian <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 0.x Backport to 0.x branch (stable branch)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement] BuiltinFunctionName does not include all built-in functions
4 participants