Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ICU-22940 MF2 ICU4C: Update for bidi support #3236

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

ICU-22940 MF2 ICU4C: Update for bidi support #3236

wants to merge 2 commits into from
+583 −118

Conversation

catamorphism
Copy link
Contributor

@catamorphism catamorphism commented Oct 8, 2024 •
edited
Loading

The tests in this PR are also included in a PR against the MF2 spec. However, some editing will have to occur unless #3198 (matching on variables instead of expressions) lands before then. There was a spec change to the syntax of .match constructs, and some of the tests include .match constructs.

Checklist

  • Required: Issue filed: https://unicode-org.atlassian.net/browse/ICU-22940
  • Required: The PR title must be prefixed with a JIRA Issue number. Example: "ICU-1234 Fix xyz"
  • Required: The PR description must include the link to the Jira Issue, for example by completing the URL in the first checklist item
  • Required: Each commit message must be prefixed with a JIRA Issue number. Example: "ICU-1234 Fix xyz"
  • Issue accepted (done by Technical Committee after discussion)
  • Tests included, if applicable
  • API docs and/or User Guide docs changed or added, if applicable

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/test/intltest/messageformat2test_utils.h is different
  • testdata/message2/bidi.json is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/test/intltest/messageformat2test.cpp is now changed in the branch

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@catamorphism catamorphism marked this pull request as ready for review October 10, 2024 16:00
@catamorphism catamorphism changed the title ICU-22940 DRAFT: MF2 ICU4C: Update for bidi support ICU-22940 MF2 ICU4C: Update for bidi support Oct 10, 2024
srl295
srl295 reviewed Oct 18, 2024
View reviewed changes
icu4c/source/i18n/messageformat2_parser.cpp Outdated
inRange(c, 0x00F8, 0x02FF) || inRange(c, 0x0370, 0x037D) || inRange(c, 0x037F, 0x1FFF) ||
inRange(c, 0x200C, 0x200D) || inRange(c, 0x2070, 0x218F) || inRange(c, 0x2C00, 0x2FEF) ||
inRange(c, 0x00F8, 0x02FF) || inRange(c, 0x0370, 0x037D) || inRange(c, 0x037F, 0x061B) ||
inRange(c, 0x061D, 0x200D) || inRange(c, 0x2070, 0x218F) || inRange(c, 0x2C00, 0x2FEF) ||
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@catamorphism i see ALM is not namestart, but this change makes U+2000…U+200B isNameStart true. they are dashes and spaces, and not ID_Start

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, I think this would be far more reliable in using a UnicodeSet. That can be created as the C++ equivalent of a static final immutable object.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@srl295 Fixed

@macchiati Done in 780a947

srl295
srl295 reviewed Oct 18, 2024
View reviewed changes
icu4c/source/i18n/messageformat2_parser.cpp Outdated
@@ -125,7 +125,13 @@ static bool isContentChar(UChar32 c) {
|| inRange(c, 0xE000, 0x10FFFF);
}

// See `s` in the MessageFormat 2 grammar
// See `bidi` in the MF2 grammar
static bool isBidi(UChar32 c) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe isBidiControl might be better?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 780a947

@catamorphism catamorphism mentioned this pull request Oct 25, 2024
Merged
7 tasks
srl295
srl295 previously approved these changes Nov 5, 2024
View reviewed changes
@srl295
Copy link
Member

srl295 commented Nov 5, 2024

needs squash but LGTM, seems like all issues addressed

@catamorphism catamorphism force-pushed the bidi-tests branch 2 times, most recently from 01d2fdc to 9054d0c Compare November 6, 2024 00:07
@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/i18n/messageformat2_formatter.cpp is now changed in the branch
  • icu4c/source/i18n/messageformat2_parser.cpp is different
  • icu4c/source/i18n/messageformat2_parser.h is different
  • icu4c/source/i18n/ucln_in.h is now changed in the branch

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/i18n/messageformat2_formatter.cpp is no longer changed in the branch
  • icu4c/source/i18n/messageformat2_parser.cpp is different
  • icu4c/source/i18n/messageformat2_parser.h is different
  • icu4c/source/i18n/ucln_in.h is no longer changed in the branch
  • icu4c/source/test/intltest/messageformat2test.cpp is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/i18n/messageformat2_formatter.cpp is now changed in the branch
  • icu4c/source/i18n/messageformat2_parser.cpp is different
  • icu4c/source/i18n/messageformat2_parser.h is different
  • icu4c/source/i18n/ucln_in.h is now changed in the branch

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@jira-pull-request-webhook
Copy link

Notice: the branch changed across the force-push!

  • icu4c/source/i18n/messageformat2_formatter.cpp is different
  • icu4c/source/i18n/messageformat2_parser.cpp is different
  • icu4c/source/i18n/messageformat2_parser.h is different
  • icu4c/source/i18n/messageformat2.cpp is now changed in the branch
  • icu4c/source/test/intltest/messageformat2test_read_json.cpp is different
  • icu4c/source/test/intltest/messageformat2test_utils.h is different
  • icu4c/source/test/intltest/messageformat2test.cpp is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@catamorphism
Copy link
Contributor Author

I added 5eb2b7d to fix a bug that actually was in a previous PR, #3239 , but wasn't caught when merging that PR because of a bug in the test runner.

This PR incidentally fixes that bug, so the test failure showed up here when I rebased this PR against main, and I'm fixing it here. (Will squash after @srl295 gets a chance to look at it, I left it unsquashed so far just so it's clear what I changed after he reviewed the PR.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@macchiati macchiati macchiati left review comments

@srl295 srl295 srl295 left review comments

@echeran echeran Awaiting requested review from echeran

@mihnita mihnita Awaiting requested review from mihnita

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@catamorphism @srl295 @macchiati