fix: add unicode flag to regex to prevent accented character false positives

[deemonic]

Summary

• Adds the /u (PCRE_UTF8) flag to generated profanity regex patterns, fixing false positives caused by multi-byte UTF-8 characters (e.g. ê, é) being matched byte-by-byte
• Fixes "être" is considered as profanity (=tit) #24: words like "tête" and "aré" are no longer incorrectly flagged as profanity

Test plan

• All 188 existing tests pass (including full detection-rate suites for EN/ES/DE/FR)
• New tests verify accented words are not flagged and actual profanity is still detected

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Improved profanity detection to correctly handle Unicode and accented characters and added input encoding normalization to ensure consistent UTF-8 processing while preserving case-insensitive matching.

• Tests

  • Added and updated tests to validate Unicode/accent handling and to confirm profanity detection remains accurate for both accented and plain inputs.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 Walkthrough

Walkthrough

Updates profanity detection to be Unicode-aware and normalizes input encoding: the profanity regex now uses the 'u' flag for Unicode matching; input strings are coerced to UTF-8 when needed; tests added/updated to cover accented French words and regex expectation.

Changes

Cohort / File(s)	Summary
Regex Unicode Flag src/Generators/ProfanityExpressionGenerator.php	Final profanity regex delimiter changed from /.../i to /.../iu, enabling Unicode-aware matching while retaining case-insensitivity.
Input Encoding Normalization src/BlaspService.php	Added an encoding guard in check() to convert non-UTF-8 input to UTF-8 before further processing.
New Tests for Issue #24 tests/Issue24Test.php	Added a PHPUnit test class with four tests verifying accented French words (e.g., "être", "tête", "aré") are not falsely flagged and that real profanity (e.g., "shit") is detected.
Updated Test Expectations tests/ProfanityExpressionGeneratorTest.php	Two assertions updated to expect regex endings with /iu instead of /i to match the generator change.

Sequence Diagram(s)

(Skipped — changes are small and do not introduce a new multi-component control flow requiring a sequence diagram.)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• PR #39: Modifies ProfanityExpressionGenerator.php regex construction and wrapping logic; directly related to the regex-flag changes in this PR.

Poem

🐰 I hopped through bytes both bold and bright,
Turned slashes to 'iu' in the moonlit night,
Accents now safe, no false alarm,
UTF‑8 snug in my furry charm,
Cheers — the words can peacefully delight!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: adding a Unicode flag to the regex pattern to prevent false positives from accented characters, which is the core objective of the PR.
Linked Issues check	✅ Passed	All code changes directly address issue #24: regex Unicode flag prevents accented characters being matched byte-by-byte, UTF-8 validation ensures compatibility, and tests verify accented words are not flagged while actual profanity is detected.
Out of Scope Changes check	✅ Passed	All changes are scoped to fixing issue #24: regex Unicode flag updates, UTF-8 validation in BlaspService, and test additions specifically verify the accented character false positive fix.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@src/Generators/ProfanityExpressionGenerator.php`:
- Around line 155-157: The generated regex in ProfanityExpressionGenerator (uses
self::SEPARATOR_PLACEHOLDER and builds $expression with '/.../iu') can fail
silently on invalid UTF-8 when used by BlaspService::check(); ensure the input
is validated/normalized to UTF-8 before any normalization/regex work (use
mb_check_encoding() and, if invalid, convert with mb_convert_encoding(...,
'UTF-8', 'auto') or reject and log), and also add post-regex error handling in
the caller (after preg_match_all in BlaspService::check()) by checking
preg_last_error() and logging or falling back to a safe behavior; update
relevant places where the string enters processing so invalid-UTF8 cases are
handled before ProfanityExpressionGenerator is used.

[coderabbitai]

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/BlaspService.php (1)

366-400: Byte-based string operations are unsafe with potentially multi-byte-containing strings.

The method uses byte-based operations (strlen(), array index access $string[$i], substr()) which do not handle multi-byte UTF-8 characters correctly. While the normalizers for French, German, and Spanish convert accented characters to ASCII equivalents, the English normalizer returns the input unchanged, and other normalizers may not cover all characters. If $normalizedString contains any multi-byte UTF-8 characters, byte-level array access like $string[$tokenStart - 1] can land on the middle of a multi-byte sequence, causing preg_match() to behave unexpectedly.

Additionally, lines 341–342 elsewhere in the file already use mb_strlen() and mb_substr() for UTF-8 safety, showing the codebase is UTF-8-aware. This method should use the same approach: mb_strlen(), mb_substr(), and either mb_str_split() or preg_match() with the /u flag for safe character iteration.

🧹 Nitpick comments (1)

src/BlaspService.php (1)

260-262: Encoding conversion assumes malformed UTF-8, not alternative encodings.

The current approach correctly sanitizes invalid UTF-8 byte sequences, but if the input is in a different encoding entirely (e.g., ISO-8859-1, Windows-1252), specifying 'UTF-8' as the source encoding will corrupt the data rather than convert it properly.

Consider detecting the actual encoding for more robust handling:

♻️ Suggested improvement

-        if (!mb_check_encoding($string, 'UTF-8')) {
-            $string = mb_convert_encoding($string, 'UTF-8', 'UTF-8');
-        }
+        if (!mb_check_encoding($string, 'UTF-8')) {
+            $detected = mb_detect_encoding($string, ['UTF-8', 'ISO-8859-1', 'Windows-1252', 'ASCII'], true);
+            $string = mb_convert_encoding($string, 'UTF-8', $detected ?: 'ISO-8859-1');
+        }

Alternatively, if you intentionally want to strip invalid sequences rather than convert from other encodings, document this behavior with a comment explaining that non-UTF-8 encoded input may lose data.