[FLINK-38244][hotfix] Fix the full snapshot phase, the field case is adjusted based on isTableIdCaseInsensitive#4284
Conversation
|
@leonardBang @loserwang1024 Please take a look when you have time. |
|
looks good but no test |
@ThorneANN I've tested it in a real environment and I'm already using it. Because the test module has not been working properly in my local environment |
test it class without your local env ,just write some test code and use maven and docker to run test applition |
#4275 |
05dedaa to
0bcf771
Compare
|
MySQL field name case sensitivity should be independent of table name case sensitivity and should not be affected by |
@lvyanquan You're right. This issue was discussed in the DingTalk group before. The |
@lvyanquan , yes, we first make the behavior of #4095 same, and then can move forward in another pr. |
There was a problem hiding this comment.
Pull request overview
Fixes MySQL pipeline connector snapshot/schema handling so that column (and PK) name casing is normalized consistently during full snapshot/schema emission when isTableIdCaseInsensitive is enabled.
Changes:
- Add an
uppercase_productsfixture table (uppercase column names) to the MySQL inventory test DDL. - Expand IT coverage to run the existing alter-statement parsing test against both
productsanduppercase_products. - Normalize emitted schema column/PK names to lower-case in
MySqlPipelineRecordEmitterwhen table IDs are case-insensitive, and plumb the flag fromMySqlDataSource.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
.../src/test/resources/ddl/inventory.sql |
Adds uppercase_products table + data to exercise uppercase column-name scenarios. |
.../MySqlTableIdCaseInsensitveITCase.java |
Parameterizes the existing IT to validate schema-change parsing for both normal and uppercase-column tables. |
.../MySqlPipelineITCase.java |
Excludes uppercase_products from an exclusion test to keep expected outputs stable. |
.../MySqlDataSourceFactoryTest.java |
Updates expected discovered tables list to include uppercase_products. |
.../MySqlPipelineRecordEmitter.java |
Lowercases column names and primary key column names when isTableIdCaseInsensitive is true. |
.../MySqlDataSource.java |
Computes isTableIdCaseInsensitive once and passes it to both deserializer and record emitter. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| (default,"hammer","14oz carpenter's hammer",0.875), | ||
| (default,"hammer","16oz carpenter's hammer",1.0), | ||
| (default,"rocks","box of assorted rocks",5.3), | ||
| (default,"jacket","water resistent black wind breaker",0.1), |
There was a problem hiding this comment.
Typo in inserted test data: "water resistent" is misspelled and should be "water resistant" (even if this is only test DDL, keeping strings correct avoids propagating typos across fixtures).
| String colName = | ||
| this.isTableIdCaseInsensitive | ||
| ? column.name().toLowerCase(Locale.ROOT) | ||
| : column.name(); | ||
| DataType dataType = |
There was a problem hiding this comment.
The new case-normalization logic for column names/primary keys is gated by isTableIdCaseInsensitive, but there doesn't appear to be test coverage for the bounded snapshot (StartupOptions.snapshot()) path where schemas are fetched via SHOW CREATE TABLE/DESC. Consider adding an IT that runs with StartupOptions.snapshot() against a table with uppercase column names (and PK) to exercise this end-to-end.
…justed based on `isTableIdCaseInsensitive`. Signed-off-by: peiyu <125331682@qq.com>
Signed-off-by: Pei Yu <125331682@qq.com>
Signed-off-by: Pei Yu <125331682@qq.com>
Signed-off-by: Pei Yu <125331682@qq.com>
Signed-off-by: Pei Yu <125331682@qq.com>
6b34990 to
a5f8680
Compare
Signed-off-by: Pei Yu <125331682@qq.com>
During the full snapshot phase, the field case is adjusted based on
isTableIdCaseInsensitive.