[FLINK-32416] Fix flaky tests by ensuring test utilities produce records w… #79
Conversation
| @@ -212,32 +214,27 @@ public void start() { | |||
|
|
|||
| private void handleNoMoreSplits() { | |||
| if (Boundedness.BOUNDED.equals(boundedness)) { | |||
| enumContext.runInCoordinatorThread( | |||
There was a problem hiding this comment.
No need to use this API. The code already runs in coordinator thread
mas-chen
force-pushed
the
hotfix-tests
branch
from
82b8bbd
to
ef0be3b
Compare
|
The only thing I'm wondering about is that we actually should file a Jira for this; otherwise LGTM |
|
Can we put this under the original JIRA issue? I think that should be ok since we haven't release in the meantime. |
Yeah that makes sense for me too, as a follow-up to actually complete the Jira |
There was a problem hiding this comment.
LGTM: I'll fix the commit message when merging and add it to the original Jira
Edit: I see that @mas-chen still wants to fix one thing, so I'll postpone merging this until he's done that himself :)
|
@MartijnVisser which thing to fix are you referring to? I think this PR is ready as-is if it blocking your other work. The ticket I'll need a few days to look into, probably get to it at the end of the week. Edit: If it wasn't clear, the ticket also affects the KafkaSource tests (not just DynamicKafkaSource ones). I modified the commit since I had it open already! |
…rds with consistency and cleanup notify no more splits to ensure it is sent
mas-chen
force-pushed
the
hotfix-tests
branch
from
ef0be3b
to
72bd59c
Compare
mas-chen
changed the title
I misread #79 (comment) as that you still wanted to fix something 🙈 I'll merge it after this CI run, thanks :) |
…ith consistency and cleanup notify no more splits to ensure it is sent
I have spent the last few days troubleshooting this and found that most issues are related to the bounded mode tests. These problems don't surface locally and I suspect that they are more likely to occur with smaller/busy CI machines. Issues:
At this commit, the tests are quite stable. 6/7 CI runs pass.
In one CI run that failed, I did see that there was a Flink error, indicating a lost source event. In separate commits, there were other instances that exceeded 50 minutes but this can be solved separately, see https://issues.apache.org/jira/browse/FLINK-34127.
Original issue that Max found after #44 was merged: https://github.com/apache/flink-connector-kafka/actions/runs/7504290046/job/20433204351
cc: @mxm @tzulitai @MartijnVisser