Skip to content

Conversation

@shuwenwei
Copy link
Member

@shuwenwei shuwenwei commented Nov 17, 2025

Description

#16765

Description

Default slowQueryThreshold should be 10s

截屏2025-11-17 11 49 57

optimize encodeBatch for multi tvlist scene

Scenario

Rows: 0–99999 (100,000 rows)
Duplicate timestamps occur at only two positions:

index 123

index 90732

Using regular BitMap

Must allocate 100,000 bits immediately.

Even though only 2 bits will be marked.

Using LazyBitMap (blockSize = 10,000)

LazyBitMap creates blocks only when needed:

Block for index 123 → block index = 0

Block for index 90732 → block index = 90

Total allocated blocks: 2 blocks
Each block has only 10,000 bits, so memory is:

2 blocks × 10,000 bits = 20,000 bits total

Savings

Regular BitMap: 100,000 bits
LazyBitMap: 20,000 bits

➡ 80% memory reduction
➡ Matches the sparse nature of duplicated timestamps

@codecov
Copy link

codecov bot commented Nov 17, 2025

Codecov Report

❌ Patch coverage is 85.71429% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 41.04%. Comparing base (7acdb30) to head (bc30a7f).
⚠️ Report is 16 commits behind head on dev/1.3.

Files with missing lines Patch % Lines
...ueryengine/plan/planner/OperatorTreeGenerator.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             dev/1.3   #16766      +/-   ##
=============================================
- Coverage      41.04%   41.04%   -0.01%     
  Complexity       198      198              
=============================================
  Files           3586     3589       +3     
  Lines         235277   235685     +408     
  Branches       28388    28452      +64     
=============================================
+ Hits           96571    96733     +162     
- Misses        138706   138952     +246     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@shuwenwei shuwenwei changed the title [to dev/1.3] Fix slow query threshold [to dev/1.3] Fix slow query threshold & fix a bug for lastQuery Nov 18, 2025
Copy link
Collaborator

@Caideyipi Caideyipi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shuwenwei shuwenwei changed the title [to dev/1.3] Fix slow query threshold & fix a bug for lastQuery [to dev/1.3] Fix slow query threshold & fix a bug for lastQuery & optimize encodeBatch Nov 20, 2025
Copy link
Contributor

@JackieTien97 JackieTien97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plz fix the sonar and code coverage, otherwise it won't be merged

@shuwenwei shuwenwei changed the title [to dev/1.3] Fix slow query threshold & fix a bug for lastQuery & optimize encodeBatch [to dev/1.3] Fix slow query threshold & fix a bug for lastQuery & optimize encodeBatch for multi tvlist scene Nov 21, 2025
shuwenwei and others added 4 commits November 21, 2025 22:15
@sonarqubecloud
Copy link

sonarqubecloud bot commented Nov 24, 2025

Quality Gate Failed Quality Gate failed

Failed conditions
B Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@JackieTien97 JackieTien97 merged commit db77455 into dev/1.3 Nov 24, 2025
20 of 21 checks passed
@JackieTien97 JackieTien97 deleted the fixSlowQueryThreshold-1.3 branch November 24, 2025 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants