feat(starrocks): translate one-phase aggregation and sorts#1110
Draft
mbrobbel wants to merge 5 commits into
Draft
feat(starrocks): translate one-phase aggregation and sorts#1110mbrobbel wants to merge 5 commits into
mbrobbel wants to merge 5 commits into
Conversation
Result batches can be arbitrarily large; the fixed 64 KiB thrift channel truncated anything bigger, surfacing as a transport error on the fetch_data path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Render decimal128, date32, and microsecond-timestamp result columns as MySQL text (plus large/view string arrays) so query results over the TPC-H schema types survive the fetch_data path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three env-gated debugging hooks for plan work against a live frontend: SIRIUS_CN_DUMP_FRAGMENTS writes each received thrift fragment and its translated Substrait plan bytes to a directory, SIRIUS_CN_TRANSLATE_ONLY accepts untranslatable fragments so the frontend dispatches (and we capture) a query's complete fragment set, and an ignored engine test replays a dumped Substrait plan in isolation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extend the expression translator with ARITHMETIC_EXPR (add/subtract/multiply/divide/modulus), DATE_LITERAL (as epoch days), IN_PRED (singular-or-list), CASE_EXPR (if-then chains), and an allowlist of FUNCTION_CALL builtins (like, if, substring, year/month/ day, byte/codepoint length) chosen so name-level mapping preserves StarRocks semantics on the DuckDB side. Shapes the engine cannot evaluate are rejected with structured errors instead of failing downstream: decimal-typed arithmetic (currently segfaults the GPU projection kernel), non-constant LIKE patterns or patterns with escapes, and substring with non-constant or non-positive bounds. NULL-semantics and byte-vs-codepoint mismatches (concat, length) are handled explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Translate AGGREGATION_NODE into a Substrait aggregate relation (sum/count/min/max/avg and multi_distinct_count as a distinct count), restricted to finalized one-phase aggregation (new_planner_agg_stage=1) so merge-phase partial states never need modeling. Grouping keys come first in the output tuple, aggregate output types come from its slots, and HAVING conjuncts become a filter over the aggregate output. Translate SORT_NODE into sort-tuple projection + sort, reading the materialization expressions from TSortInfo (the node-level field is a deprecated duplicate). Any node's non-negative limit (plus a sort offset) becomes a fetch relation, with an explicit unlimited count for offset-only fetches since the consumer reads the plain count field. Rejected with structured errors: merge aggregates, avg with a non-double result (decimal/temporal avg semantics differ), distinct aggregates without grouping keys or over multiple columns, and partitioned or rank-based top-N sorts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 7, 2026
pull Bot
pushed a commit
to Mu-L/sirius-db
that referenced
this pull request
Jul 14, 2026
…us-db#1106) Result batches can be arbitrarily large; the fixed 64 KiB thrift channel truncated them, surfacing as a transport error at `fetch_data`. Serialize into a plain growable buffer instead. ## Stack All based on `dev`, so each diff includes the levels below it until they merge (review commits, or the last commit, for the per-level change): 1. sirius-db#1106 ⬅️ this PR 2. sirius-db#1107 3. sirius-db#1108 4. sirius-db#1109 5. sirius-db#1110 6. sirius-db#1111 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
kevkrist
pushed a commit
to kevkrist/sirius
that referenced
this pull request
Jul 15, 2026
…-db#1107) Encode decimal128, date32, and microsecond-timestamp result columns (plus large/view string arrays) as MySQL text on the `fetch_data` path. ## Stack All based on `dev`, so each diff includes the levels below it until they merge (review commits, or the last commit, for the per-level change): 1. sirius-db#1106 2. sirius-db#1107 ⬅️ this PR 3. sirius-db#1108 4. sirius-db#1109 5. sirius-db#1110 6. sirius-db#1111 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Translate
AGGREGATION_NODEinto a Substrait aggregate (sum/count/min/max/avg,multi_distinct_countas distinct count), restricted to finalized one-phase aggregation (new_planner_agg_stage=1); HAVING conjuncts become a filter over the aggregate output. TranslateSORT_NODEas sort-tuple projection + sort; any node's limit (plus sort offset) becomes aFetchRel.Rejected: merge aggregates,
avgwith a non-double result, distinct aggregates without grouping keys or over multiple columns, partitioned/rank-based top-N.Stack
All based on
dev, so each diff includes the levels below it until they merge (review commits, or the last commit, for the per-level change):🤖 Generated with Claude Code