Skip to content

Support full outer join - #10947

Open
windtalker wants to merge 5 commits into
pingcap:masterfrom
windtalker:support_full_outer_join
Open

Support full outer join#10947
windtalker wants to merge 5 commits into
pingcap:masterfrom
windtalker:support_full_outer_join

Conversation

@windtalker

@windtalker windtalker commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #10777

Problem Summary:

Support FULL OUTER JOIN pushdown to TiFlash for equi-join cases.

What is changed and how it works?

Support full outer join
Translate full outer join note to English

This PR adds TiFlash support for FULL OUTER JOIN in the hash join path with non-empty equi join keys. It wires the DAG join type mapping, nullable schema handling, condition validation, and execution paths needed by FULL OUTER JOIN.

Key changes:

  • Add FULL OUTER JOIN mapping from TiDB DAG join type to ASTTableJoin::Kind::Full.
  • Make both left-side and right-side output columns nullable for FULL OUTER JOIN.
  • Allow FULL OUTER JOIN to carry left/right conditions.
  • Fix FULL OUTER JOIN + other condition correctness by using row-flagged map behavior so build-side used marks are applied only after other condition passes.
  • Add targeted gtests and an English implementation note.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Support FULL OUTER JOIN pushdown to TiFlash for equi-join cases.

Summary by CodeRabbit

  • New Features

    • Added support for full outer joins with unmatched rows, nullable columns, and additional join conditions.
    • Full outer joins without join keys are rejected with a clear error.
  • Bug Fixes

    • Improved row preservation, including during spill execution.
    • Corrected nullable output and condition-column handling across join types.
  • Documentation

    • Added guidance on full outer join behavior and limitations.
  • Tests

    • Added coverage for execution, spill behavior, schema nullability, column pruning, and validation.

close pingcap#10777

Support full outer join

- support full outer join protocol / join kind plumbing
- guard unsupported cartesian full outer join cases
- make full join output schemas nullable where needed
- support full join with non-equal other conditions
- fix full join other-condition execution path
- add targeted tests and design notes

Signed-off-by: xufei <xufeixw@mail.ustc.edu.cn>
@ti-chi-bot ti-chi-bot Bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/needs-triage-completed labels Jul 2, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign searise for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Jul 2, 2026
@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1b75c475-5fff-4d50-9f3b-e6a2c9c1deed

📥 Commits

Reviewing files that changed from the base of the PR and between 2a9bf23 and e6cf853.

📒 Files selected for processing (1)
  • dbms/src/Flash/Coprocessor/tests/gtest_join_get_kind_and_build_index.cpp
🚧 Files skipped from review as they are similar to previous changes (1)
  • dbms/src/Flash/Coprocessor/tests/gtest_join_get_kind_and_build_index.cpp

📝 Walkthrough

Walkthrough

This PR adds full outer join support to TiFlash. It maps TypeFullOuterJoin, propagates nullable schemas, validates non-equal conditions, preserves unmatched rows through row-flagged execution, and adds tests and documentation.

Changes

Full Outer Join Support

Layer / File(s) Summary
Join type mapping and validation guards
dbms/src/Flash/Coprocessor/DAGUtils.cpp, dbms/src/Flash/Coprocessor/JoinInterpreterHelper.*
Maps TypeFullOuterJoin to ASTTableJoin::Kind::Full, rejects cartesian full joins, names the join type, and permits left/right non-equal conditions.
Nullable schema propagation
dbms/src/Flash/Coprocessor/JoinInterpreterHelper.*, dbms/src/Flash/Coprocessor/collectOutputFieldTypes.cpp, dbms/src/Debug/MockExecutor/JoinBinder.cpp
Uses shared nullable-side helpers for join outputs, other-condition columns, field types, and mock executor schemas.
Row-flagged full join execution
dbms/src/Interpreters/JoinUtils.h, dbms/src/Interpreters/JoinPartition.cpp, dbms/src/Interpreters/Join.cpp, dbms/src/DataStreams/ScanHashMapAfterProbeBlockInputStream.cpp
Adds full-join row-flagged probing, unmatched-row handling, null padding, used-entry guards, and post-probe scanning.
Tests and design documentation
dbms/src/Flash/Coprocessor/tests/*, dbms/src/Flash/tests/*, dbms/src/TestUtils/tests/*, docs/note/fullouter_join.md
Adds coverage for mapping, nullability, conditions, execution, column pruning, spilling, and mock schemas. The design note records the supported scope and implementation checklist.

Estimated code review effort: 4 (Complex) | ~75 minutes

Poem

A rabbit checks each join-side row,
Full outer paths now safely flow.
Nulls appear where unmatched rows land,
Row-flagged maps keep results planned.
Tests and notes mark every way—
Hop, hop, hooray for join support! 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the primary change: adding FULL OUTER JOIN support.
Description check ✅ Passed The description covers the problem, implementation, tests, side effects, documentation, compatibility, and release note requirements.
Linked Issues check ✅ Passed The changes address all linked issue objectives, including plumbing, nullable schemas, cartesian-join guarding, conditions, execution fixes, tests, and documentation.
Out of Scope Changes check ✅ Passed The code, tests, and implementation note are directly related to FULL OUTER JOIN support and the linked issue scope.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@windtalker

Copy link
Copy Markdown
Contributor Author

/run-check-issue-triage-complete

@windtalker

Copy link
Copy Markdown
Contributor Author

/run pull-integration-test

@windtalker

Copy link
Copy Markdown
Contributor Author

/test pull-unit-test

@windtalker

Copy link
Copy Markdown
Contributor Author

/test pull-integration-test

Comment thread dbms/src/Flash/Coprocessor/JoinInterpreterHelper.h Outdated
Comment thread dbms/src/Flash/Coprocessor/JoinInterpreterHelper.h Outdated
Co-authored-by: xufei <xufei@pingcap.com>
Comment thread dbms/src/Flash/Coprocessor/tests/gtest_join_get_kind_and_build_index.cpp Outdated
Comment thread dbms/src/Flash/Coprocessor/tests/gtest_join_get_kind_and_build_index.cpp Outdated
windtalker and others added 2 commits August 11, 2026 20:52
Co-authored-by: xufei <xufei@pingcap.com>
Signed-off-by: xufei <xufeixw@mail.ustc.edu.cn>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support full outer join

1 participant