Skip to content

feat: add to_map expression constructor#7261

Open
dwallener wants to merge 6 commits into
Eventual-Inc:mainfrom
dwallener:codex/add-to-map-function
Open

feat: add to_map expression constructor#7261
dwallener wants to merge 6 commits into
Eventual-Inc:mainfrom
dwallener:codex/add-to-map-function

Conversation

@dwallener

Copy link
Copy Markdown

First time looking/working with Daft code...any process errors are strictly mine.

Changes Made

Fixes #6885

Summary

  • Add a daft.functions.to_map constructor for key/value expression pairs.
  • Support keyword arguments as string keys.
  • Add expression tests for positional pairs, keyword pairs, mixed use, map_get, coercion, and invalid inputs.

Tests

  • make build
  • DAFT_RUNNER=native pytest -q tests/expressions/test_to_map.py
  • DAFT_RUNNER=native pytest -q tests/expressions/test_to_list.py tests/recordbatch/struct/test_to_struct.py tests/expressions/test_to_map.py
  • DAFT_RUNNER=native pytest -q tests/recordbatch/test_from_py.py tests/expressions/test_to_map.py

AI usage

Used AI assistance to draft the implementation and tests; verified locally with the commands above.

Related Issues

@greptile-apps

greptile-apps Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a to_map expression constructor. The main changes are:

  • Public Python export through daft.functions.
  • Python helper for positional and keyword map entries.
  • Rust scalar UDF registration and map physical construction.
  • Tests for construction, lookup, coercion, and invalid inputs.

Confidence Score: 4/5

The new map constructor needs a fix for empty input batches with literal entries.

  • Literal keys or values can stay length 1 when the input row count is 0.
  • That can make a zero-row select return a one-row map result.
  • The public API and registration paths otherwise follow the existing function patterns.

src/daft-functions/src/to_map.rs

Important Files Changed

Filename Overview
daft/functions/init.py Exports to_map from the public functions module.
daft/functions/struct.py Adds the Python wrapper that validates pair counts and forwards entries to the builtin scalar function.
src/daft-functions/src/lib.rs Registers the new map scalar function in the miscellaneous function module.
src/daft-functions/src/to_map.rs Builds map arrays from key/value inputs, with a zero-row literal cardinality issue in the broadcast path.
tests/expressions/test_to_map.py Adds coverage for common map construction paths but not empty input batches.

Reviews (1): Last reviewed commit: "Add to_map expression constructor" | Re-trigger Greptile

Comment thread src/daft-functions/src/to_map.rs Outdated
@dwallener dwallener changed the title Add to_map expression constructor feat: add to_map expression constructor Jul 13, 2026
@github-actions github-actions Bot added the feat label Jul 13, 2026
@dwallener

Copy link
Copy Markdown
Author

All PR-related checks now appear addressed. The remaining failures seem to come from IO integration flakiness unrelated to this change:

  • integration-test-io (3.10, native)
  • integration-test-io (3.10, ray)
  • aggregate integration-tests required check

The IO logs show Hugging Face/S3 external access failures, e.g. https://huggingface.co/api/datasets/stanfordnlp/imdb/parquet returning HTTP 400 and transient S3 GET timeouts for daft-oss-public-data.

This PR only adds to_map expression construction and does not touch IO/Hugging Face/S3 code. I reran the failed checks and they still failed in the same unrelated area.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Add to_map constructor function

1 participant