PPL currently supports 29 SPL-equivalent commands.
Reference: PPL Documentation
This issue tracks the current status of supported commands, functions, and the forward roadmap for enhancements. Target dates are aligned to quarterly boundaries and OpenSearch release versions.
Current Support
- Core Commands:
search, fields, table, sort, rename, eval, where, join, append, lookup, inputlookup, outputlookup, stats, timechart, eventstats, bin, spath, regex, rex
- Functions: Aggregation, evaluation, and conditional functions with partial SPL parity.
- Time Functions: Partial support (
earliest, latest, per_second).
- Performance/Benchmarks: Initial ClickBench and large dataset validations are in place.
Roadmap Tables
Commands
| Command |
Description |
ETA (Quarter) |
Release |
Status |
| regex / rex |
Regular expression extraction, alias for parse |
Q3 |
3.3 |
In progress |
| search |
Core search command |
Q3 |
3.3 |
Code Review |
| table |
Display fields as columns (alias for fields) |
Q3 |
3.3 |
Completed |
| fields |
Field selection with SPL-style wildcard support |
Q3 |
3.3 |
Completed |
| sort |
Sorting with SPL nuances |
Q3 |
3.3 |
Completed |
| rename |
Field renaming |
Q3 |
3.3 |
Completed |
| eval |
Expression evaluation |
Q3 |
3.3 |
Completed |
| spath |
JSON path extraction |
Q3 |
3.3 |
Completed |
| where |
Filtering clause |
Q3 |
3.3 |
Completed |
| join |
Join across indices |
Q3 |
3.3 |
Completed |
| append |
Append results from subsearch |
Q3 |
3.3 |
Completed |
| lookup/input/outputlookup |
Lookup and CSV-like workflows |
Q3 |
3.3 |
Workarounds |
| stats |
Aggregation framework |
Q3 |
3.3 |
Completed |
| timechart |
Time-series charting |
Q3 |
3.3 |
Completed |
| eventstats |
Stats on event level |
Q3 |
3.3 |
Completed |
| bin |
Bucketing (alias for span) |
Q3 |
3.3 |
Completed |
Aggregation Functions
| Function |
Description |
ETA (Quarter) |
Release |
Status |
| count / distinct_count |
Count with eval expressions |
Q3 |
3.3 |
Needs Fixes |
| sum / avg |
Basic arithmetic aggregations |
Q3 |
3.3 |
Completed |
| values / list |
Multivalue stats |
Q3 |
3.3 |
Completed |
| earliest / latest |
Chronological functions |
Q4 |
3.4 |
Not Supported |
| min / max |
Numeric and string support |
Q4 |
3.4 |
Needs Fixes |
| percentile / median |
Percentile, syntax aliases |
Q4 |
3.4 |
Needs Fixes |
| first / last |
Event order functions |
Q4 |
3.4 |
Not Supported |
| estdc / estdc_error |
Approximate distinct counts |
Q1 |
3.5 |
Not Supported |
| exactperc / upperperc |
Exact/approximate percentiles |
Q1 |
3.5 |
Not Supported |
| mean / mode |
Statistical functions |
Q1 |
3.5 |
Not Supported |
| range / sumsq |
Range and sum of squares |
Q1 |
3.5 |
Not Supported |
| stdev / var functions |
Standard deviation / variance |
Q1 |
3.5 |
Needs Aliases |
Eval Functions
| Function |
Description |
ETA (Quarter) |
Release |
Status |
| coalesce |
First non-null value |
Q3 |
3.3 |
Completed |
| if / case |
Conditional evaluation |
Q3 |
3.3 |
Needs Fixes |
| like / match |
Pattern matching |
Q3 |
3.3 |
Completed |
| sum / avg |
Inline math functions |
Q3 |
3.3 |
Completed |
| strftime |
Time formatting |
Q3 |
3.3 |
In Review |
| mvjoin |
Join multivalue fields |
Q3 |
3.3 |
Completed |
| replace |
Regex-based replace |
Q4 |
3.4 |
In Progress |
| round |
Numeric rounding |
Q4 |
3.4 |
Completed |
| strptime |
Parse human time |
Q1 |
3.5 |
Not Supported |
| mvindex / mvcount |
Multivalue utilities |
Q1 |
3.5 |
Not Supported |
| isnull / isnotnull |
Null checks |
Q1 |
3.5 |
Not Supported |
| tostring / tonumber |
Conversion functions |
Q1 |
3.5 |
Not Supported |
| len / substr |
String functions |
Q1 |
3.5 |
Needs Aliases |
| json_valid |
JSON validation |
Q1 |
3.5 |
Not Supported |
| searchmatch |
Search inside eval |
Q1 |
3.5 |
Not Supported |
Time Functions
| Function |
Description |
ETA (Quarter) |
Release |
Status |
| per_second |
Values per second |
Q3 |
3.3 |
In Review |
| per_minute/hour/day |
Time bucketing by unit |
Q4 |
3.4 |
Not Supported |
| rate, rate_avg, rate_sum |
Rate functions for counters |
Q4 |
3.4 |
Not Supported |
| earliest_time / latest_time |
Return UNIX time for first/last |
Q1 |
3.5 |
Not Supported |
Performance & Benchmarks
| Task |
Description |
ETA (Quarter) |
Release |
Status |
| Expression pushdown |
Query optimization |
Q3 |
3.3 |
Planned |
| Functional testing |
Explore env with otel-demo |
Q3 |
3.3 |
In progress |
| Large-scale testing |
ClickBench + Big dataset validation |
Q4 |
3.4 |
Planned |
| Stability testing |
End-to-end regression and reliability validation |
Q4–Q1 |
3.4–3.5 |
Planned |
Q4/Q1 High Priority Commands
| Command |
Description |
GitHub Issue(s) |
ETA (Quarter) |
Release |
Status |
| dedup |
Removes duplicate documents by field; SPL also supports keepevents and sortby |
#4039, #4040 |
Q4 |
3.4 |
Completed (Pending Testing) |
| appendcols |
Appends fields of subsearch results; SPL supports extra options |
— |
Q4 |
3.4 |
Analyzing |
| multisearch |
Multiple search pipelines |
— |
Q4 |
3.4 |
Not Started |
| fillnull |
Replace nulls with specified values; SPL and OS differ in syntax |
— |
Q4 |
3.4 |
Analyzing |
| multikv |
Dynamic field extraction |
— |
Q4 |
3.4 |
Not Started |
| head |
Returns first N results; SPL supports eval-expr options |
#4249 |
Q4 |
3.4 |
Code Review |
| streamstats |
Incremental stats over events |
— |
Q4 |
3.4 |
Not Started |
| top |
Returns most common values with options (limit, useother, showperc) |
— |
Q4 |
3.4 |
Analyzing |
| collect |
Collect results for further processing |
— |
Q1 |
3.5 |
Not Started |
| chart |
Charting capabilities |
#399 |
Q1 |
3.5 |
Not Started |
| appendpipe |
Appends pipeline results |
— |
Q1 |
3.5 |
Not Started |
| mvexpand |
Expands multivalue fields |
— |
Q1 |
3.5 |
Not Started |
| addcoltotals |
Add totals across columns |
— |
Q1 |
3.5 |
Not Started |
| cluster |
Cluster results |
— |
Q1 |
3.5 |
Not Started |
| format |
Format results |
— |
Q1 |
3.5 |
Not Started |
| convert |
Type conversions |
— |
Q1 |
3.5 |
Not Started |
| fieldformat |
Apply formatting to fields |
— |
Q1 |
3.5 |
Not Started |
| rare |
Returns least common values |
— |
Q1 |
3.5 |
Not Started |
| foreach |
Iterative field transformations |
— |
Q1 |
3.5 |
Not Started |
| addtotals |
Add totals across rows |
— |
Q1 |
3.5 |
Not Started |
| union |
Combine multiple datasets |
— |
Q1 |
3.5 |
Not Started |
| from |
Define source dataset |
— |
Q1 |
3.5 |
Not Started |
| xyseries |
Convert tabular data to XY series |
— |
Q1 |
3.5 |
Not Started |
| transpose |
Transpose result table |
— |
Q1 |
3.5 |
Not Started |
| timewrap |
Time-based wrapping/comparison |
— |
Q1 |
3.5 |
Not Started |
| mvcombine |
Combine multivalue fields |
— |
Q1 |
3.5 |
Not Started |
Open Issues
Summary
The PPL engine has achieved feature parity for many SPL commands and functions, with significant progress on performance, datatype support, and query optimizations. The roadmap focuses on closing the gaps in aggregation and eval functions, expanding time-based capabilities, and ensuring scale/stability benchmarks before enabling broader adoption in OpenSearch releases.
