Skip to content

atlanhq/generic-openlineage-examples

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

OpenLineage Sample Events for Atlan

A collection of runnable OpenLineage event examples for testing and validating your Atlan Generic OpenLineage connector setup.

Send these events to your Atlan tenant to see FlowControlOperations, Processes, and lineage assets appear in the UI.


Prerequisites

  • Python 3.8+
  • An Atlan tenant with the Generic OpenLineage connector configured
  • An Atlan API key (Settings → API Keys)

Setup

# 1. Clone this repo
git clone https://github.com/atlanhq/generic-openlineage-examples
cd generic-openlineage-examples

# 2. Install dependencies
pip install -r requirements.txt

# 3. Configure your endpoint
cp .env.example .env
# Edit .env and fill in OL_ENDPOINT and API_KEY

Usage

python send_events.py examples/01_simple_dag

The script reads all .json files from the example's events/ directory in sorted order and POSTs each one to your Atlan endpoint. Status is printed for each event.


Examples

# Example Events What appears in Atlan
01 Simple DAG 4 Parent + child FlowControlOperation, no datasets
02 DAG with table lineage 4 FCOs + Process + Postgres input → Snowflake output
03 Multi-task DAG 8 3 child FCOs, Postgres → S3 → S3 → Snowflake lineage chain
04 Column-level lineage 4 FCOs + Process + ColumnProcess assets (BigQuery → BigQuery)

Event format

Events are raw OpenLineage RunEvent JSON — no additional envelope needed. Each file contains a single event.

The minimal required fields per event are:

{
  "eventTime": "2025-01-15T10:00:00.000Z",
  "eventType": "START",
  "producer": "...",
  "schemaURL": "https://openlineage.io/spec/2-0-2/OpenLineage.json#/$defs/RunEvent",
  "run": { "runId": "<uuid>" },
  "job": { "name": "<job-name>", "namespace": "<namespace>" },
  "inputs": [],
  "outputs": []
}

Parent events (DAG/APPLICATION level) use job.facets.jobType.jobType: "DAG" or "APPLICATION". Child events (task level) include run.facets.parent linking back to the parent run ID.


Connector documentation

For connector setup instructions and configuration options, see the Atlan OpenLineage connector documentation.

About

This report is for creating example OL events and the pipeline to send events to Atlan Endpoint to show how generic open-lineage assets and lineage are created in Atlan.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages