OpenLineage V2

Stable Linux Foundation OpenLineage spec 2023+.

Definition

OpenLineage JSON event format: RunEvent { eventType, eventTime, run, job, inputs, outputs, producer, schemaURL }. Facets for extensible metadata (columnLineage, schema, dataSource, ownership, dataQuality, sql). Native integrations with Apache Airflow (since 2.7), Apache Spark, dbt, Flink, Dagster. Backend: Marquez (Linux Foundation), Atlan, OpenMetadata, DataHub (LinkedIn).

Origin

Datakin (Julien Le Dem) 2021, donation to Linux Foundation AI & Data Foundation 2023.

Usage

Emerging cross-tool data lineage standard. Competes with OpenMetadata events and DataHub MCEs.

Related terms

• OpenLineage — fiche socle.
• Marquez — backend référence.
• Data lineage — concept générique.