DATAHUB-SPEC

DataHub open-source metadata platform.

Definition

DataHub uses Pegasus (PDL) metadata model, GraphQL and REST APIs, Python ingestion framework. 80+ connectors: Snowflake, BigQuery, Redshift, dbt, Airflow, Looker, Tableau, Power BI. ES-backed search, column-level lineage. Apache 2.0 license.

Origin

Created by LinkedIn 2015 ; open-sourced 2020 ; Acryl Data 2021.

Example in context

A data engineer searches "customer_id" in DataHub and finds the 47 tables containing it.

Related terms

• Apache Atlas — open-source competitor.