Apache Parquet V3

Parquet 2.10 adds encryption and variant type.

Definition

Parquet 2.10 (May 2024): Parquet Modular Encryption (PME) GA, semi-structured variant type aligned with Iceberg / Delta UniForm, geometry WKB type, column index improvements, bloom filter encoding. Parallel 2.11 + 2.12 evolutions in progress 2024. Compatible with Apache Spark, Apache Iceberg, Delta Lake, Apache Hudi, Trino, Snowflake, BigQuery.

Origin

Twitter + Cloudera 2013, Apache Software Foundation top-level 2015. 2.x versions since 2017.

Usage

Standard columnar file format for data lakes and lakehouse. Storage backbone for Iceberg / Delta / Hudi tables.

Related terms

• Parquet — fiche socle.
• Apache Iceberg V3 — consommateur principal.
• Delta Lake V4 — consommateur principal.