Loading data into pg_mooncake v0.1 USING columnstore tables from Postgres tables, Parquet/CSV/JSON files, Iceberg/Delta tables, and HuggingFace.
Legacy pg_mooncake v0.1 Archive (Delta Lake Engine)
This page documents pg_mooncake v0.1, which used direct CREATE TABLE ... USING columnstore DDL and Delta Lake storage. Starting in v0.2 and PGSTY’s v0.3.0 release, pg_mooncake uses moonlink real-time columnstore mirrors (CALL mooncake.create_table + USING mooncake) and Apache Iceberg v2. For current documentation, see Introduction and v0.1 vs v0.2 & v0.3.
-- Using COPY
COPYusers_columnstoreFROM's3://bucket/file.parquet';COPYusers_columnstoreFROM's3://bucket/folder/*.parquet';COPYusers_columnstoreFROM's3://bucket/data.csv'WITH(FORMAT'csv',HEADERtrue);-- Using INSERT with mooncake.read_parquet
INSERTINTOusers_columnstoreSELECT*FROMmooncake.read_parquet('s3://bucket/file.parquet')AS(idINT,nameTEXT,created_atTIMESTAMP);
From Iceberg, Delta Lake, or HuggingFace (v0.1)
-- Iceberg
INSERTINTOusers_columnstoreSELECT*FROMiceberg_scan('s3://bucket/table')AS(idINT,nameTEXT,created_atTIMESTAMP);-- Delta Lake
INSERTINTOusers_columnstoreSELECT*FROMdelta_scan('s3://bucket/table')AS(idINT,nameTEXT,created_atTIMESTAMP);-- HuggingFace
COPYusers_columnstoreFROM'hf://datasets/OpenLeecher/lmsys_chat_1m_clean/data/train-00000-of-00004.parquet';