(Glue-2411) Replication to Delta Lake

Owned by SNP
yesterday at 10:37 am
1 min read

SAP data could also be replicated into delta lake files only on Binary storages which support parquet files.
Proceed as follows to set it up.

Delta Lake integration setup

Set profile

The following parameters should be set in SNP Glue Profile Settings:

  • Use file type PARQUET.

  • Select Create Delta Lake metadata in the file settings.

SNP Glue table setting

When setting a Storage ID the File storage options button is displayed. Select a specific profile created above or check Custom options. Here is also required to use the PARQUET file type and check Create Delta Lake metadata.

Metadata files

Additional metadata files are created in the Glue table directory. Under directory _delta_log, there are JSON files containing information for Delta Lake, such as CREATE TABLE also with the table schema, or WRITE with a list of created files.