(Glue-2408) Release Notes 2408

Owned by SNP
Last updated: Sept 11, 2024 by Robert Adamec
5 min read

Kindly be informed that starting with SNP Glue version 2408, the SNP Java Connector now requires JRE 11.

In prior releases, the minimum required version was JRE 8; however, this has changed, and JRE 11 is now the minimal required version.

Please note that SNP plans to decommission the Enable Merge feature in the SNP Glue extraction process. Starting with release 2502, this feature will no longer be configurable and will be fully removed in release 2511. 

The "Enable Merge" feature was originally introduced to perform delta updates on connected data storage. However, its direct integration with the load process has proven to be inefficient. We now offer more efficient methods for performing merges by decoupling the load and merge processes, utilizing platform-based solutions such as our native app. This approach not only improves performance but also reduces credit consumption.

As a result, we have decided to transition to these improved functionalities and decommission the "Enable Merge" feature. This change affects only "Enable Merge" and does not impact the "Enable Update" feature or customers replicating to SQL databases.

New Features

SAP Extractor fetcher

Preview feature

In the latest release, a newly introduced SAP Extractor fetcher enables extraction from SAPI extractors. The SNP Glue SAPI fetcher harnesses SAPI technology, facilitating the use of SAP application extractors for generic data extraction.

Please note that in the current release, only full load is available.

CDC support for SAP CDS view

The CDS fetcher, introduced in release 2402, initially supported only full-load replication. The latest release introduces enhanced capabilities for delta load replication.

Various options are available for delta replication, including generic SNP Glue delta mechanisms such as timestamp, date, value, and value distinct, or a Delta-enabled CDS view. The delta type for a Delta-enabled CDS view is determined by its CDS annotation.

For more information, please refer to our documentation.

SAP mirroring

SNP Glue enables mirroring of SAP systems. In combination with SAP SLT and SNP Glue SLT Add-on, SNP Glue facilitates the replication of data between two SAP systems, providing a solution when SAP SLT itself cannot replicate tables due to structural differences in tables between S/4 and non-S/4 systems. For additional details, refer to the provided documentation.

Microsoft OneLake storage

SNP Glue can extract data to Azure OneLake using the newly implemented storage that leverages Microsoft Azure Java SDK.

Azure OneLake is a unified, enterprise-grade data lake that centralizes all data for organizations across multiple clouds and formats, making it easier to store, manage, and analyze large datasets. It is part of Microsoft's Fabric platform, offering integrated security, governance, and a simplified data management experience. In the upcoming releases, we are planning a more extensive integration with Azure Fabric.

Improvements

Extractors

  • A new column has been added to the request monitor. This displays the executing server of an extraction.

  • Parallel processing (mass execution) for SAP table fetchers and BEx Query fetchers (BICS) has been enhanced. The latest release allows the use of all available split options for parallel replication.

Scheduler

  • The SNP Glue scheduler type Sequence has been enhanced.
    Previously, the logic allowed for defining the execution order of individual extraction processes, where processes with the lowest sequence number were initiated first, and processes with higher sequence numbers were executed only after all processes with lower sequences were completed.

    In the latest release, alternative processing of extraction processes defined in the scheduler has been enabled. The logic now permits an extraction process, even from the subsequent sequence, to start as soon as a job becomes available, without waiting for the completion of all jobs specified in the current sequence.

  • A new report, SNP Glue scheduler starter (/DVD/GL_SCHEDULER_START), has been introduced. This report facilitates the initiation of specified schedulers with the streaming execution type. It is particularly useful for automatically starting schedulers after an SAP system restart.

TMS

  • If an activation of a transport in the background fails, an error message is displayed in a job log. The detailed information is available at /DVD/GL_SLG.

Snowflake streaming

  • Several enhancements have been implemented in the Snowflake Streaming connector, including the integration of logs from the Snowflake Ingest SDK into the standard SNP Java connector logs. This integration provides increased visibility into internal processes during data replication to Snowflake Streaming, facilitating more efficient troubleshooting of connectivity issues.

  • New parameters have been introduced to manage the Java memory consumption used by the Snowflake SDK.

  • Snowflake database table is now created with metadata that contains the description of the table and its columns.

Google BigQuery streaming

  • SNP Glue now supports the integration of BigQuery Storage Write API with Google Cortex.

Fixes

Extractors

  • Trigger delta capture with the parameter “Read deleted entries“ didn’t work properly for cluster tables. This has been corrected and the scenario works as expected.

  • SNP Glue queues have been reworked to optimize the performance and decrease the number of locks that the logic requires. This is important mostly for scenarios when a large number of small packages is transferred using the SNP Glue queues mechanism.

Scheduler

  • In the Extraction 2.0 scheduler, it was possible to save multiple variants of the same extraction process. This has been restricted and now only one variant is permitted.

CDS

  • In the previous release, the replication of data from a CDS view with lowercase field names was not functional. This has been corrected.

  • In the SNP Glue data dictionary, when creating an SNP Glue table from the existing CDS view, it was required to fill in the name of the view manually, as there was no Value help provided. This has been added and the Value help is available for this use case as well.

SLT

  • Previously, when replicating data using SLT from a source system with the HANA database, the replication of boolean types was unsuccessful. This issue has now been resolved, and the replication of boolean types functions correctly. Similarly, the replication of DATS values with incorrect length is supported.

Storage management

  • The issues with ADLS Gen2 storage have been resolved in the following areas:

    • OAuth Authentication: The compatibility problem with OAuth authentication when using the Java HTTP client has been fixed, ensuring proper functionality.

    • _m.json File Deployment: The Java client can now deploy _m.json files without the need to populate the destination RFC.

    • Delta Lake Metadata: The handling of Delta Lake metadata with the Java client is no longer broken.

    • Parquet Files: The use of parquet files is now possible when the HTTP client is configured to ICM.

    These updates have improved the reliability and versatility of ADLS Gen2 storage operations.

  • The installation of the SNP Java connector previously depended on the availability of the 7zip tool, leading to failures if the tool was not present on the SAP system. To address this, the latest release has transitioned to utilizing the standard unzip command, thereby circumventing such issues.