New features

SAP Extractor fetcher

The SAP Extractor Fetcher, which facilitates data extraction from SAPI extractors, has been enhanced. In the latest version, delta load functionality is now supported. This enhancement introduces an additional delta type, Datasource Delta which, when enabled, allows delta loads for datasources that support this feature. Additionally, Delta repeat (recovery) is automatically activated during a delta load failure, ensuring data consistency and reliability. 

Data aging

Data aging in SAP refers to the process of managing data within the SAP HANA database by distinguishing between current ("HOT") data and old ("COLD") data. This concept allows for the movement of less relevant operational data from the current area to a historical area, thereby optimizing memory usage and improving performance.

SAP table fetcher now supports cold data replication from SAP tables with Data aging enabled.

Microsoft Fabric

Microsoft Fabric is a distributed systems platform by Microsoft designed to simplify the development, deployment, and management of scalable and reliable microservices-based applications.

SNP Glue now provides a dedicated connector, leveraging the Open Mirroring in Microsoft Fabric to enable integration of SAP systems with the Azure Fabric.

Improvements

Scheduler

• A new search functionality has been introduced in the SNP Glue Scheduler, allowing users to locate extraction processes more efficiently.

• Additionally, within the SNP Glue Scheduler, an option to truncate data on the target storage has been implemented for the EXT2 scheduler type. This enhancement eliminates creating a separate variant for each extraction process in drop-and-reload scenarios.

CDC

• SNP Glue Trigger Management now provides the capability to display the definition of the database trigger generated by SNP Glue. This feature is particularly useful for troubleshooting issues related to database compatibility and other related investigations.

CDS views

• The stability and robustness of the solution for replicating data from SAP CDS views have been enhanced.

• Support for CDS views with names exceeding 30 characters has been introduced. Due to SNP Glue’s 30-character table name limitation, CDS view names will be truncated to the first 30 characters. To ensure uniqueness, an incremental identifier will be appended when multiple CDS views share the same truncated name.

SLT

• The configuration of a Separate Box Deployment, where SLT and SNP Glue run on separate systems, has been streamlined. The SLT-specific logic has been separated from the SNP Glue core, eliminating the need for DMIS or the Glue SLT Add-on in the target SAP system.

Storage Management

AWS Redshift

• A new version of the Redshift connector has been introduced that leverages the AWS SDK.

Kafka

• The Kafka connector now includes an option to specify sensitive connection details, such as the SSL mechanism, username, and password, through a configuration file.

• The Kafka connector now supports schema drift scenarios.

Azure SQL

• Azure SQL connection now supports page compression.
  Page compression is an advanced data compression technique in Azure SQL that minimizes storage space and improves performance by reducing I/O operations. For more information, please refer to the official Microsoft documentation.

Fixes

Extractors

• The modification of the storage type from file-based to tabular storage resulted in residual SNP Glue metadata remaining in the SAP system. The logic has been adjusted to ensure the removal of all unnecessary metadata.

• An issue affecting delta initialization when using the ADSO Fetcher has been resolved. The logic has been corrected, and the procedure now functions as expected.

• A rare scenario where the SNP Glue Scheduler could be scheduled multiple times has been addressed. The scheduling process now ensures that it can only be executed once.

CDS views

• In some cases, the option to Import Fields from CDS was not available even if the CDS plugin was installed.

• In rare cases, field descriptions were missing when using the Import Fields from CDS functionality.

• Sometimes a CLIENT field in the underlying table during TRIGGER delta operations caused a shortdump, as the CDS views didn’t include this field. The issue has been resolved and so the stability and error-free processing is guaranteed.

Storage Management

• It wasn't possible to replicate data to a Snowflake table containing a field named “EMPTY.”

• If a newline character was inserted into AWS Redshift, the operation failed.
  To address this, a new parameter, Wrap values in staged CSV files has been added to AWS Redshift storages. When enabled, all values in staged files are quoted, ensuring that all characters, including newlines, are transferred correctly without modification.

• Kafka storage didn’t work properly when using AVRO files.

• Cloudera Hadoop storage did not function correctly on Windows machines. The implementation has been adjusted to ensure proper operation in this environment.