(Glue-2111) Release Notes 2111

Owned by SNP
Last updated: Nov 22, 2021 by Marian Vidra
4 min read

New Features

SLT Integration

Datavard Glue provides an option to integrate SAP LT with its data extraction framework. This enables you to leverage the existing SLT implementation and provide the ever-growing Glue support for multiple targets,

The solution enables you to:

  • Support batch and real-time data integration

  • Write to multiple targets, leveraging queues

  • Control load and performance

  • Deploy Datavard Glue on a separate box from SLT

  • Handle new sources/targets in an automated way

The integration leverages SAP LT SDK to write to custom ABAP-based targets via BADI implementation. The end-to-end process is seamless and automated.

Azure SQL

You can now connect Azure SQL as a target in storage management 

Delta Lake support

Delta lake is file storage based on the parquet. It adds metadata information (JSON) on top of the Parquet data to enable a database-like behavior, keeping the data open and accessible from different sources.

Glue now provides an option to generate the required JSON files and thus allows the Glue folder to be easily processed by standard data lake technologies (like a spark) or accessed by query engines like Azure serverless SQL or AWS Athena.

External types mapping

Datavard Glue now provides a way to manage type mapping between SAP and external environments for both when exporting the data and importing/virtualizing them back.

The default type mapping can be now overridden with customized settings and used for the extraction or virtualization of the data.

Improvements

Extractors

  • Similar to ADSO, also DSO extraction objects support automatic adjustments of DSO structure. If you add a new object to your DSO you no longer have to adjust the structure of your Glue table manually. 

  • BEx Query (BICS) extraction enables you to transfer data from Key Figures in an external format, i.e. currency conversion can be applied.

  • Using Listcube fetcher, you can transfer data from navigational attributes of BW objects. You can also use these attributes as input for selection before the extraction.

  • When extracting the data into files using Mass extraction, the generated JSON files provide you with information that allows you to assign the files to a specific extraction process. The JSON files also contain information on whether some data has been marked as truncated during the execution of the extraction.

  • Additional field GLCHANGETIME can be added to a Glue table. This field holds the information about when the entry was captured by trigger (when using trigger delta) or when the extraction was executed (for other delta types)

  • Glue monitor allows you to filter the extraction jobs based on the execution time using from/to parameters.

HANA Native extraction

  • Glue enables you to define a DB schema used for the HANA table in the HANA view fetcher. By default, the schema is defined in the secondary DB connection (using DBCO transaction) but now you can choose a different one if needed.

  • You can now use parallel mass extraction using the HANA View fetcher.

CDC

  • When using trigger delta, the number of entries captured by the trigger is displayed in the Trigger Management screen.

Glue Queues

  • Queue watchdog enables parallel extraction of data from a queue.

  • Stability and robustness of the solution have been improved significantly.

TMS

  • You can now transfer also variants of extraction processes using Glue TMS.

Additional features

  • Glue Cockpit has been reorganized so that you can easily access the most relevant features.

  • Object Navigator enables you to perform mass operations - you can deactivate or delete all the selected Glue objects.

Fixes

Data Dictionary

  • The creation of a Glue table was failing if SAP GUI language was different from English. This has been corrected and you can successfully create a Glue table using any SAP GUI language.

Extractor

  • When replicating data into parquet files, a Java JCO connector is required. Java was not started automatically during the extraction. This has been corrected and the process now works without manual intervention.

  • Mass execution enables you to use data split based on the max number of records or size in MB even if the partitioning field for the calculation of data chunks is the same as the field on which you do the selection of data.

  • Calculation of package size was not supported for HANA View Fetcher.

  • If the HANA table contained a field with dot “.”, the extraction was failing. The problem has been corrected and the extraction now works all for this case.

  • Several improvements have been implemented for the DSO fetcher that allows you to modify records modes or add REQUEST ID to a selection.

  • Mass execution of ADSOs and BEx Queries (using BEx Query Fetcher (BICS)) was not working correctly. The logic has been adjusted to support these scenarios.

  • When extracting the data into parquet files, the generated JSON metadata files didn’t contain the information about created data files.

  • In parquet files, the date types were mapped to varchar. This has been adjusted and by default, the data is mapped directly into date format. The mapping can be now customized by using External types mapping described in the “New Features” chapter.

Glue Monitor

  • Glue monitor was not showing selection values for canceled jobs of BEx Query mass execution. This has been corrected and now all the selection values are properly displayed.

  • The monitor was not showing a Variant description in case the user was using any other language than English.

  • The performance of the monitor has been improved and the tool can now handle large amounts of Glue extraction jobs.

CDC

  • Value delta was not working on cluster tables. This has been corrected and now you can use this type of delta also on cluster tables.

  • While using Glue Delta Watchdog, if an extraction failed the watchdog terminated the running process and didn’t wait for the other running jobs to finish. The logic has been corrected and the watchdog handles such situations properly now.

Additional features

  • The parameters of the Glue Object Generator were optimized to enable stable generation of Glue objects with the desired settings.

  • Glue profiles have been reworked to enable a better user experience.

TMS

  • Transport of a fetcher with delta timestamp was failing.

  • Transport requests in the Glue TMS selection help were not sorted. They are now sorted by the time of their creation.