New Features
SLT Integration
Datavard Glue provides an option to integrate SAP LT with its data extraction framework. This enables you to leverage the existing SLT implementation and provide the ever-growing Glue support for multiple targets,
The solution enables you to:
Support batch and real-time data integration
Write to multiple targets, leveraging queues
Control load and performance
Deploy Datavard Glue on separate box from SLT
Handle new sources/targets in an automated way
The integration leverages SAP LT SDK to write to custom ABAP-based targets via BADI implementation. The end-to-end process is seamless and automated.
Azure SQL
You can now connect /wiki/spaces/ReuseLib/pages/3645014181 as a target in storage management
Delta Lake support
Delta lake is a file storage based on parquet. It adds metadata information (JSON) on top of the Parquet data to enable a database like behavior, keeping the data open and accessible from different sources.
Glue now provides an option to generate the required JSON files and thus allows the Glue folder to be easily processed by standard data lake technologies (like spark) or accessed by query engines like Azure serverless SQL or AWS Athena.
External types mapping
Datavard Glue now provides a way to manage type mapping between SAP and external environments for both when exporting the data and importing/virtualizing them back.
The default type mapping can be now overridden with customized settings and used for the extraction or virtualization of the data.
Improvements
Extractors
Similarly to ADSO, also DSO extraction objects support automatic adjustments of DSO structure. If you add a new object to your DSO you no longer have to adjust the structure of your Glue table manually.
BEx Query (BICS) extraction enables you to transfer data from Key Figures in an external format, i.e. currency conversion can be applied.
Using Listcube fetcher, you can transfer data from navigational attributes of BW objects. You can also use these attributes as an input for selection before the extraction.
When extracting the data into files using Mass extraction, the generated JSON files provide you information that allow you to assign the files to specific extraction process. The JSON files also contain information whether some data has been marked as truncated during the execution of the extraction.
Additional field GLCHANGETIME can be added to a glue table. This field holds the information about when the entry was captured by trigger (when using trigger delta) or when the extraction was executed (for other delta types)
Glue monitor allows you to filter the extraction jobs based on the execution time using from/to parameters.
HANA Native extraction
Glue enables you to define a DB schema used for HANA table in Hana view fetcher. By default, the schema is defined in the secondary DB connection (using DBCO transaction) but now you can choose a different one if needed.
You can now use parallel mass extraction using HANA View fetcher.
CDC
When using trigger delta, the number of entries captured by the trigger is displayed in Trigger Management screen.
Glue Queues
Queue watchdog enables parallel extraction of data from a queue.
Stability and robustness of the solution has been improved significantly.
TMS
You can now transfer also variants of extraction processes using Glue TMS.
Additional features
Glue Cockpit has been reorganized so that you can easily access the most relevant features.
Object Navigator enables you to perform mass operations - you can deactivate or delete all the selected Glue objects.
Fixes
Data Dictionary
The creation of a Glue table was failing if SAP GUI language was different from English. This has been corrected and you can successfully create a Glue table using any SAP GUI language.
Extractor
When replicating data into parquet files, java JCO connector is required. Java was not started automatically during the extraction. This has been corrected and the process now works without manual intervention.
Mass execution enables you to use data split based on max number of records or size in MB even if the partitioning field for the calculation of data chunks is the same as the field on which you do the selection of data.
Calculation of package size was not supported for Hana View Fetcher.
If HANA table contained a field with dot “.”, the extraction was failing. The problem has been corrected and the extraction now works all for this case.
Several improvements have been implemented for DSO fetcher that allow you to modify records modes or add REQUEST ID to a selection.
Mass execution of ADSOs and BEx Queries (using BEx Query Fetcher (BICS)) was not working correctly. The logic has been adjusted to support these scenarios.
When extracting the data into parquet files, the generated JSON metadata files didn’t contain the information about created data files.
In parquet files, the date types were mapped to varchar. This has been adjusted and by default the data is mapped directly into date format. The mapping can be now customized by using External types mapping described in the “New Features” chapter.
Glue Monitor
Glue monitor was not showing selection values for cancelled jobs of BEx Query mass execution. This has been corrected and now all the selection values are properly displayed.
The monitor was not showing Variant description in case user was using language different from English.
The performance of the monitor has been improved and the tool can now handle large amounts of Glue extraction jobs.
CDC
Value delta was not working on cluster tables. This has been corrected and now you can use this type of delta also on cluster tables.
While using Glue Delta Watchdog, if an extraction failed the watchdog terminated the running process and didn’t wait for the other running jobs to finish. The logic has been corrected and the watchdog handles such situations properly now.
Additional features
The parameters of Glue Object Generator were optimized to enable stable generation of Glue objects with the desired settings.
Glue profiles have been reworked to enable better user experience.
TMS
Transport of a fetcher with delta timestamp was failing.
Transport requests in the Glue TMS selection help were not sorted. They are now sorted by the time of their creation.