(Glue-2011) Release Notes 2011

New Features

Glue REST-API

Glue provides the REST-API interface to manage and control the Glue extraction process. This gives you an option to control Datavard Glue from a non SAP GUI environment.

Selective deletion

Using selective deletion, you can delete the selected data from the target storage.
Currently, there are two scenarios supported:

• hard deletion (available for Transparent Binary storage, Hadoop, and SQL like execution engines) which deletes the entries from the storage

• soft deletion (available for Transparent Binary storage, Hadoop Hive, and Impala) - inserts flag into a dedicated column to indicate that the entry is considered as deleted.

SAP Table Consumer

You can transfer data from external storage into the SAP table using a new extraction object called SAP Table Consumer. This way you can leverage scenarios when data from multiple sources are joined in an SAP environment and used for further processing or reporting.

Storage Management

Transparent-binary storage
Transparent-binary storage takes the advantage of cheap file storages such as HDFS, Azure Data Lake, AWS S3, and provides additional logic to accomplish classic transparent RDB-like functionality.
Using unique architecture of this storage, we can simulate table, querying, or data manipulation behavior by supporting SQL commands such as DDL, DML, or DQL.

Improvements

Mass Execution

• When configuring the mass execution to run the process in parallel, you can specify a server group that will be executing the process. 

• Mass execution is now available also for Extractor 1.0.

Mass Operations

• Using mass activation features you can now perform various operations in bulk also over Glue objects created based on SAP cluster tables.
  The specific variant can be saved and reused later.

Extractors

• FIFO extraction parametrized - New parameters were introduced for both FIFO fetcher and FIFO consumer to manage the extraction process and to prevent deadlocks during the execution.
  Also, Glue now provides additional information about FIFO extraction in generated JSON files.

• InfoCube and DSO extractors provide you additional options that give you more control over the extraction process. You can now reset the delta of an extractor or enable/disable the DB aggregation.

• The Glue request monitor now enables you to:

  • kill a job or group of running jobs in case of mass execution.

  • reload the data in the Glue request monitor using the F8 key.

CDC

• New functionality was created to ensure that the streaming jobs keep running and to restart them in case of their accidental cancellation.

• When extracting data using trigger delta from a cluster table, you can now capture all types of operations (INSERT, UPDATE, DELETE) on a table.

• Glue Trigger Management provides you information about the actual status of a database trigger and also options to restore the trigger if it is in an inconsistent state.

• We have also prepared a functionality to help you with the cleanup CDC related DB objects after the system refresh.

Glue cleanup

• A glue cleanup tool is used to remove the data of damaged Glue tables.
  This may be useful if standard deletion of a Glue table fails for various reasons - e.g. the target storage no longer exists, the metadata was corrupted or became inconsistent.

Data Browser

• Using Glue Data Browser, you can display the data stored in file storage.

Data Dictionary

• When extracting data into Transparent-binary storage, you can specify a folder in a File (binary) storage where the files will be stored.

Fixes

Data Dictionary

• You can no longer create a Glue table from an existing external table (e.g. Redshift, Hive, or BigQuery table) if there is another Glue table already created for it.

• It was not possible to change the storage of a Glue table and successfully activate it. This has been corrected and such a modification is now possible.

Data Browser

• In Glue Data Browser, the select options filter was limited to 45 characters only. This limitation has been increased to 255 characters.

• When querying the data, Glue was replacing multiple empty spaces with a single one. This misleading behavior has been removed and the original value is used.

Extractor 1.0

• The glue request monitor was not showing any controls for DSO extractors. This has been fixed and all the features are available for DSO extractors as well.

• DSO delta extraction has been corrected to transfer only the relevant data.

• Glue extractor was trying to insert an empty package to file storage if there was no data to transfer which was causing problems on the storage and eventually lead to process failure. The logic has been corrected and the process is working as expected now.

Extractor 2.0

• You couldn’t delete the FIFO Glue request from the Glue request monitor.

• FIFO extraction was failing on tables with .INCLUDE. This has been corrected and now you can extract the data from such tables as well.

• BEx Query fetcher was not able to replicate data from an SAP BEx Query if the characteristic was displayed as a hierarchy, or if compounded navigation attributes or expanded key figures were used. All the mentioned scenarios are working now and you can extract data without any issues.

CDC

• The stability of trigger CDC was improved to operate flawlessly even during the extensive usage of an SAP system.

• When using trigger delta, Glue was clearing the records captured by a trigger even after the extraction failed. This was causing data loss and full data reload was required to get data into a consistent state again. The logic has been correct and now you can restore the extraction without data loss even if the process has been stopped for some reason.

• When running delta jobs from Delta watchdog, they were not displayed in the Glue request monitor.

Transport Management

• You couldn’t import the Glue table with modified File storage options.