New Features
Dynamic Scheduler with Trigger Delta
Dynamic Scheduler introduced in the previous release now provides a new feature to schedule trigger delta replication jobs.
Similar to scheduling extractions from Glue queues, you can create your own configurations, and specify multiple parameters like the number of jobs, priority or a user for the scheduled job, extraction type (whether it is executed as a batch job or as a streaming replication), and choose extraction processes that will be executed as part of the configuration.
The Trigger Delta scheduling provides you an option to choose whether you want to replicate only the data captured by the generated DB trigger or also the complete table content.
By default, the scheduler covers the full end-to-end scheduling flow. In the beginning, the full load is executed - replicating the full table content, after it is completed, the delta mode starts automatically. This mechanism enables customers to easily configure the complete replication scenario.
Improvements
Data Dictionary
In some cases, DDIC fields contain reserved names. A new solution has been implemented to ensure that such names are automatically adjusted with predefined suffixes during the activation of the Glue table.
SLT Integration
The performance and stability of the whole solution have been improved significantly.
Data Validation mechanisms have been introduced to cover the entire replication scenario from SLT to the target platform.
Verification of whether the data obtained from SLT is correctly stored in internal SNP Glue tables
Validation of whether the data read from Glue queues matches the data in the target platform
Extractors
Glue extraction process reports contained ABAP .INCLUDE statement. This has been removed to prevent the regeneration of extraction processes after each upgrade of the Glue software.
CDC
You can now use the trigger delta directly of a cluster.
Glue triggers can now correctly capture changes in the key fields. Such a case is handled as a combination of DELETE and INSERT operations on the DB level.
The trigger implementation for all the supported DBs has been adjusted to handle this properly.
Storage Management
A Changelog for storage management parameters has been introduced. This feature allows you to see the history of changes in Storage Management.
During the replication of data to Snowflake, the data is first copied in the form of a CSV file to the Snowflake Internal Stage. The file name used to be a generated sequence of characters but has been adjusted to contain storage ID and Glue table name. This is useful when troubleshooting to assign the created file to the specific Snowflake table.
Snowflake COPY command now uses rows validation to guarantee data consistency.
Java Call Repeat and Repeat Delay parameters have been added for Google Cloud Storage and BigQuery storage.
Fixes
Glue Cockpit
The cockpit didn’t count failed jobs in the daily statistics. With the adjusted functionality, also the jobs that “Ended with error” are counted.
Extractors
You can optionally add the GLCHANGETIME field to a Glue table. By doing so, the “Change Time“ rule is automatically assigned to the field. In the previous versions, the logic was not consistent for multiple scenarios. This has been corrected and the values are now consistent for all types of extraction - be it a table replication, ODO, DSO, or ADSO.
The filtering of data in case of mass execution has been improved. The solution is now stable and consistent for all supported scenarios. Filters are now considered also when calculating the packages. This will prevent random size or even empty calculated packages.
BEx Query (BICS) mass execution was not working with a split on 2 variables. This has been corrected and the functionality now works as expected.
The replication was not working for a Glue table created from BEx Query with a characteristic that had a Display setting set to “No Display“.
BEx Query (BICS) fetcher filters were not working correctly due to incorrect mapping between field names in a fetcher and a process. This has been corrected and filters now work correctly.
GLUE-4821 Glue - Queue extraction is not using F/D information from the CLH table
Glue Scheduler
Queue Scheduler didn’t replicate all the queues if there were more tables with data in the queue than the number of jobs. The logic has been modified to correctly handle all the specified replications.
In case there was a short dump, a scheduler job after the system restart was not cleaned up, therefore with the next scheduler start, another job was scheduled, resulting in multiple scheduler instances after the system restart.
CDC
Date, Value, and Timestamp delta capture have been corrected to work properly also on S4 systems.
GLUE-4799 Watchdog schedules multiple jobs after restart
GLUE-4694 Watchdog behavior
SLT
TMS
GLUE-4843 Glue - Variant with ABAP code not transported correctly
Additional Features
The performance of mass activation and deletion of Glue objects has been improved.
GLUE-4955 GLue REST API - ODP parameters not working
GLUE-4897 Glue table disappears from gl80
GLUE-4870 Mass activation/deactivation issues
GLUE-4735 ECB- failed import\(activation\) of Glue experts settings
GLUE-4466 Discrepancy between SM37 and Glue monitor
Storage Management
GLUE-4898 Don't try to programmatically start JCO if remote
GLUE-4876 Output stream \+ empty first package \+ empty load data files
GLUE-4842 API issue: parameter load type used for database hint
Known issues
Extractors
Storage Management
Merge operation is not supported for Google BigQuery storage.
Redshift