New Features
...
Glue Scheduler with Trigger Delta
...
| Note |
|---|
A Glue Scheduler is a tool for scheduling Glue extractions and is a central place for scheduling and management of all the replication jobs. |
Glue Scheduler introduced in the previous release now provides a new feature to schedule trigger delta replication jobs.
Similar to scheduling extractions from Glue queues, you can create your own configurations, and specify multiple parameters like the number of jobs, priority or a user for the scheduled job, and extraction type (whether it is executed as a batch job or as a streaming replication), and . You can also choose extraction processes that will be executed as part of the configuration.
...
By default, the scheduler covers the full end-to-end scheduling flow. In the beginning, the full load is executed - replicating the full table content, after it is completed, the delta mode starts automatically. This mechanism enables customers to easily configure the complete replication scenario.
Standalone JCO
SNP supports a configuration when the Java connector is running standalone on a separate machine.
Java connector Management screen provides a new monitor with an overview of existing Java connections. Using the tool, you can establish a connection to a standalone Java instance, or configure and set up a local Java connector.
Improvements
Data Dictionary
In some cases, DDIC fields contain reserved names. A new solution has been implemented to ensure that such names are automatically adjusted with predefined suffixes during the activation of the Glue table.
...
The performance and stability of the whole solution have been improved significantly.
Data Validation mechanisms have been introduced to cover the entire replication scenario from SLT to the target platform.
Verification of whether the data obtained from SLT is correctly stored in internal SNP Glue tables
Validation of whether the data read from Glue queues matches the data in the target platform
SLT data validation mechanisms have been improved. You can now set up a validation per specific table.
Extractors
Glue extraction process
...
generation of the ABAP report has been
...
improved to prevent the regeneration of all extraction processes after each upgrade of the Glue software.
CDC
You can now use the trigger delta directly of a clusteron table clusters (for example RFBLG, which is the table cluster for BSEG).
Glue triggers can now correctly capture changes in the key fields. Such a case is handled as a combination of DELETE and INSERT operations on the DB level.
The trigger implementation for all the supported DBs has been adjusted to handle this properly.
...
A Changelog for storage management parameters has been introduced. This feature allows you to see the history of changes in Storage Management.
During the replication of data to Snowflake, the data is first copied in the form of a CSV file to the Snowflake Internal Stage. The file name used to be a generated sequence of characters but has been adjusted to contain storage ID and Glue table name. This is useful when troubleshooting to assign the created file to the specific Snowflake table.
Snowflake COPY command now uses rows validation The process of data insertion to the Snowflake table now contains a validation mechanism to guarantee data consistency.
Java Call Repeat and Repeat Delay parameters have been added for Google Cloud Storage and BigQuery storageGoogle Cloud Storage and BigQuery storage have been improved to properly work even in an unstable network environment.
Authorizations
There have been improvements in the authorization process for different parts of our system, including both Glue and Storage Management:
New transactions were created for Glue Scheduler (/DVD/GLS) and Glue Backup (/DVD/RL_TABLE_BACKUP) with corresponding transaction code authorizations.
A new Glue action called MAN_SETT was added to manage Glue Settings.
For storage management, a new action called EDIT_JCO was added for editing JCO Management.
Fixes
Glue Cockpit
The cockpit didn’t count calculation of failed jobs in the daily statistics. With the adjusted functionality, also the jobs that “Ended with error” are counted.Glue cockpit has been corrected.
Extractors
You can optionally add the GLCHANGETIME field to a Glue table. By doing so, the “Change Time“ Change Time rule is automatically assigned to the field. In the previous versions, the logic was not consistent for multiple scenarios. This has been corrected and the values are now consistent for all types of extraction - be it a table replication, ODO, DSO, or ADSO. The inserted GLCHANGETIME value is a timestamp in the UTC timezone.
The filtering of data in case of mass execution has been improved. The solution is now stable and consistent for all supported scenarios. Filters are now considered also when calculating the packages. This will prevent random size or even empty calculated packages.
BEx Query (BICS) mass execution was not working with a split on 2 variables. This has been corrected and the functionality now works as expected.
The replication was not working for a Glue table created from BEx Query with a characteristic that had a Display setting set to “No Display“.
BEx Query (BICS) fetcher filters were not working correctly due to incorrect mapping between field names in a fetcher and a process. This has been corrected and filters now work correctly.
...
No Display.
Glue Scheduler
Queue Scheduler didn’t replicate all the queues if there were more tables with data in the queue than the number of jobs. The logic has been modified to correctly handle the replication of all the specified replicationsselected queues.
In case there was a short dump, a scheduler job The Glue Scheduler was adjusted to work properly after the system restart was not cleaned up, therefore with the next scheduler start, another job was scheduled, resulting in multiple scheduler instances after the system restart.
CDC
Date, Value, and Timestamp delta capture have been corrected to work properly also on S4 systems.
...
GLUE-4799 Watchdog schedules multiple jobs after restart
...
SLT
TMS
...
BEx Query (BICS) variants with ABAP code for mass execution data split were not transported correctly. The logic has been fixed and now you can transfer also variants with the code.
It was not possible to import Glue expert settings using Glue TMS.
Additional Features
The performance of mass activation and deletion of Glue objects has been improved.
...
Glue REST API
...
GLUE-4897 Glue table disappears from gl80
GLUE-4870 Mass activation/deactivation issues
GLUE-4735 ECB- failed import\(activation\) of Glue experts settings
GLUE-4466 Discrepancy between SM37 and Glue monitor
Storage Management
GLUE-4898 Don't try to programmatically start JCO if remote
GLUE-4876 Output stream \+ empty first package \+ empty load data files
GLUE-4842 API issue: parameter load type used for database hint
Known issues
...
has been corrected to properly support ODP extraction.
Storage Management
AWS S3 storage contained a bug which in some cases resulted in inserting a NULL character into the generated file.
It was impossible to create a Snowflake connection with the key pair authentication.
MsSQL didn’t support case-sensitive primary keys.
SAP ASE didn’t support the INT8 data type.
Known issues
Storage Management
Merge operation is not supported for Google BigQuery storage.
Redshift storage doesn’t support direct data insertion (without using the staging table). As a workaround, you can use insertion via staging files.