(Glue-2108) Transfer Rules

Owned by SNP
Aug 17, 2021

Introduction

With Datavard GLUE Transfer rules you can modify the data during the extraction process. This process is similar to transfer routines within BW transformations. The functionality can be used for adjusting data, changing SAP abbreviations to more human-readable form, currency conversion, etc. In this document, you can find some basic information about the transfer rules and current implementation. 

Transfer rules are set for each field separately. This functionality behaves as an end routine in BW transformations, where we change/adjust data stored in a target structure.

Currently, there are these options for transfer rules: 

  • Direct assignment – just copies one to one data from a source field to a target field
  • Routine – a custom implementation which changes the value in the selected field
  • Currency – transformation of currency values according to a currency key 
  • Meaningful field values – translation of values according to a domain, mapping table, SID, or a custom conversion routine 

  • Business function – adjust values with business functions

  • Trigger operation type - in trigger delta extractor, fill operation type (INSERT, UPDATE, DELETE)
  • Trigger operation timestamp - in trigger delta extractor, fill timestamp of the operation 
  • Routine: Unique recordmode - similarly to the standard routine, this transfer rule generates include for custom adjustment of data. However, this routine contains a predefined implementation for suppressing duplicate rows with the same semantic key but a different recordmode field. Moreover, because this routine is specific for recordmode field present in the Change log table, you can use it only for Change log delta type. This routine is described in more details in the chapter (Glue-2108) BW Transfer Rules.

Except for the Direct assignment, all transfer rules are represented by an "include form" which is part of the extractor form. You can access and modify them with the transaction SE38.

After creating an extractor, the system sets by default the "Direct assignment" rule. However, in case a field is of a ‘CURR’(Currency) type, the system uses the Currency conversion rule.
Default currency rule isn't assigned to a currency field when all keys from the Reference table aren't included in the Fieldname Mapping and the reference table is different than the source table. More information can be found in the chapter (Glue-2108) Structured Data Extraction.

For easy working with the Transfer rules the Fieldname Mapping provides a toolbar with these buttons:

  • Select All - select all fields
  • Deselect All - deselect all fields
  • Delete rules - delete rules for all fields
  • Apply rules - apply default rules on fields with Direct assignment rule
  • Global Data - display the main report for the user to define additional global data (see User-defined global data section for more info)
  • Start routine - displays form in the main report for the user to define additional code that will be executed before the transfer rules on fields will be applied for each package.
  • End routine - displays form in the main report for the user to define additional code that will be executed after the transfer rules on fields have been applied for each package.

Direct assignment 

This is the default type when an extractor is created. The extractor takes a source value and copies it in an unchanged form to a target. In this case, the system sets a different rule and you change it to the direct assignment rule, the old rule (include form) with the existing code is deleted. 

Routine 

In the case that value in a particular field needs to be changed, deleted, or adjusted, you can create your custom ABAP code that will be then processed for the selected field during the load. To perform these changes:

Select Routine as a transfer rule and click  in the column Edit Rule to access the generated form. The form can be edited in the same way as any Z*/Y* ABAP report. Importing parameter is_record contains the whole record which is processed. This parameter should not be modified.  You can change the value of the selected field through the parameter cv_field which modifies the value according to your requirements. 


  • User-defined global data

Using Edit Global Data button, you are redirected into the main program which collects all rules for fields in Fieldname Mapping.

In this report, you can define your own variables, structures, tables, etc. These can be then used in all custom routines.

  • Skip the records during the extraction process

You have an option to set up a condition to skip records during the extraction process. To perform this, you should set the flag 'gv_skip_line' under your condition.

The following example shows how you can skip all the records in the table SFLIGHT, where the Connection Number (field 'CONNID') is different from '0014'.

Skip the records during the extraction process
* This form is generated for handling target field CONNID
FORM CONNID
  USING IS_RECORD TYPE TS_DATA
  CHANGING CV_FIELD TYPE TS_DATA-CONNID.
* Insert your code here:

  IF is_record-connid <> '0014'.
    " skip the line
    gv_skip_line = 'X'.
  ENDIF.

ENDFORM.

Currency 

If a target field is of a ‘CURR’(currency) data type, the system sets this type of rule automatically by its creation but you may also change or delete it manually. 

The currency value is stored in a database as a number with two decimal places. This can lead to problems with the currencies with none or more than 2 decimal places. As an example, the Japanese Yen doesn't have any decimals, so when we have for instance 1005 JPY, the amount is stored as 10,05 in a database.

SAP applications displaying these values, perform a conversion according to a currency key during the extraction. It is necessary to convert the value of the currency field to a correct form before it is stored in an external database. Our functionality enables this conversion automatically, as it finds the currency key during the data load and changes the particular amount into a proper value, and stores it in target storage. 

Currency conversion can be disabled for BW InfoCubes. This can be achieved through the GLUE customizing table /DVD/GL_CUS_PAR, where the key value must be set to 'BW_NAMESPACE_PREFIX_+' and the parameter value should be a namespace of BW InfoCubes (for example '/BI0/' or '/BIC'). This makes sense for the tables that are part of the InfoCubes since InfoObjects have a defined currency key. Conversion of the tables in InfoCubes is currently not supported by GLUE, therefore we recommend to disable all BW namespaces when creating and extracting tables. 

Business function

This option allows you to use certain Business functions to adjust values during extraction.

Meaningful field values 

This option transforms original values into more meaningful ones so that you can use a full description of a value. For example, you can easily transform the abbreviation 'AA' into a meaningful term 'American Airlines'.

On the contrary with the above-listed transformation types, this type must be already specified while creating a Glue table by selecting the checkbox Use meaningful values. This option automatically adjusts the table structure in the background, so that the fields can store longer texts.

For every data element (e.g. 'AA') there are different ways on how to transform values. This information is stored in a database table and can be a part of customizing settings. You can adjust the settings in the GLUE Cockpit. 

When you create an extractor for a target table with checked Meaningful Field Values, the Apply rules function automatically creates all meaningful field rules.

When you create an extractor for a target table without checked Meaningful field values, you can still add this transformation rule to specific fields using the pop-up menu in the Fieldname Mapping section. This step generates a transformation of the selected table column. However, it is important to note that since the target table is already active, its field size isn't adjusted and that may lead to string leaks.


In general, the system transforms the values in four different ways into meaningful values: 

  • Role –  takes short texts of fixed values from a domain 
  • Mapping – every data element contains a mapping on how an old value should be transformed into a new one 
  • SID – converts data element RSSID into a meaningful value
  • ABAP – custom code that was written by a user 

You can specify which field should be used as a source for a meaningful value in the 'Reference table'. This is part of the customization option that was mentioned above. 
Mapping of the third way (SID) can be adjusted in Glue Cockpit. 

 

Please note: If meaningful values are missing some content, run the program '/DVD/GL_BPL_SPKFL_CONTENT_GEN'.


Example without using of meaningful values: 


Example with using of meaningful values:


If a meaningful value is not found for value, the system stores the original value and puts it into brackets – please see the column TYPE in the picture below. 

 


In the case values of data, element RSSID are included in the Master data table, then by a SID conversion, they are converted into a value from this table. However, this conversion makes sense for HANA optimized cubes without dimension tables and the F table already contains SID values (this points directly to the values in the Master data table). Every part of the process (adjusting fields during table creation and making rules in extractor) is performed automatically. For example, the source table can look like this: 



And an example of converted values in the target table: