Full Extraction

This type of extraction has no restrictions or further specifications with regard to the source. All data is transferred to the target.

Timestamp

To extract data based on a timestamp, select the Timestamp field attribute. After each extraction, the highest value of the selected timestamp attribute is stored. In the subsequent extractions, only records with an attribute value higher than the stored value are transferred. The first load is always a full extraction.

Example 1

To simplify this example, we use only the time to express timestamp values, i.e. not the complete timestamp with date and time. Therefore, all mentioned times below have the same day in the complete timestamp format.
Before the first extraction, the source table contains values 09:13, 11:10 and 08:15 in the particular column.
By the first extraction, all records are transferred, and only the value 11:10 is stored as the highest value.
Further records with the values 09:50, 13:10 and 15:05 in the particular column are inserted into the source table. The source table then contains the records with the values 09:13, 11:10, 08:15, 09:50, 13:10 and 15:05.
By the next extraction, only the records with the values 13:10 and 15:05 are transferred and the value 15:05 is stored as the highest value.
Records with the value 09:50 are not transferred because this value is lower than the stored value 11:10.

In addition, you can set the Timestamp offset, which is the value of the time shift for the next delta load in seconds. This helps to lower the risk that the new records coming to the source table during the extraction will not be captured by delta loads or delta init loads.

Example 2:

During the delta load or delta init load, the source table was updated with a new record. This record is not part of the extracted data, because it was saved after the data was selected from the source. However, at the same time, it will not be transferred with the next delta load due to its timestamp value. For such cases, it is useful to have a safety internal to ensure that all records are being captured by timestamp delta.

Date

To extract data based on the date, select Date field attribute. Similar to the timestamp delta, the highest value of the selected attribute is stored after each extraction. However, the difference that records with the values of the current date or higher are ignored. This ensures that data is extracted from a complete day and not only from part of it. This behavior is also applied to the first, full extraction.

Example

In this example, the current date is the 13.04.2018.
Before the first extraction, the source table contains the values 11.03.2018, 13.04.2018 and 14.04.2018 in the particular column for Date delta.
By the first extraction, records with the value 11.03.2018 are transferred and the value 11.03.2018 is stored as the highest value.
Records with the value 13.04.2018 are not transferred, because the current day has the same value. Records with the value 14.04.2018 are not transferred, because this value is higher than the current day (14.04.2018 > 13.04.2018).
The next day, which is the 14.04.2018, 14 records with the values 10.03.2018, 29.03.2018 and 20.04.2018 are inserted into the source table. The source table then contains the records with the values 10.03.2018, 11.03.2018, 29.03.2018, 13.04.2018, 14.04.2018, and 20.04.2018 .
By the next extraction, only records with the values 13.04.2018 and 29.03.2018 are transferred and the value 13.04.2018 is stored as the highest value.
Records with the value 10.03.2018 are not transferred, because the value is lower than the stored one (10.03.2018 < 11.03.2018). Records with the value 14.04.2018 are not transferred, because the current day has the same value (14.04.2018 = 14.04.2018). Records with the value 20.04.2018 are not transferred because the value is higher than the current day (20.04.2018 > 14.04.2018).

Value Delta

To extract data based on a value delta, select a Value field attribute that contains numeric values. This process is similar to the date and timestamp delta processes. After each extraction, the highest value of the selected field is stored. In the subsequent extractions, only records with a value higher than the stored value are transferred. The first load is always a full extraction.

Distinct Value Delta

The selection is based on the values of already extracted data in a specified Delta field. To configure this mode properly you must specify this field. Only entries with new distinct values from this field are transferred during the next execution of the extractor. 

Delta Process for Distinct Value Delta

Value delta extraction is based on a delta field that you choose. After you select the delta field, the system checks all the distinct values in the particular delta field and transfers the whole records with these distinct values to the target system. The distinct values of the delta field are then stored in a blacklist.

Example the table below contains the fields First name, Last name, and Employer ID. In this example, Employer ID is chosen as the delta field. The system transfers the records (records 1 to 5) with distinctive values (121 to 124) in the Employer ID field to the target system and stores these values in the blacklist afterwards.

During the next extraction, the system checks all the values in the delta field and selects only values that are not on the blacklist. During the delta execution, the system transfers only records with these distinct values that are not included in the blacklist to the target system and stores the new distinct values of the delta field in the blacklist.

Two more records have been added to the above table. The system selects the red-framed delta field Employer ID records with the values that are not in the blacklist (values 125 and 126) and transfers the whole records (records 6 and 7) with these distinct values to the target system. Afterwards, the system stores these new distinct values (125 and126) on the blacklist.
This extraction process is repeated the same way again during the next value delta extraction.

Select this option, if you do not want to transport any historical data but only actual data, which will be generated after the initial delta run.

Trigger Delta 

The trigger delta function uses a special database table that keeps a record of new database records as they are being added. Every time a DML operation is performed on a given table, a procedure stores information about the newly inserted data to help determine the delta.

BW Delta Mechanisms

Change Log Delta

This delta reads data based on the changelog table of an InfoProvider. The initial full load reads data from the active table of an InfoProvider. Then the delta is automatically initialized with the activation request values that are currently available in the changelog table. Next load reads data from the changelog table, reading only records with activation request values that were not loaded yet. That means only records with new request values are transferred during the next execution of an extractor.

Delta properties

Properties tab contains general information about the extractor and the delta details:

• Delta type
• Status: actual status of the extractor (Saved or Active)
• Source: the name of a source InfoProvider
• Last extraction: date and time of the last extraction
• Full load: indicates whether a full load was executed

The Properties section shows

• Truncate target table: if you select this checkbox, all data in the target table is deleted before new extraction.

Set up values that will be exported from the Change log

Select at least one Record mode value which should be exported from the changelog table of the DSO. Record mode is the SAP function. You can find more information in the section 0RECORDMODE Values of the blog Recordmode and delta type concepts.

_{Delta Tools - Properties tab}

Manage

In Manage you can execute data extractions.
Perform delta run executes a standard delta extraction.
Perform full load executes a full load even if it has already been executed. During the first full load, the changelog delta is initialized. If you execute the full load repeatedly, the delta values will not be updated.
Reset delta value button resets delta value.

Request Delta

The request delta function reads data based on DTP requests loaded into an InfoProvider. During the initial full load, all data is read from an InfoProvider and the delta is initialized with the highest request value. Then the next load reads data with DTP request values greater than the current maximum. After that, the maximum DTP request value is updated to show a new value.

Properties Tab

The Properties tab contains general information about the extractor:

Delta details:

• Delta type: Type of the delta
• Status: Current status of the extractor. Possible values: Saved or Active
• Source: the name of the source InfoProvider
  
• Last extraction: date and time of the last extraction
• Full load: indicates whether a full load was executed

Properties:

• Highest loaded value - Display the highest loaded value of an InfoProvider DTP request.

Manage tab

The manage function is not currently supported for request-based delta extractions. 

0RECORDMODE Values

Technical flag with additional information about a particular record changelog table.

Empty space: The record provides an after image. The status of the record is transferred after it has been changed or after data has been added.

X: The record provides a before image. The status of the record is transferred before it is changed or deleted. All attributes for the record that can be aggregated (key figures) must be transferred with a reversed plus/minus sign. These records are ignored in a non-additive (overwriting) update of a DataStore object. The before image complements the after image.

A: The record provides an additive image. This provides the record with differences for all the numeric values that are available. The record can be updated to an InfoCube without restrictions but requires an additional update to be made to a DataStore object.

D: The record must be deleted. Only the key is transferred. This record (and therefore the DataSource too) can only be updated to a DataStore object.

N: The record provides a new image. The content of this record is equivalent to an after image without a before image. A new image should be transferred instead of an after image when a record is created. The new image complements the reverse image.

R: The record provides a reverse image. The content of this record is equivalent to a before image. The only difference occurs when updating a DataStore object: An existing record with the same key is deleted.

Y: Can only be updated to a DataStore object.