(Glue-2405) Extraction Process

Owned by SNP
May 21, 2024
5 min read

The Extraction Process object handles the transformation of the data. The transformation is executed between acquiring the data through Fetcher and storing the data through Consumer.

The Extraction Process also allows the execution of extraction and monitoring of launched extractions.

In this section:

How to Create an Extraction Process Object

  1. Run the transaction /DVD/GL80 to open Object Navigator

  2. In the context menu of the desired package, select the option Create Extractor 2.0 object

  3. Enter the name of the new Extraction Process under Object name and a short description about it under Description

  4. Select Extraction Process under the Component type dropdown and under Component implementation leave the default implementation

  5. Confirm the popup

Activation of an Extraction Process

After the creation, the object is in the saved state.

Mandatory parameters to be filled before the activation

  • Fetcher

    • name of the Fetcher

    • provided with F4 help

  • Consumer

    • name of the Consumer

    • provided with F4 help

Mandatory buttons to click before the activation

  • Suggest Transformation Mapping 

    • button in the toolbar of the Transformation part 

    • generate mapping based on the filled Fetcher and Consumer

Optional buttons to click before the activation

  • Apply currency conversion 

    • button in the toolbar of the Transformation part

    • apply currency conversion rule for all or selected fields

  • Apply conversion routine 

    • button in the toolbar of the Transformation part

    • apply currency conversion rule for all or selected fields

Optional inputs before the activation

  • Start Routine

    • button in Transformation toolbar for ABAP code editor

    • executed before the transformation part of the extraction

  • End Routine

    • button in Transformation toolbar for ABAP code editor

    • executed after the transformation part of the extraction

  • Rule

    • icon in Transformation

Additional Functionalities of Extraction Process

The Extraction Process provides a few other functionalities on top of the Common functionalities of Extractor 2.0 objects.

  • Execute extraction

  • Monitor

    • button in the Object Navigator toolbar

    • option in the floating menu of Extraction Process

    • display detailed information for each executed extraction of the current Extraction Process

    • for more information see Extraction Request Monitor

Execute Extraction

Execution of the extraction is done by the pre-generated report, based on the Execution Process. Therefore each report can contain different input parameters.

The pre-generated report has an execution button and can have 5 sections of inputs:

  • Execution type - selection of execution type

    • Standard Execution

    • Mass Execution

  • Basic parameters - section for general settings of the extraction

    • Preview mode

      • when enabled, data will be not written to the target but only the sample will be displayed on the screen

    • Package size

      • number of rows selected for each package of package-based source

    • Package size MB

      • input for several MB, with compute button Calculate

  • Selection filter - the section is generated based on the selection in the specific Fetcher

    • Text Node

      • when enabled, values in the select option will be used in selection as Text Node (only for SNP Glue™ - BEx Query Fetcher (BICS))

  • Additional Fetcher config - section generated for editable parameters of the Fetcher

    • Database hint

      • a hint that could be used in a select statement while extracting data from the SAP table

    • Load type

    • Use cursor

      • when enabled, data from the source will be processed by the packages

  • Additional Consumer config - section generated for editable parameters of the Consumer

    • Data validation (only for SNP Glue™ consumers)

      • when enabled, the data extracted to the storage will be validated with the source data. To make the checkbox visible in the extraction process, turn on the parameter ENABLE_DATA_VALIDATION in Glue Settings (transaction /DVD/GL_SETT).

    • Enable merge

      • the flag that will enable the merge operation

      • can be also enabled/disabled in the Extraction process selection screen

    • Truncate target

      • when enabled, all data will be deleted from the target before writing from the source

    • Preserve staging files in case of failure (only for SNP Glue™ consumers with SNP Glue™ table on Hive, Redshift, BigQuery, or Snowflake)

      • the checkbox is enabled by default

      • when enabled, CSV files on the staging location won't be deleted in case the extraction fails on the COMMIT phase - enables investigation and Retry commit functionality

      • for information on how to retry commit using a monitor, check Retry commit

Preserve staging files in case of failure functionality is supported for Full Load type only, delta loads are not supported.

By clicking on the execute button a pop-up will appear with a cancel button and two buttons for execution type:

  • Dialog - the extraction will be executed in a dialog process

  • Background - another pop-up with a schedule will be displayed and after the confirmation, the extraction will be executed in the background process

If the user chooses to schedule the extraction to a particular date and time, a new request is created right away and displayed with a Pending (icon ) status.