(Glue-2111) Extractor Request Monitor

Owned by SNP
Nov 22, 2021
7 min read

Extractor Request Monitor serves as a tool for monitoring ongoing or past extraction processes. 

You can access it via GLUE cockpit, transaction /DVD/GLUE  → Request monitor.

In this section:

After accessing Request monitor, a filter for requests comes up where you can specify your selection based on the following fields: 

  • Extraction process - Name of the extractor or extraction process of Extractor 2.0 

  • GLUE request - Number of GLREQUEST

  • Username - Name of the user who executed the extraction

  • Status - Status of a request (success, error, warning, running, failed)

  • From (Date/Time) - Date and time requests will be displayed from (display records that started after the set date and time)

  • To (Date/Time) - Date and time requests will be displayed up to (display records that started before the set date and time)

  • Maximum No. of Hits - Maximum number of records displayed

After clicking on the Execute button, the request monitor displays all records according to the selection.

 

Records are sorted by the date and time of extraction (newest first).

You can also access monitor with filtered records for specific Extraction Processes from Glue Workbench by choosing the Extraction Process from the context menu and selecting option Manage.

Monitor screen functionalities

On the monitor screen you can:

  • Refresh the results for a real-time monitoring

  • Display logs for each request, using a button in the Logs column

  • Delete request together with data on the target storage

  • Clean request metadata resulting in deletion of GLUE request metadata while keeping data on the target storage 

  • Display selection is used for data extraction. Where condition is displayed if it is used instead.

  • Retry commit for supported storages, if the "Preserve staging files in case of failure" option was enabled

  • Restart failed requests in case these were executed via Extractor 2.0 Mass Execution capabilities

  • Standard ALV functionalities like displaying details of the row, sorting, and filtering

  • Redirect to standard SAP transactions like Job monitor, Process overview, SAP servers

  • Job group monitor which enables the user to stop/kill the jobs when mass execution is running. More information can be found in the chapter Job Group Usage.

  • Redirect to extractor process in GL80 

    • select a row with the desired object and click the “Goto” button on the taskbar

    • or with a double-click of the Extraction object name

Refresh

You can refresh the monitor by using the Refresh button, shown in the next image: 

The Refresh button causes the monitor to load the latest state of the GLUE requests while keeping the selection you specified.

Display logs

To keep informed about the execution of the extraction, each GLUE request keeps execution logs. You can see these logs by clicking on the button in the display logs column on the row belonging to a particular request in the request monitor.

After clicking the display logs button, you will be redirected to the log screen.

In some cases e.g. when the request is running or failed, the logs are not stored in the standard application logs.
Therefore, you will be redirected to the job log of SM37 where you can find similar information about the execution of the extraction.

Delete request

The request monitor allows you to delete the data of a particular GLUE request. Using this option both metadata and data on the target storage will be deleted.
This functionality requires the field GLREQUEST to be present in the target structures and will display an error message in case it's not.

To delete the particular GLUE requests, select these in the ALV grid of the monitor and press the delete button, shown on the next image.

 

After clicking on the Delete button the following dialog will pop up:

To continue, select yes. After that, you should see the pop-up allowing you to select between the dialog or background execution.

We recommend using the Background execution for deleting requests with a large amount of data or when deleting more than a few smaller requests. 
When using Background execution, you will have to press the refresh button to see the requests being deleted. 

In case you proceed and select dialog mode, you will see the following log after the execution of the deletion:

In case any error occurs, you will see it in the logs, and the request together with its data will not be deleted.
If you chose the Background execution, you can find these logs within the standard application logs for GLUE. 

If you are deleting the requests in a state running, you will see one additional message.

You can proceed to the deletion by selecting Yes but please be aware of a few limitations.

  1. In case you are deleting the request that runs in a background job, the program attempts to stop the job to keep the data consistent.
    If this fails (e.g. due to insufficient privileges) the request won't be deleted.

  2. In case you are deleting the request that runs in dialog, the program has no way to stop the job and therefore this action might lead to data inconsistencies on the target storage.
    Therefore, we recommend you not to delete the running requests that are executed in the dialog.

Clean request metadata

This functionality is similar to the Delete request functionality. However, it keeps the data on the target storage and deletes only the metadata in GLUE.

To delete metadata for particular requests, select these in the ALV grid of the monitor and press the Clean request metadata button, as shown in the image:

After pressing the Clear request metadata button, you will see the following pop-up:

To continue select Yes. After that, you should see the log informing you about metadata deletion.

Display selection

This functionality allows you to browse the selection that was used for the execution of a particular request.

To display the selection, click the Display selection button on the ALV grid, within the line of the request you want to see the selection for.

After executing the Display selection, you will see the pop-up with the user selection or the message informing you about the fact that the process has no selection filled in case no selection was used for the particular extraction.

Retry commit

This functionality is supported for storages that use the staging location (Hive, Redshift, BigQuery, Snowflake). When the extraction fails on the COMMIT phase and the option "Preserve staging files in case of failure" was enabled, CSV files transferred during extraction are not deleted. If the error was caused e.g. by connection issues, the commit can be relaunched, saving time that is needed to transfer the files to the staging location.

If the original run was launched using Dialog Mode the same mode will be used with retry commit, similarly with Background Mode.

Restart failed requests

The Extractor 2.0 introduces the new Mass Execution capabilities. More information can be found in the chapter Mass Execution.
If the request was created via Mass Execution and for some reason failed to extract the data, you can restart it directly from the monitor screen using the button that is displayed in the column Action.

You need to press the Restart button to re-execute the failed request.

In case there is more than one failed request in the Mass Execution run, the following pop-up is displayed and letting the user choose to restart all the failed requests or just the one the user has clicked on:

If the user chooses to restart all failed requests, another pop-up is displayed with the parallel job count selection:

After confirming the selection, the restart process begins with a selected number of parallel jobs.

After the request(s) restart has started, you will see the pop-up informing you about the restart of the request(s).


Application

The column Application contains information about the application which executes the extraction. Standard Glue functionality will fill the "DEFAULT" value into this column.