(Glue-2102) Extractor Request Monitor
Extractor Request Monitor serves as a tool for monitoring ongoing or past extraction processes.
You can access it via GLUE cockpit, transaction /DVD/GLUE → Tools → Request monitor.
After accessing Request monitor, a filter for requests comes up where you can specify your selection based on the following fields:
- Extraction process - name of the extractor or extraction process of Extractor 2.0
- GLUE request - number of GLREQUEST
- Status - status of a request (success, error, warning, running, failed)
- Username - name of the user who executed the extraction
- As of Date - date data is displayed from
- As of Time - time data is displayed from
- Maximum no. of hints - maximum number of records displayed
After clicking on the Execute button, the request monitor displays all records according to the selection.
Records are sorted by the date and time of extraction (newest first).
You can also access monitor with filtered records for specific Extraction Process from Glue Workbench by right-clicking on Extraction Process and selecting option Manage.
Monitor screen functionalities
On the monitor screen you can:
- Refresh the results for a real-time monitoring
- Display logs for each request, using a button in the Logs column
- Delete request together with data on the target storage
- Clean request metadata resulting in deletion of GLUE request metadata while keeping data on the target storage
- Display selection used for data extraction
- Retry commit for supported storages, if the "Preserve staging files in case of failure" option was enabled
- Restart failed requests in case these were executed via Extractor 2.0 Mass Execution capabilities
- Standard ALV functionalities like displaying details of the row, sorting, and filtering
- Redirect to standard SAP transactions like Job monitor, Process overview, SAP servers
- Job group monitor which enables the user to stop/kill the jobs when mass execution is running. For more information see the chapter Job Group Usage.
Refresh
You can refresh the monitor by using the Refresh button, shown in the next image:
The Refresh button causes the monitor to load the latest state of the GLUE requests while keeping the selection you specified.
Display logs
In order to keep information about the execution of the extraction, each GLUE request keeps execution logs. You can see these logs by clicking on the button in the display logs column on the row belonging to a particular request in the request monitor.
After clicking the display logs button, you will be redirected to the log screen.
In some cases e.g. when the request is running or failed, the logs are not stored in the standard application logs.
Therefore, you will be redirected to the job log of SM37 where you can find similar information about the execution of the extraction.
Delete request
The request monitor allows you to delete the data of a particular GLUE request. Using this option both metadata and data on the target storage will be deleted.
This functionality requires the field GLREQUEST to be present in the target structures and will display an error message in case it's not.
In order to delete the particular GLUE requests, select these in the ALV grid of the monitor and press the delete button, shown on the next image.
After clicking on the Delete button the following dialog will pop-up:
In order to continue, select yes. After that, you should see the pop-up allowing you to select between the dialog or background execution.
We recommend using the Background execution for deleting requests with a large amount of data or when deleting more than a few smaller requests.
When using Background execution, you will have to press the refresh button to see the requests being deleted.
In case you proceed and select dialog mode, you will see the following log after the execution of the deletion:
In case any error occurs, you will see it in the logs, and the request together with its data will not be deleted.
If you chose the Background execution, you can find these logs within the standard application logs for GLUE.
If you are deleting the requests in a state running, you will see one additional message.
You can proceed to the deletion by selecting Yes but please be aware of a few limitations.
- In case you are deleting the request that runs in a background job, the program attempts to stop the job in order to keep the data consistent.
If this fails (e.g. due to insufficient privileges) the request won't be deleted. - In case you are deleting the request that runs in dialog, the program has no way to stop the job and therefore this action might lead to data inconsistencies on the target storage.
Therefore, we recommend you not to delete the running requests that are executed in the dialog.
Clean request metadata
This functionality is similar to Delete request functionality. However, it keeps the data on the target storage and deletes only the metadata in GLUE.
In order to delete metadata for particular requests, select these in the ALV grid of the monitor and press Clean request metadata button, as shown in the image:
After pressing the Clear request metadata button, you will see the following pop-up:
To continue select Yes. After that, you should see the log informing you about metadata deletion.
If you are deleting metadata for running requests, you will see one additional message.
You can proceed to metadata deletion by pressing Yes. However, such an action is not recommended unless you know what you are doing.
Deleting metadata of running requests won't stop the running jobs, neither touch the data on the target storage, and might lead to data inconsistencies on the target.
The only proper use of the deletion of the running request is in the case when the request stuck in the running state (e.g. unexpected error in custom code).
Display selection
This functionality allows you to browse the selection that was used for the execution of a particular request.
To display the selection, click the Display selection button on the ALV grid, within the line of the request you want to see the selection for.
After executing the Display selection, you will see the pop-up with the user selection or the message informing you about the fact that the process has no selection filled in case no selection was used for the particular extraction.
Retry commit
This functionality is supported for storages that use the staging location (Hive, Redshift, BigQuery, Snowflake). When the extraction fails on the COMMIT phase and the option "Preserve staging files in case of failure" was enabled, CSV files transferred during extraction are not deleted. If the error was caused e.g. by connection issues, the commit can be relaunched, saving time that is needed to transfer the files to the staging location.
Restart failed requests
The Extractor 2.0 introduces the new Mass Execution capabilities, see the chapter Mass Execution.
If the request was created via Mass Execution and for some reason failed to extract the data, you are able to restart it directly from the monitor screen using the button that is displayed in the column Action.
You need to press the Restart button in order to re-execute the failed request.
In case there is more than one failed request in the Mass Execution run, the following pop-up is displayed and letting the user choose to restart all the failed request or just the one the user has clicked on:
If the user chooses to restart all failed requests, another pop-up is displayed with the parallel job count selection:
After confirming the selection, the restart process begins with a selected number of parallel jobs.
After the request(s) restart has started, you will see the pop-up informing you about the restart of the request(s).
After a successful restart, the new request(s) will be created and the possibility to restart the original one(s) is removed. It means that you can restart the request (with a particular request number) only once.
In case the new request would fail as well, you will be able to restart this request using the same logic however as the new request (different request number),
Application
The column Application contains information about the application which executes the extraction. Standard Glue functionality will fill the "DEFAULT" value into this column.