(DV-1808) Drill Down Execution & Backend Testing

DRILL DOWN TEST CASE EXECUTION

When a Drill Down Test Case is added to the Test Plan a new backend Test Run is automatically created and a Run ID is generated. The description holds the technical name of the Test Plan and the Test Case. All Drill Down Variants selected during the creation of Drill Down Test Case are automatically added into Drill Down Variant selection for the generated Test Run. 
To execute a Drill Down Test Case from Test Management click on  'Execute' afterwards the standard Drill Down Testing scenario is displayed. Please refer to 8.3.1 ('Test Scenario EQS_DRILL (Drill Down testing)') section for a more detailed description about each scenario step.

Drill Down testing

The Validate Drill Down Test Scenario uses the ListCube functionality in a similar way as in the ListCube test scenario and uses the RSA1 'Display Data' functionality to get the data from the InfoProviders to be tested. 
Recommendations:

• For one DrillDown testing run, a maximum of 1000 Drill Down variants should be used.
• It is recommended to use of the Validate setting 'DrillDown maximum records' to under the value of 100 in order to reduce the amount of data to process and reduce the chance of errors.

Test Scenario EQS_DRILL (Drill Down testing)

_{Drill Down Testing Scenario} 

Drill Down testing contains following steps:

1. Selecting Drill Down Variants
2. Generation of execution tasks
3. Execution of Drill Down Tasks
4. Display of results

Select Drill Down Variants

You can select which of the Drill Down Variants are to be used or create new Drill Down Variants to be added into the Test. When the Test Run is generated through the Validate Test management the variants are already preselected for the Drill Down Test Case.
Once the variants are selected for the Test Run, you can save the selection by clicking on 'Save' button (Ctrl + s).
In the following ways it is possible to add Drill Down Variants to the list:
Create New Drill Down Variant - (SHIFT + F1). For detailed information about creation of Drill Down variant please refer to 7.4.1 ('Create new Drill Down Variant ') section, 
Add Existing Drill Down Variant - (SHIFT + F4) displays all of the existing Drill Down Variants in the system, and you can select one or more variants to be added into the current Test Run. 
Add Drill Down Variants of Run - (SHIFT + F5) you can select the Variants used in another (distinct from current) run and add them into current the run as well. 
Generate Drill Down Variants - (SHIFT + F6) please refer to 7.4.2 ('Generate Drill Down Variants') section for more information.
Copy Variants - Please refer to7.1.14 7.4.7 ("Copy DrillDown Variants") section for function details.

Generate Tasks for Drill Down

The tasks are generated for the following step 'Execute Drill Down Tasks'. You can specify the key figures go be ignored in the InfoObjects during the comparison of the Drill Down Variants specified for this run. These ignored key figure columns are not visible in the 'Display Results. The Generation of the tasks is executed by clicking on 'Create Tasks' (F8) button.
You can enable Automated Root Cause Analysis for comparison tasks. Refer to Automated Root Cause Analysis section for details. 

_{Selection of ignored key figure columns}

Execute Drill Down Tasks

You double click to execute, first you can define the number of jobs/ background jobs to be used for tasks execution each task executes a Drill Down test for one Drill Down Variant. Drill Down scenario testing is explained below.
The Drill Down test variant compares the data from two different InfoProviders, in most cases it's the same InfoProvider but on different systems. Based on selected Drill Down characteristics in the variant definition the execution starts by selecting the first specified Drill Down characteristic, and adds to a ListCube, which then read on both InfoProviders as a characteristic. The ListCube reads the outputs and these are immediately compared. For two out of three scenarios the Drill Down test execution ends here.

1. One InfoProvider did not returned data while the other one did. In such case the first InfoProvider has no data, we advise not to continue as further comparisons will fail.
2. The data returned by both InfoProviders are same. If everything is correct the Drill Down task stops the execution to free up resources for other tasks.

In the third scenario some or all data returned by InfoProviders are not same. In this case a new test cycle begins, and the erroneous data is checked. Using the Validate setting 'DrillDown maximum records', for the first added characteristic (the one added at the start of first cycle) the X number of distinct values that belongs to erroneous records are the selected. These values act as a filter value for this characteristic in next test cycle (drill down into erroneous records). 
The next characteristic in the order of specified Drill Down characteristics (if no more are available then the execution stops here) is added to ListCube for reading and then another characteristic is selected for the output. The ListCube reads are repeated for both InfoProviders with the new settings and the data is then compared again. Afterwards either a new test cycle begins using the same logic just described, or execution ends here, depending on the result of comparison. 
If Validate setting 'DrillDown additional aggr.' is set to 'X' in each cycle when InfoProviders are read the returned data is aggregated again. This functionality can be used for cases when ListCube returns multiple rows with same key because of unsupported functions. 

Display results

By double clicking on this step you can view the DrillDown Variants executions outputs and their comparison results are displayed. Please refer to (Results overview) chapter for more details.