(DV-2205) Query Execution & Backend Testing

Owned by SNP
May 20, 2022

Query Test Case Execution

When added to a Test Plan a new backend Test Run is automatically created. The Test Run has a generated Run ID, while the description holds the technical name of Test Plan and Test Case. 
All Query Variants that were selected during the creation of a Query Test Case are automatically added into the Query Variant selection for the generated Test Run. You can execute the Query Test Cases from the Test Management by clicking on  'Execute' button and the screen for a standard Query Testing scenario will be displayed.

Query Based Testing

The Query-based testing uses the query output as the test data and the before image of the query output is then compared with the output of an after image of the query output. 
The actions that lead to the creation of both these images can vary based on the purpose of each scenario. SNP Validate currently supports one testing scenario with this type of testing. When the queries are executed using SNP Validate the cache is not used, the reason for this is to get as precise as possible, the runtime values for both the before and after image. 
Recommendations:

  • A maximum of 1 000 Query Variants should be used in each created run of the Query Testing scenario.
  • Processing time for each step depends on queries that are processed. E.g. if there are lots of queries resulting in a huge output (more than 100 000 rows) the processing time may take longer than that of the query with a smaller output.

Test Scenario EQS (Query Testing)



Query Testing scenario steps


The Query Testing Scenario contains the following steps:

  1. Select Query Variants
  2. Create the Before Image
  3. Performing some specific tasks in the system outside SNP Validate (like archiving, migration, etc.)
  4. Create the After Image
  5. Compare Before and After Image
  6. Display of results

You can navigate and execute all steps by double-clicking on them.

Select Query Variants

You can choose which Queries/Bookmarks are to be used for testing, the definition and selection of query/bookmarks variants are necessary and can be done in the main window of this step.



Selection of Query Variants


All of the Query Variants that are added to this list are then used in the subsequent steps of the Query Testing scenario. There are multiple ways to add/create Query Variants into the list:


Create New Query Variant – Please refer to Create new Query Variant section for a detailed description of Query Variant's creation. The only difference is in the variant creation the confirmation button is not the 'Continue' button (Enter) but the 'Save' button (Enter). 
By double-clicking on the created Query Variant, you can reenter and edit the Variant properties. Remember, it is not possible to edit variant properties if the same variant is already in use for another Test Run.
Create New Bookmark Variant – Please refer to Create new Bookmark Variant section for a more detailed description as the screen for defining Bookmark Variant directly in the run. The only difference is in the variant creation confirmation button, which in this case is not the 'Continue' button (Enter) but the 'Save' button (Enter). 
Add existing Query Variant – A list of all the existing Query Variants in a system is displayed and you can select a set of variants to be added by using the 'Copy' button (Enter). Please note that each Query Variant can only be used once in each Test Run. 
Add Query Variant from Test Run ID – By clicking the 'Add Query Variant from Test Run ID' button a list of all the existing Query Testing Runs is displayed. You can a Test Run and add all variants used in one run to the current Test Run.
Copy Query Variant of Run – By clicking the 'Copy Query Variant of Run' button a list of all the existing Query Testing Runs is displayed. You can select a Test Run and add all variants used in another run to the current Test Run as copies.
Generate Query Variants from HM (Optional) – You can automatically generate new Query Variants, to use this option you must have the HeatMap tool is present in the system. 
Add Variants of Test Case – By clicking the 'Add Variants of Test Case button a list of all the existing Query Test Cases is displayed. You can select a Test Case and add all variants used in it to the current Test Run.
Copy Variants of Test Case – By clicking the 'Copy Variants of Test Case button a list of all the existing Query Test Case is displayed. You can select a Test Case and add all variants used in it to the current Test Run as copies.
Create Based on Web Template – Click on the 'Create Based on Web Template' button (Shift + F7). Please refer to Create Variants of Web Templates section
Copy Variants – Please refer to the Copy Query Variants section for function details.

Generate tasks for before image

This step is executed automatically by double-clicking on it, afterwards the executable tasks for each defined Query/Bookmark Variant in the 'Select Query Variants' step are generated and visible in the following step 'Create before image'.

Create before image

By double-clicking on this step, you first define the number of background jobs to be used for the execution of tasks. After each task is completed a before image of the corresponding query output is created and saved for later comparison.

Generate task for after image

This step is executed automatically by double-clicking on it, after the generation of the tasks for each of the defined Query/Bookmark Variant from the 'Select Query Variants' step. These are to be executed in the following step 'Create after image'.

Create after Image

By double-clicking on this step, you first define the number of background jobs to be used for the execution of tasks. After each task is completed an after image of the corresponding query output is created and saved for later comparison.

Generate tasks for comparison

This step is executed by double-clicking on it, the tasks for each of the defined Query/Bookmark Variant in the 'Select Query Variants' step are generated and can be found under the following step 'Compare before and after image'.

Compare before and after image

By double-clicking on this step, you define a number of background jobs to be used for the execution of tasks. Each task compares the before image data with the after image data for one Query Variant.

Display results

To view the Query Variants' execution outputs and their comparison results. For more information please refer to the Results overview section for more details.