(DV-2008) Test Management

You can start the Test Management via the Dashboard or by calling the transaction /DVD/KATE_TM.

_{Validate Dashboard}

Test Management is divided into two tabs:

1. Test Cases
2. Test Plans

_{Test Management, Test Cases tab}

It is good to understand the differences between Test cases and Test plans. Test plan is basic organizational entity. Test plan allows combining different types and any number of test cases together. There are several specific actions that can be done on top of Test Plan reminded further in text. Definition of directories may seem similar to Test plan definition but they have less/different options and are more folder like. Any Test Case cannot be executed without Test Plan. Test plan serves as a frame for its test cases. Test plan holds further definition of execution, settings, testers, statuses, documents and other. Test case carries basic information about type (type of tested subject) and variants (particular details about tested objects).

Test Cases Tab

Here you can do the following:

• New Test Case (Shift + F1)
• Create new directory (Shift + F2)
• Delete selected Test Case/Directory (Shift + F4)
• Find (Shift + F7)
• Find next (Shift + F8)
• Go to Variant Editor (this option is only available if Backend Test Cases, like Query Test Case for example, is enabled in Validate Setup /DVD/VALIDATE_SETUP)

New Test Case

One of the first steps while using Validate benefits is Test case creation. 

There are 3 ways to create a new test case,

• New test case button,
• Right-clicking at directory and selecting Add Test Case.
• Hotkey Shift + F1

_{Creating new test case}

The technical name for the test case (e.g. TST_CASE) has a Description field and a Directory field that can be selected to be pre-populated with the current directory, if not selected you can manually add the directory. Note: this field is mandatory and you cannot create test case outside of the directory.

_{New Test Case window}

The Fields marked with a tick box are mandatory. 
The Test Case Type is a drop down menu, which contains the different types of Test Cases. Test Case Type 'Manual Test Case' is always preselected by default.

Depending on your Validate setup, only certain Test Case Types may be visible. 
For example, if the Validate setup were to also contain BW Test Case Types, you will see an additional button at the bottom of the 'New Test Case window', where you can define additional information, based on these Test Case Types. 

_{Variants Management for Validate BW Test Cases}

Create new directory

By pressing the Create new directory, (Shift + F2) a new directory is created and you can add subdirectories by right clicking on the existing directory and choosing Add Subdirectory. 

_{Creating new directory}

A New Directory window looks like this:

_{New Directory window}

Dir. Technical Name contains the technical name (e.g. FIELD_TST_DIR), whereas the Dir. Description is what is displayed in the navigation tree panel.
The Parent Directory field is pre-populated with the current directory, if this option is not selected the Parent directory needs to fill in manually.

Delete directory and Test Cases

If Test Cases and/or directories are no longer necessary these can be deleted.

• By right clicking on an existing test case/directory and choosing delete.
• By Selecting Test Case/Directory and pressing the delete button, (Shift + F4)

_{Deleting items}

Transport Test Cases

Via the use of transports Test Cases and their content can be transported from the system to system. Therefore you can define Test Cases in one system and then reuse the same test case in another system. If there already exists the same Test Cases or directories, these will be overwritten when transported into the system.

_{Transport Test Cases for given directory}

Find and Find next

If you have many test cases and/or directories you can use the Find and Find next search features, to do so you can use the Find button or (Shift + F7).

_{Search window}

In the tree structure you can search for elements individually or jointly by using the  asterisk for replacing a character or character sequences. If it's a broad search and has a high number of hits, you can search through these by using Find next (Shift + F8).

Where used for Test Cases

If you want to check all Test Plans where a particular Test Case is used, use the "Where Used" feature found in the context menu displayed by right clicking on the Test case. The Result table contains all Test Plan information where this variant is used.

_{Where-Used for given Test Case}

_{Where-Used results list}

Test Plans Tab

This part of the Test Management is dedicated to the test planning and test monitoring.

_{Test Management, Test Plans tab}

In the Test Plans tab you can find these buttons:

• New Test Plan (Shift + F1) / Edit Test Plan (Double-click on existing plan)
• Manage Test Plan Scope
• Assign testers (Shift + F2)
• Set Status (Shift + F4)
• Worklist view (Shift + F5)
• Filter (Shift + F6)
• Refresh (F5)
• Find (Shift + F7)
• Find next (Shift + F8)
• Go to Backend Variant Editor (F9)

New Test Plan / Edit Test Plan

Test Plan creation is another step to take when you want to order your Test cases and explore other possible operations with Test cases.

A new Test Plan is created by pressing the New Test Plan button (Shift + F1).

_{Creating Test Plan}

_{New Test Plan window}

Mandatory fields are marked with a checkbox, the Dialog box now contains new button, Scope, and allows you to edit the content of the Test Plan immediately. Test Cases can be also assigned/edited afterwards via the Edit command or by right clicking on the created plan and choosing Edit. 

In the Edit Test Plan dialog box both Technical name and Description must be filled out.

_{Edit Test Plan window}

Manage Test Plan Scope

From this dialog window you can edit existing Test Plans and create new Test Plans.

_{Manage Test Plan Scope window}

This dialog window contains two main parts:

• Left part containing tree of Existing test cases
• Right part containing your currently selected Test Plan tree

You can add new items to the Test Plan either by drag and drop or double-clicking on desired item. 
Please be aware there are some actions, which are invalid:

• Deleting Test Plan itself,
• Dragging items from Test Plan tree to Test Cases tree,
• Dragging parent onto its children,
• Dragging item outside of the plan,
• Dragging already incorporated test case / directory again
• Dragging or deleting variants of test case.

Except above mentioned, the items are fully moveable. You can add new items / directories, change their order or parent, delete undesired items / directories, or select variants that are to be used in the testing. However, you are unable to create a new folder in the test plan or rename any of the items.

Assign testers

You can assign existing Test Plans to a specific person or to a list of people responsible for the testing. By clicking on the Assign testers icon or right clicking on the plan and choosing Assign testers (Shift + F2).

_{Assign testers window}

The User Name is the SAP login for the tester and here you can use the SAP F4 help. You can assign one or more testers to a Test Plan, a Test Case directory and/or individual Test Cases.

Set Status

When setting the status of a Test Plan, it is necessary to have the status on Active so the testers can see this in their Worklist. 
Test Plans can have the following statuses:

• Active and Closed.

Test Cases can have the following statuses:

• Untested, In process, Failed, Successful.

_{Set Status window}

Test Cases and Test Plans can have a Status Document, here you can add amore detailed description of the testing results. To create a status document right click on a Test Case/Test Plan and select „Create Status Document" option.

_{Status Document}

Worklist view

In the worklist view you can display all the Test Cases assigned to a given user, and are visible in their worklist. You can navigate via the worklist view button (Shift + F5).

_{Worklist window}

Simply enter the users SAP login and press confirm, if you want to reset the view, leave the User name field blank and press confirm.

Filter

You can filter the Test Cases by pressing Filter icon (Shift + F6). Test Cases or Test Plans can both be filtered by their Technical name, and Test Cases can be filtered according to their status. It is also possible to use an  asterisk to broaden the filter.

Filter window

Find and Find next

When you have many test cases and/or directories, use the Find and Find next are search by pressing the Find button (Shift + F7).

_{Find window}

Test Plan Report

You can generate documentation from the Test Plan in the form of MS Word document or HTML document. This report contains a list of all the Test Cases, including their statuses, Status Documents and additional information depending on the type of the Test Case.

_{Generating Test Plan Report}

Before actual generation, you can select Test Cases to be part of Test Plan Report, type of generated document for Test Plan Report and what kind of information to be generated.

_{Test Plan Report Options}

_{MS DOC Test Plan Report}

HTML document is more suitable for reports with large volumes of data, e.g. variant details. Default directory for HTML document is Desktop. You can select different directory by clicking on Browse button. HTML document does not contain Status Document.

_{HTML Test Plan Report}

Test Plan Scheduling

You can schedule Test Plans to run on the regular basis and send notifications about the result.

Prerequisites

For Scheduling to take place, you need to start Scheduling Checker in the Test Management Settings.

_{Test Management Settings icon}

The Scheduling Checker is reoccurring background job that checks for all the scheduled Test Plans every 15 minutes (i.e. if Scheduling Checker is executed on 17:40 it will check for Test Plans that are scheduled between 17:25 and 17:40).
You can set the Checking Period, which defines how often the Scheduling Checker checks for the scheduled Test Plans.
The actual precision of the scheduled time is dependent on this Checking Period – the smaller the number, better the precision. It is recommended to set a limit of up to 10 minutes. For a Checking Period of 10 minutes, the deviation is +10 minutes (so a Test Plan executed for 16:30 can be actually executed between 16:30 and 16:40).
You can also set the number of background jobs that are to be executed with in a given Test Case.
Each run of the Scheduling Checker is logged and these logs are available from the Test Management Settings.

_{Test Management Settings}

In order for the notification functionality to be working properly, the mail server has to be functioning on the system in which Validate is installed on. To test whether Validate can send emails correctly, go in to the Test Management Settings and in the 'Notification' section, type your email address. If a test email arrives, Validate can then send notification emails as expected.
Validate uses Business Communication Service (BCS) for sending emails.

Scheduling Overview

To see only the Test Plans that have been scheduled, you can change the view by customizing the filter.

_{Filter only scheduled Test Plans}

Test Plans with this filter enabled have additional columns that are relevant to scheduling, such as logs, execution time, email address etc.

_{Overview of scheduled Test Plans (with filter enabled)}

How to Schedule Test Plan

To schedule a given Test Plan, right-click on the Test Plan and select 'Schedule Test Plan'.

_{Creating new Scheduling for the Test Plan}

You can schedule a Test Plan to be run just once, daily or weekly. There is also the option to execute given Test Plan up to 3 times per day.

_{New Test Plan Scheduling window}

For each Test Plan you can defined scope - select which Test Cases have to be executed in scheduled Test Plan.

^{Selection of Test Cases}

For each Test Case type, execution steps are available and therefore specific process of execution to each Test Case type can be defined.

^{Selection of Test Steps}

Test Plans that are scheduled, have the icon of the calendar with a clock, near the Test Plan.

_{Scheduled Test Plans with the icon}

Once the Test Plan has been scheduled and executed, you can check the logs by right-clicking on the Test Plan.

_{Show Scheduling Logs}

_{Scheduling Logs for given Test Plan}

If the email address has been set, a notification about the results will be sent. 

_{Notification mail sent after finished Test Plan}

Scheduling Checker Algorithm

When Scheduling Test Plans, it is useful to know the algorithm for the Scheduling Checker to better understand its behavior.
Scheduling Checker is scheduled as a background job every X minutes (X is Checking Period in Test Management settings). During the Scheduling Checker execution, two actions are being performed:

1. Scheduling Checker will execute in the background all Test Plans that are scheduled within a given date range and within 15 minutes intervals (i.e. if Scheduling Checker is executing at 15:45, it will execute all Test Plans that are scheduled between 15:30 and 15:45).
2. Scheduling Checker will send email notifications for finished Test Plans where an email address is set.

Test Plan Snapshot

You can create a historical snapshot of Test Plan backend Test Cases. This snapshot will save information about selected Test Plan and the actual execution status. To create the snapshot right click on desired Test Plan and select 'Variants Overiew' -> 'Create New Snapshot' option from the context menu. You can then select which Test Cases should be included in the snapshot.

_{Creating New History Snapshot for Test Plan}

It is possible to create snapshots automatically after each scheduled Test Plan execution by selecting 'Create Snapshot After Execution' option with in scheduling options of Test Plan. The Snapshot will be created only for Test Cases executed in scheduled execution.

_{Automatic Snapshot Creation for Scheduling}

In current version of Validate snapshots are created only for backend Test Cases Types:

• ListCube
• Table
• Query

To review historical snapshots that were created, right click on desired Test Plan and select 'Variants Overview'->'Show Variants Overview' option from the context menu. A New screen will be displayed with the already created snapshots.

_{History Overview of Test Plan}

At the top of the screen you can select a period for which you want to display created snapshots as you can display multiple test plan history snapshots at once. By using F4 help you can change the date range and then by using F8 refresh button you will load the data of appropriate snapshots. The History overview screen contains 5 main tabs:

1. Test Case Overview
2. Variant Overview
3. ListCube Variants Details [optional]
4. Table Variants Details [optional]
5. Query Variants Details [optional]

Test Case Overview Tab

In this tab you can see one record for each Snapshot/Test Case pair. It contains information about the total number of variants present in the Test Case when snapshot was created, overall status of Test Case, untested/successful/failed variant counts, and rows information.

_{Test Case Overview Tab}

In the 'User Comment' column you can provide any custom comment that will be saved there for later review.

Variant Overview Tab

Here is you will see an overview of each Snapshot/Test Case/Variant combination. The information displayed includes the variant description, compare/reporting status, tested/error rows count and reporting status text if any.

_{Variant Overview Tab}

ListCube Variants Details Tab

This tab is only displayed, if any of the selected snapshots actually contain one or more ListCube Test Cases. For each Snapshot/Test Case/Variant combination you can see details of its variant settings and execution status of all steps.

Table Variants Details Tab

This tab is only displayed, if any of the selected snapshots actually contain one or more Table Test Cases. For each Snapshot/Test Case/Variant combination you can see details of its variant settings and execution status of all steps.

Query Variants Details Tab

This tab is only displayed if any of the selected snapshots actually contain one or more Query Test Cases. For each Snapshot/Test Case/Variant combination you can see details of its variant settings and execution status of all steps.

Test Plan Replication to HP ALM

When replication to HP ALM server is correctly setup in the Datavard Validate settings, you can replicate Datavard Validate Test Plans in the HP ALM server. To replicate a Test Plan, right click on it and select 'Others' -> 'Replicate to HP ALM'. To replicate multiple Test Plans, right click on a Test Plan Folder and select 'Replicate to HP ALM'.

_{Replicating Test Plans to HP ALM server}

Choose template from listbox to fill set of settings needed for replication. For immediate customization of settings click on 'Edit' button. These changes will not affect any template. For replication click on 'Continue' button. Last used template is saved for each replicated Test Plan and it will be preselected during next replication.

_{Selecting template before replication of Test Plans to HP ALM server}

The following Validate objects are replicated:

• Test Case
  • generates Test object in HP ALM Test plan area
• Test Plan
  • generates Test Set object in HP ALM Test lab area
• Test Case in scope of Test Plan
  • generates Test Instance under Test Set
• Test Case status in Test Plan
  • generates HP ALM Run

Application log is generated during the replication process. It contains information about the replication status and possible errors that occurred during the replication.

_{Replication log}

The replication works only from Validate to HP ALM. There is no transfer of data or synchronization from HP ALM back to Validate.

Worklist

This part of documentation explains basic navigation and functions of the Worklist section.

_{Worklist in Validate Dashboard}

This area with in Validate is focused on the testers, and those who see their Test Cases/Test Plans that are assigned to them.

_Worklist

Test Plans with the status New are not displayed here, in order to make New test plans visible their status needs to be changed to Active. You can do this through the Test Management -> Test Plans tab. Users will only see their assigned Test Cases. Also directories are not displayed here.
To indicate which phase of testing the Test Cases are in, you can change their status. To change the status, click on the Set Status icon (Shift + F4), or right click on the test case, in the following window where you can add a description for the Test Cases and set the status to Untested, In process, Failed or Successful.

_{Set Status window}

Test Case Mass Execution

In Test Plans tab, there is option to execute all Test Cases in given Test Plan. For each Test Case Type, execution options are given and user can select which parts to execute and which do not (i.e. execute only Before Image for all ListCube Test Cases). 

_{Mass Execution in Test Plans tab}

For each Test Case type, execution options are available and therefore specific execution to each Test Case type can be defined.

_{Mass Execution – Main Screen}

After clicking on "Change Selection" button, scope of Mass Execution can be defined. Functionality is supported for all Test Cases that can be selected (i.e. Manual Test Case type cannot be selected, because it does not support the Mass Execution option).

_{Mass Execution – Selection of Test Cases} 

Backend Test Cases Mass Execution

When backend Test Cases are selected for mass execution you can define which test steps should be executed from each test case type. List of test steps depends on chosen backend scenarios. Please refer to chapter 8 for details of test steps for each type. Setting 'Number of jobs per Test Case' displayed on Figure 119 defines in how many background job for each chosen step are to be executed. 
Each backend test case type contains the option 'Run Sequentially'. If this setting is set all chosen test steps will be executed in the order in which they are defined within the list. Execution of a step will be started once the preceding step is finished. There is a maximum time setting that once it is reached, while waiting for the preceding step to finish will end the execution of the test case. Refer to Settings chapter for details. If 'Run Sequentially' option is not used all selected test steps are executed in parallel each using the specified number of jobs. No wait is used for these cases. 

_{Mass Execution of backend test case}