Zephyr allows kicking off or launching of automation scripts on remote target machines. For this to happen, testcases must have been marked as Automated and details about the automation script must have been added to that testcase in the Test Repository process. It is also required to have a remote agent - ZBot - installed on the target machine in order to launch the automated script on that remote machine.
Automation testcases can be executed by clicking on the "E" button that shows up for a testcase that has been marked Automated. A single test script can be executed or a batch of them can executed at one time.
Executing Multiple Automation Scripts
Multiple testcases can be selected in the Test Execution tool and then clicking on the Execute Multiple button. Upon doing that, the following window is displayed with the batch of test scripts added to it. They can be re-ordered by selecting them and using the Move up and down arrows.
System level variables can be sent to the ZBot/ZIP for this particular execution run. These are replaced by the actual values before being handed off to the ZBot/ZIP. These are entered in the ZBot Parameter field but are not mandatory. The available system variables to send are:
@@uid@@ = User ID who has been assigned to execute the testcase
@@rid@@ = Release ID of this particular release
@@cid@@ = Cycle ID of this particular execution cycle
@@cname@@ = Name of this particular execution cycle
@@cpid@@ = Cycle-Phase ID of this particular execution cycle
@@build@@ = Build information of this particular execution cycle
@@env@@ = Environment information of this particular execution cycle
@@nid@@ = ID of the tree node to which this testcase(s) belongs to
@@nname@@ = Name of the tree node to which this testcase(s) belongs to
Multiple system level parameters can be added to this field separated by a space. This is not a mandatory field.
The ZBot can then be selected, initial execution status can be set and potentially, the option to execute multiple scripts in parallel can be chosen. This last option kicks of multiple scripts without waiting for the previous one to complete.
The rest of the process is very similar to executing single test scripts as described above, including updating results manually or automatically.
Executing Single Automation Scripts
On selecting a single automation script to run, the following window is presented. Here the Path field can be modified (just in case it was entered incorrectly in the Test Repository or needs to be changed for other reasons). Changes made to this path here will not be written back to the original testcase.
System level variables can be sent to the ZBot/ZIP for this particular execution run. These are replaced by the actual values before being handed off to the ZBot/ZIP. These are entered in the ZBot Parameter field but are not mandatory. The available system variables to send are:
@@uid@@ = User ID who has been assigned to execute the testcase
@@rid@@ = Release ID of this particular release
@@cid@@ = Cycle ID of this particular execution cycle
@@cname@@ = Name of this particular execution cycle
@@cpid@@ = Cycle-Phase ID of this particular execution cycle
@@build@@ = Build information of this particular execution cycle
@@env@@ = Environment information of this particular execution cycle
@@nid@@ = ID of the tree node to which this testcase(s) belongs to
@@nname@@ = Name of the tree node to which this testcase(s) belongs to
Multiple system level parameters can be added to this field separated by a space. This is not a mandatory field.
The next thing to do is to select a ZBot from the drop-down list of available ZBots (or target machines) on which to run the automation script. You can also select a Status for the testcase to be shown while it is being executed (WIP or Work-in-Progress in this example).
At this point, clicking on Run executes the script on the remote machine. Real-time Execution Messages are shown right next to the script in the grid above and ZBot Real-time status. Upon completion of test script execution, results can be updated either manually or automatically.
Updating Automation Execution Results
Automation test scripts can be kicked off singly or in a batch as described above. Based on the type of script written, automation execution can be over in seconds or could days to complete. Either way, once the automation is complete, results can be updated into Zephyr in one of two ways.
1. Manually update or promote results
This is a good option to employ in the initial stages of Automation. If your automation scripts are prone to being disrupted by external factors (environment, data, unstable code etc.), you will have a lot more failures (potential false-positives or false-negatives). If you were to automatically update these results, they might not represent the true state of your execution. Remember, once results are updated in Zephyr, metrics are automatically calculated and this information is pushed out to gadgets. So, in these scenarios, its best to watch and complete your automation execution and when you are comfortable with the results, manually update the status of that execution.
2. Automatically update results
When you are comfortable with the state of your automation execution and confident in its ability to run without being disrupted by external factors (environment, data, unstable code etc.), automatically updating results is a great option. This can be done via the execution utility available in ZBots that use a webservices API call to update the results in Zephyr. Please refer to that section for more details.