Suite Automation

Starting October 11, 2024 (Zephyr Enterprise 8.2), the Zephyr Enterprise documentation moved from its current location on Atlassian to a dedicated, standalone Zephyr Enterprise documentation page. Please see: https://support.smartbear.com/zephyr-enterprise/docs/en/zephyr-enterprise/zephyr-user-guide/zephyr-test-automation/zephyr-vortex/suite-automation.html

About

Suite Automation jobs are used to post a bunch of test reports to Zephyr Enterprise. When you execute a Suite Automation job, test cases are created in Zephyr, then they are scheduled and executed according to the test report received from the test automation framework. If there are some failed test cases, attachment or logs are added to the test case on the Test Execution page.

Create a job

To create a Suite Automation job:

1. Select Vortex > Suite Automation and click the “+“ icon:

2. In the Automation Jobs Creation form, populate all the highlighted (required) fields and click Add:

The Automation Jobs Creation form has the following fields:

Field name	Description
Job Name (required)	The name of your suite automation job.
Job Description	The description of your job.
Automation Framework	The test automation tool you are using. Possible values: SoapUI PRO TestComplete Cucumber JUnit Selenium TestNG Tricentis Tosca UFT Eggplant
Script Path	The check box is used to invoke a script. You can then specify the path to it in the field below.
Result Path (required)	The path to the folder that will store the results.
Create Package	Select the check box if you are using the Selenium automation framework.
ZBot (required)	The Zbot you want to use for automation.
Cycle Name (required)	The name of the cycle that will be created. Here you can decide whether you want to reuse the same test cycle for all the automation runs or create a new test cycle every time the automation job is run (you can include a timestamp in the test cycle as well).
Folder in Test Repository for Test Cases (required)	Test repository folder where the created test cases will be stored.
Select User	The Zephyr user to assign the job to.
Cycle Start Date (required)	The test cycle start date.
Cycle End Date (required)	The test cycle end date.

After creating a Suite Automation job, you can execute it to automate your test cases.

Execute a job

To execute a Suite Automation job, open the Suite Automation tab in Vortex and click E on the job line, or select a job and click Execute in the top-right corner:

Zephyr will show a message box informing that you have scheduled an automation job successfully.

When the job run is completed, Zephyr will create test cases in the Test Repository, create a cycle with the given name and will post the test results to the Test Execution page:

If there are some failed test cases, they will have an attachment:

Suite Automation Job Notes

Since Cucumber JVM supports the BDD framework, in Zephyr Enterprise, it will create test cases with test steps. For the remaining frameworks, only test cases will be present and not the test cases with test steps.
In Eggplant, Zephyr will give the result path as a CSV file instead of an XML file. For example, D:\egg plant\Files\EggPlant\test.suite\Results\SuiteStatistics.csv.
For every failed test case, it will attach screenshots or logs on the Test Execution page.
For Cucumber, if any of the steps fail, the next steps will not be executed and the test case status will be set to fail.
User can create their automated scripts through any framework, and they can execute their scripts by eclipse, bat file, sh file, Jenkins, Maven, etc.
- This depends from one user to another in user configuration.
- After executing their script, they must have generated reports in any format like JSON, XML, PDF, etc.
- Zephyr is only supporting JUnit XML now and only the above-mentioned frameworks.

Reuse existing cycles and phases

Reuse Scenarios	Reuse Actions and Impact
Currently using an existing test cycle and phase	A new test cycle and phase aren't created and instead reuse the same testing cycle and phase that was already created from the previous job in the Test Planning and Test Execution pages.
Currently using an existing test cycle and phase but you need to update the cycle start and end dates	A new test cycle and phase aren't created and instead will update/append the changes to the cycle's start and end dates for the test cycle and phase that already exists in the Test Planning and Test Execution pages.
Currently using an existing test cycle and phase but you need to update the person that they are assigned to	A new test cycle and phase aren't created and instead will update/append the changes to the assignee for the specific testing cycle and phase that already exists in the Test Planning and Test Execution pages.
Creating a new test phase but using an existing test cycle	When editing the phase name, the new test phase name will be created under the existing test cycle in the Test Planning, and Test Execution pages.
Using an existing test phase but creating a new test cycle	When editing the cycle name, a new test cycle name will be created under the Test Planning and Test Execution pages but there won't be a new phase created in the Test Repository.
Creating a new test phase and new test cycle	When editing both the cycle and phase name, a new test cycle name will be created in the Test Planning and Test Execution but there will not be a new test phase created in the Test Repository.

You can reuse an existing cycle and phase you created earlier.

1. Edit the existing job by selecting the job from the automation table in Vortex.

2. Ensure that the Reuse same Test Cycle for all automation runs is selected and then click Add.

Vortex jobs support multiple automation engineers running the same automation jobs on different machines and publishing results to the same Test Cycle/Phase in Zephyr.

3. Click E to execute the automation job again.

:

5. The status should be set to complete after applying the notifications:

A new test folder will not be created in the test repository, the folder will remain the same:

No new test cycle or phase will be created on the Test Planning and Test Execution pages. They will remain the same:

When reusing the same Vortex job, the execution history may display the execution changes made by Vortex.

Zephyr Enterprise 8.1.0