How to setup a Test Repository

The test repository consists of a detailed tree organized in a way that reflects how you undertake your testing activities and contains nodes and testcases.  The test repository within Zephyr can be accessed from the Test Repository tool.
 
Zephyr provides considerable flexibility in how you can organize your testcases and your ability to change this structure from project to project and release to release.  Your testcase organization will impact how you organize and schedule testcases for execution while setting up execution cycles in the Test Planning tool. This section provides users with several options available to organize your testcases.  These options utilize the flexibility of organizing the testcase tree structure but also make use of Projects and Release tabs in various ways to provide greater options. A specific option here might not exactly reflect how you organize, but reading through the various options will give you some great ideas as you think through this process.

Traditional Waterfall

In Development and QA environments that follow a traditional waterfall methodology, release cycles are typically several weeks to several months long, and usually have multiple phases of testing (Functional, System test, Performance, User Acceptance Test, etc) during any given release cycle.   Execution cycles are planned and scheduled based on these phases and metrics are tracked for each phase as well.
 
For such environments, users can organize their testcases in a couple of ways:

• By Test Phase

• By Functionality or System and then sub categorized by Phase

Agile Environments

Typical Agile Development and QA environments release products in shorter and frequent release cycles, each consisting of multiple Sprints in which one or more User Stories are targeted for development completion.
 
In this scenario, users have a couple of options to organize and track their testcases:

• Organize testcases by Sprints and further subcategorize testcases by User Stories or functionality.  This is useful in scenarios where you have a large number of releases with shorter Sprint cycles and many overlapping release cycles

• At the Project level, define a specific release for a given Project.  Define the Release tabs under the Project as Sprints (e.g. Sprint 1, Sprint 2, etc).  This structure can be useful in those cases where you have larger release cycles with many Sprints

Testing of large platforms/complex systems with multiple systems and subsystems

For those organizations that are responsible for testing of large, complex platforms that may contain multiple systems or sub-systems each with its own development and QA tracks, and there may be a need to track testing progress on a per system bases followed by System wide testing.
 
Zephyr provides a couple of options best suited for such environments.

• Define a Project per release and use the Release tabs to track testing for individual systems followed by System level testing (System Integration test, Platform Regression, Performance, UAT, etc.)  This structure is useful when different teams are tracking progress for different systems and each requires their own metrics views

• Define a Project for the overall platform, use the Release tabs to track each release cycle and for any given release, organize testcases first by module or system then at a sub-category level, by feature or phase.  This is useful when the platform has larger number or frequent releases and a single Project Management group oversees QA progress across any given release

Testing of web tools

Web tool testing typically entails unique challenges such as requiring to run a group of testcases against multiple browsers on multiple Operating Systems and even against multiple versions of a single browser.  Here, users need to account for minor differences amongst various browsers or browser versions and the number of target browsers can change from release to release.
 
Zephyr allows for organization in this scenario in a couple of different ways:

• Under the release tab, organize testcases at the root level by browser and create linked testcases across browsers.  This is useful when there are a larger number of differences in testcases per browser and the number of target browsers is limited

• Organize testcases by functionality or phase of testing and using the tag or a custom field, identify which testcases are to be run against specific browsers.  Users can then track testing against each browser by Execution cycle in the EAS tool.  This structure is useful when on set of testcases can be used to test any given browser or when the number of target browsers changes from release to release

Games and Mobile tool testing

A key challenge that QA groups responsible for testing mobile tools or mobile games is that releases are often very fast paced and target devices against which the tool must be tested can be quite large in quantity.  In these scenarios, teams must often execute entire test runs in a matter of days or even hours while still being able to track progress on a per tool, per device basis.
 
For this, Zephyr provides multiple options to organize your testcases:

• Each release tab corresponds to variations of the tool.  For any given tool, in TCR, users can organize testcases at the root level by console, tablet or device

• Or, at the root level in TCR for an tool, testcases can be organized by functionality and test runs for each device can be tracked as a separate cycle in EAS

Customer specific / customized releases

Organizations that are responsible for testing customized releases of their core product or platform (such as Professional Services teams) can use Zephyr to track individual customer QA projects.  For such teams, ease of access to and re-use of testcases used by the Product team is very beneficial.  They will typically need to track progress of each customer customization and may even need to share metrics and dashboards with their customer.
 
In this scenario, users can either:

• Define a Project as a specific Release and use the release tab to represent individual customers.  For any given customer, they can then organize testcases in any number of ways.  This is useful for tracking metrics on a release basis when there are fewer releases per customer and when testcase variation between customers is minimal

• Alternatively, users can define a Project by customer and for any given Customer Project, can represent each Release tab with a release to be deployed for that customer.  Here too, testcases can be organized in the repository in any of the options described (by Phase, by Functionality, etc).  This structure is useful when easy review of historical data for any given customer is needed and sharing Dashboards with customers is required