Once installed, the Testy plugin allows you to create and maintain a special type of Structure column called 'Test Run'. You can take advantage of Structure's ability to manage hierarchical lists to build up testsheets, using Testy to quickly mark and view test step results in separate columns.
Adding Test Runs to the Structure
Once you have prepared a test sheet in a separate structure, it's good idea to create a separate view as well.
That's because the test runs you see on the screen are defined by the column configuration that is saved in the view.
Test run columns are added using the standard add column dialog. To add a column, click on the + button at the right corner of the table header. A drop-down with available column presets appears. Scroll down to see the Testy commands group
Either one of the available commands will create a Test Run column, which can later be changed to display any available test run.
Creating a New Test Run
Choose Create new test run... from the column list or column configuration dialog to create a new test run. Enter a descriptive name for the test run, e.g. "Test Run 1 12/03/2014 Chrome 33.0". The name of each test run should be unique within any one structure. Test runs with the same name can exist only in different structures.
Press enter or the √ icon to save the new test run. If the name is invalid, you will be alerted to this problem.
Choosing an Existing Test Run
You can choose from the existing test runs associated with a particular structure by adding the Existing test run... column to the widget and selecting the required test run from the drop-down list in the column configuration dialog. This dialog will also let you change the name of the run or delete it completely.
Please note that if you remove the column itself using trashcan icon in the dialog footer, this does NOT delete the test run. It merely removes the column from the structure board. You can add a new column and choose this test run again later. If you delete the test run itself, all associated test results are gone forever.
By default, the drop-down list displays only test runs which are associated with the current structure. You can, however, also add test data associated with the same issues but in another structure. To do this, start typing the test run name to see suggestions. Test run modification permissions are tied to structure permissions, so it may be available only in read-only mode to you.
Changing Test Results
Once you have created a new or existing test run column, you can start setting test results. To do so, simply click on the empty space or on the already selected result corresponding to the appropriate issue. A result selector popup will be displayed. Choose one of the options accordingly.
If you change the result for any group issue, the selected result will be propagated down through the whole sub-tree of all its descendants.
Filtering Structure by Test Run Status
Structures can be filtered by test runs statuses using special structure automation rule (available for Testy 2.3+).
To show only issues with specific statuses in a certain test run, do the following:
- Click Automation | Filter | Test Statuses.
Select the statuses which should be shown and the test run which will be used for filtering. Click Apply.
If you want to filter based on several test runs - several generators should be added.
Test Run Column Display Options
By default, a group issue's test result is displayed as a distribution of its children's results. You may choose to display it as a single result by unchecking the Show numbers option. In this case, by default it is calculated by the following rules:
- if all included results are PASSED (save for possibly a few SKIPPED), the group result is PASSED
- if at least one included result is FAILED, the group result is FAILED
- if at least one included result is BLOCKED, the group result is BLOCKED
- if all included results are SKIPPED, the group result is SKIPPED
- otherwise it's considered ambiguous and isn't displayed at all
You can choose to display the avatar of the user who set the status by checking the Show users option. Note that an issue at group level will display the avatars of all users who changed statuses for its sub-issues. You can also use this feature to ‘assign’ a test to yourself by setting the status to ‘none’, after which your avatar will be displayed beside the empty status field.
Check the Show notes option if you want to add and view notes for test statuses. When this feature is enabled, the notes icon is displayed beside all status fields in the column (if the status is empty, the icon will only be visible when you place your mouse cursor over the field).
Move your mouse cursor over the icon to open the notes dialog. When the dialog opens, you can immediately start typing your comment. Notes support jira-markup!
When your comment is ready, click the Add button or press Ctrl+Enter to submit the note.
You can read existing notes by placing your mouse over the note icon, edit them by clicking anywhere on the text, and delete them using the Delete button.
The additional Auto expand notes option appears when notes are enabled. When checked, it makes the notes dialog open automatically every time a test status is set to ‘Failed’ or ‘Blocked’. Use this feature to remind testers to provide details when selecting these statuses.
Users with jira-administration permission are able to create new custom statuses via Administration | Structure | Structure Testy | Manage Statuses (or via Get Started button for Testy plugin on the Manage add-ons page)
You can create new status by clicking on +Create status button and Enter. Just start typing a new value to edit the name. You can choose the color by clicking on the colored lozenge.
The order of the statuses on this page is used to calculate the aggregate status for items that have sub-items. (if “show numbers” in columns settings is unchecked). The status of the parent item is defined by the highest-order status among its children.
The Suggesting Notes checkbox allows you to specify whether Testy should automatically open the dialog after this status is set. Remember, that the notes dialog will be opened automatically for this status only if Auto expand notes for notes-suggesting statuses is checked in Testy column settings.
Also you can remove your custom statuses by clicking the recycle bin icon in the Remove column. The removed status will be replaced with ‘None’ in test run columns.