Once installed, the Testy plugin allows you to create and maintain a special type of Structure column called 'Test Run'. You can take advantage of Structure's ability to manage hierarchical lists to build up testsheets, using Testy to quickly mark and view test step results in separate columns.
Adding Test Runs to the Structure
Once you have prepared a test sheet in a separate structure, it's good idea to create a separate view as well.
That's because the test runs you see on the screen are defined by the column configuration that is saved in the view.
Test run columns are added using the standard add column dialog. To add a column, click on the + button at the right corner of the table header. A drop-down with available column presets appears. Scroll down to see the Testy commands group
Either one of the available commands will create a Test Run column, which can later be changed to display any available test run.
Creating a New Test Run
Choose Create new test run... from the column list or column configuration dialog to create a new test run. Enter a descriptive name for the test run, e.g. "Test Run 1 12/03/2014 Chrome 33.0". The name of each test run should be unique within any one structure. Test runs with the same name can exist only in different structures.
Press enter or the √ icon to save the new test run. If the name is invalid, you will be alerted to this problem.
Choosing an Exsiting Test Run
You can choose from the existing test runs associated with a particular structure by adding the Existing test run... column to the widget and selecting the required test run from the drop-down list in the column configuration dialog. This dialog will also let you change the name of the run or delete it completely.
Please note that if you remove the column itself using trashcan icon in the dialog footer, this does NOT delete the test run. It merely removes the column from the structure board. You can add a new column and choose this test run again later. If you delete the test run itself, all associated test results are gone forever.
By default, the drop-down list displays only test runs which are associated with the current structure. You can, however, also add test data associated with the same issues but in another structure. To do this, start typing the test run name to see suggestions. Test run modification permissions are tied to structure permissions, so it may be available only in read-only mode to you.
Changing Test Results
Once you have created a new or existing test run column, you can start setting test results. To do so, simply click on the empty space or on the already selected result corresponding to the appropriate issue. A result selector popup will be displayed. Choose one of the options accordingly.
If you change the result for any group issue, the selected result will be propagated down through the whole sub-tree of all its descendants.
Test Run Column Display Options
By default, a group issue's test result is displayed as a distribution of its children's results. You may choose to display it as a single result by unchecking the Show numbers option. In this case, it is calculated by the following rules:
- if all included results are PASSED (save for possibly a few SKIPPED), the group result is PASSED
- if at least one included result is FAILED, the group result is FAILED
- if at least one included result is BLOCKED, the group result is BLOCKED
- if all included results are SKIPPED, the group result is SKIPPED
- otherwise it's considered ambiguous and isn't displayed at all