The test writing engine (TWE) lets users request batches of 20 tests and get these tests back in a passing state. This means that the TWE gives users tests that can be added to their test suite for immediate execution without a need for optimization.
To learn more about the Test Writing Engine, read our help article here.
You’ve triggered or are about to trigger a test writing request. What happens next?
The most critical component that can define a successful or unsuccessful Test Writing Engine run is the input submitted. To ensure a high quality output and speed up the process so you get your tests back fast, we recommend the following guidelines.
A working environment.
This is a very common source of issues preventing successful test delivery. This could mean anything from allowing testers to log in, all the way to the environment not throwing up serious issues while testers are using related features or functionalities.
- Specify the environment and let us know if you’re using more than one.
- Ensure that the environment is accessible 24/7 - most test writers are in different time-zones and they should be able to access the environment at any point. If there are any known limitations (for e.g. down time during weekends for maintenance), please let us know when you submit your tests.
- If required, the environment should be able to support multiple concurrent testers.
- Does the environment have any IP limitations? If yes, let us know so we can proceed accordingly.
- Communicate any major known defects in the app or test environment.
Credentials. This goes hand in hand with the point related to the environment, above - provide a working set of credentials, when required.
Good instructions. What qualifies as a good set of instructions?
- A descriptive title: Think of something that will help you identify it and use it later when you add it to a feature folder and/or run group(s).
- A brief but descriptive test outline: There should be detailed instructions on what the tester should do. If testers do not have enough information, they will make assumptions to move forward and this may not align with the desired outcome. Ensure that there is no ambiguity in the outlines and that there is a helpful indication of what the expected outcome of the test should be, so test writers know if something is working as intended or not.
- No business-specific or sector-specific jargon: To make the outlines as test writer friendly as possible, do not use business jargon in the instructions as much as possible. There’s a good chance that this will be the first time test writers are encountering the app and may not be familiar with it. Be as obvious as possible, try not to use terms that are not already reflected in the app and use business jargon only if there’s a clear explanation of what it means.
Good examples of Test Outlines that have yielded green tests on first run
“Ensure that when the user is in the My Pay section, that clicking on the dependent will expand to reveal more information. Also make sure that the eye icon will show/hide the SSN of the dependent as well as the SSN of the employee in the upper right corner of the my profile page.” "Click on "my pay" from the dashboard. Click on all of the links under Financial tools and verify that each page takes you to the correct calculator. Ensure that each page loads properly."
“Please verify the following elements on the page work: All Links/buttons on the main article of the page. This includes all buttons/links below/next to/on the image carousel/hero image. Snippets (links to other articles) below the main article. Clicking on the image, clicking article's title, and Read more link. Links to other pages like Related articles, or any other links to the right of the main article. Do not check any dynamic content like news articles, events, videos.”
“The context of this test is to check the Manage Business Hours function for both Doctors and Assistants. Go to www.fakeurl.com. Doctors and assistants are able to set the Business hours. It can be set for each clinic associated to the user, they can be set for a day or they can be repeated in a defined interval. Also users are able to remove the availability by hand. [Relevant screenshot inserted here]”
- Use video, if possible: The preferred method of input is video as this provides better clarity for test writers because the video contains a visual of what the test should cover. Video walkthroughs should be a narrated recording of your screen or window that walks through the functional test you want created. To help clarify what needs to be written, be sure to narrate the actions you are taking and the expected results you need verified in the test. Verify that the video is accessible and that test writers will not have to request permission.
For more information on how video walkthroughs work, read our help article here.
An example of a video walkthrough can be found here.