Our best practices on selecting Multiple Platforms for Rainforest Exploratory Testing
With Rainforest Exploratory, it is possible to select up to 3 supported platforms for any single bug-hunting Exploratory run. In this article, we will cover:
- How Exploratory runs work works when multiple platforms are selected.
- How to understand the results from a multiple browser Exploratory run.
- Discuss best practices on selecting multiple platforms for an Exploratory run.
How Multiple Platform Exploratory Works
By default, all Exploratory runs will be executed by our testers in the latest version of Google Chrome. When enabled for an account, Multiple Platform Exploratory will allow a user to choose up to 3 different platforms that have been enabled for your account. When that run is triggered, testers will understand to test out the areas specified in the areas of focus section of the Exploratory instruction form.
Results of Multiple Platform Exploratory runs
Results from a Multiple Platform Exploratory run will include all the browsers in which a particular bug was identified. When testers encounter a bug on one platform, they will verify whether that bug can be replicated in the other platforms selected for the run. Videos documenting the issue uncovered by Rainforest Exploratory testers will depict the first platform on which the issue was identified.
Best Practices on Platform Selection
Assess Scope of an Exploratory run against multiple platforms
The scope of the assignment detailed in the areas of focus is the most impactful consideration in a multiple platform Exploratory run. When multiple platforms are selected for an exploratory run, the scope and amount of work is multiplied by the number of platforms selected.
When choosing more than 1 platform for a given Exploratory run, we recommend as a best practice to designate 1 platform as the primary platform on which testers should do their testing on and then have them verify the issue is also against the other selected platforms.
Select one type of Platform (either Desktop or Mobile) per run
Rainforest desktop and mobile platforms will show up in the platform selection row for all bug-hunting Exploratory runs. As we note in our Mobile Exploratory FAQ, it is definitely possible to select a combination of desktop and mobile platforms for an Exploratory run.
In terms of best practices, in selecting multiple platforms of the same type have produced more detailed results. Given how a workflow may function differently in a mobile versus a desktop environment, how a test case is written to document the bug in one experience versus another may differ.
With the differences in how the the steps to reproduce play out on a desktop versus on a mobile environment, we recommend - when triggering Exploratory runs where multiple platforms are selected - to select the same type of platform either all desktop or all mobile whenever possible to produce the most pointed results.
iOS v.s. Android Mobile Web Exploratory runs
As Mobile Exploratory runs utilize Rainforest mobile VMs, when Android and iOS VMs are selected, tests will be executed on mobile Chrome and mobile Safari respectively.
While there is nothing preventing the selection of some combination of these mobile platforms, we recommend keeping Android Exploratory runs separated from iOS Exploratory runs as this will keep results clearly delineated.
iOS v.s. Android Native Application Exploratory runs
It is possible to test both the iOS and Android version of a mobile application simultaneously in a Mobile Exploratory run, given that they're properly set up. As iOS and Android applications are set up differently for testing in Rainforest, if both are selected for the same mobile app Exploratory run, both source URLs will need to be supplied to Rainforest testers.
Like with desktop and mobile versions of an application, the iOS and Android versions of an application may differ slightly in how workflows play out. Given this, our best practice with mobile application Exploratory runs is to separate iOS from Android runs whenever possible. Not only does reduce the possibility of tester error in platform selection, it also provides more pointed and useful results.
If you have any questions regarding Multiple Platform Exploratory, please let us know at firstname.lastname@example.org!