An overview of state-based test accounts
Why do I need unique test accounts?
Rainforest tests should be optimized for speed and efficiency. Logging testers into separate, unique test accounts of your creation allows for simpler test writing, faster tests and deterministic results. While creating sufficient unique test accounts requires initial work upfront, you greatly improve the quality of your results in many ways.
There are many reasons why the benefits more than justify the small amount of setup labor:
Isolate testers' work
- Imagine two testers logged into the same account, performing a test to update a blank field with an address. If one tester enters a value first, the second will fail the test, but the process works fine. Unique accounts will completely avoid this shared state risk.
Eliminate time-consuming state setup steps
- Without preset accounts, testers must go through some number of steps before even starting to test the process of interest. This is guaranteed to delay your results, without much meaningful data from the extra steps.
Guarantee correct account configuration
- By defining the state of the account internally, you eliminate the risk of mistakes in configuration. Results will be more reliable and issues can be reproduced more accurately.
Ensure deterministic failures
- State setup decreases the chances that testers will run into problems before they ever get to the process that you want to test. State set up eliminates the need for extra extensive evaluation of the results to find the problem, and save you time in the process.
What kind of test accounts do I need?
You will likely need a few different account types. There is a bit of strategy behind thinking through the details.
Start by creating a large number of unique accounts in a baseline state. These test accounts should be used for all tests that require testers to be logged in.
Pre-seeded accounts are a bit more nuanced. Consider these example scenarios you may want to test:
- Perhaps it is essential that a power user, who has purchased over 1,000 dollars of goods is shown a specific advertisement.
- A manager account with 25 employees should be able to view the employee detail pages, but the employees themselves should not see this page.
- An insurance startup might need to confirm sending and receiving a completed document, but want to avoid the time-consuming process of filling it in
All of these scenarios are made simple with pre-seeded accounts.
Consider any processes which require a lot of state setup before testing the core functionality. These are likely candidates for preset account types.
How do I create test accounts?
Once you've determined the account types that you require, you (or your developers) should script the creation of user accounts in the appropriate state. Save the accounts to your database, and save all of the individual login credentials for each different account type into a different CSV per account type. To make these available for testers, simply upload each CSV file into your rainforest account as new tabular variables.
How many test accounts do I need?
Be sure to create sufficient accounts so that all testers assigned to your test will receive a unique account. Use the following equation to ensure that you will have sufficient unique accounts:
Minimum number of test accounts:
Tests * Browsers * (3 Testers * 2 Safety)
- tests = number of tests using this variable
- browsers = number of browsers you're running these tests against
- testers = default number of testers we send to each job
- safety = safety margin we use in case we need to add more testers to a job
Using this equation, to successfully run 1 profile update test against 5 browsers, the we would need a minimum of 30 logins.
How do I assign test credentials to my testers?
Rainforest makes this easy to manage. To allow testers to access your pre-created accounts:
- Upload the spreadsheet of login credentials as a new tabular variable
- Create standalone 'Login' tests for each account type
- Direct testers through your login process and select the correct tabular variable
- Clearly name each test so that you know what account type it will log into
- Embed the appropriate login test into your process tests
Can I reuse the accounts I've created?
Yes! We recommend that you reset your database to a known baseline in preparation for every run. This allows you to reuse the test accounts you have in place again and again, saving time, and reducing work.
Managing this reset is most efficient if you've integrated Rainforest with your CI tool. Resetting your database either before or after each run should be included in your existing CI configuration.
If you are not using any sort of CI tool, or haven't integrated with Rainforest, you can trigger a database reset using our webhooks.