Test prioritization: variations
July 2019
Once you’ve prioritized pages, elements, and types of change, you’ve got a test. It just needs variations. And some might not make the cut.
Do you need to prioritize variations?
I mean, maybe not.
You know where you’re testing, and what type of changes you’re making; let’s say this is a homepage hero section design test.
You’ll know in advance how many variations your site can support; let’s say it’s 5.
You might only have the design and developer resources to produce 5 variations - possibly only 3 or 4. If that’s the case, go for it! No prioritization needed. Your test is good, and you should feel good.
But if you’re able to bask in the luxury of more designs that you can test 😌, you’ll need some way to discard a few without hurting feelings.
Differentness
You asked your designer(s) to give you homepage hero mockups that were “as different as possible.” So that’s the first criteria to apply in ruling out options.
Find the two most similar variations, and discard one. Repeat until you’re down to your target number.
Which one, though? Or what if all your variations are quite different, but you still have too many?
If you find yourself making this kind of decision frequently, you might want a framework. More on that tomorrow!
But for small teams who only need to eliminate the occasional surplus variation, I’ll suggest that any one of the following methods will work just fine:
- Roll some dice
- Let the most junior person choose
- Keep the easiest ones
- Keep the ones with the most supporting evidence
- Discard your personal favorite
What do you think? Would the internet explode if you and your team used a pair of dice to decide which variations to test? Hit Reply and set me straight.