Stealth testing

Analytics it’s all nonsense, you are tired to watch recordings of your user actions, A/B testing is working to slow, so one thing remains: direct user testing.

You gather a group, you write your test case scenario and you come up with some pretty decent questions for them. Yet you are afraid they will be influenced by you. And you’re right, even if they don’t realize it, knowing they’re being watched makes them act differently with your product. So what can you do?

I like to go in stealth mode, I like to watch them when they feel most comfortable and not knowing the test has already begun.

Let’s see how I do it:

1. Meet with the subjects one by one, tell them that you want to let them know what will happen during the testing day 
2. When the meeting starts say to them that you have to make a quick phone call, something urgent 
3. Ask them to help you, for you to make the call you have to obtain some information from your website, and you have your hands full so you can’t do it yourself (it’s quite useful to give them a tablet or smartphone on which they can search what you need) 
4. Start talking on the phone with your imaginary friend. It’s time to pay attention on what the users are doing with the task you given them, while you keep talking. It’s ok to stand up and move around, place yourself behind them to have a better look, and it’s ok to put pressure on them, asking from time to time if they have found the information you need (in real life your users will use the site under pressure from time to time) 
5. End the phone conversation after you have your answer from them and explain your test subjects what will happened for the rest of the test 

By the end of the day you’ll have concluded two test case scenarios with the same users. Not only you can conclude how they acted differently when someone was watching them and when not but also you’ll see how they behaved when they saw your product for the second time.

This was just one example of stealth testing, the process can become more simplified or more elaborate, it can be transformed in a game, and it can be anything you want as long as your test subjects don’t know they’re being watched.