Other disadvantages

Another disadvantage of remote unmoderated studies as compared to moderated testing is the lack of immediate observation. Body posture and other physical expressions that could provide the moderator with additional insight during a study are not available. We don't view this as an absolute argument against remote unmoderated testing because you could also argue that, especially in globally distributed tests, the UX researcher may not be familiar with local gestures, and thus potentially interpret them wrongly. In India, for example, shaking your head does not mean "no"—it actually means "ok" or "I understand". Thus, asking participants to explicitly express their opinions in writing in an unmoderated study is less subject to bias than interpreting body language.

The lack of personal contact could also be viewed as a disadvantage because the unmoderated study does not allow for building a strong personal rapport between the UX researcher and the study participants. To some extent, it is possible to build a rapport in a remote unmoderated study just through how the participants are addressed in the script. As we saw with the previously mentioned disadvantage, though, what might help to build rapport with members of one culture might not work with others, or might even have the opposite effect.

One last disadvantage of the remote unmoderated methodology is that it is not possible to get reliable time-on-task values. Some unmoderated testing platforms may record the time taken on a task, but there is no guarantee that this will be valid because it is not possible to determine whether the participant continuously worked on the task or was distracted by a phone call during the execution of the task, for example.