Remote Usability Testing
Inge De Bleecker Rebecca Okoroji更新时间：2021-07-16 18:18:46
最新章节：Leave a review - let other readers know what you think封面
Title Page
Copyright and Credits
Remote Usability Testing
Dedication
Packt Upsell
Why subscribe?
PacktPub.com
Contributors
About the authors
About the reviewers
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the color images
Conventions used
Get in touch
Reviews
Why Everyone Should Run Remote Usability Studies
Usability testing methodologies
In-person or lab usability testing
Remote usability testing
Advantages of remote methods
Extended reach
Typical devices
No travel required
No travel costs
No lab costs
No lab environment
Familiar equipment
User's natural environment
In the wild testing
Larger number of participants
Disadvantages of remote methods
Distributing the product under testing
Other disadvantages
Remote moderated usability testing
Advantages of remote moderated studies
Body language
Tailored follow-up questions
Disadvantages of remote moderated studies
Remote unmoderated usability testing
Advantages of remote unmoderated studies
Natural behavior
Time zone independent
Less effort for the UX researcher
Eliminates influence
No schedules to manage
Testing with minors
Faster turnaround
Disadvantages of remote unmoderated studies
Guidance
Other disadvantages
Types of usability studies
Formative and summative usability studies
Qualitative and quantitative usability studies
Longitudinal and single-session studies
Comparative and single test object studies
Other study types
Summary
What Not to Forget When Planning Your Study
Who are the stakeholders?
The product being tested
The goal of the study
Determining the status quo
A regular study to measure against a baseline
Why? (Finding answers to questions about increasing conversion/decreasing drop-off)
Understanding the users
Global suitability
Comparing design options
Comparing to competitors
Feature validation
Concept validation
Product validation
What to do next
Predicting future behavior
Classic usability goals
The budget
The scope of the study (the tasks and questions)
Objective benchmark scores
Task completion rate
Time-on-task
Subjective benchmark scores
Customer Effort Score
Single Ease Question
Task satisfaction rate
Net Promoter Score
System Usability Scale
USERindex
The test environment
Which are the target devices for the study?
What state is the product in?
How can the product be accessed?
Are special credentials required?
Are there any known bugs?
Any other requirements
The participants
Demographic requirements
Device requirements
Other criteria
The number of participants
Participant incentives/compensation
The study methodology
Moderated or unmoderated study
Video-based and survey-based studies
The schedule
When will the product be available for testing?
Are there any hard deadlines?
Are there any update cycles that need to be taken into consideration?
Do the participants need to be equipped with physical devices?
What time of year is the study planned for?
Is a pilot run with participants/stakeholders necessary?
Is external approval required?
Do external factors impose a schedule on the UX study?
The deliverables
Study framework for sign off
Participant screener
Discussion guide/script
Report
Next steps
Summary
How to Effectively Recruit Participants
Who to recruit
Multiple target user groups
When the target user (almost) doesn't exist
Recruiting for specific target user groups
Minors
Seniors
People with disabilities
Subject matter experts
Who is really participating in your study?
How to find participants
Self-recruiting
Coworkers
Friends and family
Recruiting Ads
Customers
Panel companies
Representations of target users
Expectations
Screening the participants
Screener methods
The number of participants
Challenges
Informing the participants
Time commitments
Location
Online versus offline
Devices and tools
Expectations
Compensation
Monetary compensation
Cash
Gift cards
Donations
Non-monetary compensation
Lottery drawings
Promotional items
Sheer gratitude
How much is the right amount?
When to provide compensation
Extra costs
Next steps
Re-screening participants
Building a database for future recruiting
Summary
Running a Remote Moderated Study
Discussion guide
Anatomy of a discussion guide
Introduction
Warm-up
Tasks
How many tasks and questions?
Task descriptions
Topics
Questions
Post-session questions
Wrap-up
Writing tips
Tone
Style
Preparing for the study
Number of participants
Days for the sessions
Product being tested
Internal team
Number of sessions per day
Time of day
Dry run
Backup slots
Floaters
Sample schedule
Communicating
Communicating with observers
Communicating with participants
Communication tools
Being prepared for changes
Pre-session tech setup
Running the study
Running the sessions
When to abort a session
The participant is not getting it
Mismatches in participants
Someone other than the recruited participant joins the session
Participant misrepresented themselves
Debriefing
When enough is enough
Blocking usability issues
Consistent feedback
Moderator techniques
The rules of the game
Understanding your participants
Level of engagement
Participant disposition
Managing observers
Moderator note-taking
Basic tool functionality
Next steps
Summary
Running a Remote Unmoderated Study with User Videos
About user videos
Screen capture
Audio capture
Think-aloud protocol
Formulating thoughts
Continued thinking aloud
Strengths of user videos
Observation of user actions
Think-aloud verbal cues
Facial expressions
A picture is worth a thousand words
Suitable throughout the development cycle
Limitations of user videos
Privacy of user data
Digital rights management (DRM)
Less deliberate feedback
Session organization
Session duration
Number of participants
Number of tasks
Number of videos
The script
Introduction
Instructions
Think-aloud
Give examples
Greeting and task description
Video recording
Video file format size and delivery
Access to the product under testing
Device to be used
Read task instructions
Tasks
Tasks that are easy to complete
Tasks that are difficult to complete
Complex tasks
Long tasks
Goal-oriented tasks
Post-task or post-session questions
Writing tips
Provide a clear starting point for each task
Tone
Preparing for the study
Scheduling participants
Running the study
Next steps
Summary
Running a Remote Unmoderated Study with a Survey
About surveys
Write-down-what-you-think protocol
Strengths of surveys
Long tasks
Deep dive questions
Suitable throughout the development cycle
Larger number of participants
Limitations of surveys
Not ideal for path analysis
Fraud
Self-reported data
Survey questions
Open questions
Closed questions
Single-answer multiple-choice questions
Multi-answer multiple-choice questions
Semantic differential scale
Ranking questions
Grouping questions
Study organization
Session duration
Number of participants
Number of tasks
The script
Introduction
Instructions
Write-down-what-you-think protocol
Access to the product under testing
Device to be used
Tasks versus questions
Tasks
Task context
Task flow
Tasks that are easy to complete
Tasks that are difficult to complete
Complex tasks
Long tasks
Goal-oriented tasks
Post-task questions
Time-on-task measurement
Post-session questions
What makes for a good question?
What is the desired answer?
One question per question
Avoiding leading questions
Meeting the goals
Clarity
Tone
Unblocking participants
Providing clear anchors
Providing a clear starting point for each task
Facilitating sentiment expression
Follow-up questions
Obvious questions
Preparing for the study
Dry run
Scheduling participants
Running the study
Next steps
Summary
Running a Remote Unmoderated Study with a Hybrid Approach
About the hybrid method
Strengths of the hybrid method
Self-reported data informed by user videos
Device and product version confirmation
Reducing fraud
Limitations of the hybrid method
Increased analysis effort for the UX researcher
Study organization
Session duration
Number of participants
Number of tasks
The script
Introduction
Instructions
Standalone responses
Tasks
Recording start and end
Balance between open questions and user videos
User videos for relevant tasks only
Post-task or post-session questions
Writing tips
Preparing for the study
Scheduling participants
Running the study
Next steps
Summary
What to Consider When Analyzing and Presenting the Study Results
Analyzing the data
Preparing the raw data
Remote moderated studies
Remote unmoderated studies with videos
Remote unmoderated studies with surveys
Hybrid remote unmoderated studies
Compiling the findings
Observed or recorded data
Self-reported data
Interpreting the findings
Visualising the data
(Stacked) column/bar chart
Line chart
Pie chart
Table
Word cloud
Lists
Best practices when representing data
Identifying issues
Where did the study participants struggle with a task?
Where did the participants voice (or respond with) insecurity?
Were there any big emotions whether negative or positive?
Are there any outliers?
Are participants consistently using different terminology?
Are there any inconsistencies in the responses?
Were there any "false positives"?
Did anyone encounter bugs?
Identifying recommendations
Make recommendations constructive and direct
Provide detail and illustrate
Address only the original usability problem
Speak the readers' language
Provide alternatives
Solve the problem
Assign a severity
Focus on the user
Reporting the data
Audience
Content of the report
Cover page
Summary
The goal(s)
Any critical issues
The bottom line
Optional details
Study context
Benchmark scores
Findings
Improvement/remediation recommendations
Verbatim participant comments
Conclusion and next steps
Appendix
Summary
Thanks! And What Now?
The debrief session
Next steps
Summary
Sample Material and Further Reading
Sample material
Further reading
Other Books You May Enjoy
Leave a review - let other readers know what you think
更新时间：2021-07-16 18:18:46