Prevalence and state of user research
As in all other global locations, usability testing in the US is conducted with an emphasis on understanding user behavior and capturing user feedback.
Availability and location
Since the US is home to thousands of focus group facilities and dedicated usability test labs, most usability test sessions will take place in locations specifically designed for this function. It is rare for usability test sessions to be conducted at a hotel or convention center outside of a designated conference or large-scale meeting.
Drop out rates
Although most test participants treat their appointments seriously, work requests and weather conditions usually play a more important role in their days. Participant no-show rates in the US range between 5-15%. Even if US participants say “Yes” to a study request with full intent to show up, many rely on their employer’s flexibility when they take a couple of hours off from work. This means that last-minute work requests can limit a participant’s ability to actually show up for a study.
When scheduling test sessions, it is important to note that Tuesday, Wednesday, and Thursday typically have the highest show rates for usability studies. Monday tends to have a lower show-rate because participants might not have the opportunity to check appointment reminders during the weekend. Friday tends to have a higher rate of last-minute cancellations simply because weekend plans sometimes take priority over a usability test appointment.
Language and translation considerations
Most business-grade translators find it relatively straightforward to translate usability test sessions. US-based translators often work in pairs when translating more than a couple of hours. Based on our experience, translators for Asian languages (or Spanish, when testing with Hispanic participants) are most commonly used.
Testing design and protocols
We typically conduct usability test sessions in teams of two, with a moderator in the test room with the participant and a notetaker in the observation. Both team members are equally trained in the study protocol, which allows us to alternate moderation and notetaking and avoid moderator burnout over a multi-day study. We also use instant messenger (IM) to allow the moderator and notetaker to remain in communication during sessions, which allows for more thorough probing with participants on a task-by-task basis, instead of saving follow-up questions for the end of a test session.
In the US, we typically recruit participants not only for their match to the study criteria but also their ability to clearly articulate their thoughts and provide feedback. This, in combination with US participants’ general willingness to be critical, has led to feedback from some Asian clients that US participants seem to talk noticeably more than they expected. The US tendency to talk more has an impact when localizing study protocols to the US. Usability test sessions that may have taken 60 minutes in Asia may take 75-90 minutes in the US.
Although US participants may provide extremely blunt or negative feedback on an interface, it is not unheard of for them to then turn around and award relatively positive satisfaction ratings because “once I figured it out it was easy to use.” This inconsistency can be surprising to our global clients and colleagues. We suspect that US participants may feel a somewhat greater obligation to be polite in their numerical rankings than during their spur-of-the moment qualitative feedback.
Comfort using technology
It will not come as a surprise that the way US participants use their mobile devices is sometimes about 5-6 years behind Japanese users and about 2-3 years behind European users. It has an undeniable impact on US participants’ understanding of why they might use a device in particular ways, especially “novel” mobile features. Among iPhone users, however, this gap is starting to narrow, so it is important to know participants’ experience.
US participants usually expect to be recorded and do not mind signing consent or non-disclosure (NDA) forms. However, there is a greater sensitivity to facial recordings for healthcare studies, especially when patients are being asked to provide feedback on a healthcare device or interface that is directly related to a medical condition that they have. In this case, we may focus the recording on their hands instead of their faces while they are using a healthcare device.
Holiday, seasonality, and timing considerations
It is usually very difficult to schedule usability studies on the weekend. Most US participants place a high premium on their weekend time and reserve it for family, friends, or weekend errands and chores. If a weekend study is required, we usually need to recruit for it much further in advance and provide higher incentives to encourage study participants to show up.
Due to Christmas and New Year’s, it is often best not to schedule fieldwork for a usability test any later than the second week of December. In addition to these end-of year holidays, there are often several holidays that the US federal government and all school systems observe, which makes it difficult for many participants to attend test sessions on those dates.
In the US, we typically average 6-7 hours of test sessions per day. Since our working days are actually longer – with early morning final equipment checks, a lunch break, and end-of-day briefing discussions – we try to schedule usability test sessions to run between 9:00am and 5:00pm.
When we test with under-age participants (students who are under 18 years old, for example), we require participants to fax in parental consent forms prior to their study session. Depending on the study, some under-age participants will also arrive at their test session with a parent. Although we have had rare situations where the parent wants to check the test room to make certain that nothing is improper about the setup, most parents are content to sit in the waiting area and read.