Prevalence and state of user research
Usability testing is the most popular usability evaluation method conducted in Japan, followed by heuristic evaluation. The usability test is conducted not only as a summative evaluation technique, but also as a formative evaluation technique that is frequently combined with paper prototyping. Sometimes, it is conducted in the very early phases to determine problems of a current product so that the new design will be an improvement.
Recruiting costs are relatively high compared to US and Europe, starting from $300 (plus $80 to $150 remunerations) per person to recruit general consumers from an online panel. Targeting special panels such as doctors, lawyers and patients can be difficult and expensive, with costs ranging from $500-$1,500 per person.
Drop out rates
No-shows are very rare
Language and translation considerations
If you are going to test a localized product, it is recommended to fully translate the interface into the Japanese language. Although Japanese people begin studying English in junior high school, and there are many foreign people living in Japan, we are still not completely familiar with foreign languages, including English. Therefore, most participants will expect to see a user interface or test stimuli in the local Japanese dialect.
The cost for simultaneous translation is quite expensive in Japan. A full-day session usually requires two simultaneous translators and can cost approximately $2,000 USD. Well-trained translators are generally very busy and it is recommended to schedule them long before the day of testing.
Testing design and protocols
It is often a surprise how serious participants are about working on tasks and answering questions. This seriousness sometimes causes unnatural behavior during the testing. We see some participants who are too influenced by the task goals, too obliged to complete the tasks, or who blame themselves when they are at a loss. A careful test design, such as one that gives more natural scenarios or open-ended tasks, as well as a friendly pre-test orientation to ease their sense of obligation, will help avoid this. It is also important to emphasize that the “test” is not for the participant but for the product or the system.
Travel and transportation
Travel within this country
Middle or high rated hotels ($150-$250 USD per day) in big cities such as Tokyo and Osaka offer fast internet access (either free or paid) in guest rooms or at the hotel’s business center. Inexpensive hotel chains such as Tokyo-Inn or APA Hotel will costs less than $100 USD per day, but the rooms are often very small.
The public transportation in these big cities is very well developed, and using trains/metros is the recommended method to get around. Except for during peak hours (7:30-9:30), they are quite punctual. However, for first comers to Japan, the train network may be very confusing, requiring them to walk quite a distance to switch trains or have seek help in purchasing tickets. It is necessary to estimate enough time to arrive at the testing site on time or early. In local cities such as Sendai or Okayama, taxis are recommended.
It used to be said that Japanese were shy and awkward during usability testing. Today, you will find many people quite relaxed and talking naturally during sessions. Of course, the amount of verbal response differs depending on the personality. Serious people and speculative people usually give fewer verbal responses. Additionally, there is a cultural bias for men not to say “I don’t know” or “I don’t understand,” as answers such as these are seen to make him seem less intelligent in the eyes of others. If you feel you do not get enough comments from the participants, it would be best to ask questions during/after the session about what you would like to clarify.
The Japanese in general tend to be quite conscious about privacy. Participants are sensitive about having their faces captured on video. However, this is not a major problem as Japanese participants tend to be less expressive in thier facial expressions during testing.
Holiday, seasonality, and timing considerations
The Japanese usually have long holidays for about a week from the year-end to the New Year season, during “Golden Week” from the end of April to the beginning of May, and during “Obon,” a ritual holiday period in the middle of August. It is recommended to avoid these periods as the recruiting of participants could be difficult.
The school year begins in April and ends in March, and most students have long holidays from mid-July to the end of September, from mid-December to the first week of January and from the beginning of February to the end of March. These are good periods to conduct testing with students. However, it is recommended to book their schedules in advance because they may be planning to go back to their hometowns during the holidays.
Evening and weekend sessions are common in Japan because usually the office workers are unable to attend daytime sessions during weekdays.
Test leaders may find it beneficial to collaborate with Japanese usability professionals who can provide information on the cultural issues of Japan and their impact on user testing. The rate for this type of consultation is approximately $800-$1,000 USD per day.
HCD-Net, the major organization of usability professionals in Japan, offers resources to support user research initiatives.