CAPTCHA: Necessary Evil

Everyone has had a problem with standard security technologies - Completely Automated Public Turing test to tell Computers and Humans Apart (CAPTCHAs) - at least once in life. The technologies are said to protect websites from bots and Optical Character Recognition (OCR) software, so that only humans are able to set up an e-mail account, download software, etc. Nevertheless, CAPTCHA technologies often stop the Internet users and do not stop bots and software.

CAPTCHAs hacking

Nowadays, easy CAPTCHAs are not difficult to hack in many different ways. Some examples include: using OCR software, statistic analysis (words and pictures from database with CATPCHAs), neural networks (better than classic algorithms used for discernment of shapes variety), and Turing Farm (employment of people from Third World who rewrite CAPTCHAs). 

The basic way to prevent codes from being hacked is by a distortion, or an intentional disorder of code elements, so they become unrecognizable to OCR software. Most often, disorders used by CAPTCHAs are:

• dislocating elements up or down
• turning
• calibrating
• bending

Currently, there are no CAPTCHAs which are absolutely safe from an attack, but the technology is very popular when it is applied for users’ verification. The question is: Why do we still use them?

Problems with usability and accessibility

Usability and accessibility difficulties with CAPTCHAs started from the very beginning of CAPTCHA technology. What is more, the problems are getting more and more complex and complicated. Some of the drawbacks are:

• low readability
• degree of code complication – mathematic riddle, etc.
• problems with readability by disabled people
• no refresh button – page is being refreshed automatically which causes loss of data in a form
• lack of information why CAPTCHA is required
• lack of audio button
• no alternative way of user’s identification without using the Internet – such service is provided by Yahoo!.

Usability of CAPTCHAs may be considered in many aspects. The most important aspects were classified by Jeff Yan and Ahmad Ahmad in their thesis “Usability of CAPTCHAs or Usability Issues in CAPTCHA Design.”

(Above) J. Yan, A. Ahmad, Usability of CAPTCHAs or Usability Issues in CAPTCHA Design; http://cups.cs.cmu.edu/soups/2008/proceedings/p44Yan.pdf.

Another problem with CAPTCHAs is accessibility. Not so long ago, blind users could use Google’s codes and reCAPTCHAs only. Fortunately, today we have more friendly websites for the disabled. They can click one special button and hear CAPTCHA. However, understanding spoken code can be a real challenge, especially for non-natives.

Symetria’s survey

1.   Introduction

Symetria did an eye tracking test related to CAPTCHAs. The aim of the review was to check if CAPTCHA is problematic and hard to read for users.

We chose three popular websites for our survey:

1. Sign-up form on Gazeta.pl [Poland]
2. Account on YouTube.pl. [USA – polish version]
3. Sign-up form on O2.pl [Poland]

The goal for users was to set up a new account. The real aim of the survey was hidden, as to not influence users’ natural behavior.

In data analysis we took into consideration effective fixations and the time that users needed to read CAPTCHA’s code.

The participants of the survey were:

• women and men
• between 20-30 years of age
• everyday Internet users
• living in a city above 500 thousand citizens

The results of the survey should be regarded as inspiration and exemplary instruction for future research.

2.   Survey results

Table 1 presents statistical results of Symetria’s survey. The best reading time was on O2.pl e-mail account. Users needed only 0.6 second and 5 fixations to read CAPTCHA.  Much worse times were achieved on YouTube and Gazeta.pl.

Table 1 Statistical Symetria's survey results-- i.e. CAPTCHA reading time and fixation number

We used independent samples (between subjects) t-test to compare means: fixation number and reading time. We found that:

1.  Fixation number:

• statistically there is significant difference between O2.pl and Gazeta.pl
• statistically there is significant difference between O2.pl and YouTube
• statistically there is no significant difference between Gazeta.pl and YouTube

2.  Mean time:

• statistically there is significant difference between O2.pl and Gazeta.pl
• statistically there is significant difference between O2.pl and YouTube
• statistically there is significant difference between Gazeta.pl and YouTube\ 

As you can see, there can be big differences with CAPTCHA’s readability. It’s quite obvious that the more difficult CAPTCHAs are to read, the more often users refresh the page and make mistakes. Taking into consideration that even the most sophisticated CAPTCHAs can be hacked, it may be not a bad idea to make them less difficult to read and more functional for users. 

During the survey we often observed users saying “those digits are very blurred” and users who were refreshing CAPTCHAs, hoping the new one will be easier to read.

Conclusions and good practices

To sum up results of the review, we characterized good CAPTCHA, which:

• ensures security for the website and, at the same time, doesn’t cause problems for its users
• ensures access to the website for the disabled
• contains a refresh button
• informs about the number of elements that are necessary
• reflects cultural specific character
• informs why it is required
• gives an additional opportunity of user’s verification (i.e. the telephone)
• is a subject for temporal security controls

You should bear in mind that all the elements above may not always be used in single CAPTCHA.

Good practices for AUDIO CAPTCHA are:

• possibility of hearing CAPTCHA as an accessibility requirement
• disorders of background sounds must be limited
• existence of audio / disabled mark
• reasonable use of javascript and flash