Testing, Assessment and Quality Assurance

Task: User test a design or prototype

For general guidance on user testing, read the section

About User Testing

In addition, specific guidance on how to test for compliance is provided with the individual guidelines. Each guideline has a link to a full explanation and help, which may include suggested test methods and advice on user testing. Go to the guidelines.

Task: Assess a design concept or prototype

Designs can be assessed for accessibility at any stage, from the initial design concepts through successively more functional prototypes.

Step 1. Consider the following high-level questions.

These questions will help you identify many of the potential accessibility issues without looking at the HTML code. You can therefore use them to assess design concepts at the earliest stage, when they may still be only drawings.

Question: Is the purpose and organisation clear?

If you had never seen the site before, how would you know what it was for, what you could find there and how you would navigate it?

Question: Is content simple enough?

Is there any jargon, terminology or concepts which make sense to you but could confuse users? Are there complex sentences or big blocks of text that could be broken down?

Question: Are style and navigation consistent?

Are the overall arrangements of page elements consistent? Do different pages share common features? Do controls behave consistently?

Question: Is the colour, texture and contrast okay?

If someone has difficulty perceiving colour, will they see the controls and text? Are there any patterned backgrounds that reduce text legibility?

Question: Where should there be alternative text and what should it say?

Where are images used and what alternative text descriptions should they have for users who are blind and use a screen reader? In particular, look out for images used for buttons and navigation links and images of text such as logos and headings. These may be difficult to distinguish from real text. If in doubt, ask the designers or developers what they intend to use.

Question: Does the site use frames?

It may be difficult to know by looking at a design whether it uses frames. If in doubt, ask the developers. If it does use frames, ask what the developers intend to name the frames and whether they intend to add summaries. This may be important for people who are not using a visual browser and cannot therefore see what the frames contain without entering each one.

Question: What problems might there be with tables?

Is important information presented in table format or are HTML tables are used to create formatting effects? It may be difficult to know by looking at a design whether it uses tables and how they are arranged. If in doubt, ask the designers or developers. Consider all the guidelines relating to tables in the web accessibility guidelines.

Question: Are there any context-dependent links?

Link titles should always have meaningful names. Read them and ask yourself if they clearly explain the consequences of following them. Links are often listed by screen reader users, so they are read without the surrounding context. This also applies to alternative text descriptions of images used for links, controls and buttons.

Question: Are there scripts, applets or other programming techniques?

This could include technologies like Flash, Java, JavaScript or any other programming or scripting language used to provide interactive features. Sometimes, these are used inappropriately or implemented in a way that can make a site inaccessible. It may be difficult to know by looking at a design whether it uses scripts or applets. If in doubt, ask the developers.

Step 2. Apply the Web accessibility guidelines.

The above questions will cover the major accessibility issues. This may be enough at an early design stage. However, if the design has advanced to a prototype stage, particularly if it has been partially implemented as HTML, you can go further by assessing it against the NDA web accessibility guidelines.

The steps you should go through are described in the Assess a current web site for accessibility task.

Task: Assess a current offering for accessibility

Step 1. Determine the required level of compliance.

Accessibility requirements for the site may be written into the design specification. This may reference the NDA or WAI guidelines, saying something like "The site should meet all priority 1 accessibility guidelines from the Web Accessibility Initiative". The NDA web guidelines are the same as the WAI guidelines, so saying "NDA priority 1" and "WAI priority 1" are the same thing. The purpose of the NDA guidelines is to present the WAI guidelines in a way that is easier for you to understand and use.

If the design specification states accessibility requirements in terms of NDA or WAI guidelines, these will be your target and you can proceed to step 2.

If there is no design specification, or if the specification does not state accessibility requirements in terms of the NDA or WAI guidelines, you will have to decide now which of the guidelines the site should meet. The steps you should go through are described in the Scope accessibility requirements task.

Step 2. Use the Web accessibility checklist.

This is the NDA web accessibility guidelines, presented in the form of a printable checklist. When you are satisfied that the site meets a specific guideline, you can tick it off the checklist. Go to the checklist.

Step 3. Consult the test methods for each guideline

Many of the guidelines include suggested test methods that can be performed by an assessor. Consider carrying out these tests as part of your assessment.

Step 4. Test with real users, where appropriate

The suggested test methods for each guideline may include user testing. For a description of user testing, read About User Testing.

Task: Scope accessibility requirements

Step 1. Consult users or user advocates.

The first stage of scoping requirements is to build up a picture of the user population. You should aim to find out who will be using the site and in what situations. This information can best be gathered by talking to users themselves and will prove invaluable when it comes to deciding what accessibility features are needed. Requirements definition is described in more detail in How to Create Accessible Products and Services.

Step 2. Decide which of the NDA guidelines the site should meet

When you are sure of the user requirements, refer to the guidelines themselves to determine which should be met. The guidelines are divided into three priority levels.

• Following priority 1 will ensure that the site can be used by most people with impaired mobility, vision, hearing, cognition and language understanding and is compatible with their assistive technologies.
• Also following priority 2 will remove further barriers for these people.
• Also following priority 3 will ensure that the site is as easy, efficient and enjoyable to use as possible.

However, in order to assess whether a site is "accessible", you will have to decide how accessible it should be.

• Should it meet priority 1 only?
• Should it also meet priority 2 and 3?
• Should it perhaps meet a selection from priority 1, 2 and 3?

This is a decision for whoever decides the requirements for the site, and the NDA guidelines are not intended to state accessibility requirements for any given site, whether public sector or private. Although they may be referenced in a public body's policy (Irish Government policy currently requires priority 2), the NDA guidelines are not themselves policy. Although they may be referenced by the requirements specification for a web site, they are not a requirements specification for any given web site.

So, it is up to the persons who are responsible for setting the requirements for the site to decide what level of accessibility is appropriate. The NDA guidelines are provided to help with that decision by stating what makes a site usable by people with impairments. They are accompanied by detailed descriptions and rationale which explain the motivation and justification for each guideline. They also give guidance on how to achieve compliance.

It is unlikely that a site that does not follow at least all of the priority 1 guidelines could be considered to be accessible in a broad sense. Above this basic level, you will have to decide which priority 2 guidelines the site should also meet. This will depend on its purpose, the kind of content and services delivered and the project context. Questions like the following may help determine which guidelines are appropriate.

Question: How extensive or complex is the content and functionality?

Sites with more complex functionality or a larger body of information are generally more difficult for users to comprehend and navigate. If the site is large or complex, you may therefore decide to include in the requirements the priority 2 guideline "2.19 Provide information about the general layout of the site (e.g. a site map or table of contents)".

Question: Will users have support or help available to them when they use the service, or is it up to them to learn the interface?

If users access the site from an office environment, initial support and help may be available from colleagues or training. In contrast, home users may not have any support available to them, so it will be important to ensure that the site is easy to learn. This may be another reason for including guideline 2.19, to provide a site map or table of contents.

Question: Is the site subject to legislation or policy requirements?

Priority levels for public sector sites may be determined by policy. See Legislation and policy.

Question: What resources are available?

How much time is available to implement the site? Should maximum accessibility be implemented all at once or can it be rolled out in stages, starting with the minimum requirements? What skills will be needed on the development team to implement solutions for the various requirements? For example, if the developers already have the skills required to implement cascading style sheets (CSS), it may be wise to take advantage of that by including in the requirements the priority 2 guideline "2.4 Use style sheets to control layout and presentation". However, you may have to extend the skill set of the development team before adding in this requirement.

Question: What is the most important content or functionality?

You may decide to specify different accessibility requirements for different parts of the site. Although the entire site should conform at least to the priority 1 guidelines, it may be very wise to concentrate extra resources on the most important parts to take in some of the priority 2 and 3 requirements. For example, you may require only the minimum accessibility for archive or legacy material which few users will need to access, but specify further requirements for the most current content or the core functionality.

For more about the importance of the context of use, see What is accessibility?.