Testing, Assessment and Quality Assurance

Task: user test a design or prototype

For general guidance on user testing, read the section about user testing.

In addition, specific guidance on how to test for compliance is provided with the individual guidelines. Each guideline has a link to a full explanation and help, which may include suggested test methods and advice on user testing.go to the guidelines.

Task: Assess a design concept or prototype

Designs can be assessed for accessibility at any stage, from the initial design concepts through successively more functional prototypes.

Step 1. Consider the following high-level questions.

These questions will help you identify many of the potential accessibility issues without using the application. This can be done even at the earliest stage, when the design is still only drawing.

question: is the colour, texture and contrast okay?
if someone has difficulty perceiving colour, will they see the controls and information? are there any patterned backgrounds that reduce text legibility? will they be able to change the colours used to suit their visual abilities?

question: does the interface contain any fixed size windows or interface objects?
people with poor eyesight, poor quality screens or who are in environments where there is a lot of lighting glare often need to resize text and objects to make them visible. People who use on-screen assistive software, such as screen magnifiers and on-screen keyboards, need to keep a portion of the screen available for that. Will they be able to resize the window so it does not take up the whole screen?

question: is there anything that you might not be able to do using the keyboard?
for operations that you would normally do by clicking or dragging, how would you do them using only the keyboard?

question: how is the application going to be made compatible with assistive technologies?
this is a question you can ask the developers. Are they going to use appropriate operating system tools and apis (application programming interfaces)? more information on this can be found within the specific guidelines relating to assistive technologies.

question: does it use custom controls or cursors?
these can cause problems because they may not be visible to assistive technologies. If the designers have chosen not to adopt with the standard platform look & feel and use the standard system tools, they will still be able to make it accessible, but they will need to do more work. Are they aware of what they will need to do?

Step 2. Apply the NDA application software accessibility guidelines

The above questions will cover the major accessibility issues. This may be enough at an early design stage. However, if the design has advanced to a prototype stage, particularly if it has been partially implemented as a working application, you can go further by assessing it against all the NDA accessibility guidelines.

The steps you should go through are described in the assess a current application for accessibility task.

Task: Assess a current offering for accessibility

Step 1. Determine the required level of compliance.

Accessibility requirements for the application may be written into the design specification. This may reference the NDA guidelines, saying something like "the application should meet all priority 1 NDA accessibility guidelines for application software". If so, these will be your target and you can proceed to step 2.

If there is no design specification, or if the specification does not state accessibility requirements in terms of the NDA guidelines, you will have to decide now which of the guidelines the application should meet. The steps you should go through are described in the scope accessibility requirements task.

Step 2. Use the accessibility checklist.

This is the NDA accessibility guidelines, presented in the form of a printable checklist. When you are satisfied that the application meets a specific guideline, you can tick it off the checklist.go to the checklist.

Step 3. Consult the test methods for each guideline

Many of the guidelines include suggested test methods that can be performed by an assessor. Consider carrying out these tests as part of your assessment.

Step 4. Test with real users, where appropriate

the suggested test methods for each guideline may include user testing. For a description of user testing, read about user testing.

Task: Scope accessibility requirements

Step 1. Consult users or user advocates.

The first stage of scoping requirements is to build up a picture of the user population. You should aim to find out who will be using the application and in what situations. This information can best be gathered by talking to users themselves and will prove invaluable when it comes to deciding what accessibility features are needed. Requirements definition is described in more detail in how to create accessible products and services.

Step 2. Decide which of the NDA guidelines the application should meet

When you are sure of the user requirements, refer to the guidelines themselves to determine which should be met. The guidelines are divided into two priority levels.

  • Following priority 1 will ensure that the application can be used by most people with impaired mobility, vision, hearing, cognition and language understanding, using their assistive technologies.
  • Also following priority 2 will make it easier to use for these people and will include more people with cognitive impairments or multiple disabilities.

However, in order to assess whether an application is "accessible", you will have to decide how accessible it should be.

  • Should it meet priority 1 only?
  • Should it meet both priority 1 and priority 2?
  • Should it perhaps meet a selection from priority 1 and 2?

This is a decision for whoever decides the requirements for the application, and the NDA guidelines are not intended to state accessibility requirements for any given application, whether used in the public sector or private sector. Although they may be referenced in a public body's policy, the NDA guidelines are not themselves policy. Although they may be referenced by the requirements specification for an application, they are not a requirements specification for any given application.

So, it is up to the persons who are responsible for setting the requirements for the application to decide what level of accessibility is appropriate. The NDA guidelines are provided to help with that decision by stating what makes an application usable by people with impairments. They are accompanied by detailed descriptions and rationale which explain the motivation and justification for each guideline. They also give guidance on how to achieve compliance.

It is unlikely that a application that does not follow at least all of the priority 1 guidelines could be considered to be accessible in a broad sense. Above this basic level, you will have to decide which priority 2 guidelines the application should also meet. This will depend on its purpose, the kind of content and functionality delivered and the project context. Questions like the following may help determine which guidelines are appropriate.

question: how extensive or complex is the functionality?
applications with more complex functionality are generally more difficult for users to comprehend and navigate. If the application is complex, you may therefore decide to include guideline 2.2 "ensure that the user interface and task flow is similar across different functions".

question: will users have support or help available to them when they use the application, or is it up to them to learn the interface?
if the application is used in an office environment, initial support and help may be available from colleagues or training. In contrast, home users may not have any support available to them, so it will be important to ensure that the application is easy to learn. One of the best practices in making applications easy to learn is to make sure they behave like other applications by adhering to the standard operating system interface guidelines. Then methods learned on other applications can be transferred to the new application. For this reason, you may want to include guideline 2.3 "adhere to the operating system user interface guidelines".

question: might users have to install the application themselves?
if the application is used in an office environment, it will usually be unpacked, installed and configured by a technical support team. But home users may have to do these tasks themselves. If this is likely, you may want to include guideline 2.4 "provide accessible packaging, installation and configuration tools".

question: are many of the users elderly?
elderly people in particular may have multiple impairments. They may have reduced vision, hearing and movement all at the same time. If this is the case, consider including guideline 2.5 "provide for users with multiple impairments".

For more information about the importance of the context of use, see what is accessibility?.