Tying Procurement Language to IT Accessibility Measurement
Chris M. Law, PhD
Procurement language: where are the details?
Somewhere towards the end of the typical contract is the obligatory but brief statement:
“The <software product> will comply with the accessibility requirements of Section 508 of the Rehabilitation Act (36 CFR Part 1194).”
The problem comes with the term “will comply”. What does that mean? How is compliance going to be measured to the satisfaction of both vendor and customer? Who will do this measurement, when and how?
Currently in government purchasing, the most common thing that acquisition personnel rely on is the VPAT
(Voluntary Product Accessibility Template). Vendors use the VPAT to indicate their level of conformance with the Section 508 standards. For agencies that are short on accessibility resources—perhaps they do not have a dedicated IT accessibility team, or dedicated IT accessibility testers—the VPAT may be the only thing that they rely on to determine whether the product is likely to be accessible to the end users. The problem is that there is no requirement for vendors to state how they measured their product to come up with their VPAT claims. So the question remains: What does that mean… compliance was measured by whom, when, and how?
An alternate approach is now possible. This alternate approach can be very effective, even in agencies that have traditionally thought of themselves as ‘under resourced’ / ‘under-funded’ in terms of their accessibility testing. The alternative contract language looks something like this:
“The <software product> will comply with the accessibility requirements of Section 508 of the Rehabilitation Act (36 CFR Part 1194). Prior to acceptance of the final product, the product will fully pass the ‘XYZ Agency accessibility test process’ as tested by certified personnel. The vendor’s personnel will be at liberty to use and follow the test process and use the associated code-inspection testing tools during their development.”
Backing Up the Specific Details
There are three components that enable this level of specificity, and each has some advantages over traditional approaches to IT accessibility testing:
1. Accessibility Code Inspection Tools
Programmers are accustomed to using code inspection tools to diagnose and fix issues with things like syntax, security, and quality (google search ‘code inspection tools’ to see how wide a variety exists). There are now accessibility code inspection tools that can be used by developers. These inspection tools are used to assess whether code will work with assistive technology (AT). Many developers may not have any amount of familiarity with AT. Learning how to use AT for testing purposes can be quite involved. For example, screen reader usage requires familiarization with how it interacts within a webpage that includes a number of keyboard commands. Code inspection tools, on the other hand, are more familiar to developers and can instantly reveal many of the coding errors.
2. Test Processes for using Code Inspection Tools
A code-inspection based test process should present step-by-step instruction on how to use the code inspection tools, and include specific failure conditions that can be referenced and shared between the tester and the developer. For example: “Use ‘Show Data Tables’ tool to reveal header programming… Inspect header row cells for inclusion of HTML codes (SCOPE="col/row"; or ID="x")… Failure condition #10.A.: Not Conformant if an HTML data table's row or column header is not correctly identified programmatically.”
A shared understanding between personnel is essential, and code inspection methods allow this. This is not always possible when using AT testing tools such as screen readers for testing purposes, since the knowledge and skills are highly specialized. Code inspection tools also provide a quantifiable method for sharing and understanding data related to accessibility. Testing with AT solutions such as screen readers is a specialized niche activity, and will likely require additional time for testing.
It is essential that code-inspection methodologies are validated such that passing the test means that the products should work with screen readers and other assistive technologies. Therefore, people who author code-inspection test methods need to be experts in interoperability of code with AT. People who use the tests should not be obligated to learn to use AT. However, if possible, manual testing with AT solutions such as screen readers is encouraged in order to compliment the automated code inspection tools.
3. Certification as a Quality Control Option
As part of your acquisition process, there is a point in time where you want to know whether this product is of sufficient quality for acceptance. It helps to have a certification system to ensure greater reliability on the results of a given tester (i.e., they have passed a certification course). While a certification system provides a mechanism for quality control, not everyone involved in development needs to be certified in order to understand the test process and use the test tools. Most programmers, when given a code-inspection based test method, will be able to follow it easily because it matches the way they use other inspection tools. This has important implications for agencies with ‘few accessibility resources’.
For those who currently rely only on VPATs, sharing a code-inspection test process with developers allows for much more specific measurement. The measurement can be conducted by the vendor’s development teams. For those who have very small IT accessibility testing teams, being able to share a test process with developers means that accessibility is going to be addressed throughout development as standard practice. This has great advantages over the alternative, where developers present their product for AT testing near the end of the development process, when fixes are much harder to implement. This also has the advantage of placing the IT team in more of a quality assurance role. They can be the certified person to do the QA test at the end, knowing that the developers have had to provide completed test process checklists at earlier gates in their systems development life cycle.
Freeing up the IT accessibility test team from the requirement to conduct lengthy AT tests for standards conformance allows those teams to use their AT instead for conducting usability tests. You may want to know whether the product is standards conformant, but also whether the product is actually providing a good user experience for individuals who use screen readers and other AT.
When the development teams have successfully incorporated code-inspection methods, additional quality assurance checks can be implemented via automated testing tools that ‘trawl’ through published sites looking for code problems. These tools are best used as an ongoing QA check—verifying that one-off errors are not being generated by content providers and programmers—rather than as a ‘first line of defense’.
Automated Inspection Tools and Their Limitations
Automated inspection tools clearly have a number of advantages, including their ease of use, the speed at which they can render results, and quantified and easily shareable information. However, it is important to keep in mind that comprehensive accessibility testing cannot be provided with automated testing alone. For example, automated testing tools cannot determine whether alternative text to an image is effectively labeled, or whether a hyperlink is always contextually meaningful. These tools are also unable to test software applications that are not web-based, and/or that reside on a server or hard drive. That being said, automated inspection tools for accessibility are certainly better than no testing at all, and their implementation demonstrate a good faith effort toward achieving greater accessibility. The Worldwide Web Consortium (W3C) provides additional information related to automated and manual testing: Accessibility Testing
The details as a Hook for Organizational Change
Vague procurement language begets vague quality. Implementing a shareable test process with associated test tools—tools that are familiar to programmers and other developers—can be a hook for rethinking the way IT accessibility is tackled in organizations. It can go from a niche activity to a shared activity, with the developers responsible for the level of accessibility of their products. There is no reason why software accessibility cannot be thought of in the same way as software security: it becomes everyone’s responsibility, there are sources of expertise when needed, and there are quality controls in place.
Contributing Author: Dr. Law has extensive experience in the area of IT accessibility. He has worked with Department of Homeland Security in the area of IT accessibility. He is currently the President and Owner of Accessibility Track Consulting, a company that provides consulting services to federal government and industry clients on their organizational responses to accessibility issues.
Archived Webinar: Procurement as a Vehicle for Meeting Organizational IT Accessibility Objectives
View the recent webinar provided by Dr. Chris Law, President and Owner of Accessibility Track Consulting, LLC: Procurement as a Vehicle for Meeting Organizational IT Accessibility Objectives