Accessibility evaluation methods and inclusive recruiting

Why it matters

Accessibility cannot be achieved without evaluation, and evaluation is most valuable when it is integrated throughout the development process — not bolted on at the end. This lesson covers when and how to evaluate, the critical limits of automated tools, and how organizations can build accessibility expertise by hiring people with disabilities. (BoK p. 68)

Evaluation timing: formative, summative, and continuous

The BoK distinguishes three evaluation approaches based on when they occur: (BoK p. 68)

Formative evaluation occurs during development — while a product is still being built and decisions can be changed cheaply. It might involve reviewing wireframes, testing a prototype, or checking a component in isolation. Findings from formative evaluation prevent barriers rather than fixing them after the fact.

Summative evaluation occurs at or after release — testing the completed product against a standard (such as WCAG 2.1 or EN 301 549) to measure conformance. It is typically more formal and produces an audit report with pass/fail findings.

Continuous evaluation involves ongoing monitoring after release — automated scanning, user feedback channels, periodic re-audits. It ensures that updates, content changes, and third-party integrations do not introduce new barriers over time.

Human evaluation and testers with disabilities

Automated accessibility scanning tools can check structural properties — the presence of alt attributes, color contrast ratios, form label associations — and can do so at scale and speed. But they have significant limitations:

• They cannot assess whether alt text accurately describes the purpose of an image.
• They cannot tell if a form’s instructions are understandable.
• They cannot test the interaction experience of a screen reader user navigating a complex data table.
• They cannot detect cognitive barriers or cultural assumptions.

Human evaluation by trained accessibility reviewers fills these gaps. Trained reviewers using assistive technology, following WCAG 2.1 and document standards like PDF/UA, can identify issues that no automated tool finds. (BoK p. 69)

Testers with disabilities add another dimension: real users who depend on assistive technology daily bring lived experience that trained reviewers without disabilities may not replicate. They identify friction points that affect actual usability, not just technical conformance. The BoK recommends including people with disabilities in evaluation programs. (BoK p. 69)

Inclusive recruiting

Organizations that want to improve their evaluation programs and build accessibility expertise should actively recruit people with disabilities. The BoK notes several strategies for inclusive recruiting: (BoK p. 69)

• Disability job boards: Posting vacancies on specialist job sites that target candidates with disabilities increases the visibility of roles to this talent pool.
• Accessible application processes: Job applications, interview processes, and onboarding must themselves be accessible — otherwise candidates with disabilities cannot participate.
• Reasonable accommodations in recruiting: Offering interview adjustments (extended time, sign language interpreters, accessible formats) signals genuine commitment.
• Partnerships with disability organizations: Working with disability employment organizations to source candidates and receive guidance on inclusive practices.

Inclusive recruiting serves both an accessibility and an organizational capability goal: people with disabilities who use assistive technology are some of the most expert testers and evaluators an organization can have.

Real-world example

A government agency contracts an annual WCAG 2.1 audit. The audit report identifies 47 issues — all of them flagged by the automated scanner. The auditor also spends three hours testing with a screen reader user who has low vision. That session reveals 18 additional issues: meaningful images with generic alt text, a modal dialog that traps keyboard focus, and a date picker that is technically labeled but confusing to navigate. The agency updates its evaluation protocol to include quarterly manual testing sessions with users who have disabilities, in addition to the automated monthly scans.