Accessibility Audits: What We Actually Check Beyond Automated Tools
Automated accessibility scanners are a genuinely useful first pass, and we run them on every project, but on a government services portal we redesigned, the automated report came back nearly clean while a manual audit found dozens of real problems the scanner had no way to catch: a modal that trapped keyboard focus incorrectly so users couldn't tab back out of it, a multi-step form where the error summary at the top didn't actually match the fields it claimed were invalid, and a set of icon-only buttons that had technically valid alt text but text so generic, 'icon,' that a screen reader user got no useful information from it.
So every audit we run now includes a full keyboard-only pass, unplugging the mouse entirely and navigating the whole flow with tab, shift-tab, enter, and arrow keys, on every interactive screen, not a sample. This alone catches a large share of what automated tools miss, because keyboard traps and illogical tab order are structural problems that a static scan of the HTML often can't detect without actually simulating the interaction sequence.
We also do a real screen reader pass, using VoiceOver and a Windows screen reader like NVDA, on the actual product, not a transcript of the DOM. This is where we catch things like decorative images that get read aloud unnecessarily, cluttering the experience, or form fields where the visible label and the accessible name have drifted apart because a developer changed the visible text without updating the aria-label to match.
Where possible, we bring in users who rely on assistive technology daily rather than relying solely on our own simulated testing, because there's a real gap between a sighted designer using a screen reader carefully and someone who uses one as their primary mode of navigating every day and has developed much faster, more specific expectations about how things should behave. Feedback from that group on the government portal project surfaced a navigation pattern that technically passed every automated and internal manual check but was still meaningfully slower to use than it needed to be.
Automated tools remain valuable for catching baseline issues at scale and preventing regressions in CI, and we keep them running continuously. But we've stopped treating a clean automated report as evidence of an accessible product. It's evidence of an absence of a specific, narrow category of markup errors, which is a much smaller claim than 'this works for people using assistive technology,' and conflating the two is how real barriers slip through.
