Choosing among web accessibility testing tools is easier when each tool has a defined job. This guide compares axe, WAVE, Pa11y, and manual audit methods, then turns them into a repeatable workflow for local development, authenticated pages, CI checks, and release review.
Overview
Accessibility testing works best as a layered process rather than a search for one perfect scanner. Automated tools can identify many common implementation problems quickly, but they cannot determine whether a workflow makes sense with a keyboard, whether instructions are understandable, or whether a screen reader user can complete a task efficiently. Manual review supplies that context.
The main tools in this comparison serve different purposes:
- axe: useful for browser-based inspection, developer workflows, component testing, and automated checks that can be integrated into test suites.
- WAVE: useful for visual, page-level inspection in a browser. Its annotations help developers understand where detected issues, alerts, and structural information appear in the interface.
- Pa11y: useful for command-line and scripted testing, especially when a team wants accessibility checks to run against selected URLs or as part of a build process.
- Manual auditing: essential for keyboard access, focus behavior, meaningful interaction sequences, screen-reader review, content clarity, and issues that automated rules cannot reliably judge.
These tools generally evaluate implemented pages against selected automated rules. They do not certify that an application is accessible, and a clean scan should be treated as evidence that certain checks passed—not as a complete accessibility review. Rule sets, browser behavior, application state, and test configuration all affect the result.
| Need | Best starting point | Important limitation |
|---|---|---|
| Inspect a page while developing | axe or WAVE browser workflow | Results apply to the rendered state you inspected |
| Run checks from a terminal | Pa11y or an axe-based test setup | Requires deliberate URL, state, and configuration management |
| Check reusable UI components | axe integrated with component or browser tests | Component tests may miss page-level and journey issues |
| Gate a release candidate | Automated CI checks plus manual review | Automation alone cannot cover all user experiences |
| Review complex interaction flows | Manual keyboard and assistive-technology testing | Needs a documented scenario and trained reviewer |
Checklist by scenario
For local page development
- Open the page in its expected responsive states and run an automated scan.
- Use the browser inspector or WAVE-style visual annotations to locate each finding in context.
- Confirm whether the finding is a real defect, a limitation of the selected rule, or a result of incomplete page state.
- Fix the underlying markup or interaction pattern rather than hiding the report entry.
- Retest after the change and check nearby states, such as an expanded menu, validation error, dialog, or loading result.
Use this stage to establish accessible patterns in shared components. A button, form field, dialog, table, or navigation component should be tested in its meaningful states—not only in its initial appearance.
For a command-line or CI check
- Choose a stable set of routes that represents important templates and user journeys.
- Make the application available in a predictable test environment before the accessibility command runs.
- Define how authentication, cookies, seeded data, feature flags, and client-side navigation will be handled.
- Store reports as build artifacts or test output that developers can inspect.
- Decide which findings fail the build and which are recorded for follow-up. Apply the same policy consistently.
- Keep the tool configuration in version control so changes to included rules or exclusions are reviewable.
Pa11y can be a practical choice when a team wants a straightforward scripted URL check. An axe-based setup can fit naturally where accessibility assertions already run beside component or end-to-end tests. The right choice depends less on brand preference than on whether the tool can reach the application state that matters.
For authenticated or stateful applications
- List the routes that anonymous scans cannot reach, including account, checkout, administration, and error states.
- Use a controlled test account with representative permissions and data.
- Test both valid and invalid states: empty forms, rejected submissions, expired sessions, and permission boundaries.
- Verify that automated checks run after the page has finished rendering the relevant content.
- Protect credentials and personal data in logs, screenshots, and CI artifacts.
For a release or manual WCAG review
- Run automated scans against representative pages and record the tool, configuration, date, and tested state.
- Navigate the complete task with only a keyboard. Check that focus is visible, logical, and not trapped unexpectedly.
- Inspect headings, landmarks, labels, link text, instructions, error messages, and status updates.
- Test dialogs, menus, tabs, comboboxes, drag alternatives, and other custom controls with keyboard interaction.
- Review text and interface contrast using an appropriate contrast-checking method, while also checking non-color cues.
- Use a screen reader or other assistive technology for high-value journeys, especially where content updates dynamically.
- Retest fixes at the same viewport, route, account state, and interaction sequence that produced the original issue.
For broader browser automation coverage, an end-to-end framework can provide the navigation and state setup around accessibility assertions. The comparison in Playwright vs Cypress vs WebdriverIO can help with that surrounding test strategy.
What to double-check
Rule coverage and standards mapping
Check which rules are enabled, what standard or success criterion a report references, and whether the mapping is appropriate for your project. Tool labels are useful for triage, but they do not replace reading the requirement in context. Keep a record when a team intentionally excludes a rule, including the reason and a review date.
False positives and incomplete context
A finding can be technically accurate but still require human interpretation. For example, an automated result may identify a missing or questionable label without understanding the purpose of the control. Conversely, a page can pass an automated check while presenting an unusable focus sequence or confusing instructions. Every reported item should be confirmed in the rendered interface.
Dynamic content and timing
Single-page applications can change after the initial load. Test after menus open, results arrive, validation runs, or a dialog appears. If a tool scans too early, it may miss content; if it scans an artificial state, it may report problems users never encounter. Make the wait condition part of the test design.
Component versus page coverage
Component testing is valuable for catching regressions in shared controls, but it does not prove that the assembled page has correct headings, landmarks, reading order, or task flow. Pair component checks with representative templates and end-to-end journeys.
Common mistakes
- Treating a zero-issue scan as approval: automated coverage is only one layer of evidence.
- Scanning only the home page: prioritize forms, authenticated areas, transactional flows, errors, and content-heavy templates.
- Ignoring keyboard behavior: test tab order, focus visibility, skip mechanisms, escape behavior, and focus restoration.
- Testing only the default state: include expanded, disabled, invalid, empty, loading, and completed states.
- Suppressing findings without ownership: document exceptions and assign a follow-up decision.
- Changing tools instead of fixing patterns: switching scanners can change the report but not the user experience.
- Running CI checks without actionable output: a failed build should identify the route, element, rule, and available evidence.
Keep accessibility findings alongside normal engineering work, with enough detail to reproduce the issue. A useful ticket states the page or component, user impact, steps to reproduce, expected behavior, observed behavior, and retest conditions.
When to revisit
Revisit this workflow before each release candidate and after changes to navigation, forms, design-system components, authentication, routing, or client-side rendering. Repeat the review when a new browser, framework, testing library, or automated rule set enters the toolchain. A quarterly check of configurations, excluded rules, route coverage, and manual scenarios helps prevent the process from quietly becoming stale.
When updating the article or an internal checklist, verify the current WCAG reference used by the team, tool documentation, browser-extension behavior, CI examples, and any project-specific conformance guidance. Do not copy an old configuration without checking its assumptions.
For the next audit, use this short action list:
- Select representative public, authenticated, error, and dynamic states.
- Run an automated scan with the project’s recorded configuration.
- Confirm findings manually and remove duplicates.
- Perform keyboard, focus, screen-reader, and contrast checks on critical journeys.
- Fix issues, rerun automation, and repeat the manual step that exposed each defect.
- Archive the reports and note the tools, versions, routes, accounts, and date.
That combination gives axe, WAVE, Pa11y, and manual review clear roles: fast detection, visual investigation, repeatable scripting, and human judgment. The result is a maintainable accessibility testing workflow rather than a one-time scan.