InterA11y gives LLM agents interactive tools for web accessibility evidence
Submitted October 29, InterA11y combines rules, criterion-specific rubrics and browser interaction tools to evaluate accessibility-error detection across ten commonly violated WCAG criteria, 682 curated conformance cases and 197 expert-labeled cases derived from 54 real webpages.
Why it matters
Generative interfaces change components, states and interaction paths. Static DOM rules cannot observe every outcome, while visual or general-purpose agents can miss violations and raise false alarms. Controlled interaction paired with deterministic checks moves accessibility from a pre-release checklist toward runtime evidence gathering.
A closer look
On the expert-labeled set, the paper reports 79.0% recall versus 44.4% for the strongest LLM checker, with 93.1% precision close to rule checkers at about 95.7%. Adding its tools to general agents raises AgentLab recall from 22.2% to 44.9% and lowers FDR from 22.7% to 11.7%; Browser Use moves from 36.0% to 42.3% recall and 24.4% to 16.6% FDR. The official repository includes datasets, case-level predictions, runtime, adapters and commands that recompute statistics without model calls.
Who this is for
Generative-UI, design-system, web-platform, QA and accessibility engineering teams
Original sources
- InterA11y arXiv paper and official replication repository ↗arxiv.org
- Official replication repository with datasets, saved predictions and runners ↗github.com
Revisions & corrections (1)
— Publish a primary-source accessibility study with a public replication package because it adds interactive evidence gathering, quantified counterevidence, cost limits and a reproducible saved-result path relevant to generated interfaces.