The Testing Gap Isn't the Problem. Organizational Capacity Is.

In their recent analysis, David makes a structurally sound argument: the accessibility industry has built its evaluation infrastructure around measurable proxies rather than actual user experience. The split button case is a clean illustration of that gap. Automated tools miss state management failures. Manual testing catches them. Therefore, we need more manual testing.

The logic is correct. The prescription is incomplete.

After fifteen years covering accessibility compliance, I've watched organizations absorb this exact argument — automated testing has limits, manual testing is essential, user testing is the gold standard — and then do almost nothing differently. Not because they disagree. Because they lack the operational capacity to act on it.

That's the conversation the field keeps avoiding.

What Operational Capacity Actually Means

When I use the term operational capacity, I mean something specific: the combination of budget, staffing, institutional knowledge, procurement leverage, and executive mandate that determines whether an organization can translate accessibility findings into remediated products. You can read about our analytical framework at length, but the short version is that compliance outcomes depend less on audit methodology than on what happens after the audit.

Most organizations that commission accessibility audits — whether automated, manual, or user-tested — share a common structural problem: findings accumulate faster than remediation capacity. A single manual audit of a complex enterprise application can generate 200 to 400 discrete issues. According to research published by the Nielsen Norman Group (opens in new window), the average time to remediate a known accessibility defect in enterprise software exceeds 90 days. That's not a testing methodology problem. That's a capacity and prioritization problem.

The split button failures Patricia documents — and that David analyzes so precisely — are genuinely consequential for the people who can't use those components. But they're also, in the taxonomy of organizational remediation queues, medium-complexity component-level fixes. They require developer time, QA cycles, and cross-browser screen reader validation. In organizations where accessibility competes with feature development for engineering resources, medium-complexity fixes often sit in backlogs for quarters.

More testing doesn't shorten that queue. It lengthens it.

The Audit-Remediation Disconnect

The DOJ's web accessibility guidance under Title II of the ADA (opens in new window) establishes a legal obligation for effective communication — not a testing methodology standard. That distinction matters more than it might appear. Organizations can be fully compliant with every recommended testing protocol — automated scans, manual keyboard audits, screen reader testing — and still maintain websites that are functionally inaccessible to disabled users because remediation never closes the gap that auditing keeps reopening.

This is the structural reality that David's piece, for all its analytical precision, touches lightly. The observation that the 37% automated detection ceiling has become a floor we've learned to live with is accurate. But the reason organizations have learned to live with it isn't primarily methodological complacency. It's that expanding to manual testing and user testing multiplies findings without multiplying the remediation capacity to address them.

Section 508 guidance from GSA (opens in new window) makes this implicit in how it structures testing requirements: automated testing, manual testing, and user testing are presented as a hierarchy of rigor. What the guidance doesn't address is the organizational infrastructure required to operationalize that hierarchy at scale. Federal agencies with dedicated Section 508 program offices struggle with this. Private sector organizations without that infrastructure face it acutely.

Where the Capacity Problem Concentrates

The organizations most likely to have split button failures — and least likely to catch them through manual testing — are mid-market companies and public sector entities operating without dedicated accessibility engineering staff. Large technology companies have accessibility teams. They run internal screen reader testing. They catch the component-level failures Patricia describes before products ship, or at least before they persist for years.

The Great Lakes ADA Center's technical assistance resources (opens in new window) consistently show that the organizations seeking compliance guidance are not Fortune 500 companies with mature accessibility programs. They're school districts, municipal governments, regional healthcare providers, and mid-size retailers — organizations where accessibility responsibility is often distributed across a web team of two or three people who also manage content, analytics, and platform maintenance.

For those organizations, the argument that they should be conducting manual keyboard audits and screen reader testing across their entire digital footprint is correct in principle and operationally disconnected from their actual capacity. Telling a three-person web team at a county health department that automated tools only catch 37% of barriers is accurate. It is also, without accompanying structural support, not particularly actionable.

Capacity Assessment Before Methodology Selection

The field's dominant conversation about testing methodology — automated versus manual versus user testing — assumes that organizations have or can acquire the capacity to implement whatever approach the evidence supports. That assumption is worth examining directly.

A more productive frame starts with organizational capacity assessment before audit methodology selection. What is the organization's realistic remediation throughput? How many engineering hours per quarter are available for accessibility work? What is the decision-making authority of whoever owns accessibility findings? Those questions determine what testing methodology is actually useful — not the other way around.

WebAIM's survey of web accessibility practitioners (opens in new window) has consistently documented that the primary barriers to accessibility progress are organizational — lack of knowledge, lack of resources, and lack of management prioritization — not technical. The testing gap is real. But it sits downstream of a capacity gap that the field addresses less directly.

You can read more about how our editorial team approaches the intersection of compliance frameworks and organizational reality at our methodology page. The short version: legal standards establish what's required; operational capacity determines what's achievable; the gap between them is where most organizations actually live.

What This Means for Split Buttons Specifically

Building on the framework David establishes, the split button case is useful precisely because it's concrete. A split button with an unlabeled arrow and unannounced state changes is a discrete, fixable problem. It doesn't require organizational transformation to remediate. It requires a developer with clear requirements, time to implement them, and a QA process that includes screen reader validation.

The question worth asking is: what organizational conditions make that straightforward remediation likely? Not what testing methodology would catch the failure — manual testing would, as Patricia demonstrates — but what capacity structures make acting on that finding probable.

Accessibility journalism, including this publication's work, tends to focus on the detection side of that equation. The remediation side — procurement requirements, accessibility acceptance criteria in development workflows, executive accountability structures, budget allocation — is where the field's analytical attention is less concentrated. That's the gap worth closing.

For practitioners advising organizations right now, the most actionable step isn't expanding the testing stack. It's auditing the remediation pipeline first: how many open findings exist, how long they've been open, and who has authority to prioritize closing them. That assessment tells you more about what testing methodology will actually produce results than any methodology comparison does.

The testing methodology conversation is necessary. It's not sufficient.