Actually, that sounds like one of the better startup ideas I've heard around AI. Automated accessibility compliance (or something close to it) would be very useful and definitely something people would pay money for.
I fear LLMs are only about 80% up to the task, though, which is actually a very unpleasant place to be in that curve; sort of the moral equivalent of the uncanny valley. Whatever comes after LLMs though, I bet they could do it, or get very close.
80% sounds way to optimistic to me. The problem is that screen readers (and other assistive technology) have bugs and different behaviors, and some people use older versions of those tools with even more bugs and quirks. The only way to make sure that a website has a high level of accessibility is to perform manual testing in different environments. I don’t see how AI can solve this problem. And the people who perform the manual testing need to be experts in semantic HTML and ARIA to be able to identify problems and create reports. That means that semantic HTML remains important.
>80% sounds way to optimistic to me. The problem is that screen readers (and other assistive technology) have bugs and different behaviors, and some people use older versions of those tools with even more bugs and quirks. The only way to make sure that a website has a high level of accessibility is to perform manual testing in different environments.
That's if you want actual accessibility support on a wide range of old and new devices.
But the business idea the parent proposes is automated accessibility for compliance, which is the real thing that could be sold, and has a much lower bar.
My estimate of 80% included 6-12 months of serious development first, and a certain amount of budget for manual intervention for the first several dozen jobs. Certainly just flinging HTML at ChatGPT as it stands today would do nothing useful at all. Providing manual testing could easily be done as part of a higher service plan. Not only is there no rule that a startup using AI has to be just in the form of "throw it at the AI and then steadfastly refuse to do anything else", that's probably a pretty good way of filtering out the ones that will make it from the ones that won't.
Do assistive technologies have more "bugs" and "quirks" and "different behaviors" than natural text? I don't really think so. In fact I'd expect they have qualitatively fewer such things.
Semantic HTML would be important in this case... but it would be important as the output, not the input.
This hypothetical startup could also pivot into developing a better screenreader pretty easily once they built this, but there would be a good few years where an AI chewing on the HTML and HTML templates in use by a server would be practical but you can't expect every assistive technology user to be using a 64GB GPU to run the model locally. Certainly that would factor into my pitch deck, though.
I'd give more credence to the "it has to be perfect to be useful at all" argument you're going with here if it weren't that I'm pretty sure every user of such technology is already encountering a whole bunch of suboptimal behavior on almost every site today.
Unless it’s 100% reliable or near 100% reliable, you’d still need manual testing. Right now, automatic accessibility testing can’t even detect most accessibility issues. So we haven’t even reached the stage where all issues are detected by tools, and probably never will. Fixing all issues automatically is significantly harder than detecting them.
>Unless it’s 100% reliable or near 100% reliable, you’d still need manual testing.
Not unless:
(a) it's X% reliable now, and it would be Y% < X% if done via LLMs.
(b) businesses actually care for increased reliability, and not just for passing the accessibility requirements.
Most businesses could not give less f...., and don't do "manual testing" today either. Just add the token required tags. That's true even when they do business with the government (which mandates this even more highly).
LLM-driven accessibility info would be an improvement.
The idea generalizes. Imagine an archiver which applies a transform to a site. Adding semantic markup - or censoring parts that someone finds offensive. If the original author agrees, they might offer an api so the transformation is linked to by the original. Or perhaps the transformer could make an agreement with/fool Google into linking to their version rather than the original. Perhaps because it's "safer".
I fear LLMs are only about 80% up to the task, though, which is actually a very unpleasant place to be in that curve; sort of the moral equivalent of the uncanny valley. Whatever comes after LLMs though, I bet they could do it, or get very close.