A corpus linguistic study can produce datapoints on empirical questions of ordinary meaning—like the question in the Snell case on whether the installation of an in-ground trampoline falls within the ordinary meaning of "landscaping." But LLM AIs do not and cannot do so, for reasons developed in detail in our draft article.
We highlighted some of the reasons for our conclusions in yesterday's post. But the problem is even bigger than we let on there.
It's not just that AIs are black boxes. Or even that their bottom-line responses obfuscate any empirical detail on how a term like "landscaping" is ordinarily used. We must also come to terms with the fact that AI responses are easily affected by even slight variations in search terminology and vary a lot even when the identical query is presented multiple times.
