There are still many tasks which people can perform easily but computers cannot.

One of these tasks is understanding ordinary or natural language. We still don’t know how to program a computer to understand what we say in ordinary language, just as we don’t know how to program a computer to reliably recognize a face, or a road, a bomb, a pistol, or an ice-cream.

The need is so great that software exists for such tasks, but with very few exceptions it works too poorly to be useful. Nothing approaches human ability.

Theoretical Outline

Linguistics attempts to find a system for language.

In common with broader science this search has moved in the direction of seeking general laws, or rules, broadly speaking grammar.

The conjecture of this website is that such rules cannot be abstracted for natural language.

Looked at more broadly this seems to be part of a general revision of the place of theory in science. Godel's mathematical incompleteness appears to be a version of it. Stephen Wolfram's "computational irreducibility" appears to be a version of it.

The Solution

The solution is to keep finding structure. We can't capture the structure of natural language with rules or generalizations. But this need not be a problem. It is only our expectations which are frustrated. Instead of seeing this constant source of ungeneralizable structure as a problem, we can see it as a resource.

End of Theory?

Wired editor Chris Anderson has written an article "The End of Theory" hints at broader issues such as our ability to abstract "truth" into theory, and the utility of drawing conclusions directly from data:


The End of Theory

As I said in the Funknet thread where this article came up, I don't think this indicates an "End of Theory" so much as "the birth of the
theory that there can be lots more theories buried in a set of data than we've ever imagined we needed to look for before."

Observed limitations to "rules" in physics

R. B. Laughlin* and David Pines†‡§

"For better or worse we are now witnessing a transition from the science of the past, so intimately linked to reductionism, to the study of complex adaptive matter, firmly based in experiment, with its hope for providing a jumping-off point for new discoveries, new concepts, and new wisdom."

Similarity modeling

Similarity modeling - Similarity modeling uses the same parameters as distributional analysis, but it assumes nice clusters are not possible. Instead it makes ad-hoc analogies by averaging sets of properties, as required:

E.g. Dagan, Marcus, Markovitch '95: (p.g. 32, long version) "It has been
traditionally assumed that ... information about words should be
generalized using word classes ... However, it was never clearly shown
that unrestricted language is indeed structured in accordance with
this assumption."

(and previously on p.g. 4) "... our method assumes that

Paul Hopper's Emergent Grammar

Berkeley Linguistics Society, vol. 13 (1987), 139-157

"I am concerned in this paper with the ... the assumption of an
abstract, mentally represented rule system which is somehow implemented when
we speak.

...'Culture is temporal, emergent, and disputed'
(Clifford 1986:19). I believe the same is true of grammar, which like speech
itself must be viewed as a real-time, social phenomenon, and therefore is
temporal; its structure is always deferred, always in a process but never
arriving, and therefore emergent; and since I can only choose a tiny fraction