The solution is to keep finding structure. We can't capture the structure of natural language with rules or generalizations. But this need not be a problem. It is only our expectations which are frustrated. Instead of seeing this constant source of ungeneralizable structure as a problem, we can see it as a resource.
Generalizations over language just keep on creating new structure. If we are willing to forget the expectation this should not happen, we can use the fact it does happen, to constantly generate useful structure and solve problems. Problems like distinguishing an acceptable utterance from an unacceptable one, or matching language with "meaning".