As a norm for attributing content to mental states - W&F 124:
Understanding is not a fixed relationship between a representation and the things represented, but is a commitment to carry out a dialog within the full horizons of both speaker and hearer in a way that permits new distinctions to emerge.
Heidegger's description of the phenomenology of thought and action - W&F 97:
Heidegger argues that the basis for an understanding of cognition is being-in-the-world. Our ability to treat our experience as involving present-at-hand objects and properties is derived from a pre-conscious experience of them as ready-to-hand. The essence of our Being is the pre-reflective experience of being thrown in a situation of acting, without the opportunity or need to disengage and function as detached observers. Reflection and abstraction are important phenomena, but are not the basis for our everyday action.
Whenever we treat a situation as present-at-hand, analyzing it in terms of objects and their properties, we thereby create a blindness. Our view is limited to what can be expressed in the terms we have adopted. This is not a flaw to be avoided in thinking - on the contrary, it is necessary and inescapable. Reflective thought is impossible without the kind of abstraction that produces blindness. Nevertheless we must be aware of the limitations that are imposed.
W&F 97:
The program is forever limited to working within the world determined by the programmer's explicit articulation of possible objects, properties, and relations among them. It therefore embodies the blindness that goes with this articulation.
W&F 121:
[In a wide range of NL systems], the program responds on the basis of a fixed set of patterns provided by a programmer who anticipated certain inputs. This anticipation may be clever (as in the DOCTOR's response to sentences mentioning "everybody"), but it still represents a permanent structure of blindness. This limitation is not one of insufficient deductive power. It applies equally to programs like SHRDLU that include routines for reasoning with representations, and holds as well for systems with 'frame-like' reasoning. It lies in the nature of the process by which representations are fixed in a computer program.
It is important to recognize that this limitation is not dependent on the apparent breadth of subject. SHRDLU operates in a microworld in which the set of objects, properties, and relations are fixed and limited in an obvious way. The DOCTOR apparently deals with all aspects of human life, but it is really working with an even more limited set of objects and properties, as specified in its patterns. Given the sentence, "I am swallowing poison," it will respond "How long have you been swallowing poison?" rather than responding as a person would to implications that were not anticipated in creating the pattern.
Remember W&F's key desiderata: agent makes commitments and allows new distinctions to emerge.
You don't get this from parameter adjustment, because the underlying representation is basically preprogrammed:
a fixed structure is in place, and the learning consists of adjusting some kind of weights to achieve a higher measure of performance.
You don't get this from combinatorial concept formation, because the programmer creates the basic representation and the kinds of rules that the program learns to classify objects just work with the ingredients of those basic representations.
You don't get it from evolution - because the search space is so intractably large that in practice no meaningful learning can occur.
Question: what about more recent learning approaches?
W&F 106:
We treat other people not as merely 'rational beings' but as 'responsible beings'. An essential part of being human is the ability to enter into commitments and to be responsible for the courses of action that they anticipate. A computer can never enter into a commitment (although it can be a medium in which the commitments of its designers are conveyed), and can never enter as a participant into the domain of human discourse. Our earlier chapters point out the centrality of commitment for those aspects of intelligent behavior that initially seem to be based on more objective ideals of rationality. Even the ability to utter a 'true statement' emerges from the pontential for commitment, and the absence of this potential gives computers a wholly different kind of being.