Most thought is unconscious, and the usual estimate is around 98 percent. But if you believe the work that Stan Dehaene talked about the other night, it is more than 98 percent. Consciousness is the tip of the iceberg of thought. It is there that things are put together in an interesting way and the interesting way is the following, that before consciousness, what happens is that your brain unconsciously changes what you perceive or what you think. This is something remarkable. I think one of the best papers I heard on this was by Shin Shimojo who is a vision scientist at the Caltech. He came to Berkeley a couple of months ago and gave a truly remarkable overview of experiments that showed this, many of them which were his. Let me give you a sense of this. Suppose you know that if there are flashing lights and they are going along and they are going fast enough, they look like a single stream.
No problem. You all know that. But, if the same thing happens with touch, say little machines that will touch you along your arm very closely, very quickly and it will feel like someone’s stroking your arm. Fine, then they set the machine to hit you here and then underneath the first one and then to the right of the first one. What you feel is the second one, not underneath but in the middle. That is you don’t feel what was actually done. You assimilated to a pattern you already know. That means not recognizing what was presented to you on your arm through touch but only representing something and you feel on your arm something that was not touched there. Okay. This is like the McGurk effect which I’m sure you probably all know and if you don’t, you didn’t take a phonology course. Basically, it goes like this. You have a picture, a video of someone saying ba and the same person saying ga. You show those and then you show the picture of them saying ga while he sounds like ba. That’s what people are presented with and what they hear is da with the D which is in the mouth in between. It’s the same thing. You’re creating something that was not either presented to you visually or orally and you’re hearing something that wasn’t there. This is completely normal and it happens in other kinds of cases. There’s a classic case where they present you with a black dot and underneath the black dot is a flash that moves to the right. What people see always is next to the black dot, the flash is to its right then moves to the right. It’s not underneath. If you tell them that, they still see it that way. The same thing happens in another case. There will be a beep and a flash at the same time. There will be two beeps and a flash with each of the beeps, fine. Then there are two beeps with a flash with the first beep and people see two flashes and then you tell them that this is the case and they still see two flashes. This is perfectly normal and it happens in politics which we’ll get to, I hope. But it happens all the time when you’re thinking about things as well as when you’re perceiving things. This is normal. It’s just the kind of thing that Stan Dehaene was talking about when he talked about the difference between what’s unconscious and what’s conscious. We’ll try to explain what’s going on in these cases and there’s a reason for it. The next thing is that all meaningful thought is embodied. Now, there’s a very simple way to think about this. First of all, you have connections to your body and all parts of your brain, but more on the periphery sort of below and above and around within connections going front toward the prefrontal part where there are more generalizations toward the front. That is there’s a result that’s very cool. If you learn two rules, one of which is more general than the other, it’s more to the front for reasons we will talk about in a while. It follows from other principles and that’s why they studied it. They knew it follows from other principles. This was done at Berkeley. Go Berkeley. The idea is this.
Imagine that you had a complete brain map of all of the connections in the brain that were not connected to the body. They’re just connected to each other. Could you have meaningful thought? No. There will be nothing to think about. Think about that. There will be nothing to think about. You can only have meaningful thought through connections to the body. That is absolutely necessary. There’s no chance that it could just be connections among themselves. The next thing to realize of course is that when you see brain maps and they show you all of these networks connected to each other that of course is a computer model because nothing in the brain touches like that. There are synapses and everything is going across the synapse. There are occasional cases of touching on the side. There are a few cases like that but they’re rare. The real cases are you’re going across a synapse. So all the pictures where things are, quote, connected are not connected. They’re working through chemistry and physics. It’s very important to know that for reasons we’ll talk about, because those models of networks actually are interesting for a number of reasons that we’re going to get to. Then there’s another part that we will get to which is that all the circuitry we’re going to be talking about is there in animals. We’re going to point out that the circuitry needed to run thought and language is all there in animals too and it has been what is called exapted or repurposed for thought and language. That’s cool. I like that. I like knowing that this is a case and that this is an important part, an important feature of what our thinking is. Of course, it suggests animals think, which they do. That’s the next part. Then there are a few basics to know about. The first is this. We’re born, as you probably know, with about 100 billion neurons. Each connects to between 1,000 and 10,000 to others so if you multiply that out, it’s about a quadrillion connections. That’s a lot of connections and they’re all over the brain. Now, the other part is that we’re born with all sorts of embodied structures in place. When we’re born, we have topographic maps of the visual field. That’s there. It’s there as you’re developing in the womb. You have topographic maps of the visual field. You have all sorts of other things. For example, a fetus in the womb will move its arms, will move its legs, will put its thumb in its mouth, will turn over. In order to do that, it has to have a neural system that enables him to put his thumb in his mouth, to move his arms, to move his legs. That means there have to be some neural connections that allow that to happen. Every time they move their arms in the same way, those neural connections get strengthened via Hebbian learning. Every time they put their thumb in their mouth, those connections get strengthened by Hebbian learning. Every time they turn over, that is in the womb, the child is developing something rather important. Developing – starting probably in the sixth month or so, if Dehaene’s work is right and other people’s work is right – developing those connections that allow the baby when born to have already all sorts of neural structures in the brain ready to go. That’s quite remarkable. These are embodied structures. They’re there in the body. Now, the way they get there is called neural recruitment. That is if in the womb, you can put your thumb in your mouth and you do it over and over, what happens? The neurons for doing that have to be used over and over and strengthened and then they at birth, you can put your thumb in your mouth. The neural structure has to be there and it is, quote, recruited. That is it’s strengthened because it’s used. This happens whenever you learn anything. What happens is it already has to be there in order to be recruited, if it isn’t already there and firing, nothing can get strengthened. That’s a remarkable fact. Everything you have learned depends upon both the structure that’s there that you’re born with. Things like topographic maps of the visual field and other parts of things that you’re born with. Things like a motor cortex that allows you to move your body, a somatosensory cortex that allows and connects you to everything in your body that feels anything. All of those things are there and they structure what you learn next. You don’t randomly learn anything. There’s a not very interesting paper that was published recently that claim that all categories are all ad hoc categories because everything you do changes. It doesn’t just change randomly. It changes relative to what you already have. That’s a big deal and you need to know that. The name for this is called Neural Darwinism. It says if your connections are in the right place to be able to do it, they will be strengthened and then you’ll learn those. That’s what Neural Darwinism is about. It’s a very, very important principle.
So next, basic, there are certain constraints on the brain’s neural system so here are some. We have the embodiment before you’re born and certain primitives for controlling force and space. We’ll go through a list of those primitives in a while. You have synaptic changes and some of them are Hebbian where you, you know, neurons that fire together, wire together as you know, but some work by STDP. That is spike-time-dependent plasticity. What is that? You got a neuron with an axon, another one with an axon. If they happen to come together like that, the one that regularly fires first to strengthen in its direction, the other is weakened in the opposite direction. That allows you to do things in sequence. You can’t do anything in sequence if it doesn’t have an asymmetry. You can’t move anything if it doesn’t have an asymmetry. Spike-time plasticity is allowing you to do anything at all in sequence and you’ll see in a minute, it also permits metaphor. We’ll get to that in a while. But that is crucial. You need to know how synaptic changes work and it’s not just Hebbian. These are sort of like an extension on Hebbian learning. Then there’s the issue of energy maximization which happens in two ways by either rate coding. Rate coding just means things fire at base rate normally, but then they fire above or below. They get strengthened if they fire above and weakened if they fire below, but that also is time coding because of spike-time-dependent plasticity as well. So time coding also changes these things so you have energy minimization. What happens is the least energy it takes to fire the more likely the firing will take place, and that means the easier it is to recruit. It’s easier to recruit things that can fire more easily. That’s a very important principle for language and thought which we’re going to get to.
Next, there is the neural reward system. Dopamine, norepinephrine, good and bad rewards, which are there and very, very important for the moral system which we’re going to get to for understanding what morality is and we’ll talk about that after a while. That’s a very, very basic set of constraints. What that means is you don’t just acquire any random circuitry. What is permanently acquired is constrained by these mechanisms and these are commonalities that we all share. Now, they allow you to learn a certain set of ideas that appear to be universal, that appear to be there, and then there are others that combine differently in different cultures. You’ve seen lots of examples of cultural difference in this conference and they’re real examples. This is a very important thing to understand. You don’t just learn anything at random. You learn things on the basis partly of what is already there even at birth and partly of what’s used. Also, many of these cases are things that you learn because everybody lives on earth. You live in a gravitational field. The sun rises every day. You’re born to parents, if you’re lucky. You have to eat. You have to excrete. You have to breathe. You have to do all these things and they shape how you think. We’re going to talk about how that works. Now, one of the things that’s very important has to do with generalizations. When you talk about framing, what is framing? A frame is a neural structure that allows you to structure your experience and structure it either in some of the things you’re born with like the notion of motion itself – where there’s a source, there’s a goal, an endpoint to the motion. It is a course of motion. That is done in MT [phonetic] and it’s there and you have parts of the brain that record that. You’re born with that. But there are other ones that are fit together so you have a frame like a commercial event. A commercial event has four elements. It’s a whole event and it has a buyer, a seller of goods, and money. There’s a structure to it, a little scenario. Before the event happens, you have a buyer who wants some goods and has some money, hopefully, or can loan it, and borrow it, or whatever, or convince people they have it. Then they exchange. The seller has the goods, hopefully, and wants the money. They exchange and then the opposite is true. Then the seller has the money and the buyer has the goods. That’s a scenario. We’ll talk about how scenarios develop and how they work, but this is a major part of what a frame is. It has semantic roles – buyer, seller, goods and money – and it has these things. Now, in addition to that, you can put these together. If you’ll go to a restaurant, which you all do, you will see that a restaurant is partly a business, partly a food service. If it’s a fancy enough restaurant, if it’s not McDonald’s or something, you will go and it will be a host-guest relation. You’ll have a binding between the customer, you, either you, you’re the same person, and the guest. What happens is you have put these frames together but in the brain, you can’t mush them together. They don’t move. They’re in different places in the brain. How did they get together? The answer is this has a name. It’s called neural binding. What we have and we’ll talk about it in a few minutes is a theory of how that works. What happens is that you have a neural connection between the two roles. Now, these roles can apply separately. You can have businesses independent of restaurants, and food service at your house independent of businesses and so on, or you can bring them together. In order to bring them together, you have to have connections that go in both directions and give you simultaneous firing. But in order for that to happen, these connections have to be gated. There has to be something that gates it either a group of neurons that say hey, this is a restaurant. What gating can be is you could have something that connects and then the gate inhibits and then it can be disinhibited, or it can be connecting but not strongly enough and it needs more. This is called modulation and gates modulate. That’s how gating works. Gates are all over the place because you need to be bringing together the embodied ideas that you have in order to have any complex idea at all and most of your ideas are very complex. We’re going to talk about how those things get brought together and how this kind of gating works.
Next, embodiment, for the first dozen years of my work as a linguist, I did generative linguistics. I was in MIT and my senior year was the year of Chomsky’s first year of his department. I learned all of that stuff as an undergraduate but I was in mathematics and literature so I went off to graduate school in English literature. Then I wound up in linguistics which was interesting because I got fired from my job as a TA. I was teaching freshmen composition. It is a true story. I was teaching freshmen composition and we were assigned to read essays to the students, to go over certain essays. One of the essays was by George Orwell on the Spanish Civil War. I had a group of students in Indiana who had never heard of the Spanish Civil War. I had to explain what it was. It just so happened at that time, it was the fall of 1962 which was the time of the Cuban Missile Crisis. In the middle of the Cuban Missile Crisis, people were saying what’s the Spanish Civil War? And I was saying, well, here are the bad guys and here are the good guys. Guess who the good guys were? They were the communists. So someone writes, someone says there’s a communist teaching us and I get fired. I go up to the head of the linguistics department. I’ve been talking to him and I’ve gotten to know him pretty well. I explained what happened. He said well, I’m an interim head. I’m here because last year, I got fired from Ohio State for taking part in a civil rights demonstration. Interesting, then the phone rings. He picks it up, talks, puts it down and he says oh, that’s somebody turning down a three-year fellowship to our department. Would you like to go into linguistics? It’s true. The best thing that ever happened to me was getting fired. Now, this is interesting for a number for reasons. Very often the best thing that ever happens to you is that you learn a lot and you fail. What I did for the first dozen years of my work as I found examples with Chomsky’s theory of grammar didn’t work because it was purely formal not based on meaning. It just had symbols which were uninterpreted symbols, independent of meaning and communication. That sounded pretty weird to me so I started finding examples that didn’t work. I said well, how would you make up a theory of meaning – and at that time which was 1962, 1963, around there – meaning meant logic. I was trained as a mathematician. I’ve learned logic so what I’ve done was to try to fit them together with logic, but there isn’t one logic. There are hundreds of logics, maybe thousands of logics. The job of a logician is to make up logic to deal with some phenomenon. That’s what logicians do. They make up logics. If you go and read the Australasian Journal of Logic or The Journal of Symbolic Logic, there are all these people making up logics. They have to make them up and they’re all different. There are a lot of very smart people who make up logics about particular individual issues and they don’t fit together. What I found was that they never fit together very well. After a while, it got very frustrating. I worked on this between about 1963 and 1975. The whole idea of generative grammar fell apart. There were things that you could not do with it even if it was meaningless, you still couldn’t do certain sentences. I’ll be happy to go through with that if anyone really cares but there are examples where there can’t be a deep structure, for example. That was discovered in ’74 and nobody has found a way to do it since then. It fits together very well from a neural perspective but not from a generative perspective. In 1975, I was teaching at Berkeley and I happened to have a group of remarkable colleagues. People like Paul Kay, Eleanor Rosch, Leonard Talmy and Charles Fillmore. I heard from them an amazing set of lectures in 1975, all during a summer session when we had a group of people coming to an underground linguistic institute. You know, this is Berkeley, right? You announce that you got a grant for eight people who are going to come and do something interesting and 188 showed up. And so my research assistant gets a bunch of sorority houses and forms communes, which was great. We had everybody giving talks on two days’ notice and so on. Among those talks were the following. There was a talk by Paul Kay who was studying the terms for colors. It turns out that that was just after Russ De Valois had figured the nature of the physiology of color vision. That depended on – not surprisingly now but then it was news – on wavelengths in the world – which are not colors, which are reflectances out in world – your color cones in your eye, and connections to the neural circuitry in your brain which creates colors. It was discovered shortly after that the color cones are dependent on X chromosomes. There are two versions of these for men and 16 for women. Women have a larger color vocabulary if they’re going to talk to each other because there are different types. But then also since then it’s been discovered that certain women have four color cones. I’m married to one of those women. She’s an artist. She sees colors that I can’t even imagine are there. But this is important. If you’ve ever had an argument with someone in the opposite gender about whether something is blue or green or orange or brown or something, you’re both right because you don’t see the same thing, a very basic thing to know. What that means is there’s no color in the world. Well, at that time, if you’re thinking about logic, logic is defined in terms of truth conditions. If you have something that says the chair is green, it means that that thing you’re referring to as a chair is in the set of green things in the world, but if there is no descriptive green things in the world, this can’t work. Logic can’t work. I almost fell off my chair when I heard this. In the next few weeks, I heard Eleanor Rosch give her first lecture on basic level categories and what she pointed out was if you have category hierarchies like furniture, chair, rocking chair or a vehicle, car, sports car – the one in the middle has special properties. Those properties are a car or a chair is defined by three things: mental imagery, gestalt perception and motor programs. So you’re all sitting in a chair using those motor programs. You can then close your eyes and imagine a chair or imagine a car. You can’t get a mental image of a generalized piece of furniture that’s neutral between chairs, beds, tables and so on. And you can’t get a mental image of something that’s neutral between cars and airplanes and boats and motorcycles and so on. You can’t do it. Try it. The question is why does this happen. At the time we only knew that it happened. We now know why it happens. Part of this comes from the work of Martha Farah who back in 1989 did a remarkable paper where she showed that mental imagery and visual perception use the same structures in the brain, the neural structures. The second part has to do with the work on mirror neurons circuitry. The mirror neurons circuitry, it’s not just the neurons, you have circuits that connect from the premotor cortex to the motor cortex, but they also connect to the parietal lobe behind that which integrates with vision. So that you have connections integrating the motor system, your motor programs, with your gestalt perception, with your mental imagery in a single loop. That is why you have basic level categories. Is that cool? There’s a reason for this and it doesn’t apply up above because you can’t get the same things in those visual systems. That’s important. The next week, we heard a talk from Leonard Talmy. Leonard Talmy was then a graduate student finishing a dissertation on spatial terms in various languages. He was also at the time going blind. He knew he had tunnel vision and he would eventually go blind. He prepared himself for this by becoming the world’s expert on space, and he’s still the world’s expert on space. He does folk dancing, blind. He raises flowers, blind. He spent 20 years running a cognitive science department at SUNY Buffalo blind and so on, an absolutely remarkable human being, a wonderful human being, a sweet, sweet guy. There was something also Ron Langacker at San Diego had pointed out is that if you look at spatial relations terms in different languages like Dutch versus English, they’re all different but they have the same primitives. They can be broken down into the same primitives like motion, a source-path-goal, containment, things like nearness, how close things are, how tightly connected they are, whether they’re going around or not and so on. Various things he called force dynamics. Force dynamics include a long list of forces. I’ll come back to that in a minute but let’s — oh, I want to go one other thing that he pointed out is what is called fictive motion and then we’ll get on to his list. Fictive motion is a sentence like the road runs through the woods. The road is not moving. Now, this is basically a metaphor. It says the way you understand a line or a curve is by motion tracing it. Now, Teenie Matlock took this and, quote, ran with it. What she did was if you take something like the road runs through the woods and the road meanders through the woods, guess which one takes longer. Meander takes longer. Well, she then has done a series of experiments over the years that show that all of the properties of motion come into this metaphor, a very crucial metaphor. You get things like the wall curves or the roof slopes downward and so on. I like Route 1 turns inland after Tomales Bay as you’re going along it. It’s not moving, but, you know, and so on. We understand this all the time. Now, in this, you have a bunch of conceptual primitives. The week after Len talked, we got a talk from Charles Fillmore who observed that grammar is based in simple sentences and in the case of languages like Finnish and Hungarian and so on, on notions like action and events in the world. So you have an agent, you might have somebody who is fighting against an agent. You have a patient who the agent works on. You have an experience or you experience something. Something happens to you. You have observers. You have beneficiaries. You have instruments, recipients, sources and goal. You go from here to there. You have inhibitions. Things can stand in your way and stop you and so a word like despite in a sentence has to do with an inhibition. You did this despite this happening. You have reference where you have pronouns that refer to the same thing. You have accompaniment with words like with. You have resultant expressions and so on. Locations, directions, manner, measures, ranges and so on, all of these are what are called basic semantic roles and actions and events and grammars are structured around them. That’s what Chuck found. It’s very important. This is part of your everyday experience and it’s the experience of the most basic actions and events that structure simple grammars then things get more complicated. What Len pointed out was you have other things. You have deictic centers. Actually, Fillmore has the greatest book and it’s called Deixis. What is deixis? It’s a location. It’s point of view from here. If I say come here, it’s come to here. Go is someplace else. But if you talk about here versus there, that’s deixis and so on, or I can talk about home or your home which is you are now the deictic center. You can shift deictic centers.
What does this have to do with neuroscience? Think about who won the Nobel Prize in neuroscience last year. The people discovered space cells and grid cells which are responsible for deixis. It’s a neural thing. You also have the notion of cardinal directions and you have that in the space cells. So you know that the sun rises in the east and sets in the west and so on, if you live in certain places like Australia and you can see them. You can actually also see them from Berkeley but not from any other places. But there are some places in the world where you can actually see the sun rising in the east and setting in the west. In many of those places, there are languages where that’s the only thing you can say. Where you say something like he is sitting to the west of her – and that’s all you can do. You can’t do it but you think about it and you know all the time what is east and what is west. There are languages where that works that way. But also as Len pointed out, you have all these notions like part-whole, center-periphery, an axis. So if you’re going across the stream, you’re going across an axis, along the axis. You have a front, a back. Now, in physics, there’s no notion of front and back. You have to have a body to have a front and a back. This is a little bit tricky for crabs because you can see why. They move and see in different directions. But we move and see in the same directions and that’s fronts and backs, and then we project them on to other things. You can put something in front of the tree where the tree is understood as facing you. But there are some languages where the tree is focused in the other direction. It depends on which language you’re in. You have notions like side, and left and right, and top and bottom, adjacent regions, figure and ground, verticality, horizontality, regions, links between things. What Len pointed out is that these ideas which are called basic primitive image schemas structure thought in every language in the world. This is how you think in these terms and they’re there and they’re there for everybody. The question is exactly how they’re embodied but they have to be embodied. There’s more of course. Near, far, facing opposite, intermediate, attached, separated and so on. There are a lot more even there, these additional ones. Now, in addition, Len pointed that there is what is called force dynamics, and so