Professor Robert Gallager is arguably one of the best-known figures in information theory. He first came to MIT as a graduate student in the fifties, and has been a Professor Emeritus since 2001. Gallager has garnered considerable recognition throughout his career, including a recent International Marconi Fellowship, and is the author of Information Theory and Reliable Communication (John Wiley & Sons, 1968), considered one of the foremost texts on the topic. Notably, Gallager also collaborated with the late Claude Shannon, an MIT professor known as the father of modern digital communication.
I had the opportunity to sit down with Bob Gallager and ask him about his work, his experience at MIT, and his thoughts on the nature of research and his field. His humor, insight, and extensive knowledge filled a lively hour. Read on to find out what it was like to work with Claude Shannon, why people once told Gallager to get out of information theory all together, and how he came to MIT in the first place. And that's just the beginning. Equal parts eloquent and irreverent, honest and incisive, the excerpts that follow are sure to stay with you.
What brought you to MIT originally?
I went to the University of Pennsylvania as an undergraduate and then worked at Bell Labs, which was the research and development arm of AT&T. In that era, AT&T had a monopoly on telephone service in the U.S., so they could afford a fine national research lab. In many ways, the research there was better than that in any of the universities then.
Then in 1954 I got drafted into the army, which I didn’t mind initially. It just turned out to be such an awful waste of time. I was in a scientific and professional unit, which was supposed to look at communication needs of the army. This was directed by a few non-army people who were engineers, but not very well educated engineers, by junior officers from southern military schools—nothing against southern military schools, but military schools are military schools—and by non-commissioned officers back from the Korean War, some of whom were trying to turn themselves into alcoholics before they got discharged. So this was a totally inverted society. There was a program in the army then where you could get out three months early if you went to graduate school. So I applied to electrical engineering at MIT and to mathematics at Princeton. I got into both of them. MIT started a week before Princeton, and that was really the only reason I went to MIT. I wanted to get out of the army in the worst way.
I wanted to go back to Bell Labs at that point; I didn’t see the need for a PhD. I wasn’t very turned on by the academic life. I viewed myself as a more practical engineer. But then I got to MIT, and it was just a wonderful atmosphere.
What was it like?
Mostly young faculty members. The best word to use is intellectual, but intellectual doesn’t carry the right connotation today, because people think of intellectuals as being somewhere off in the wild blue yonder. These were people who really loved to play mathematical games, but they also loved to understand why things behaved the way they did. They would look at a communication system, and what turned them on were not all the details, but the general architecture of it. They weren’t really interested in how you can make it a little bit cheaper. They were interested in major ways of making systems better.
And is that what you came in doing? Communication research?
That’s where I quickly wound up. I didn’t know what I wanted to do, really. My background was in communication, but I really didn’t know who I wanted to work with. Most graduate students now come with a research assistantship with some faculty member, so they’re sort of locked in. But back then it was much more open. For the first year I was here I wasn’t really working with anyone. I was looking for interesting projects for a Master’s thesis. The courses I was taking that were most interesting were the ones in information theory and various related subjects. So I wound up doing a Master’s thesis for Bob Fano, who is still on the faculty here. He was an early information theorist, then head of Project MAC (which is now the Computer Science and Artificial Intelligence Laboratory - CSAIL). And by that time, I was thinking that this was really fun!
What was your Master’s thesis about?
I can’t even remember anymore. It was really kind of silly when I look back on it. It was trying to analyze something mathematically which shouldn’t have ever been analyzed. But you have to do these things, and you have to understand them before you realize that they shouldn’t be looked at. So, it was in a sense a failure.
At that point I was really into the research going on at the lab. The communication part of MIT was growing very rapidly. Claude Shannon had just come here, in 1957, as a faculty member. It seemed that all the best graduate students were drawn to this one area. Look around at the best-known people in the field today, andmany of them were here back in the late fifties, early sixties.
What were the problems in information theory then?
There was one major problem, namely, how do you use it? There was a theory that said, on these particular communication media—for example, a wire going from one place to another—how much data can you transmit? How much information? Back in those days that was a confusing question. Today it’s not so confusing, because we all think of having a 64K modem, or even a two gigabyte modem. That makes sense to us because we think of information as being translated into bits.
This was really Claude Shannon’s contribution. He said that everything can be translated into bits, and that the real problem with communication is, how do you get bits from one place to another? He had this theorem that said if you could describe the channel, you could find out how many bits per second you could send on it. You could send that many bits reliably, with no errors, and you couldn’t send anymore. But nobody knew how to do it.
His proof of this was not to find a specific way of doing it, but to say, let’s choose all possible ways of sending information. We’ll choose them randomly. We’ll analyze over this average, and we’ll show that the average is very good. For people like me that was just fascinating. Here was something where you couldn’t think of any one way of doing it, but you knew if you just chose something at random it would probably be good.
People used to say that all ways of communicating are good, except the ones you can think of! So we were busy trying to think of specific ways that would work.
Was the laboratory called LIDS then?
This work was not being done in LIDS. It was being done in RLE (Research Laboratory for Electronics) which existed then also. LIDS at that point was the control systems lab. It did a lot of project work, with machines and things like this. The kind of engineering that I never wanted to be involved with: building big things.
What happened? How did that change?
Gradually, I guess in the late sixties, they started to hire some young engineers who were more interested in theory. There was a lot of interesting theory going on concerning the control of large systems. A group of people who became the best known control theorists of the day were here, along with all these information theorists and communication theorists.
Then, by the early seventies, people were starting to think that information theory had worked itself through all its interesting problems.
Well, by 1973 the field was twenty-five years old, and people thought that if nobody had invented something by then, they weren’t going to. Also, people who’d been working on these problems for twenty years were tired of working on them. If you’re tired of working on a problem, what do you do? Do you say, ‘well, the problem is too hard for me; let somebody else work on it?’ Or do you say, ‘this field really isn’t very interesting, and all of us should start doing something else?’
A few people at that time were really starting to find out how to use these new ideas. Irwin Jacobs, who is now the CEO of Qualcom, was trying to take all of these ideas in information theory and control theory and put them together to make wireless communication systems and tracking control systems. The revolution in solid state was far enough along by then that people could build complicated things cheaply. So while the academics were saying communication theory is dead, the people who had moved from theory into practice were saying, gee! This is all ready to be applied.
Where were you in the midst of that?
I was sort of in the middle of it. Even back in 1960 when I got my PhD, a lot of people were telling me that most of the interesting problems had been solved in communication theory, which was strange because it was just the beginning of the golden age of that field. It’s surprising; you talk to technical people about their field, and they just have no idea! I think it’s always been this way. If you’ve been deeply involved in it your
self, you see where you’re going, but you don’t have very good perspective. Somebody who was very well known in the field at that time told me that I ought to go into vacuums tubes, which were the things they used to make radios out of. They looked like light bulbs, and you used them for amplifying and all the things you use solid state circuitry for now. That was a field that really was dying, but people said that’s a field I ought to get into. I said, well no, I think I’ll stay in information theory for a few years.
By 1970 or so I was starting to work in computer science for the most part, so I would go back and forth between communication and computers, partly because I was listening to other people too much. Everyone seemed to feel that computer science was the wave of the future at that time. The computer field and the communication field were roughly the same size, but MIT was putting all of its resources into computer science. Suddenly by about 1980 there was almost nobody left in the communication field, which is when I went back into it.
What drew you back to it?
Well, the fact that it wasn’t so crowded at that point, and the fact that lots of interesting things were going on. It just seemed that the problems were more interesting there. The communication people still in the field and the control people got together and changed this lab. It changed its name to the Electronic Systems Lab, working more on what people call systems. We were applying both communication ideas and control ideas to big systems such as military command and control.
Most of the people doing that were never really interested in militaristic things. I was particularly uninterested in militaristic things after being in the army and seeing what a disaster area it was at that time. But the U.S. government was putting a lot of money into research for military projects, and in most of these projects, the real problems were important to both non-military and military areas.
What about the research going on now? What kind of impact do you think it has, or could have, on the world?
I don’t know. I’ve always tried to answer that question. I’ve always felt guilty because I can’t answer it. When I look back five years, and I see what I was thinking then, and I see what other people were saying then, I realize that nobody understands how those things are going to evolve. Maybe three years in the future you can see what’s happening. Five years in the future, probably not. Ten years in the future you don’t have a clue. At least I don’t have a clue. Maybe some people do, but I don’t.
Really, all you can say is that research is interesting because it gives you a better basic understanding of these underlying problems. To me that’s what research should be doing.
Have the students that are drawn to this field changed over the years?
I think students are much more interested in writing papers, in getting recognition, going to conferences. There seems to be more of a need to be successful in lots of different things. The MIT ‘super success’ student is often involved in a lot of activities. In my day, if you could write one good paper while you were a graduate student, people would be very happy. But it had to be a very good paper, because you spent so much more time on it.
I don’t think that’s a change in the student, I think that’s a change in the way people perceive the world. Students who want to become faculty members perceive the process for getting tenure as being a competition between a huge number of people. And it probably is, because back when I was in the information theory field, the field was very small. You had a conference, and there were maybe thirty or forty people there. Now you have a conference and there are three hundred to one thousand people.
Do you think more students now are going into industry, rather than becoming faculty?
My sense is that it’s stayed more or less constant. Back during the bubble, a lot of people were attracted into industry. I don’t see people having that sense of very easy money anymore, and I don’t think they have the sense of enormous fun anymore. When you have a company with hundreds of people, either a very small number of them are the ones who decide what to do and everybody else has to follow orders, or else they’re all squabbling all the time. It clearly isn’t as much fun. A place like Bell Labs, where I worked before MIT, was different.
If you’re doing engineering, and you’re good at research, even if your ideas aren’t useful, you can still think they’re very interesting. And in fact that’s all you can ask for, because most of the good research ideas aren’t really useful very quickly. My PhD thesis was not useful for about forty years.
What was it about?
Something called low density parity check codes. People recognized it as an interesting way of doing coding, which is what you had to do to send those high data rates over communication channels. And this was a neat way of doing it, except it cost too much. Finally, it got so cheap to build things that it got to be attractive at a certain point.
How would you describe information theory today to someone who doesn’t have a technical background in it?
It’s the theoretical side of communication technology, and communication technology is all of the fields connected with how you get data from one place to another. Like telephones, televisions, cameras, cell phones that take pictures, all the wireless networks people have in their homes now. People have their clock, their toaster, their coffee maker, their pacemaker; we seem to be moving towards a society where all these things are networked together. So that your pacemaker will start to wake up and send a message to your coffee maker: It’s time to make coffee!
I’m not sure I want to imagine it!