What Is the Most Important Problem in Your Field?
Hamming describes eating lunch with physicists at Bell Labs for years. When their conversation grew too comfortable, he began asking: What are the important problems in your field? What are you working on that is important?
Eventually he became direct: 'If what you are working on is not important and not likely to lead to important things, then why are you working on it?'
He was not invited back to the physics table.
One chemist stopped him in the hall months later: 'What you said caused me to think for the whole summer about what the important problems are in my field. While I have not changed my research, it was well worth the effort.' That chemist became head of his group. Then a member of the National Academy of Engineering.
Hamming's observation: no other person at the table responded to the question. No other person at the table became notable.
His formulation: 'If you do not work on important problems, you have little chance of doing important work.'
This sounds obvious. The evidence: most scientists spend most of their time on problems they believe are neither important nor likely to lead to important things. The question is not asked. The question is avoided.
Ask the Question
Hamming's question, directed at you:
The Courage to Work on Hard Problems
Hamming identifies fear of failure as the primary reason most people avoid important problems. Hard problems fail more often. Failure is visible. Easy problems succeed more often. Success is rewarded.
The result: most researchers accumulate a long list of successful small results while the important problems remain untouched. The illusion of productivity is real — they are productive, just not on the problems that matter.
Shannon had courage. Hamming describes him: who else would think of averaging over all possible random codes and claim the average code would be good? Shannon knew what he was doing was important and pursued it intensely. He was not afraid to look foolish.
Shannon's chess mantra: 'I ain't scaird of nothing.' Hamming copied it deliberately. When stuck, he repeated it. At times it enabled him to continue and find a solution.
His prescription: look at your successes, not your failures. Pay less attention to learning from mistakes than is usually advised. Build confidence by cataloging your wins. Use that confidence to pursue the next hard problem.
Newton on the subject: 'If others would think as hard as I did, they would be able to do the same things.' Edison: 'Genius is 99% perspiration.' Hard work over long periods — not exceptional talent — produces the important results.
Recognizing Fear of Important Problems
Hamming's observation: the avoidance of important problems is usually not conscious. Researchers convince themselves that the work they are doing is important, necessary, or a prerequisite for attacking the harder problem later. The later never arrives.
Luck Is Not Sufficient, But Not Irrelevant
Hamming takes luck seriously without treating it as the primary factor. Shannon got lucky: information theory was in the air and many people were working on it. But Shannon had asked an early question about the relationship between information and uncertainty that primed him to understand what was happening better than anyone else.
The prepared mind is Hamming's bridge between luck and preparation. Luck = an opportunity appearing. Prepared mind = being positioned to recognize and use it. The combination produces great work; neither alone is sufficient.
His open door policy: Hamming kept his office door open, was constantly interrupted, and constantly exposed to problems, people, and ideas from across Bell Labs. The closed door produced focused work. The open door produced serendipitous collisions.
He knew he sacrificed focus. He considered it worthwhile. Zone melting (a materials purification technique essential to transistors) crossed his desk as an open-door conversation with Bill Pfann — who Pfann's own department had dismissed. Hamming helped him, taught him computing, gave him machine time, and let him have all the credit. Pfann ended up with all the prizes; his old lab became a National Monument.
The open door is a strategy for increasing your exposure to lucky accidents. Not waiting for luck — manufacturing the conditions under which luck can find you.
Engineering Your Exposure to Luck
Hamming's open door is a specific, concrete practice for increasing serendipity. He lost focus; he gained breadth.
The Compound Question
Hamming closes with a compound challenge that draws on everything in the course.
He distinguishes three types of questions everyone in science and engineering must answer separately:
1. What is possible? — the scientific question.
2. What is likely to happen? — the engineering question.
3. What is desirable? — the ethical question.
Most practitioners ask only the first. Occasionally the second. Rarely the third.
His argument: all three are required for doing important work. Knowing what is possible without asking what is desirable leads to contributing to projects with negative value. Knowing what is desirable without asking what is possible leads to wishful thinking. The third question — what is desirable — is the one most systematically avoided.
He adds: the desire for excellence is an essential feature of doing great work. Without a vision of excellence, effort is like a random walk: each step independent, progress proportional to √n. With a vision, steps compound: progress proportional to n. For large n, the difference is everything.