A colleague asked me for a short list of heuristics or principles for doing good creative research work. Here's three hours of stream-of-consciousness thinking in response, lightly revised -- I was surprised by the strength of my feeling! It would ideally be heavily revised and improved, but it seemed better to release an imperfect set of notes than to await a hypothetical improved version. Context: informed by my work in theoretical physics, computer science, tools for thought, and metascience; the latter two are very much proto-fields. Caveat emptor: it feels like bragging to be putting such thoughts to screen, especially since I'm so often frustrated by the progress (or lack thereof) in my work! But I also believe creative research is something we researchers are all in together, and that we'd all be better off if everyone shared their thoughts on how to do it, what we find works for us, what doesn't, where we get frustrated, and so on. So I'd love to see other documents like this! Written January 2023, released July 2023.

Beware general principles: Principles of work aren't like the laws of nature. What works for one person is often terrible for another. What works at one time in your life may work terribly at another. So you need a patchwork of (sometimes contradictory) heuristics. When things are going well, keep doing what you're doing. When you're in a rut, try other heuristics. It can be hard to do that: if you're stubborn and determined, you'll likely double down, and try to tough out the rut. And it's hard to be a good researcher if you're not stubborn and determined. Indeed: good researchers are often a remarkable combination of stubborn, determined, and astonishingly flexible and willing to re-orient.

Related: you must figure out how you need to work. There will likely be some overlap with other people – we're all near enough to East African plains apes(!) – but also many idiosyncrasies. The great researchers I've known were mostly extremely unusual individuals, and often had extremely unusual approaches to work.

Beware advice: The trouble with advice is often that the people who most need to take it seriously, don't, and the ones who most need to ignore it, don't. Which category are you in? How do you know? If someone you respect gives you advice, and it resonates and inspires you, run with it! If it slows you down, sometimes it's because you need to change what you're doing. But sometimes it's because it's the wrong advice for you. Even the very best people will dispense advice that's mismatched for you. Play with it. Consider the equal and opposite advice. Is there some other way to metabolize it? And if not, it's fine to disregard it. But don't let it get you down.

(A surprising fraction of my early physics teachers told me I wasn't suited to physics, explaining in detail why. In retrospect it's clear to me that those people didn't know what they were talking about. But this experience made things very difficult for a long time.)

Momentum: In general: momentum rules, momentum rules, momentum rules. And the corollary is: do everything you can to cultivate motivation¹.

Collaboration as mutual mentorship: The best collaborations often involve mutual mentorship². That is, your collaborator helps you grow, and you help them grow. This is partially a function of your respective capacities. But it's also in part a reflection of your willingness to learn. A strange thing: I often didn't notice the people from whom I learned the most, at the time. What I was learning was so unfamiliar that I didn't even realize I was learning, until much later. An example was my collaboration on quantum information with Ben Schumacher: Ben approached the world and asked questions in a way unlike anyone I knew. This was far more valuable than even the (very valuable) technical things I learned from him.

Find mentors much younger and much older than you. Obviously, this may not apply at the beginning or end of a career! One of the worst patterns afflicting older researchers is that they believe they know better than younger people. Their younger collaborators often end up as peons – vessels for work. The way to remain creatively alive into your 60s and 70s and 80s seems to be to find people in their 20s and 30s from whom you can learn deeply³. That may mean helping them with their projects, not yours.

No matter what your age, learning to talk to people of many different ages is a competitive advantage. This can be intimidating, especially at first.

John Mack, one of the biographers of T. E. Lawrence, remarked that Lawrence's essential quality was "a power of enablement". He found many instances where Lawrence's life would briefly intersect some other person, sometimes for just a few minutes, and yet it would change the course of that person's life. The pattern was this: Lawrence would question his new acquaintance deeply, elicit what they wanted – sometimes desires they had barely acknowledged to themselves – and then hold that desire up in crystalline form, a kind of mirror, and explain to them a very concrete way in which they had the power and capability to immediately act toward it. Not everyone is Lawrence. But you can consciously approach people with an attitude of enablement, and cultivate the skill. It is a skill.

The frontier can move faster than any human: One moment the exciting research frontier is over here, where you are, the next it's in some distant area. This is very similar to the way a wavefront can move (much!) faster than any particle in a wave. There are several reasons this happens. It is, of course, enabled by the enormous ocean of people with many different combinations of expertise; while you are no longer at the exciting frontier, someone else woke up one day to find that their expertise is now hot, when just a month earlier it was sleepy. It's also partly because capital can move much faster than individuals can acquire expertise⁴.

This is an emotionally challenging situation. Different people respond in different ways. Some are always chasing after the wave. This usually leads to boring, me-too research. One alternate approach is to ask: is there any secret thesis I can find, something that gives me a unique edge? Often, the answer is no. But sometimes people find a way to get ahead of the wave. In any case, it's better than pure wave chasing. And a second approach is to try to cultivate your own vision, deliberately routing around what is fashionable.

Personally, I find work meaningful when I can make a unique contribution of importance. Most of my work has been in areas that I hope will be full-fledged fields decades in the future, doing what I hope is foundational work. Some people instead love the thrill of competition in hot fields, and enjoy narrowly winning races. I had a conversation with a CRISPR scientist who told me that he loved submitting a paper a day or two ahead of his competitors. He said: "Isn't that [competition] why we all do science?" I'd rather gouge my eyes out. Doesn't mean he was wrong, exactly: for him, I suspect, it's the only way he'd be motivated to do science. But it does mean: understanding your own (honest!) motivations matters a lot. And then figuring out the best way to work in response.

Bits and pieces

1. The highest order bit is the fertility of the overall area you are working in, not you yourself. Work on quantum mechanics in the 1920s, or CRISPR in the 2010s, or interface design in the 1970s, and you would likely do well. Suppose: "I worked in a subject area that was incredibly fertile, before it became super-fashionable, in a pretty good environment, and I worked hard and with imagination and courage and determination". You would do extremely well. By contrast: you can be astoundingly talented and hard working, but if you're in the wrong area you will only make a minor contribution. Even duffers can make significant contributions in areas that are incredibly fertile, but under-subscribed. Related: old researchers who did great work in their youth are often interviewed about how they did it. They often give all kinds of process explanations ("see, our seminar culture was…"). It's interesting stuff, and can be helpful, but low-order bits compared to: "I worked very hard in a subject area that was incredibly fertile [etc]".
2. The next highest order bit – closely related! – is developing good taste. This does not mean, as some people think, figuring out what's fashionable, and going where the crowd is going. Your work will, at very best, be work that someone else would have done anyway. Rather, it means developing your own sets of heuristics for what is of fundamental importance, what are the big overall drivers and opportunities, and what special skills you can develop that will enable you to contribute. Something I love about Einstein's career is that he published (IIRC) six methods for determining Avogadro's number. To our modern eye, this looks strange. But I believe it reflects Einstein's underlying taste well, an obsession with a certain way of understanding the world. It's the same kind of taste that led him to special and general relativity and his other major discoveries.
3. Good environment != branded environment. All other things equal, brand helps. But often the best environments are in no-name places.
4. Courage, imagination, and determination are usually more important than being smart. To an astounding extent, research is about emotional self-management.
5. What do you know that no-one else does? What can you do that no-one else can? If it's early in your career, and the answer is "not much", then can you find routine projects that will help other people, and help you develop unique capabilities? What capabilities will enable us to solve the deepest problems over the next 3-30 years? Can you find ways of beginning to develop those, while still doing something useful?
6. Given all that: the problems you work on matter enormously. I say "work on" but that's not the same as "pick". Einstein got interested in some strange transformation properties of Maxwell's equations; in trying to resolve that, he discovered that space and time, mass and energy, are not as they seemed, but fundamentally different. Darwin went on a geology expedition, but was so curious and observant that he ended up understanding the origin of species. Great problems have to be discovered; often the solution of the problem is only a tiny part of the story, most of it is really about discovering the problem.
7. A shocking fraction of what you learn from the best people is tacit knowledge. How they respond emotionally; when they work hard, when they relax; the way they play with ideas; what they ignore; how they respond to fashion; how they respond to news; how they approach problems; how they structure their day; how they structure their year; how they structure their decade. And a million other hard-to-articulate things. Gabriel García Márquez said of reading Kafka: "I thought to myself that I didn't know anyone was allowed to write things like that. If I had known, I would have started writing a long time ago". More generally, close examination of unusually creative people will expand your sense of what you're allowed to do, and what may be beneficial to do.
8. Cynicism about your work is incredibly destructive. If you find yourself getting cynical, be prepared to make drastic changes to your environment. Start small, but: change supervisor, change city, change field, do something. Ditto burnout, though for different reasons.
9. Do not, do not, do not lie in your grant applications. This is partially for public-spirited reasons, but there's a really big selfish reason: if you lie it will confuse you about what you're doing, and that will damage your work. This is true no matter how much you think it isn't. This does not, by the way, mean you are then tied to do what you said you were going to do (unless that's a condition of the grant): if you're learning interesting things, then your project will change, sometimes radically. But clarity and sincerity are valuable intellectual fuel.
10. Research involves tremendous amounts of routine janitorial work. Even the smartest people usually do enormous amounts of routine work. Doing it lays the groundwork for the less routine work. I've often witnessed grad students who are reluctant to get their hands dirty doing routine calculations, hoping they can solve the problem with a stroke of insight; it's sobering to then see a Fields Medallist or Nobelist roll up their sleeves and do page after page of very elementary calculations, just chasing down every single lead. The golfer Jack Nicklaus supposedly claimed that the harder he practiced the luckier he got. The same seems to be true in research, in two different ways: the harder you work, the luckier you will get; and the harder you work at routine things, the more insight you will have⁵.
11. "Motivation follows action", as a friend likes to say. If you keep waiting to be motivated, you'll never get anything done. On the other hand, if you repeatedly act and aren't getting motivated, you probably need to find a different project, a different environment, or a different subject area.
12. Honest post mortems are surprisingly helpful. So is being honest about your fears and shortcomings, in advance and during the course of the project.
13. Courage matters. An example, one of infinitely many: large research collaborations are usually thought of as being arranged by senior researchers. But one of the two collaborations that discovered the accelerating universe was largely organized and driven by two postdocs (Adam Riess and Brian Schmidt). They saw the opportunity to use type 1A supernovae to determine whether the expansion of the universe was accelerating, and just went and rounded up all the people and resources they needed to do it, despite not having the usual seniority. And, as a result, transformed our understanding of the universe, showing that Einstein was wrong in an important way. And won a Nobel Prize.
14. Teaching a subject well is a superb way to learn it. This is a cliche because it's true. It has a curious corollary, though, which is that by challenging yourself to teach better you can learn more. I'm sure Feynman learned more from The Feynman Lectures on Physics than any of his students. And you can pour more creativity and effort in than even Feynman did. Look at people like Brian Moriarty or Vi Hart or Grant Sanderson to see just how much creativity someone can pour into explanation. Andrei Kolmogorov decided to teach a lecture class about one of Hilbert's problems; that class led to the solution of the problem. This is an extreme example, but not an unusual pattern.
15. Difficult is not at all the same as important.
16. Learn what it means to work hard: this is surprisingly difficult. It means knowing when to push, when not. It means knowing how to vacation well and how to enjoy your life. You must not become a drudge – a surprisingly common problem among the ambitious. You must become alive to moments of leverage and creative opportunity and insight. And, of course, it also means knowing how to work incredibly hard and with great courage and determination.
17. Related: Take care of your body. Take care of your energy levels.
18. Do some damn-fool work: the co-discoverer of graphene, Andre Geim, set aside Friday afternoons to do foolish experiments that he couldn't quite get out of his head. It led to both his Nobel Prize and an igNobel Prize, as well as several other important lines of work.
19. Groucho's law of creative work: "You should never work on any project for which you can get funding": Obviously somewhat tongue-in-check. But there's some underlying truth. If it's easy to get funding, yours is probably the kind of work someone else will do anyway. What can you do that won't otherwise be done? There's a kind of analog of the efficient market hypothesis in research, the notion that there's a space of research projects our institutions are pretty much just going to get done. You want to escape out of that, and bend the market, while doing something important. Is there anything you can think of that existing institutions don't have the courage or imagination to support? Is there some way you can figure out how to do it anyway?
20. Vacation – real vacation, the kind where you do things purely for enjoyment – often helps get out of a rut: For me, the pattern of good vacation is giving myself 2-3 days of doing almost nothing. And then going to explore the world. Do not, do not, do not, go on a half vacation, where you still answer your email, do bits and pieces of work, etc. It's fine to plot and plan and to work on some other project – I accidentally wrote the first draft of my book "Reinventing Discovery" in an enormous flurry of work while on (a very relaxing!) vacation in Hawaii. But don't do anything that you think of as routine work.
21. When you're getting started it's easy to be paralyzed. There's so much you don't know! In fact, you barely need to know anything. Find something which matters, and where you can make a contribution, and work on it. This is a major benefit of working with terrific researchers. They will help you find things that matter, and where you can contribute. PhD students often think their advisors will help them with how to solve a problem, but the much more important thing they do is help them find problems which: (a) are tractable; (b) of some importance; (c) will help them develop; and (d) help them know what to ignore.
22. Courage is a key ingredient⁶. And it must be taken: insofar as you only follow paths tacitly approved by some pre-existing community, your work will be of a kind that could have happened anyway. To go beyond this, you must construct your own belief. I struggle greatly with this.
23. That said, insofar as you can: socialize as much of your understanding and creative work as you can. Talking to supportive others – even if they don't understand – is immensely helpful. I don't think I've ever run a class that didn't give back tenfold; despite that, I often hesitate ("it's too much work!")
24. The perils of process advice: I've given tonnes of process advice here. It can be paralyzing. In the end, you've just got to get on with it. Keep reading lots of advice, from many different sources, act on anything that seems immediately helpful, reflect on the advice, and then try to forget it much of the time!

Acknowledgements

Thanks to Paul Graham for the question which stimulated these notes. Conversations with hundreds of people informed the notes; my thanks to them all! Thanks in particular to the marvellous collaborators from whom I've learned over the past few years: Shan Carter, Patrick Collison, Andy Matuschak, and Kanjun Qiu.

Footnotes