Discovery fiction

To help me understand a scientific result, I often find it helpful to write what I call discovery fiction. By this I mean: a plausible story of how I could have discovered that result – an arc of small questions and ideas, false starts and backtracking, incremental steps eventually leading to the result. None of these things should come out of thin air: they should all be simple, almost-obvious steps.

Although it perhaps seems an expository form, I write discovery fiction for a more selfish reason: as a way of deepening my own understanding. For instance, in the 1990s I got interested in cryptocurrency, when I became aware of the cypherpunks and David Chaum's invention of digital cash. In 2013, after several years of hearing about Bitcoin, I wanted to understand it in detail. I read the Bitcoin white paper, and found it easy to "understand" in broad strokes. But even after several close reads my understanding was shallow. I could follow the steps, but if you'd taken the white paper away from me I couldn't have reinvented the protocol from scratch, even though many of the ideas were familiar because of my prior interest. I just didn't understand the white paper well enough.

To deepen my understanding I wrote a discovery fiction essay that begins with the question: how could we design a digital currency? And then describes an extremely simple, obvious idea for such a currency – an obviously flawed back-of-the-napkin idea, the kind of thing anyone familiar with computers would come up with if they took the question seriously. Then the essay points out an obvious problem with the idea. And proposes a simple, obvious fix. Then points out another obvious problem. And another simple fix. And so on, over and over, until eventually we "discover" Bitcoin! This approach shows very concretely how and why each element of Bitcoin is introduced, and what purpose it serves; it also makes clear that some features of Bitcoin are a little arbitrary, and how they could have been different. By contrast, the Bitcoin white paper (mostly) presents the ideas in a more complex jumble.

Discovery fiction is often much longer than conventional presentations. This is not surprising: it's typically much easier to verify the solution to a problem than it is to find the solution¹. My experience is that discovery fiction is usually 2-5 times longer than a conventional explanation, if written at similar levels of detail. This may explain why it is not more common: it superficially appears inefficient, even long-winded. But the benefit of discovery fiction, as both writer and reader, is that it leaves you understanding much better where the solution came from. And I find well-written discovery fiction easier and faster to read, even given the greater length.

Of course, I certainly didn't originate discovery fiction as a form. Galileo's 1632 Dialogue Concerning the Two Chief World Systems isn't strictly discovery fiction, but it has much overlap. My first conscious exposure to what I now call discovery fiction was Imre Lakatos's wonderful 1976 book Proofs and Refutations. In the book, Lakatos develops an understanding of the Euler characteristic for an object, and of the Euler formula. It's not pure discovery fiction – Lakatos is a philosopher and has a particular agenda for his work – but close enough that it counts as an example of the form. Many scientists have since told me they use discovery fiction in both their teaching and their research work. And I've since learned of a handful of published examples of discovery fiction. The best collection I know comes from this StackExchange post by Alessandro Cosentino. Tim Gowers, in particular, has written many pieces which are adjacent to discovery fiction, and in some cases may be considered examples of the form – see, for instance, this discussion of where the idea of a normal subgroup comes from². It would be interesting to collect together many more examples of discovery fiction, or even to write a history of discovery fiction.

Although the form is old, the term "discovery fiction" seems to be my coinage³. I wrote this brief note because I've found the term extremely useful⁴. It refers to an idealized literary form for scientific results⁵. It has a different goal from the most commonly used literary forms for science. For instance, the goal of a scientific paper is (typically) to explain the claim (or claims) of a new scientific result, and the evidence and supporting reasoning. The paper does not need to explain how one could have arrived at the discovery oneself, except insofar as that is essential for verification of correctness. If something is not part of the chain of evidence and argument, it's not crucial for the paper! The effect is often to remove many (and sometimes most) signposts that would allow a reader to understand how the result was ever obtained. None of this is a criticism of scientific papers as a literary form, nor is it intended to suggest that papers "should" be discovery fiction. I just wish to stress that they have genuinely different goals from discovery fiction⁶.

Perhaps more surprisingly, discovery fiction is also very different from actual history of science. The real history of discovery is often extraordinarily complex, involves large amounts of chance and highly contingent local knowledge and fortunate happenstance. Indeed, sometimes reading the history of science leaves one almost as bewildered as before about how a discovery is made. To paraphrase Toynbee, the history of science sometimes seems to be just one random fact after another⁷. By contrast, while good discovery fiction is entirely free to borrow from history – as inspiration or illustration to enliven – it has no allegiance to history. Its only obligation is to construct a satisfying idealized story about how the discovery could plausibly have been made.

Not all discoveries need discovery fiction. Sometimes, people do a well-motivated experiment in an easy-to-understand way, and get a striking result. No discovery fiction needed! But when there's a tricky conceptual path, and it's not obvious what led to a discovery, then discovery fiction is more useful. Much of my work has been in computer science, design, and theoretical physics. These are all oriented toward discovery of new concepts and hard-to-find arguments. As such, they're well suited to discovery fiction. I expect the need in other parts of science to vary quite a bit.

I wrote this note to encourage other people to create more discovery fiction. This may be done in expository papers and books and essays; in newer forms like Twitter and TikTok and YouTube, and in forms yet to be discovered. And, I hope, people will find ways of using the form more in research papers. I personally enjoy reading discovery fiction, and often learn more (and more rapidly!) from it than from conventional presentations.

I also wrote this note to stress how useful discovery fiction is in research and creative work. It's a forcing function or point of view that can be used to deepen one's own understanding⁸. I wrote my short Twitter discovery fiction about quantum teleportation in 2019. Prior to that I'd written three papers about quantum teleportation, did one of the first quantum teleportation experiments in the laboratory, and gave countless talks about it. I would have sworn I knew quantum teleportation extremely well. And yet writing that thread greatly improved my understanding. That's really the main reason I'm interested in discovery fiction: as a point of view that forces me to deepen my own understanding. The exposition is merely a side effect.

You may wonder, if I think of discovery fiction primarily as a tool for personal understanding, then does that mean most of my discovery fiction is private? I do have some private discovery fiction, but I've found it most useful to write in public: writing in public – in any form – is also a reliable way to improve my thinking. There are many reasons writing in public can improve one's thinking, including: (a) you can be embarrassingly wrong; and (b) you hold yourself to the standards of your imagined audience. Provided you've chosen your imagined audience well, both these things greatly improve the quality of your writing and so of your thinking⁹. When I write just for myself, too often it becomes a low fidelity rederivation of the result. That's useful, but I can fool myself into believing I've understood it better than I have. A less forgiving imagined audience helps me avoid fooling myself.

There is far more that could be said about writing discovery fiction as a way of improving one's own understanding. I'll make just two remarks. First, there is a variant form, multiple discovery fiction, where you describe multiple possible stories of discovery, usually (though not always) by starting with quite different initial ideas. It's a lot of work, but can be immensely enjoyable – I get a sense of really knowing my way around the subject, especially when I have three or more stories of discovery. And second: I've found discovery fiction surprisingly pleasurable to write when combined with the use of memory systems, as described here.

As with any other literary form, there is good discovery fiction, and bad. The science fiction writer Theodore Sturgeon was once asked why 90% of science fiction is crap. He supposedly replied that of course 90% of science fiction is crap: 90% of everything is crap. Many years ago, Bret Victor wrote a marvelous essay introducing the idea of "explorable explanations". After the essay, many people wrote attempts at such explorable explanations. In keeping with Sturgeon's Law, most weren't very good. But the problem was perhaps worse even than Sturgeon's Law would suggest. Many of the people attempting explorable explanations seemed much more interested in the form than in what they were writing. Not surprisingly, the results in such cases were not very good¹⁰. I must admit to some trepidation about discovery fiction on this account: I'd love for the term to take off, and (even if the term doesn't take off) for the form to be much more widely used. But I hope people writing discovery fiction are genuinely interested in just doing a great job, independent of experimenting with the form!

I've described discovery fiction as an idealized literary form meeting certain criteria. Of course, I don't mean to imply that it "must" be pure. It's not a formal specification; this isn't haiku or iambic pentameter. Rather, I'm giving a name to and describing a literary form that is already routinely used in small doses, and occasionally used in large doses. Indeed, people will often say a "good writer" of scientific papers is someone who includes insightful "motivating remarks". Often this means they're mixing in some discovery fiction. But what I'd like to see is much more discovery fiction, and more awareness of how useful it is for improving one's own understanding.

Acknowledgments

Thanks to Andy Matuschak and Grant Sanderson for conversations about discovery fiction. And thank you to the good people of Twitter for encouraging me to write up my thinking about discovery fiction. This work was supported by the Astera Institute.

Footnotes

I don't mean to imply that discovery fiction is solely about how one might discover a solution to a well-posed problem. Indeed, often the most interesting part of the discovery journey is the discovery of the problem to be solved. It may start with a half-baked hunch, or with an apparently unrelated problem – so the discovery fiction includes discovering the problem, and perhaps even the basic concepts underlying the problem, along with the solution. You see this in my discovery fiction for Hindu-Arabic numerals, for instance.↩︎
I suspect – though I am not sure of my own motivation – that this is a bit part of what has stoked my interest in the form over the years.↩︎
I've used the term informally in quite a number of places. This note is the first extended discussion, however.↩︎
In a similar way I've found Timothy Chow's notion of "solving an unsolved exposition problem" helpful.↩︎
The examples I know are in science (including mathematics). I presume the form could be applied elsewhere, too – any subject where arguments or concepts may be hard-to-find but easy-to-verify would be a good candidate – but I don't know of examples. I've found it amusing to push this thinking: how could one "discover" the novels of Jane Austen or Douglas Adams, for instance? What would it even mean?↩︎
Of course, a paper could also function as discovery fiction. I know of a few examples which mix in quite a bit of discovery fiction, but don't know of any example which is intended both as a research paper and as discovery fiction.↩︎
August Kekulé famously discovered the structure of benzene after daydreaming of a snake swallowing its own tail. Ramanujan claimed that his famous mathematical ideas were visions from the Goddess Mahalakshmi of Namakkal. It is fortunate that there are other ways of verifying such ideas, and we don't personally need to have the same experiences. Facts such as these are irrelevant in a scientific paper (except as color), highly relevant in history of science (though what to make of them I'm not sure), and would be optional-but-potentially quite useful in discovery fiction.↩︎
Past experience suggests that it will be labeled an "expository technique", no matter how forcefully I describe it as principally a technique for deepening understanding. I find this chafes a little.↩︎
This is probably true of most imagined audiences. But I'll bet that if you choose your audience really badly you can actually make your writing worse. People who write for audience engagement or reach on Twitter sometimes make their thinking worse in this way.↩︎
This is a problem with much work on "tools for thought": so many poorly-conceived "hammers" invented by people so uninterested in actual carpentry that they don't realize the tool they've made is of little conceivable use.↩︎

Acknowledgments

Footnotes

Comments