Useful Vices for Wicked Problems

*Click lower right to download or find on Apple Podcasts, Spotify, Stitcher, etc.*

I've claimed that the best way to learn is [by writing](https://www.cold-takes.com/learning-by-writing/) about important topics. (Examples I've worked on [include](https://www.cold-takes.com/learning-by-writing/): [which charity to donate to](https://www.cold-takes.com/the-wicked-problem-experience/#flashback-to-2007-givewell), [whether life has gotten better over time](https://www.cold-takes.com/has-life-gotten-better/), [whether civilization is declining](https://www.cold-takes.com/cost-disease-and-civilizational-decline/), [whether AI could make this the most important century of all time for humanity](https://www.cold-takes.com/most-important-century/).)

But I've also said this can be "hard, taxing, exhausting and a bit of a mental health gauntlet," because:

* When trying to write about these sorts of topics, I often find myself needing to constantly revise my goals, and there's no clear way to know whether I'm making progress. That is: trying to write about a topic that I'm learning about is generally a [wicked problem](https://www.cold-takes.com/the-wicked-problem-experience/).

* I constantly find myself in situations like "I was trying to write up why I think X, but I realized that X isn't quite right, and now I don't know what to write." and "I either have to write something obvious and useless or look into a million more things to write something interesting." and "I'm a week past my self-imposed deadline, and it feels like I have a week to go, but maybe it's actually 12 weeks - that's what happened last time."

* Overall, this is the kind of work where I can't seem to tell how progress is going, or stay on a schedule.

This post goes through some tips I've collected over the years for dealing with these sorts of challenges - both working on them myself, and working with teammates and seeing what works for them.

A lot of what matters for doing this sort of work is coming at it with open-mindedness, self-criticality, attention to detail, and other *virtues.* But a running theme of this work is that it can be deadly to approach with *too much* virtue: holding oneself to self-imposed deadlines, trying for too much rigor on every subtopic, and otherwise trying to do "Do everything right, as planned and on time" can drive a person nuts. So this post is focused on a less obvious aspect of what helps with wicked problems, which is *useful vices* - antidotes to the kind of thoroughness and conscientiousness that lead to unreachable standards, and make wicked problems impossible.

I've organized my tips under the following vices, borrowing from [Larry Wall](https://wiki.c2.com/?LazinessImpatienceHubris) and extending his framework a bit:

* **Laziness.** When some key question is hard to resolve, often the best move is to just ... not resolve it, and change the thesis of your writeup instead (and change how rigorous you're trying to make it). For example, switching from "These are the best charities" to "These are the charities that are best by the following imperfect criteria."

* **Impatience.** One of the most crucial tools for this sort of work is *interrupting oneself*. I could be reading through study after study on some charitable activity (like building wells), when stepping back to ask "Wait, why does this matter for the larger goal again?" could be what I most need to do.

* **Hubris.** Whatever I was originally arguing ("Charity X is the best"), I'm probably going to realize at some point that I can't actually defend it. This can be demoralizing, even crisis-inducing. I recommend trying to build an unshakable conviction that one has something useful to say, even when one has completely lost track of what that something might be.

* **Self-preservation.** When you're falling behind, it can be tempting to make a "heroic" effort at superhuman productivity. When a problem seems impossible, it can be tempting to fix your steely gaze on it and DO IT ANYWAY. I recommend the opposite: instead of rising to the challenge, shrink from it and fight another day (when you'll solve some problem other than the one you thought you were going for).

Overall, it's tempting to try to "boil the ocean" and thoroughly examine every aspect of a topic of interest. But the world is too big, and the amount of information is too much. I think the only way to form a view on an important topic is to do a whole lot of simplifying, approximating and skipping steps - aiming for a step of progress rather than a confident resolution.

Laziness
--------

### Hypothesis rearticulation

My [previous piece](https://www.cold-takes.com/the-wicked-problem-experience/) focused on "hypothesis rearticulation": instead of defending what I was originally going to argue, I just change what I'm arguing so it's easier to defend. For example, when asking [Has Life Gotten Better?](https://www.cold-takes.com/has-life-gotten-better/#fnref4), I could've knocked myself out trying to pin down exactly how quality of life changed in each different part of the world between, say, the year 0 and the year 1000. Instead, I [focused](https://www.cold-takes.com/did-life-get-better-during-the-pre-industrial-era-ehhhh/) on saying that that time period is a "mystery" and focused on arguing for why we shouldn't be confident in any of a few tempting narratives.

My [previous piece](https://www.cold-takes.com/the-wicked-problem-experience/#flashback-to-2007-givewell) has another example of this move. It's one of the most important moves for answering big questions.

### Questions for further investigation

This is really one of my favorites. Every GiveWell report used to have a big section at the bottom called "Questions for further investigation." We'd be working on some question like "What about the possibility that paying for these services (e.g., bednets) just causes the government to invest less in them?" and I'd be like "Would you rather spend another 100 hours on this question, or write down a few sentences about what our best guess is right now, add it to the Questions for Further Investigation section and move on?"

To be clear, sometimes the answer was the former, and I think we eventually did get to ~all of those questions (over the course of years). But still - it's remarkable how often this simple move can save one's project, and create another fun project for someone else to work on!

### What standard are we trying to reach? How about the easiest one that would still be worth reaching?

If you're writing an academic paper, you probably have a sense of what counts as "enough evidence" or "enough argumentation" that you've met the standards for a successful paper.

But here I'm trying to answer some broad question like "[Where should I donate](https://www.cold-takes.com/the-wicked-problem-experience/#flashback-to-2007-givewell)?" or "[Is civilization declining](https://www.cold-takes.com/cost-disease-and-civilizational-decline/)?" that doesn't fit into an established field - and for such a broad question, I'm going to run into a huge number of sub-questions (each of which could be the subject of many papers of its own). It's tempting to try for some standard like "Every claim I make is supported by a recognizably rigorous, conclusive analysis," but that way madness lies.

I think it's often better to aim for the *minimum* level of rigor that would still make something "the best available answer to the question." But I'm not absolutist about that either - a frustrating aspect of working with me on problems like this is that I'll frequently say things like "Well, we don't *need* to thoroughly answer objection X, but we should do it if it's *pretty quick* to do so - that just seems like a good deal." I think this is a fine way to approach things, but it leads to shifting standards.

Here's a slightly more nuanced way to think about how "rigorous" a piece is, when there's no clear standard to meet. I tend to ask: “How hard would it be for a critic to demonstrate that the writeup's conclusion is significantly off in a particular direction, and/or far less robust than the writer has claimed?”[1](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#fn1)

The "how hard" question can be answered via something like:

* "A-hardness": minimum hours needed by literally anyone in the world

* "B-hardness": minimum hours needed by any not-super-hard-to-access person, including someone who’s very informed about the topic in question

* "C-hardness": minimum hours needed by a reasonably smart but not very informed critic, looking on their own for flaws

I seem to recall that with GiveWell, we got a lot more successful (in terms of e.g. donor retention) once we got to the point where we could get through an hour-long Q&A with donors with a “satisfying” response to each question - a response that demonstrated that (a) we had thought about the question more/harder than the interlocutor; (b) we had good reason to think it would take us significant time (say 10-100 hours or more) to get a better answer to the question than we had. At this point, I think the C-hardness was at least 10 hours or so - no small achievement, since lots of not-very-informed people know something about some random angle.

(By now, I'd guess that GiveWell’s A-hardness is over 100 hours. But a C-hardness of 10 hours was the first thing to aim for.)

These standards are very different from something like “Each claim is proven with X amount of confidence.” I think that’s appropriate, when you keep in mind that the goal is “most thoughtful take available on a key action-guiding question.”

Impatience
----------

Many people dream of working on a project that puts them in a [flow state](https://en.wikipedia.org/wiki/Flow_(psychology)):

> In [positive psychology](https://en.wikipedia.org/wiki/Positive_psychology), a flow state, also known ... as being in the zone, is the [mental state](https://en.wikipedia.org/wiki/Mental_state) in which a person performing some activity is fully immersed in a feeling of energized focus, full involvement, and enjoyment in the process of the activity. In essence, flow is characterized by the complete absorption in what one does, and a resulting transformation in one's sense of time.

But if you're working on wicked problems, I recommend that you avoid flow states, nice though they may be. (Thanks to [Ajeya Cotra](https://www.openphilanthropy.org/about/team/ajeya-cotra) for this point.) Maybe you instead want a [Harrison Bergeron](http://www.tnellen.com/cybereng/harrison.html) state: every time you're getting in a groove, you get jolted out of it, completely lose track of what you were doing, and have to reassemble your thoughts.

That's because one of the most productive things you can do when working on a wicked problem is rethink what you're trying to do. The more you interrupt yourself, and the less attached you are to the plan you had, the more times you'll be able to notice that what you're writing isn't coming out as planned, and you should change course.

### Checkins and virtual checkins

I think the ideal way to interrupt yourself is to be working with someone else who's engaged in your topic and has experience with similar sorts of work (at Open Philanthropy, this might mean your manager), and constantly ping them to say things like:

* I've started to argue for point X, but I don't think my arguments are that great.

* I'm thinking I should deeply investigate point Y - sound right?

* I'm feeling dread about this next section and I don't really have any idea why. Any thoughts? (A lot of people are hesitant to do this one, but I think it often is exactly the right move!)

I think this is helpful for a few reasons:

* You may have gotten subconsciously attached to the vision you had in your head for what you were going to write, and it's good to get a reaction from someone else who has less of that attachment.

* It's generally just hard to make yourself look at your work with "fresh eyes" as your goal is constantly changing, so bringing in another person is good.

* It's easy to get caught up in a "virtue" narrative when doing this work - "I'm thorough and rigorous and productive, I'm going to answer this question thoroughly and rigorously and do it on time." It's tempting ([as I'll get to](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#self-preservation)) to try to overcome hard situations with "heroic effort." But another person is more likely to ask questions like "Well, how long does it usually take you to do this sort of thing?" rather than "Can you make an incredible heroic effort here?" and “What do we think we can do and by when and is it worth it?” rather than “What would failure to do the thing you thought you could do say about you as a person?”

With [early GiveWell](https://www.cold-takes.com/the-wicked-problem-experience/#flashback-to-2007-givewell), I got a huge amount of value from [Elie](https://www.linkedin.com/in/elie-hassenfeld-92072a9/), who consistently wanted to do things far less thoroughly than I wanted to. I probably ended up doing things 3x as thoroughly as he wanted and 1/3 as thoroughly (and so 3x faster!) as I originally wanted - a nice compromise.

These kinds of checkins can be very vulnerable (especially when the topic is something like "I can't accomplish what we both said I would"), and it can be hard to have the kind of relationship that makes them comfortable. It's best if the manager or peer being checked in with starts from a place of being nonjudgmental, remembering the wicked nature of the problem and not being attached to the original goals.

I also recommend *imagining* an outsider interrupting you to comment on your work - I think this can get you some of those same benefits.

### Outline-driven research

I recommend always working off of a *complete outline of what you are going to argue and how*, which has ideally been reviewed by someone else (or [your simulation of someone else](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#checkins-and-virtual-checkins)) who said "Sure, if you can defend each subpoint in the way you say you can, I'll find this valuable."

That is:

* [As soon as possible after you start learning about a topic](https://www.cold-takes.com/learning-by-writing/#my-steps), write an outline saying: "I think I can show that A seems true using the best available evidence of type X; B seems true using the best available evidence of type Y; therefore, conclusion C is true (slash the best available guess)." Don't spend lots of time in "undirected learning" mode.

* As soon as your attempt to flesh out this outline is failing, prioritize *going back to the outline*, adjusting it, getting feedback and being ready to go with a new argument. It's easy to say something like "I'm not actually confident in this point, I should investigate it" (as I did [here](https://www.cold-takes.com/the-wicked-problem-experience/#flashback-to-2007-givewell)), but I think it's better to interrupt yourself at that point; go back to the outline; redo it with the new plan; and ask whether the whole new plan looks good.

* Outlines don't need to be correct, they just need to be guesses, and they should be constantly changing. They're end-to-end plans for gathering and presenting evidence, not finished products.

### Constantly track your pace

I think it's good to consistently revisit your estimate of how quickly you're on pace to finish the project. Not how quickly you *want* to finish it or *originally said* you would finish it - how quickly it will be finished if you do all of the remaining sections at about the pace you've done the current ones.

I think a common issue is that someone looks very thoroughly into the first 2-3 subquestions that come up, without noticing that applying this thoroughness to all subquestions would put the project on pace to take years (or maybe decades?) Consistently interrupting yourself to re-estimate time remaining can be a good prompt to [re-scope the project](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#hypothesis-rearticulation).

### Don't just leave a fix for later; duct tape it now

This tip comes from [Ajeya](https://www.openphilanthropy.org/about/team/ajeya-cotra). When you reach some difficult part of the argument that you haven't thought about enough yet, it's tempting to write "[to do]" and figure you'll come back to it. But this is dangerous:

* It creates an assignment of unknown difficulty for your future self, putting them in the position of feeling obligated to fill in something they may not remember very well.

* It makes it harder to [estimate how much time is remaining](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#your-productivity-is-fine) in the project.

* It poses the risk that you'll come back to fill it in, only to realize that you can't argue the subpoint as well as you thought - meaning you need to change a bunch of other stuff you wrote that relies on it.

Instead, write down the shortest, simplest version of the point you can - focusing on [what you currently believe](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#your-take-is-valuable) rather than doing a fresh investigation. When you read the piece over again later, if you're not noticing the need for more, then you don't need to do more.

Hubris
------

### Your take is valuable

A common experience with this kind of work is the ["too-weak wrong turn"](https://www.cold-takes.com/the-wicked-problem-experience/#flashback-to-2007-givewell): you realize just how much uncertainty there is in the question you're looking into, and how little you really know about it, and how easy it would be for someone to read your end product and say things like: "So? I already knew all of this" and "There's nothing really new here" and "This isn't a definitive take, it's a bunch of guesswork on a bunch of different topics that an expert would know infinitely more about" and such.

This can be demoralizing to the point where it's hard to continue writing, especially once you've put in a lot of time and have figured out most of what you want to say, but are realizing that "what you want to say" is covered in uncertainty and caveats.

It can sometimes be tempting to try to salvage the situation by furiously doing research to produce something more thorough and impressive.

When someone (including myself) is in this type of situation, I often find myself saying the following sort of thing to them:

* "If what you've got so far were trivial and worthless, you wouldn't have felt the pull to write this piece in the first place."

* "Don't find support for what you think, just explain why you already think it."

I think it can be useful to just take "My take on this topic is valuable" as an almost axiomatic backdrop (once one's take has been developed a bit). It doesn't mean more research isn't valuable, but it can shift the attitude from "Furiously trying to find enough documentation that my take feels rigorous" to "Doing whatever extra investigation is worth the extra time, and otherwise just finishing up."

### Your productivity is fine

**Understanding deadlines.** One of the hardest things about working on wicked problems is that it's very hard to say how long a project is supposed to take. For example, in the [first year of GiveWell](https://www.cold-takes.com/the-wicked-problem-experience/#flashback-to-2007-givewell):

* We felt that we absolutely had to launch our initial product by Thanksgiving 2007. Our initial product would be our giving recommendations for our initial five causes: saving lives in Africa, global poverty (focus on Africa), US early childhood care, US education, US job opportunities.

* As we got close to the deadline, we were *both* pulling all nighters *and* cutting huge amounts of our planned content - things we had intended to write up or investigate were getting moved to [questions for further investigation](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#questions-for-further-investigation). At some point we gave up on releasing all five causes and hoped we would get one out in time.

* We got “saving lives in Africa” up on December 6, and “global poverty” sometime not too long after that.

* We hoped to get the remaining causes out in January so we could move on to other things. I believe we got them out in May or so.

The "deadline miss" didn't come from not working hard, it came from having no idea how much work was ahead of us.

Working on wicked problems means navigating:

* **Not enough deadline.** I think if one doesn't establish expectations for what will get done and by when, one will by default do everything in way too much depth and take roughly forever to finish a project - and will miss out on a lot of important pressure to do things like [cutting](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#questions-for-further-investigation) and [reframing](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#hypothesis-rearticulation) the work.

* **Too much deadline.** On the other hand, if one *does* set a "deadline," it's likely that this is based on a completely inaccurate sense of what's possible. If one then makes it a point of personal pride to hit the deadline - and sees a miss as a personal failing - this is a recipe for a shame spiral.

Early in a project, I suggest treating a deadline mostly as a "deadline to have a better deadline." Something like: "According to my wildly uninformed guess at how long things should take, I should be done by July 1; hopefully by July 1, I will be able to say something more specific, like 'I've gone through 1/3 of my subquestions, and the remaining 2/3 would take until September 1, which is too long, so I'm re-scoping the project.'"

At the point where one can *really reliably* say how much time should be remaining, I think one is usually done with the hardest part of the project.

For these sorts of "deadline to have a deadline"s, I tend to make them comically aggressive - for example, “I’m gonna start writing this tomorrow and have it done after like 30 hours of work,” while knowing that I’m actually several months from having my first draft (but that going in with the attitude “I’m basically done already, just writing it down” will speed me up a lot by making me articulate some of the key premises). So I'm both setting absurd goals for what I can accomplish, and preparing to completely let myself off the hook if I fail. Hubris.

**Understanding procrastination/incubation.** For almost anyone (and certainly for myself), working on wicked problems involves a lot of:

* Feeling "stuck."

* Not knowing what to do next - or worse, feeling like one knows what one is *supposed* to do next, but finding that the next step just feels painful or aversive or "off."

* Having a ton of trouble moving forward, and likely procrastinating, often a huge amount.

(More at my [previous piece](https://www.cold-takes.com/the-wicked-problem-experience/).)

In fact, early in the process of working on a wicked problem, I think it's often unrealistic to put in more than a few hours of solid work per day - and unhelpful to compare one's productivity to that of people doing better-defined tasks, where the goals are clear and don't change by the hour.

Working on wicked problems can often be a wild emotional rollercoaster, with lots of moments of self-loathing over being unable to focus, or missing a "deadline," or having to heavily weaken the thing one was trying to say.

It's a tough balance, because I think one really does need to pressure oneself to produce. But especially once one has completed a few projects, I think it's feasible to be simultaneously "failing to make progress" and "knowing that one is still broadly on track, because failing to make progress is part of the process." I think it's *sometimes* productive to have a certain kind of arrogance, an attitude like: "Yes, I cleared the whole day to work on this project and so far what I have done is played 9 hours of video games. But the last 5 times I did something like this, I was in a broadly similar state, and then got momentum and finished on time. I'm doing great!" The balance to strike is feeling enough urgency to move through the whole procrastinate-produce-rethink process, while having a background sense that "this is all expected and fine" that can prevent excessive personal shame and fear from the "procrastination" and "rethink" parts.

(Personally, I often draft a 15-page document by spending 4 hours failing to write the first paragraph, then 1 hour on the first paragraph, then 1 hour failing to continue, then 1 hour on the rest of the first page, then 4 hours for the remaining 14 pages. If someone tries to interrupt me during the first 4 hours, I tell them I'm working, and that's true as far as I'm concerned!)

Self-preservation
-----------------

As noted above, working on wicked problems often involves long periods of very low output, with self-imposed deadlines creeping up. This sometimes leads people to try to make up for lost time with a "heroic" effort at superhuman productivity, and to try to handle the hardest parts of a project by just working that much harder.

I'm basically totally against this. An analogy I sometimes use:

> Q: When Superman shows up to save the day and realizes his rival is loaded with kryptonite, how should he respond? What’s the best, most virtuous thing he can do in that situation?
>

A: Fly away as fast as he can, optionally shrieking in terror and letting all the onlookers say “Wow, what a coward.” This is a terrible time to be brave and soldier on! There are so many things Superman can do to be helpful - the single worst thing he can do is go where he won’t succeed.[2](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#fn2)

If the project is taking "too long," it might be because it was impossible to set a "schedule" for in the first place, and trying to finish it off at a superhuman pace could easily just leave you exhausted, demoralized and still not close to done. Additionally, the next task sometimes seems "scary" because it is actually a bad idea and needs to be rethought.

I generally advise people working on wicked problems to aim for "jogging" rather than "sprinting" - a metaphor I like because it emphasizes that this is fully consistent with trying to finish as fast as possible. In particular, I prefer the goal of "Make at least a bit of progress on 95% of days, and in 100% of weeks" to the goal of "Make so much progress today that it makes up for all my wasted past days." (The former goal is not easy! I think aiming for it requires a lot of [interrupting oneself](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#impatience) to make sure one isn't spiraling or going down an unproductive rabbit hole - rather than a lot of "trying to pedal to the metal," which can run right into those problems.)

### It’s a bird, it’s a plane, it’s a schmoe!

This section is particularly targeted at [effective altruists](https://en.wikipedia.org/wiki/Effective_altruism) who feel compelled to squeeze every ounce of productivity out of themselves that they can, for moral reasons and not just personal pride. I think this attitude is dangerous, because of the way it leads people to set unrealistic and unsustainable expectations for themselves.

My take: "Whenever you catch yourself planning on being a hero, just stop. If we’re going to save the world, we’re going to do it by being schmoes." That is:

* Plan on being about as focused, productive, and virtuous as people doing similar work on other topics.

* Plan on working a normal number of hours each day, plan on often getting distracted and mucking around, plan on taking as much vacation as other high-productivity people (a lot), plan on having as much going on outside of work as other high-productivity people (a lot), etc.

* (This is also a standard to hold oneself to - try not to lose productivity over things, like guilt spirals, that other people doing similar work often don't suffer from.)

If [effective altruists](https://en.wikipedia.org/wiki/Effective_altruism) are going to have outsized impact on the world, I think it will be mostly thanks to the unusual questions they’re asking and the unusual goals they’re interested in, not unusual dedication/productivity/virtue. I model myself as “Basically like a hedge fund guy but doing more valuable stuff,” not as “A being capable of exceptional output or exceptional sacrifice.”

Be virtuous first!
------------------

I don't think you're going to get very far with these "vices" alone. If you aren't balancing them with the *virtues* of open-mindedness, self-criticality, and doing the hard work to understand things, it's easy to just lazily write down some belief you have, cite a bit of evidence that you haven't looked at carefully or considered the best counter-arguments to, and hit "publish." I think this is what the vast majority of people "investigating" important questions are doing, and if I were writing tips for the average person in the world, I'd have a very different emphasis.

For forming opinions and writing useful pieces about important topics, I think the first hurdle to clear is being determined to examine the strongest parts of both sides of an argument, understand them in detail (and with [minimal trust](https://www.cold-takes.com/minimal-trust-investigations/)), and write what you're finding with [reasoning transparency](https://www.openphilanthropy.org/reasoning-transparency). (All of this is much easier said than done.) But in my experience, many of the people who are strongest in these "virtues" veer too far in the virtuous direction and end up punishing themselves for missing unrealistic self-imposed deadlines on impossible self-imposed assignments. This piece has tried to give a feel for when and how to pull back, skip steps, and go easy on oneself, to make incremental progress on intimidating questions.

Footnotes
---------

---

1. In practice, for a report that isn't *claiming* much rigor, this often means demonstrating “This isn’t even suggestive, it’s basically noise.” [Here's a long debate about exactly that question](https://astralcodexten.substack.com/p/biological-anchors-a-trick-that-might?s=r) for one of the key inputs into my views on transformative AI! [↩](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#fnref1)- My favorite real-life example of this is [Barry Bonds in 2002](https://www.baseball-reference.com/players/b/bondsba01.shtml). So many star players try to play through injuries all year long, and frame this as being a "team player." I remember Barry Bonds in 2002 taking all kinds of heat for the fact that he would sit out whenever he got even moderately injured, and would sometimes sit out games just because he felt kinda tired. But then the playoffs came around and he played every game and was out-of-this-world good, in a season that came down to the final game, at age 38. Who's the team player? [↩](https://www.cold-takes.com/p/bbfdc0ca-cb1c-4972-904a-2459a3af3c65/#fnref2)


Ideal governance (for companies, countries and more)

*Click lower right to download or find on Apple Podcasts, Spotify, Stitcher, etc.*

I'm interested in the topic of **ideal governance:** what kind of governance system should you set up, if you're starting from scratch and can do it however you want?

Here "you" could be a company, a nonprofit, an informal association, or a country. And "governance system" means a Constitution, charter, and/or bylaws answering questions like: "Who has the authority to make decisions (Congress, board of directors, etc.), and how are they selected, and what rules do they have to follow, and what's the process for changing those rules?"

I think this is a very different topic from something like "How does the US's Presidential system compare to the Parliamentary systems common in Europe?" The idea is not to look at today's most common systems and compare them, but rather to generate options for setting up systems radically different from what's common today.

I don't currently know of much literature on this topic (aside from the literature on [social choice theory](https://en.wikipedia.org/wiki/Social_choice_theory) and especially [voting methods](https://plato.stanford.edu/entries/voting-methods/), which covers only part of the topic). This post **describes [the general topic](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#common-governance-structures-today) and [why I care](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#why-do-i-care), partly in the hopes that people can point me to any literature I've missed.** Whether or not I end up finding any, I'm likely to write more on this topic in the future.

Outline of the rest of the piece:

* I'll outline some common governance structures for countries and major organizations today, and highlight **how much room there is to try different things** that don't seem to be in wide use today. [More](www.cold-takes.com/ideal-governance-for-companies-countries-and-more/#common-governance-structures-today)* I'll discuss **why I care about this question.**  I have a few very different reasons:
	+ *A short-term, tangible need:* over the last several years, I've spoken with several (more than 3) organizations that feel no traditional corporate governance structure is satisfactory, because the stakes of their business are too great and society-wide for shareholder control to make sense, yet they are too early-stage and niche (and in need of nimbleness) to be structured like a traditional government. An example would be an artificial intelligence company that could end up with a normal commercial product, or could end up bringing about the [most important century of all time for humanity](https://www.cold-takes.com/most-important-century/). I wish I could point them to someone who was like: "I've read all of, and written much of, the literature on what your options are. I can walk you through the pros and cons and help you pick a governance system that balances them for your needs."

	+ *A small probability of a big future win.* The world today has lots of governments, but they seem to mostly follow a very small number of basic governance templates. At some point, there will be new states with new Constitutions - maybe via space settlements, maybe via collapse of existing states, etc. - but I expect these moments to be few and far between. A significant literature and set of experts on "ideal governance" could lead to a radically different kind of state government, potentially with radically different policies that the rest of the world could learn from.

	+ *A weird, out-of-left-field application.* Some of my interest in this topic actually comes via my interest in [moral uncertainty](https://80000hours.org/articles/moral-uncertainty/): the question of what it's ethical to do when one is *struggling between more than one theory of ethics,* with radically different implications. This is hard to explain, but I try [below.](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#MoralUncertainty)* I'll describe a bit more what I think **literature on this question could look like** (and what already exists that I know of), partly to guide readers who might be able to help me find more.

Common governance structures today
----------------------------------

All of these are simplified; I'm trying to illustrate the basic idea of what questions "ideal governance" is asking.

* A standard (e.g., public) corporation works like this: it has shareholders, assigned one vote per share (not per person), who elect a board of directors that governs by majority. The board generally appoints a CEO that it entrusts with day-to-day decisions. There is a "constitution" of sorts (the Articles of Incorporation and bylaws) and a lot more wrinkles in terms of how directors are selected, but that's the basic idea.

* A standard nonprofit is like a corporation, but entirely lacking the shareholder layer - it's governed directly by the board of directors. (I find something weird about a structure this simple - a simple board majority can do literally anything, even though the board of directors is often a somewhat random assortment of donors, advisors, etc.)

* The US federal government is a lot more complex. It splits authority between the House of Representatives, the Senate, the Presidency and the Supreme Court, all of which have specific appointment procedures, term limits, etc. and are meta-governed by a Constitution that requires special measures to change. There are lots of specific choices that were made in designing things this way, and lots of things that could've been set up differently in the 18th century that would probably still matter today.

* Other democracies tend to have governments that differ in a lot of ways ([e.g.](https://borgenproject.org/parliamentary-system-versus-presidential-system/)), while being based on broadly similar principles: voters elect representatives to more than one branch of government, which then divide up (and often can veto each other on) laws, expenditures, etc.

* When I was 13, the lunch table I sat at established a Constitution with some really strange properties that I can't remember. I think there was a near-dictatorial authority who rotated daily, with others able to veto their decisions by assembling supermajorities or maybe singing silly songs or something.

![](https://www.cold-takes.com/content/images/2022/04/governance-diagrams.png)

In addition to the design choices shown in the diagrams, there are a lot of others:

* Who votes, how often, and what voting system is used?

* How many representatives are there in each representative body? How are they divided up (one representative per geographic area, or [party-list proportional representation](https://en.wikipedia.org/wiki/Proportional_representation#How_party-list_PR_works), or something else)?

* What term limits exist for the different entities?

* Do particular kinds of decisions require supermajorities?

* Which restrictions are enshrined in a hard-to-change Constitution (and how hard is it to change), vs. being left to the people in power at the moment?

One way of thinking about the "ideal governance" question is: **what kinds of designs could exist that aren't common today? And how should a new organization/country/etc. think about what design is going to be best for its purposes, beyond "doing what's usually done"?**

For any new institution, it seems like the stakes are potentially high - in some important sense, picking a governance system is a "one-time thing" (any further changes have to be made using the rules of the existing system[1](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fn1)).

Perhaps because of this, there doesn't seem to be much use of innovative governance designs in high-stakes settings. For example, here are a number of ideas I've seen floating around that seem cool and interesting, and ought to be considered if someone could set up a governance system however they wanted:

* **Sortition,** or choosing people randomly to have certain powers and responsibilities. An extreme version could be: "Instead of everyone voting for President, randomly select 1000 Americans; give them several months to consider their choice, perhaps paid so they can do so full-time; then have them vote."
	+ The idea is to pick a subset of people who are both (a) representative of the larger population (hence the randomness); (b) will have a stronger case for putting serious time and thought into their decisions (hence the small number).

	+ It's solving a similar problem that "representative democracy" (voters elect representatives) is trying to solve, but in a different way.* **Proportional** **decision-making.** Currently, if Congress is deciding how to spend $1 trillion, a coalition controlling 51% of the votes can control all $1 trillion, whereas a coalition controlling 49% of the votes controls $0. Proportional decision-making could be implemented as "Each representative controls an equal proportion of the spending," so a coalition with 20% of the votes controls 20% of the budget. It's less clear how to apply this idea to other sorts of bills (e.g., illegalizing an activity rather than spending money), but there are plenty of possibilities.[2](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fn2)* **[Quadratic voting](https://www.radicalxchange.org/concepts/quadratic-voting/)**, in which people vote on multiple things at once, and can cast more votes for things they care about more (with a "quadratic pricing rule" intended to make the number of votes an "honest signal" of how much someone cares).

* **Reset/Jubilee:** maybe it would be good for some organizations to periodically redo their governance mostly from scratch, subject only to the most basic principles. Constitutions could contain a provision like "Every N years, there shall be a new Constitution selected. The 10 candidate Constitutions with the most signatures shall be presented on a ballot; the Constitution receiving the most votes is the new Constitution, except that it may not contradict or nullify this provision. This provision can be prevented from occurring by [supermajority provision], and removed entirely by [stronger supermajority]."

* More examples in a footnote.[3](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fn3)

If we were starting a country or company from scratch, which of the above ideas should we integrate with more traditional structures, and how, and what else should we have in our toolbox? That's the question of ideal governance.

Why do I care?
--------------

I have one "short-term, tangible need" reason; one "small probability of a big future win" reason; and one "weird, out-of-left-field" reason.

**A short-term, tangible need: companies developing AI, or otherwise aiming to be working with huge stakes.** Say you're starting a new company for developing AI systems, and you believe that you could end up building [AI with the potential to change the world forever](https://www.cold-takes.com/most-important-century/).

* The standard governance setup for a corporation would hand power over all the decisions you're going to make to your shareholders, and by default most of your shares are going to end up held by people and firms that invested money in your company. Hopefully it's clear why this doesn't seem like the ideal setup for a company whose decisions could be world-changing. A number of AI companies have acknowledged the basic point that "Our ultimate mission should NOT just be: make money for shareholders," and that seems like a good thing.

* One alternative would be to set up like a nonprofit instead, with all power vested in a board of directors (no shareholder control). Some issues are that (a) this cuts shareholders out of the loop completely, which could make it pretty hard to raise money; (b) according to me at least, this is just a weird system of governance, for reasons that are not super easy to articulate concisely but I'll take a shot in a footnote[4](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fn4) (and possibly write more in the future).

* Another alternative is a setup that is somewhat common among tech companies: 1-2 founders hold enough shares to keep control forever, so you end up with essentially a dictatorship. This also ... leaves something to be desired.

* Or maybe a company like this should just set up more like a government from the get-go, offering everyone in the world a vote via some complex system of representation, checks and balances. But this seems poorly suited to at least the relatively early days of a company, when it's small and its work is not widely known or understood. But then, how does the company handle the transition from the latter to the former? And should the former be done exactly in the standard way, or is there room for innovation there?

Over the last several years, I've spoken with heads of several (more than 3) organizations that struggle between options like the above, and have at least strongly considered unusual governance setups. I wish I could point them to someone who was like: "I've read all of, and written much of, the literature on what your options are. I can walk you through the pros and cons and help you pick a governance system that balances them for your needs."

But right now, I can't, and I've seen a fair amount of this instead: "Let's just throw together the best system we can, based mostly on what's already common but with a few wrinkles, and hope that we figure this all out later." I think this is the right solution given how things stand, but I think it really does get continually harder to redesign one's governance as time goes on and more [stakeholders](https://www.cold-takes.com/empowerment-and-stakeholder-management/) enter the picture, so it makes me nervous.

Similar issues could apply to mega-corporations (e.g., [FAANG](https://www.investopedia.com/terms/f/faang-stocks.asp)) that are arguably more powerful than what the standard shareholder-centric company setup was designed for. Are there governance systems they could adopt that would make them more broadly accountable, without copying over all the pros and cons of full-blown representative democracy as implemented by countries like the US?

**A small probability of a big future win: future new states.** The world today has lots of governments, but they seem to mostly follow a very small number of basic governance templates (e.g., I believe you see almost none of [the things I listed above](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#AssortedIdeas)), and probably relatedly, there seems to be remarkably little variety and experimentation with policy. Policies that many believe could be huge wins - such as [dramatically expanded immigration](https://smile.amazon.com/dp/B07YRKYKZ3/ref=dp-kindle-redirect?_encoding=UTF8&btkr=1), [land value taxation](https://en.wikipedia.org/wiki/Land_value_tax), "consumer reports"-style medical approvals,[5](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fn5) [drug decriminalization](https://michaelpollan.com/books/how-to-change-your-mind/), and [charter cities](https://chartercitiesinstitute.org/intro/) - don't seem to have gotten much of a trial anywhere in the world.

At some point, there will be new states with new Constitutions - maybe via space settlements, maybe via collapse of existing states, etc. - but I expect these moments to be few and far between.

By default I expect future Constitutions to resemble present ones an awful lot. But maybe, at some future date, there will be a large "ideal governance" literature and some points of expert consensus on innovative governance designs that somebody really ought to try. That could lead to a radically different kind of state government, potentially with radically different policies that the rest of the world could learn from.

**An out-of-left-field application for "ideal governance."** This is going to veer off the rails, so remember to skip to the next section if I lose you.

Some of my interest in this topic actually comes via my interest in [moral uncertainty](https://80000hours.org/articles/moral-uncertainty/): the question of what it's ethical to do when one is *struggling between more than one theory of ethics,* with radically different implications.

For example, there are arguments that our [ethical decisions should be dominated by concern for ensuring that as many people as possible will someday get to exist](https://www.cold-takes.com/debating-myself-on-whether-extra-lives-lived-are-as-good-as-deaths-prevented/). I really go back and forth on how much I buy these arguments, but I'm definitely somewhere between 10% convinced and 50% convinced. So ... say I'm "20% convinced" of some view that says preventing human extinction[6](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fn6) is the [overwhelmingly](https://www.cold-takes.com/defending-one-dimensional-ethics/) most important consideration for at least some dimensions of ethics (like where to donate), and "80% convinced" of some more common-sense view that says I should focus on some cause unrelated to human extinction.[7](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fn7) How do I put those two together and decide what this means for actual choices I'm making?

The closest thing I've seen to a reasonable-seeming answer is the idea of a [moral parliament](https://www.overcomingbias.com/2009/01/moral-uncertainty-towards-a-solution.html): I should act as though I'm run by a Parliament with 80 members who believe in "common-sense" ethics, and 20 members who believe in the "preventing extinction is overwhelmingly important" idea. But with default Parliament rules, this would just mean the 80 members can run the whole show, without any compromise with the 20.

And so, a [paper on the "moral parliament" idea](https://www.fhi.ox.ac.uk/wp-content/uploads/2021/06/Parliamentary-Approach-to-Moral-Uncertainty.pdf) tries to make it work by ... introducing a completely new governance mechanism that I can't find any other sign of someone else ever talking about, "proportional chances voting" (spelled out in a footnote).[8](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fn8) I think this mechanism has its own issues,[9](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fn9) but it's an attempt to ensure something like "A coalition controlling 20% of the votes has 20% of the effective power, and has to be compromised with, instead of being subject to the tyranny of the majority."

My own view (which I expect to write more about in the future) is that governance is roughly the right metaphor for "moral uncertainty": I am torn by [multiple different sides of myself](https://www.cold-takes.com/defending-one-dimensional-ethics/), with different takes on what it means to be a good person, and the problem of getting these different sides of myself to reach a decision together is like the problem of getting different citizens (or shareholders) to reach a decision together. The more we can say about what ideal governance looks like, the more we can say about how this ought to work - and the better I expect this "moral parliament"-type idea to end up looking, compared to alternatives.[10](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fn10)

The literature I'm looking for
------------------------------

Ideal governance seems like the sort of topic for which there should be a "field" of "experts," studying it. What would such study look like? Three major categories come to mind:

**Brainstorming** ideas such as [those I listed above](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#AssortedIdeas) - innovative potential ways of solving classic challenges of governance, such as reconciling "We want to represent all the voters" with "We want decisions to be grounded in expertise and high engagement, and voters are often non-expert and not engaged."

I've come across various assorted ideas in this category, including [quadratic voting](https://www.radicalxchange.org/concepts/quadratic-voting/), [futarchy](https://en.wikipedia.org/wiki/Futarchy), and [proportional chances voting](https://www.fhi.ox.ac.uk/wp-content/uploads/2021/06/Parliamentary-Approach-to-Moral-Uncertainty.pdf), without seeing much sign that these sit within a broader field that I can skim through to find all the ideas that are out there.

**Economics-style theory** in which one asks questions like: "If we make particular assumptions about who's voting, what information they have and lack, how much they suffer from [bounded rationality](https://en.wikipedia.org/wiki/Bounded_rationality), and how we define 'serving their interests' (see below), what kind of governance structure gets the best outcome?"

[Social choice theory](https://en.wikipedia.org/wiki/Social_choice_theory), including on [voting methods](https://plato.stanford.edu/entries/voting-methods/), tackles the "how we define 'serving their interests'" part of this. But I'm not aware of people using similar approaches to ask questions like "Under what conditions would we want 1 chamber of Congress vs. 2, or 10? 100 Senators vs. 500, or 15? A constitution that can be modified by simple majority, vs. 2/3 majority vs. consensus? Term limits? Etc. etc. etc."

**Empirical research (probably qualitative):** Are there systematic reviews of unusual governance structures tried out by companies, and what the results have been? Of smaller-scale experiments at co-ops, group houses and lunch tables?

To be clear, I think the most useful version of this sort of research would probably be very qualitative - collecting reports of what problems did and didn't come up - rather than asking questions like "How does a particular board structure element affect company profits?"

One of the things I expect to be tricky about this sort of research is that I think a lot of governance comes down to things like "What sorts of people are in charge?" and "What are the culture, expectations, norms and habits?" A setup that is "officially" supposed to work one way could evolve into something quite different via informal practices and "soft power." However, I think the formal setup (including things like "what the constitution says about the principles each governance body is supposed to be upholding") can have big effects on how the "soft power" works.

If you know where to find research or experts along the lines of the above, please share them [in the comments](https://www.lesswrong.com/posts/slug/ideal-governance-for-companies-countries-and-more#comments) or [using this form](https://forms.gle/J4yCHcRL8VAC6mKR6) if you don't want them to be public.

I'll likely write about what I come across, and if I don't find anything new, I'll likely ramble some more about ideal governance. So either way, there will be more on this topic!

---

Footnotes
---------

1. Barring violent revolution in the case of countries. [↩](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fnref1)
- An example would be the "proportional chances voting" idea described [here](https://www.fhi.ox.ac.uk/wp-content/uploads/2021/06/Parliamentary-Approach-to-Moral-Uncertainty.pdf). [↩](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fnref2)
- * **Proxying/[liquid democracy](https://en.wikipedia.org/wiki/Liquid_democracy)**, or allowing voters to transfer their votes to other voters. (This is common for corporations, but not for governments.) This could be an alternative or complement to electing representatives, solving a similar problem (we want lightly-engaged voters to be represented, but we also want decisions ultimately made using heavy engagement and expertise). At first glance it may seem to pose a risk that people will be able to "buy votes," but I don't actually think this is necessarily an issue (proxying could be done anonymously and on set schedules, like other votes).

	* **Soft term limits:** the more terms someone has served, the greater a supermajority they need to be re-elected. This could be used to strike a balance between the advantages of term limits (avoiding "effectively unaccountable" incumbents) and no-term-limits (allowing great representatives to keep serving).

	* **Formal technocracy/meritocracy**: Using hard structures (rather than soft norms) to assign authority to people with particular expertise and qualifications. An extreme example would be [futarchy](https://en.wikipedia.org/wiki/Futarchy), in which prediction markets directly control decisions. A simpler example would be structurally rewarding representatives (via more votes or other powers) based on assessments of their track records (of predictions or decisions), or factual understanding of a subject. This seems like a tough road to go down by default, as any mechanism for evaluating "track records" and "understanding" can itself be politicized, but there's a wide space of possible designs.  [↩](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fnref3)- Most systems of government have a sort of funnel from "least engaged in day to day decisions, but most ultimately legitimate representatives of whom the institution is supposed to serve" (shareholders, voters) to "most engaged in day to day decisions, but ultimately accountable to someone else" (chief executive). A nonprofit structure is a very short funnel, and the board of directors tends to be a somewhat random assortment of funders, advisors, people who the founders just thought were cool, etc. I think they often end up not very accountable (to anyone) or engaged in what's going on, such that they have a hard time acting when they ought to, and the actions they do take are often kind of random.

 I'm not saying there is a clearly better structure available for this purpose - I think the weirdness comes from the fact that it's so unclear who should go in the box normally reserved for "Shareholders" or "Voters." It's probably the best common structure for its purpose, but I think there's a lot of room for improvement, and the stakes seem high for certain organizations. [↩](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fnref4)- Context in [this Marginal Revolution post](https://marginalrevolution.com/marginalrevolution/2015/08/is-the-fda-too-conservative-or-too-aggressive.html), which links to [this 2005 piece on a "consumer reports" model for the FDA](https://www.independent.org/news/article.asp?id=1500). [↩](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fnref5)- Or "existential catastrophe" - something that drastically curtails humanity's future, even if it doesn't drive us extinct. [↩](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fnref6)- This isn't actually where I'm at, because I think the leading existential risks are a big enough deal that I would want to focus on them even if I completely ignored the philosophical argument that the future is overwhelmingly important. [↩](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fnref7)- Let's say that 70% of the Parliament members vote for bill X, and 30% vote against. "Proportional chance voting" literally uses a weighted lottery to pass bill X with 70% probability, and reject it with 30% probability (you can think of this like rolling a 10-sided die, and passing the bill if it's 7 or under).

 A key part of this is that the members are supposed to negotiate before voting and holding the lottery. For example, maybe 10 of the 30 members who are against bill X offer to switch to supporting it if some change is made. The nice property here is that rather than having a "tyranny of the majority" where the minority has no bargaining power, we have a situation where the 70-member coalition would still love to make a deal with folks in the minority, to further increase the probability that they get their way.

 Quote from the paper that I am interpreting: "Under proportional chances voting, each delegate receives a single vote on each motion. Before they vote, there is a period during which delegates may negotiate: this could include trading votes on one motion for votes on another, introducing novel options for consideration within a given motion, or forming deals with others to vote for a compromise option that both consider to be acceptable. The delegates then cast their ballots for one particular option in each motion, just as they might in a plurality voting system. But rather than determining the winning option to be the one with the most votes, each option is given a chance of winning proportional to its share of the votes." [↩](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fnref8)- What stops someone who lost the randomized draw from just asking to hold the same vote again? Or asking to hold a highly similar/related vote that would get back a lot of what they lost? How does that affect the negotiated equilibrium? [↩](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fnref9)- Such as "[maximize expected choice-worthiness](https://static1.squarespace.com/static/5506078de4b02d88372eee4e/t/5bc7224a0852299b5cd60e86/1539777103255/Why+Maximize+Expected+Choice-Worthiness%3F.pdf)," which I am not a fan of for reasons I'll get to in the future. [↩](https://www.cold-takes.com/p/7a67e036-1684-43bc-ab2b-5e997783f125/#fnref10)


Debating myself on whether “extra lives lived” are as good as “deaths prevented”

*Click lower right to download or find on Apple Podcasts, Spotify, Stitcher, etc.*

If we could do something to lower the probability of the human race going extinct,[1](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fn1) that would be really good. But **how** good? Is preventing extinction more like “saving 8 billion lives” (the number of people alive today), or “saving 80 billion lives” (the number who will be alive over the next 10 generations) … or "saving 625 quadrillion lives" (an [Our World in Data estimate](https://ourworldindata.org/longtermism) of the number of people who could ever be born) ... or “saving a comically huge number of lives" ([Nick Bostrom](https://www.nickbostrom.com/astronomical/waste.html) argues for well over 10^46 as the total number of people, including [digital people](https://www.cold-takes.com/how-digital-people-could-change-the-world/), who could ever exist)?

More specifically, is “a person getting to live a good life, when they otherwise would have never existed” the kind of thing we should value? Is it as good as “a premature death prevented?”

Among [effective altruists](https://en.wikipedia.org/wiki/Effective_altruism), it’s common to answer: “Yes, it is; preventing extinction is somewhere around as good as saving [some crazy number] of lives; so if there’s any way to reduce the odds of extinction by even a tiny amount, that’s where we should focus all the attention and resources we can.”

I feel conflicted about this.

* I am sold on the importance of specific, potentially extinction-level risks, such as risks from [advanced AI](https://www.cold-takes.com/most-important-century/). But: this is *mostly* because I think the risks are really, really big and really, really neglected. I think they’d be worth focusing on even if we *ignored* arguments like the above and used much more modest estimates of “how many people might be affected.”

* I’m less sold that we should work on these risks if they were very small. And I have very mixed feelings about the idea that “a person getting to live a good life, when they otherwise would have never existed” is as good as “a premature death prevented.”

Reflecting these mixed feelings, I'm going to examine the philosophical case for caring about “extra lives lived” (putting aside the first bullet point above), via a **dialogue between two versions of myself: Utilitarian Holden (UH) and Non-Utilitarian Holden (NUH).**[2](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fn2)

This represents actual dialogues I’ve had with myself (so neither side is a pure straw person), although this particular dialogue serves primarily to illustrate UH's views and how they are defended against initial and/or basic objections from NUH. In future dialogues, NUH will raise more sophisticated objections.

This is part of a set of dialogues on [future-proof ethics](https://www.cold-takes.com/future-proof-ethics/): trying to make ethical decisions that we can remain proud of in the future, after a great deal of (societal and/or personal) moral progress. (Previous dialogue [here](https://www.cold-takes.com/defending-one-dimensional-ethics/), though this one stands on its own.)

A couple more notes before I finally get started:

* The genre here is philosophy, and a common type of argument is the thought experiment: "If you had to choose between A and B, what would you choose?" (For example: "is it better to prevent one untimely death, or to allow 10 people to live who would otherwise never have been born?")

* It's common to react to questions like this with comments like "I don't really think that kind of choice comes up in real life; actually you can usually get both A and B if you do things right" or "actually A isn't possible; the underlying assumptions about how the world really works are off here." My general advice when considering philosophy is to avoid reactions like this and think about what you would do if you *really had to make the choice* that is being pointed at, even if you think thI ae author's underlying assumptions about why the choice exists are wrong. Similarly, if you find one part of an argument unconvincing, I suggest pretending you accept it for the rest of the piece anyway, to see whether the rest of the arguments *would* be compelling under that assumption.

* I often give an example of how one could face a choice between A and B in real life, to make it easier to imagine - but it's not feasible to give this example in enough detail and with enough defense to make it seem realistic to all readers, without a big distraction from the topic at hand.

* Philosophy requires some amount of suspending disbelief, because the goal is to ask questions about (for example) what you *value*, while isolating them from questions about what you *believe.* (For more on how it can be useful to separate values and beliefs, see [Bayesian Mindset](https://www.cold-takes.com/the-bayesian-mindset/).)

Dialogue on “extra lives lived”
-------------------------------

To keep it clear who's talking when, I'm using **-UH-** for "Utilitarian Holden" and **-NUH-** for "non-Utilitarian Holden." (In the audio version of this piece, my wife voices NUH.)

Let’s start here:

* Let's say that if humanity can avoid going extinct (and perhaps [spread across the galaxy](https://www.cold-takes.com/most-important-century/)), the number of people who will ever exist is about one quintillion, also known as "a billion billion" or "10^18." (This is close[3](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fn3) to the [Our World in Data estimate](https://ourworldindata.org/longtermism) of the number of people who could ever live; it ignores the possibility of [digital people](https://www.cold-takes.com/how-digital-people-could-change-the-world/), which could lead to a [vastly bigger](https://www.nickbostrom.com/astronomical/waste.html) number.)

* If you think that’s 1% likely to actually happen, it’s an [expected value](https://www.cold-takes.com/expected-value/) of 10^16 people, which is still huge enough to carry the rest of what I'm going to say.

* You can think of it like this: imagine all of the people who exist or ever could exist, all standing in one place. There are about 10^10 such people alive today, and the rest are just “potential people” - that means that more than 99.999% of all the people are “potential people.”

* And now imagine that we’re all talking about where you, the person in the privileged position of being [one of the earliest people ever](https://www.cold-takes.com/all-possible-views-about-humanitys-future-are-wild/), should focus your efforts to help others. And you say: “Gosh, I’m really torn:
	+ I think if I focus my efforts on today’s world, maybe I could help prevent up to 10,000 untimely deaths. (This would be a very high thing to aim for.)

	+ Or, I could cause a tiny decrease in extinction risk, like 1% of 1% of 1% of 1%. That would help about 100 million of you.[4](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fn4) What should I do?”

	+ (Let’s ignore the fact that helping people in today’s world could also reduce extinction risk. It could, but it could also increase extinction risk - who knows. There are probably actions better targeted at reducing extinction risk than at helping today’s world, is the point.)

	In that situation, I think everyone would be saying: “How is this a question? Even if your impact on extinction risk is small, even if it’s uncertain and fuzzy, there are just SO MANY MORE people affected by that. If you choose to focus on today’s world, you’re essentially saying that you think today’s people count more than 10,000 times as much as future people.

“Now granted, most of the people alive in your time DO act that way - they ignore the future. But someday, if society becomes morally wiser, that will look unacceptable; similarly, if you become morally wiser, you'll regret it. It's basically deciding that 99.999%+ of one’s fellow humans aren’t worth worrying about, just because they don’t exist yet.

“Do the forward-looking thing, the [future-proof](https://www.cold-takes.com/future-proof-ethics/) thing. Focus on helping the massive number of people who don’t exist yet.”

I feel like you are skipping a very big step here. We’re talking about what *potential people who don’t exist yet* would say about *giving them a chance to exist?* Does that even make sense?

That is: it sounds like you’re counting every “potential person” as someone whose wishes we should be respecting, including their wish to exist instead of not exist. So among other things, that means a larger population is better?

 Yes.

 I mean, that’s super weird, right? Like is it ethically obligatory to have as many children as you can?

It’s not, for a bunch of reasons.

The biggest one for now is that we’re focused on [thin utilitarianism](https://www.cold-takes.com/future-proof-ethics/#thin-utilitarianism) - how to make choices about actions like donating and career choice, not how to make choices about everything. For questions like how many children to have, I think there’s much more scope for a multidimensional morality that isn’t all about [respecting the interests of others](https://www.cold-takes.com/future-proof-ethics/#thin-utilitarianism).

I also generally think we’re liable to get confused if we’re talking about reproductive decisions, since reproductive autonomy is such an important value and one that has historically been undermined at times in ugly ways. My views here aren’t about reproductive decisions, they’re about avoiding existential catastrophes. Longtermists (people who focus on the long-run future, as I’m advocating here) tend to focus on things that could affect the *ultimate, long-run* population of the world, and it’s really unclear how having children or not affects that (because the main factors behind the ultimate, long-run population of the world have more to do with things like the odds of extinction and of [explosive civilization-wide changes](https://www.cold-takes.com/most-important-century/), and it's unclear how having children affects those).

So let’s instead stay focused on the question I asked. That is: if you prevent an existential catastrophe, so that there’s a large flourishing future population, does each of those future people count as a “beneficiary” of what you did, such that their benefits aggregate up to a very large number?

OK. I say no, such “potential future people” do not count. And I’m not moved by your story about how this may one day look cruel or inconsiderate. It’s not that I think some types of people are less valuable than others, it’s that I don’t think *increasing the odds that someone ever exists at all* is benefiting them.

Let’s briefly walk through a few challenges to your position. You can learn more about these challenges from the academic population ethics literature; I recommend [Hilary Greaves’s short piece on this](http://users.ox.ac.uk/~mert2255/papers/population_axiology.pdf).

### Challenge 1: Future people and the “mere addition paradox”

So you say you don’t see “potential future people” as “beneficiaries” whose interests count. But let’s say that the worst effects of climate change won’t be felt for another 80 years or so, in which case the vast majority of people affected will be people who aren’t alive today. Do you discount those folks and their interests?

No, but that’s different. Climate change isn’t about whether they get to exist or not, it’s about whether their lives go better or worse.

Well, it’s about both. The world in which we contain/prevent/mitigate climate change contains completely *different people* in the future from the world in which we don’t. Any difference between two worlds will ripple chaotically and affect things like which sperm fertilize which eggs, which will completely change the future people that exist.

So you really can’t point to some fixed set of people that is “affected” by climate change. Your desire to mitigate climate change is really about causing there to be *better off* people in the future, instead of *completely different worse off* people. It’s pretty hard to maintain this position while also saying that you only care about “actual” rather than “potential” people, or “present” rather than “future” ones.

I can still take the position that:

* If there are a certain number of people, it’s good to do something (such as mitigate climate change) that causes there to be better-off people instead of worse-off people.

* But that doesn’t mean that it’s good for there to be *more* people than there would be otherwise. Adding more people is just neutral, assuming they have reasonably good lives.

That’s going to be a tough position to maintain.

Consider three possible worlds:

* **World A:** 5 billion future people have good lives. Let’s say their lives are a 8/10 on some relevant scale (reducing the quality of a life to a number is a simplification; see footnote for a bit more on this[5](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fn5)).

* **World B:** 5 billion future people have slightly *better* than good lives, let’s say 8.1/10. And there are an additional 5 billion people who have not-as-good-but-still-pretty-good lives, let’s say 7/10.

* **World C:** 10 billion future people have good lives, 8/10.

![](https://i.imgur.com/4cL8Ctv.png)

My guess is that you think World B seems clearly better than World A - there are 5 billion “better-off instead of worse-off” future people, and the added 5 billion people seem *neutral* (not good, not bad).

But I’d also guess you think World C seems clearly better than World B. The change is a small worsening in quality of life for the better-off half of the population, and a large improvement for the worse-off half.

But if World C is better than World B and World B is better than World A, doesn’t that mean World C is better than World A? And World C is the same as World A, just a bigger population.

I admit my intuitions are as you say. I prefer B when comparing it to A, and C when comparing it to B. However, when I look at C vs. A, I’m not sure what to think. Maybe there is a mistake somewhere - for example, maybe I should think that it’s *bad* for additional people to come to exist.

That would imply that the human race going extinct would be *great*, no? Extinction would prevent massive numbers of people from ever existing.

That is definitely not where I am.

OK, you’ve successfully got me puzzled about what’s going on in my brain. Before I try to process it, how about you confuse me more?

### Challenge 2: Asymmetry

Sure thing. Let’s talk about another problem with the attempt to be “neutral” on whether there are more or fewer people in the future.

Say that you can take some action to prevent a horrible dystopia from arising in a distant corner of the galaxy. In this dystopia, the vast majority of people will wish they didn’t exist, but they won’t have that choice. You have the opportunity to ensure that, instead of this dystopia, there will simply be nothing there. Does that opportunity seem valuable?

It does, enormously valuable.

OK. The broader intuition here is that *preventing* lives that are *worse than nonexistence* has high ethical value - does that seem right?

Yes.

Now you’re in a state where you think preventing bad lives is good, but preventing good lives is neutral.

But the thing is, every time a life comes into existence, there’s some *risk* it will be really bad (such that the person living it wishes they didn’t exist). So if you count the bad as bad and the good as neutral, you should think that each future life is purely a bad thing - some chance it’s bad, some chance it’s neutral. So you should want to *minimize* future lives.

Or at the civilization level: say that if humanity continues existing, there’s a 99% chance we will have an enormous (at least 10^18 people) flourishing civilization, and a 1% chance we’ll end up in an equally enormous, horrible dystopia. And even the flourishing civilization will have *some* people in it who wish they didn’t exist. Confronting this possibility, you should hope that humanity doesn’t continue existing, since then there won’t be any of these “people who wish they didn’t exist.” You should, again, think that extinction is a great ethical good.

Yikes. Like I’ve said, I don’t think that.

In that case, I think the most natural way out of this is to conclude that a huge flourishing civilization would be good enough to compensate - at least partly - for the risk of a huge dystopia.

That is: if you’re fine with a 99% chance of a flourishing civilization and a 1% chance of a dystopia, this implies that *a flourishing civilization is at least 1% as good as a dystopia is bad.*

And *that* implies that “10^18 flourishing lives” are at least 1% as good as “10^18 horribly suffering lives” are bad. 1% of 10^18 is a lot, as we’ve discussed!

Well, you’ve definitely made me feel confused about what I think about this topic. But that isn’t the same as convincing me that it’s good for there to be more persons. I see how trying to be neutral about population size leads to weird implications. But so does your position.

For example, if you think that adding more lives has ethical value, you end up with what's called the [repugnant conclusion](https://plato.stanford.edu/entries/repugnant-conclusion/). Actually, let’s skip that and talk about the [very repugnant conclusion](https://plato.stanford.edu/entries/repugnant-conclusion/#AccRepCon). I’ll give my own set of hypothetical worlds:

* **World D** has 10^18 flourishing, happy people.

* **World E** has 10^18 horribly suffering people, plus some even larger number (N) of people whose lives are mediocre/fine/”worth living” but not *good.*

There has to be some “larger number N” such that you prefer World E to World D. That’s a pretty wacky seeming position too!

### Theory X

That’s true. There’s no way of handling questions like these (aka population ethics) that feels totally satisfactory for every imaginable case.

That you know of. But there may be some way of disentangling our confusions about this topic that leaves the anti-repugnant-conclusion intuition intact, and leaves mine intact too. I’m not really feeling the need to accept one wrong-seeming view just to avoid another one.

“Some way of disentangling our confusions” is what Derek Parfit called [theory X](https://plato.stanford.edu/entries/repugnant-conclusion/#ArrRepCon). Population ethicists have looked for it for a while. They’ve not only not found it, they’ve produced [impossibility theorems](https://plato.stanford.edu/entries/repugnant-conclusion/#AccImpSatPopEth) heavily implying that it does not exist.

That is, the various intuitions we want to hold onto (such as “the very repugnant conclusion is false” and “extinction would not be good” and various others) collectively contradict each other.

So it looks like we probably have to pick *something* weird to believe about this whole “Is it good for there to be more people?” question. And if we have to pick something, I’m going to go ahead and pick what’s called the *total view:* the view that we should maximize the sum total of the well-being of all persons. You could think of this as if our “potential beneficiaries” include all persons who ever could exist, and getting to exist[6](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fn6) is a benefit that is capable of overriding significant harms. (There is more complexity to the total view than this, but it's not the focus of this piece.)

I think there are a number of good reasons to pick this general approach:

* It’s simple. If you try to come up with some view that thinks human extinction is neither the best nor the worst thing imaginable, your view is probably going to have all kinds of complicated and unmotivated-seeming moving parts, like the asymmetry between good and bad lives discussed above. But the idea behind the total view is simple, it just counts everyone (including potential someones) as persons whose interests are worth considering. Simplicity fits well into my goal of [systemizing](https://www.cold-takes.com/future-proof-ethics/#systemization) ethics, so that my system is more robust and relies on fewer intuitions.

* Having considered the various options for “which weird view to take on,” I think “the very repugnant conclusion is actually fine” does pretty well against its alternatives. It’s totally possible that our intuitive aversion to it comes from just not being able to wrap our brains around some aspect of (a) how huge the numbers of “barely worth living” lives would have to be, in order to make the very repugnant conclusion work; (b) something that is just confusing about the idea of “making it possible for additional people to exist.”[7](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fn7)* And is it really so unintuitive, anyway? Imagine you learned that some person made a costly effort to prevent your ancestors’ deaths, 1000 years ago, and now you are here today. Aren’t you glad you exist? Don’t you think your existence counts as part of the good that person accomplished? (More of this kind of thinking in [this blog post by my coworker Joe Carlsmith](https://handsandcities.com/2021/03/14/against-neutrality-about-creating-happy-lives/).) Is your take on the fact that 10^18 people might or might not get to exist really just “It doesn’t ethically matter whether this happens?”

Maybe part of what’s confusing here is something like:

* I’m not *indifferent* to extra happy lives - they are better than nothing, after all.

* But if the *only* or *main* kind of way I was improving the world was allowing extra happy lives to exist, that wouldn’t be right.

* So maybe extra lives matter up to some point, and then matter less. Or maybe it’s true that “an extra life is a good thing” but not that “lots of extra lives can be more important than helping people who already exist.”

That approach would contradict some of the key principles of “other-centered ethics” discussed [previously](https://www.cold-takes.com/defending-one-dimensional-ethics/).

I previously argued that once you think something counts as a benefit, with *some* amount of value, a *high enough amount* of that thing can swamp all other ethical considerations. In the example we used previously, enough of “helping someone have a nice day at the beach” can outweigh “helping someone avoid a tragic death.”

Hmm.

If this were a philosophy seminar, I would think you were making a perfectly good case here.

But the feeling I have at this juncture is not so much “Ah yes, I see how all of those potential lives are a great ethical good!” as “I feel like I’ve been tricked/talked into seeing no alternative.”

I don’t need to “pick a theory.” I can zoom back out to the big picture and say “Doing things that will make it possible for more future people to exist is not what I signed up for when I set out to donate money to make the world a better place. It’s not the case that addressing today’s injustices and inequities can be outweighed by that goal.”

I don’t need a perfectly consistent approach to population ethics, I don’t need to follow “rules” when giving away money. I can do things that are uncontroversially valuable, such as preventing premature deaths and improving education; I can use math to maximize the amount of those things that I do. I don’t need a master framework that lands in this strange place.[8](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fn8)

Again, I think a lot of the detailed back-and-forth has obscured the fact that there are simple principles at play here:

* I want my giving to be about benefiting others to the maximum extent possible. I want to spend my money in the way that others would want me to if they were thinking about it fairly/impartially, as in the “veil of ignorance” construction. If that’s what I want, then giving that can benefit enormous numbers of persons is generally going to look best. (Discussed [previously](https://www.cold-takes.com/defending-one-dimensional-ethics/))

* There’s a question of who counts as “others” that I can benefit. Do potential people who may or may not end up getting to exist? I need a position on this. Once I let someone into the moral circle, if there are a ton of them, they’re going to be the ones I’m concerned about.

* On balance, it seems like “potential people who may or may not end up getting to exist” probably belong in the moral circle. That is, there’s a high chance that a more enlightened, ethical society would recognize this uncontroversially.

I might end up getting it wrong and doing zero good. So might you. I am taking my best shot at [avoiding the moral prejudices of my day](https://www.cold-takes.com/future-proof-ethics/) and focusing my giving on helping others, defined fairly and expansively.

*For further reading on population ethics, see:*

* *[Population Axiology](http://users.ox.ac.uk/~mert2255/papers/population_axiology.pdf) - a ~20 page summary of dilemmas in population ethics** *[Stanford Encyclopedia of Philosophy on the Repugnant Conclusion](https://plato.stanford.edu/entries/repugnant-conclusion/) - shorter, goes over many of the same issues** *Chapter 2 of [On the Overwhelming Importance of Shaping the Far Future](https://rucore.libraries.rutgers.edu/rutgers-lib/40469/PDF/1/play/)*

### Closing thoughts

I feel a lot of sympathy for the closing positions of *both* UH and NUH.

I think something like UH’s views do, in fact, give me the best shot available at an ethics that is highly “other-centered” and [“future-proof.”](https://www.cold-takes.com/future-proof-ethics/) But as I’ve pondered these arguments, I’ve simultaneously become more compelled by some of UH’s unusual views, and *less* convinced that it’s so important to pursue an “other-centered” or “future-proof” ethics. At some point in the future, I’ll argue that these ideals are probably unattainable anyway, which weakens my commitment to them.

Ultimately, if we put this in a frame of "deciding how to spend $1 billion," the arguments in this and previous pieces would move me to spend a *chunk* of it on targeting existential risk reduction - but probably not the majority (if they were the only arguments for targeting existential risk reduction, which I don't think they are). I find UH compelling, but not wholly convincing.

However, there is a different line of reasoning for focusing on causes like AI risk reduction, which doesn’t require unusual views about population ethics. That’s the case I’ve presented in the [Most Important Century](https://www.cold-takes.com/most-important-century/) series, and I find it more compelling.

---

Footnotes
---------

1. I’m sticking with “extinction” in this piece rather than discuss the subtly different idea of “existential catastrophe.” The things I say to extinction mostly apply to existential catastrophe, but I think that adds needless confusion for this particular purpose. [↩](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fnref1)- In this case, "Utilitarian Holden" will be arguing for a particular version of utilitarianism, not for utilitarianism generally. But it's the same character from a [previous dialogue](https://www.cold-takes.com/defending-one-dimensional-ethics/). [↩](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fnref2)- It's about 1.5x as much as the Our World in Data estimate, and not everyone would call that "close" in every context, but I think for the purposes of this piece, the two numbers have all the same implications, and "one quintillion" is simpler and easier to talk about than 625 quadrillion. [↩](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fnref3)- "1% of 1% of 1% of 1%" is 1%\*1%\*1%\*1%. 1%\*1%\*1%\*1%\*10^16 (the hypothesized number of future lives, including only a 1% chance that humanity avoids extinction for long enough) = 10^8, or 100 million. [↩](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fnref4)- For this example, what the numbers are trying to communicate is that all things considered, some of these lives would rank more highly than others if people were choosing what conditions they would want to live their life under. They're supposed to implicitly incorporate reactions to things like inequality - so for example, World B, which has more inequality than Worlds A and C, might have to have better conditions to "compensate" such that the 5 billion people with "8.1/10" lives still prefer their conditions in World B to what they would be in World A. [↩](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fnref5)- (Assuming that existence is preferred to non-existence given the conditions of existence) [↩](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fnref6)- For one example (that neither UH nor NUH finds particularly compelling, but others might), see [this comment](https://www.lesswrong.com/posts/prEZkHYawwnhzswyf/the-mere-cable-channel-addition-paradox?commentId=mYDH35bZz8h45gS26). [↩](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fnref7)- This general position does have some defenders in philosophy - see <https://plato.stanford.edu/entries/moral-particularism/>  [↩](https://www.cold-takes.com/p/d1daba51-feba-4a28-a277-935581dee515/#fnref8)


Cold Takes reader survey - let me know what you want more and less of!

I'm interested in readers' takes on what I should be doing more and less of on this blog. I will generally write what I feel like writing, even if my audience has asked for less of it, but knowing what my audience wants will have some effect on what I feel like writing.

If you're up for weighing in on this, I appreciate it! Please use [this Google form](https://forms.gle/swTzVDZRQvokgyKy9). It also has some basic questions about my readers that I'm interested in the answers to (all questions are optional).

Estimated time: 10 minutes.


Programming note

I'll be posting somewhat less frequently for a while, as I've gone through a lot of my backlog (though not 100% of it) and want to focus for a while on trying to make progress on some thorny, important-seeming questions about [making the best of the most important century](https://www.cold-takes.com/making-the-best-of-the-most-important-century/).

I have a writeup of what sorts of questions I want to focus on, and why I think they're important, [here](https://forum.effectivealtruism.org/posts/zGiD94SHwQ9MwPyfW/important-actionable-research-questions-for-the-most). It's a bit less general-interest-oriented than my normal Cold Takes posts, so I only put it on the Effective Altruism Forum, but [check it out](https://forum.effectivealtruism.org/posts/zGiD94SHwQ9MwPyfW/important-actionable-research-questions-for-the-most) if you're curious about what I think is most holding us back from knowing what actions to take to improve the future of humanity.

Some things I'll probably post reasonably soon are (a) a reader survey on what topics you'd like to see more of; (b) tips for working on [wicked problems](https://www.cold-takes.com/the-wicked-problem-experience/); (c) another dialogue or two to follow [this one](https://www.cold-takes.com/defending-one-dimensional-ethics/) (topic is [future-proof ethics](https://www.cold-takes.com/future-proof-ethics/)). I expect to post on a bunch more topics later. In the meantime, check out the [archives](https://www.cold-takes.com/)!