Any study of the authoritarian nightmares of the 20th century is usually enough to make anyone wary of governments – a glimpse into the excesses of both state power and incompetence, and the vast human toll it can collect. Certainly in the last few years I’ve become more and more wary of any bureaucratic institution – the capacity of systems to carelessly crush human lives seems virtually limitless, but there is another consideration we need to remember.
As one runs along the spectrum from unlimited freedom on one end to cold, iron order on the other, we are subject to systems at every step of the way. Wherever we free ourselves from the man-made rules and regulations controlled by states, bureaucracies, and companies, we are simply accepting a new set of rules. We are, as far as we know, always subject to the laws of physics. And to a considerable extent, we are always subject to evolutionary pressure. This pressure isn’t purely biological, though that’s certainly significant – evolutionary pressure shows itself in any competitive system which selects for one thing over another. Ideas, or memes, are subject to evolutionary pressure just like we are, and those that can replicate will flourish. Indeed, those ideas that can brutally exterminate their rivals – and their hosts – are viable too. And with the advent of Artificial Intelligence we must realize that despite the fact they may not be biological entities, the realities of evolution will still apply to machines, if they begin to act as agents in the world subject to selective pressure. Machines that can self-improve and self-replicate will be more competitive than those that cannot.
Personal freedom may give you a perception that you have more individual discretion on how you interact with these systems, and that if you want to maximize your personal freedom, you must always resist the imposition of human systems (such as regulations). Certainly ceding authority to the state brings to my mind both bureaucratic ineptitude and literally murderous efficiency, depending on the state and the subject of their ire, but ceding authority to the law of the jungle has its own set of problems. Enter Moloch.
Moloch, an ancient god of child sacrifice, was the subject of a poem by Allen Ginsberg and an excellent essay by Scott Alexander. In the context of Ginsberg and Alexander’s works, he’s used as a metaphor for the brutality of certain systems – not merely limited to states and bureaucracies, but also including more natural systems, like evolutionary competition. Moloch is the race to the bottom – the cruelty of the deals he cuts are that they are not built on deceit. Even when the participant knows that what he offers makes them worse off, Moloch’s offers entice. Game theory has shown several clear examples of Moloch’s malign influence in practice. Consider an arms-race. In this arms-race, two hypothetical neighbouring states are concerned that their rival will produce a military advantage over them which makes them vulnerable to conquest. Both believe that if given the opportunity to subjugate them, their opponent would likely take it. They presently exist in a stalemate – both have sufficient defences to thwart an attack, and neither considers it viable to invade.
If that was where it ended, then this system would stabilize at peace – but a new advance in technology, or a change in circumstance, has produced a way to overcome their defences. You have come up with one – and why would you ever trust that they are not working on similar ways to best you? Your own defences must be shored up. Your rivals must be spied upon, to understand what they’re up to and ensure they cannot overcome you. Even if you didn’t mean to crush them, this would only be reasonable. Moloch, using the pale light of cynical reasoning, reveals to you that they’d be mad not to spy on you, too. For your own security, their spies must also be rooted out.
Now, what had previously been an unsatisfying but peaceful compromise has become an expensive and perhaps brutal competition. Constant probing, spying, counter-spying, murder, and betrayal. Sabre-rattling and brinksmanship. Expensive R&D projects with inflated budgets. Weapons tests that spread radiation and fear. Biolabs manufacturing vaccines for diseases that don’t yet exist – and the disease. Bloody occupations of neighbouring states, to make sure valuable resources don’t fall into the hands of your rivals. And after all this expense and misery, you’re still in a stalemate. Moloch laughs at you both. You’ve been played, he says. You look at this malignant god hatefully, but why should he care? You’re welcome to stop this stupid little game at any time. Moloch has plenty of other playthings – just today, he’s cheerfully talked a monkey into shouting a false alarm to steal fruit, a fly into laying eggs in the eye of a cow, and a cancer cell into multiplying. Even if you two decide to get along, Moloch hums, there’s a perfectly good up-and-comer on the geopolitical scene, eager to make a splash and undermine your new peaceful order. Maybe you need to do something about that. Or don’t, what does it matter to him?
The problems of competition and incentives far transcend human society. Indeed, society, in its own flawed ways, has often done its best to act in opposition to Moloch’s incessant whispering. The establishment of laws to regulate behavior, of cultural norms, of religions to dictate morality in no uncertain terms – while all of these things are themselves subject to selective pressure, they are also capable at times of looking at the race to the bottom from the outside, and attempting to halt it.
One familiar example of game theory is the prisoner’s dilemma, wherein two prisoners can either collaborate or defect. The game is simple. If both prisoners collaborate, they are given a small penalty – let’s say 6 months in prison. If one chooses to collaborate and the other defects, the defector goes free, and the collaborator gets ten years. If both defect, they both get five years. Collaboration results in a net one year of prison time for the two, whereas any defection results in a net ten years one way or another – but that another matters, because in one of those scenarios, the defector has completely fucked you, and freed himself. If you don’t trust the criminal you’re expected to collaborate with, he could get off scott-free and you’re in the slammer for ten years. Moloch suggests you wisely halve your prison time, and just take the fiver – or if he’s a complete imbecile, you’ll get to go free.
This game changes, however, when you add another participant. Some people call him the mob boss. The mob boss doesn’t like rats. He has told both of you in no uncertain terms that if you defect, you’ll be sleeping with the fishes. This significantly changes the game. Not only does your incentive to defect vanish, but your expectation that the other prisoner will defect – based on the same incentive – vanishes too. Thanks to this murderous mob boss, your analysis of the situation is now six months in prison or death. You’ve both been spared a difficult decision and, possibly, up to ten years in prison. Suddenly Moloch has lost his teeth.
So yes, in theory the presence of an external authority onto these games to put their finger on the scale and adjust incentives does, genuinely, alter the game and help avoid a race to the bottom, where both parties accept sub-optimal results due to competitive pressure. This scenario, it’s worth noting, is still not perfect for you. In a perfect world – at least if you were amoral – you would have defected, he would have collaborated, and you’d have been spared any punishment at all. The best of all worlds for your selfish interests is in fact the one where you are the only defector in a world of collaborators. Yet this externally imposed position has still spared all participants a stable state of punishment ten-to-twenty times longer than it needs to be. This, in a sense, is the argument for a lawful society over one of individual freedom. This mob boss of law, culture, and ethics is part of what takes the incentive out of robbing or killing your neighbour, by forcing you to re-evaluate the incentives of theft and murder. Even if the boss isn’t your friend, and even if he might not actually be doing this for you, you still might be better off.
The game clearly doesn’t end there, though. Moloch shrugs his shoulders, and leisurely remarks to the boss that a rival gang is moving into his territory… The game continues, at a new scale, and Moloch smiles. The international arms race is between two states which, themselves, are constantly acting as the mob boss for the societies they are governing – sometimes for better, sometimes for worse. And if tomorrow a world government, alien, or AI mob boss were to arrive with the power to immediately halt that arms race, they may themselves be subject to their own competitive pressures. And let’s not forget that many regulating systems will extract their own price – like protection money. It’d be better if we didn’t need a mob boss to threaten to murder us in the first place, you’d think.
The arms-race of the two states can be seen everywhere, without any need for rational contemplation among its participants. One need only look at nature, and the vast number of ways species have evolved in competition with one another. These adaptations increase the fitness of one creature – and in any context where resources are scarce, that is at the expense of another. Creatures that don’t play this game are out-competed, and vanish from the gene-pool. And it’s worth noting, too, that while these adaptations can increase fitness, they don’t necessarily increase well-being. In an island of plenty, the species that can replicate most numerously will thrive on that plenty. Those not adapted to reproduce plentifully won’t, and those that are, will – the outcome is predictable. This is true even when the inevitable result of that is reaching a Malthusian hell wherein there is no longer enough for everyone, starvation reigns, and rats eat one another. Without some external force – such as a new predator, or a caretaker for the island – the game settles into a stable configuration of abject misery.
What does this say for us humans? It suggests to me that, as much as the idea concerns me, we do need a mob boss. Moloch’s reign is not necessarily any better than that of a tyrant king, even though it’s clear that tyranny is not desirable either. Scott Alexander uses the more agreeable metaphor of a gardener, or caretaker, rather than a mob boss. Alexander observes:
“In the very near future, we are going to lift something to Heaven. It might be Moloch. But it might be something on our side. If it’s on our side, it can kill Moloch dead.
And if that entity shares human values, it can allow human values to flourish unconstrained by natural law.
I realize that sounds like hubris – it certainly did to Hurlock – but I think it’s the opposite of hubris, or at least a hubris-minimizing position.
To expect God to care about you or your personal values or the values of your civilization, that’s hubris.
To expect God to bargain with you, to allow you to survive and prosper as long as you submit to Him, that’s hubris.
To expect to wall off a garden where God can’t get to you and hurt you, that’s hubris.
To expect to be able to remove God from the picture entirely… well, at least it’s an actionable strategy.
I am a transhumanist because I do not have enough hubris not to try to kill God.”
Of course, any student of history will view these dreams with trepidation – to elevate some transcendent being with power over us, to regulate our behavior for our own good, paves the road for a truly horrid authoritarian dystopia at even a minor miscalculation. What are human values? This is a subject we clearly cannot agree on. And to presume that whatever we create won’t be vulnerable to Moloch’s whisper is itself naive. But it’s true, as well, that to leave it entirely to that murderous deity of unbridled competition is untenable. With the technology we’ve developed today, we cannot persist in a prisoner’s dilemma. We will reach a point where a misjudgment isn’t “ten years in prison”, it’s extermination. Then we will need to make a deal – let’s try to make it a good one.