Humans as Pixels, Death Beyond the Screen
The story begins in 2024, in front of a monitor reflecting a refugee camp in Gaza. Instead of family photos, a cold screen sits on the desk of an Israeli soldier in his twenties. His mission is not to run through a dusty battlefield. Instead, he sits in a cool control room chair, staring at numerous dots and lines on the screen, and at the pixel mass that artificial intelligence (AI) has labeled as ’targets.'
The AI called ‘Lavender’ has created a list of 39,000 individuals suspected of being Hamas members, while another AI named ‘The Gospel’ has pinpointed buildings where they might be found. The soldier has only 20 seconds. He merely needs to verify the AI’s judgment with ‘human eyes’ and press the approval button to complete his mission. With a single click, an apartment that was once a warm home for someone disappears like dust from the map. There are no explosions, no screams. Only the data throughput displayed on the screen speaks of his achievements.
This is no longer a story from a distant future. In the vast plains of Ukraine, AI drones hunt each other, and in the heart of Silicon Valley, the world’s top talents are developing faster and more accurate ways to kill, namely the ‘Kill Chain’ algorithm. The way humanity conducts war is being shaken to its roots.
The paradigm of war, which has evolved from spears and swords to gunpowder and then to nuclear weapons, is now attempting to push the last variable, ‘human,’ out of the system. We stand at the massive threshold of an era where artificial intelligence becomes the main player in ‘algorithmic warfare.’
This new war is not merely a matter of stronger weapons. It shakes the speed and scale of war, and most importantly, the ’ethics’ of war. When human fingers leave the trigger, and the rapid calculations of silicon chips take their place, what will we gain and what will we lose? Is it acceptable to entrust the weight of countless lives hidden behind the cold term ‘collateral damage’ to AI’s probability calculations?
This article is a journey to find answers to this chilling question. We must face how the ‘algorithm of death’ we designed ourselves is knocking at our door and ponder together what we should do before we fully open that door. The future of humanity depends on what moral shackles we will place on this terrifying technology.
Chapter 1: The God of the New Battlefield, Code
The history of war is often said to be the history of technology. To go further, it is about going further, more accurately, and more destructively. However, the change we face now is somewhat different. It is because the ‘decision-making subject’ is leaving human hands.
1.1 Killing Machines Breaking Free from the ‘Human Loop’
To understand AI warfare, one must first grasp the concept of ‘Human-in-the-Loop.’ Depending on how much humans are involved in weapon systems, it can be divided into three stages.
- Human-Centric Stage (Human-in-the-Loop)
- This is the most traditional method. Humans directly identify targets and decide whether to fire. AI plays the role of a helpful assistant, aiding in aiming or providing information. It is easy to imagine a fighter pilot using AI assistance to target enemy aircraft and pressing the missile button himself.
- Human-Supervised Stage (Human-on-the-Loop)
- In this stage, AI identifies targets and suggests attacks, while humans supervise and can ultimately reject the proposal. Israel’s ‘Iron Dome’ defense system is a good example. The system automatically identifies incoming rockets and prepares to intercept, but the final launch approval comes from a human in the control room. However, in situations where numerous missiles are incoming, the human role gradually becomes a mere formal supervisor.
- Human-Excluded Stage (Human-out-of-the-Loop)
- This is the most controversial final stage. AI makes all decisions and executes actions from target identification to destruction without human intervention. These are the ‘Lethal Autonomous Weapons Systems (LAWS),’ commonly referred to as ‘killer robots.’ Once activated, these weapons autonomously assess and eliminate ’enemies’ without further human commands.
The problem is that as the speed of war accelerates, it becomes impossible for humans to intervene in every instance. Imagine a ‘drone swarm’ attack where hundreds or thousands of drones attack like a swarm of bees. Can a human judge and stop each one? Ultimately, the defending side has no choice but to respond with the same AI swarm. This leads to a ‘war of machines’ that escapes human control, where humans may become mere bystanders, pressing the start button and watching the results unfold.
1.2 Ukraine: A Massive AI War Experiment
The war in Ukraine, which began in 2022, has become a massive experiment that compresses the appearance of future warfare, often referred to as the ‘first AI war.’
In particular, the Ukrainian military is changing the course of the war through the AI platform ‘Gotham’ from the American data analytics company Palantir. Palantir describes itself as a ’technology partner for Western democracy’ and has actively engaged in the war. Their technology has woven together all scattered information into one massive ‘Kill Web.’
AI analyzes everything in real-time, from reconnaissance drone footage and civilian satellite images to communication interception data and even videos of Russian troops posted on TikTok. Information that would have been analyzed separately by various units in the past is now combined on a single platform, predicting the location, size, and next actions of Russian troops. Commanders can oversee the battlefield like gods, receiving recommendations for optimal attack points and timings. Thanks to this, Ukraine has been able to effectively resist the overwhelming Russian forces.
However, behind this remarkable technology lies uncomfortable questions. If a civilian hospital was located at the artillery strike point recommended by Palantir’s algorithm, who bears the responsibility? If an AI drone attacks a school bus mistaken for a tank, whose fault is that? Is it a program error, or the responsibility of the human who decided to use that program? Behind Ukraine’s desperate resistance lurk such terrifying questions.
Chapter 2: The Gospel of ‘Lavender,’ The Tragedy of Humans Turned into Numbers
If the potential of AI warfare was tested in Ukraine, the horrific reality was proven in the Gaza Strip. The Israel Defense Forces (IDF), under the name of ’efficient warfare,’ have crossed a dangerous line, turning human lives into statistical probabilities.
2.1 The Birth of the Mass Assassination Factory
The existence of two AI systems, ‘Lavender’ and ‘The Gospel,’ was revealed through investigative journalism in Israel. Testimonies from former intelligence officers were shocking.
- Lavender
- The mission of this AI is singular: to identify ’everyone suspected of being Hamas members.’ Lavender learns from vast data, including the call records of 2.3 million Gaza residents, social media activities, and membership in specific group chat rooms, scoring ‘Hamas affiliation potential.’ Once a certain score is exceeded, individuals automatically enter the ‘kill list.’ The IDF approved the use of the system, knowing that Lavender has an error rate of about 10%, meaning that one in ten could be innocent civilians. This implies that the list of 39,000 names could include around 3,900 innocent lives.
- The Gospel
- While Lavender decides ‘who’ to kill, The Gospel determines ‘where’ to strike. This AI rapidly generates hundreds of potential targets among numerous buildings and facilities.
The combination of these two systems has completely transformed the IDF’s operational methods. Unlike the past, where care was taken to minimize civilian casualties, they now intentionally target homes where lower-level operatives are with their families. There is even an AI named ‘Where’s Daddy?’ that tracks the moment a target enters their home and alerts the airstrike team.
2.2 The Statisticalization of ‘Collateral Damage’ and the Numbing of Humanity
The introduction of AI systems has shattered the standards for civilian casualties. In the past, operations would be canceled if the sacrifice of one or two civilians was anticipated to capture a terrorist. Now, to eliminate one lower-level operative, up to 15 to 20 civilian casualties are pre-approved, and for high-ranking commanders, over 100 civilian casualties are also approved in advance.
This signifies that the tragedy of war has shifted from an ’ethical dilemma’ to a matter of ‘statistical calculation.’ A former officer confessed:
“Everything was statistical and neat. I pressed the button, and the machine told me it had completed the mission. I killed many people, but because I didn’t see the blood, I felt less guilty.”
This confession illustrates how algorithmic warfare destroys the human heart. The act of erasing pixels on a screen provides a completely different psychological experience than killing real people. As the ‘algorithm’ interjects between the act of murder and its consequences, feelings of responsibility and guilt fade, and humanity becomes numb. The cases of Lavender and The Gospel demonstrate that AI is a terrifying ‘game changer’ rewriting the rules of war itself.
Chapter 3: The Black Participation of Silicon Valley, Hands Nurturing Monsters
The monster of algorithmic warfare cannot be nurtured by the military alone. Behind it are the big tech companies in Silicon Valley, which once claimed to make the world a better place. Their idealistic slogans often lose their way in the face of massive defense budgets.
3.1 The End of ‘Don’t Be Evil’
In 2018, Google faced fierce backlash from its employees after participating in the U.S. Department of Defense’s drone video analysis AI development project, ‘Project Maven.’ The ethical resistance from engineers insisting that “Google should not be involved in war business” ultimately led the company to withdraw from the project.
However, the void left by Google’s departure was quickly filled by companies like Palantir. Born from the outset to collaborate with defense and intelligence agencies, Palantir received investments from the CIA and was considered an ‘outlaw’ in Silicon Valley. Palantir’s CEO Alex Karp criticized the culture in Silicon Valley that avoids defense cooperation, stating, “Our mission is to provide the best technology to protect Western democracy.” For him, technology was a powerful weapon to uphold values. (Ultimately, there are reports suggesting that Google is also keeping the possibility of cooperation with the Department of Defense open.)
Recently, even OpenAI, famous for ‘ChatGPT,’ quietly removed the clause prohibiting use in ‘military and warfare’ from its terms of service, leaving the door open for cooperation with the Department of Defense. In the end, a massive trend is forming where everyone follows the path paved by Palantir.
3.2 AI as a Pawn in the Competition for Technological Hegemony
The fundamental reason big tech companies are deeply involved in defense is the technological hegemony competition between the U.S. and China. China is mobilizing its big tech companies under state leadership for military modernization. In this context, the U.S. Department of Defense pressures companies to participate, stating, “If the best companies do not participate, we will have to fight China’s AI army with technologically inferior weapons.”
Ultimately, big tech companies find themselves at a crossroads. Participating in defense projects can yield enormous profits but invites criticism for joining the ‘banality of evil,’ while refusing risks being labeled unpatriotic and falling behind in competition.
The problem is that within this massive trend, the best technology, which should enrich human life, is transforming into the sharpest spear that could destroy humanity.
Chapter 4: Ghosts in the Machines, The Evaporation of Human Ethics
The most dangerous change brought about by algorithmic warfare is not the form of weapons but the transformation occurring within the ‘human’ who conducts the war. As machines take center stage in decision-making, the concepts of responsibility and ethics are dissipating like fog.
4.1 The Black Hole of ‘Accountability Gap’
Imagine a scenario where an autonomous lethal drone mistakenly attacks a wedding venue, believing it to be a terrorist gathering, resulting in numerous civilian casualties. Who bears the responsibility for this horrific tragedy?
- The Programmer? He would argue that he merely wrote code according to the given conditions.
- The Manufacturer? The company would claim it only produced the product according to military requirements and bears no operational responsibility.
- The Commander? He would say he merely deployed the drone and could not predict its specific actions.
- The Drone (AI)? It is impossible to hold a machine accountable.
Ultimately, a state of ‘Accountability Gap’ arises, where no one fully takes responsibility. To punish war crimes, ‘intent’ must be proven, but the decision-making process of AI is often a ‘black box’ that even developers cannot fully explain. This accountability gap provides soldiers with a dangerous excuse of ‘I was just following the system’s orders,’ numbing moral reflection.
4.2 ‘Emotionless Killing’ and the Paradox of PTSD
Remote pilots conducting warfare through screens from safe bases perceive their acts of killing as unreal, akin to a video game. This ‘psychological distancing’ may reduce guilt in the short term.
However, paradoxically, they also experience severe post-traumatic stress disorder (PTSD). The disconnect between killing people on screens during the day and having dinner with family in the evening as if nothing happened destroys their psyche. When AI recommends targets and ‘proves’ the legitimacy of attacks with data, humans relinquish their moral judgment to machines and gradually degrade into mere components of the system.
4.3 Future Dilemma: Is Ethical AI Possible?
As an alternative to these issues, there are attempts to create ’ethical AI’ that learns the laws of war and rules of engagement to minimize civilian casualties.
However, how can we algorithmically create judgments about balancing ‘military advantage’ and ‘civilian casualties,’ which are profoundly human values? This may be a dangerous arrogance, attempting to solve the ’trolley dilemma’ that humanity has not resolved for thousands of years with just a few lines of code. We must fundamentally question whether what we truly desire is ‘a more ethical way to kill’ or ‘a way to avoid killing altogether.’
Chapter 5: An Uncontrollable Blaze, Towards Humanity’s Last War
The advancement of AI warfare technology opens a ‘Pandora’s Box’ that could lead humanity into an uncontrollable catastrophe, beyond merely stronger weapons.
5.1 The War of Machine Speed and the Fear of ‘Flash War’
Future wars could escalate into full-scale conflicts in just a few minutes or even seconds, without any human intervention.
For instance, suppose an AI alert system in one country detects a heat source suspected of being a missile launch from an enemy nation. This information is passed to the AI command system, which determines it to be a clear attack and automatically executes a retaliatory strike without human approval. However, the initial heat source could have been a wildfire or a meteor. All of this could happen within minutes, leaving the leaders of both nations no time to grasp the situation. This type of war, proceeding at a speed beyond human control, is referred to as ‘Flash War.’
This fear is driving nations into an ‘AI arms race.’ The logic of “If we don’t do it, our enemies will” recalls the nuclear arms race of the Cold War, but AI is far cheaper and easier to proliferate than nuclear weapons, making it even more dangerous.
5.2 The Proliferation of AI Technology and New Terrorism
AI technology is no longer the exclusive domain of great powers. Now, even terrorist groups can launch attacks in urban areas targeting specific individuals or groups with dozens of small drones equipped with facial recognition technology. The horrific future described in the warning video ‘Slaughterbots’ by AI experts has become a reality.
Moreover, fake news and propaganda using deepfake technology can undermine a nation’s social trust and paralyze democracy. The battlefield is now expanding into each of our ‘minds’ and ‘hearts.’
5.3 A Rampant Monster, The Absence of International Norms
Despite these evident dangers, international efforts to control AI weapons have been sluggish. The United Nations (UN) has engaged in discussions for years, but due to opposition from major developing countries, no binding agreements have been reached. In the regulatory vacuum, AI weapon technology is advancing at an uncontrollable pace. Our time is running out.
Questions Posed to Machines, Time for Humans to Answer
We stand at a significant turning point in human history. The era of algorithmic warfare is approaching like an unavoidable future, and its destructive potential is heading towards the dehumanization of war and uncontrollable catastrophe.
Machines are asking us, “How far will you allow me?”
This is not merely a technological issue; it is a philosophical question about human values and dignity. Will we succumb to the sweet temptation of efficiency and speed, handing over the gravest ethical decision of murder to the cold logic of machines?
The answer to this question is the responsibility of us all. Before it is too late, scientists, ethicists, and citizens must come together to establish a ‘clear red line’ regarding the development of AI weapons. A binding international treaty prohibiting the development and use of autonomous lethal weapons that escape direct human control will be the first step.
AI is not a value-neutral tool. If we teach AI ’efficient killing,’ it will become the most terrifying instrument of destruction in history. However, if we teach AI the value of life and the importance of peace, it could become the greatest invention contributing to human prosperity.
The choice is in our hands. We must remember the dignity of a human being beyond the pixels on the screen. Before the algorithm of death fully opens our door, we must firmly lock it in the name of humanity. The future of humanity depends on what answer we give to this machine right now.