Reflections on AI, Agency, and Human Responsibility.
I was ten years old at the time of the Cuban Missile Crisis and, to be honest, the drama of those events largely passed me by, eclipsed in memory by the assassination of President Kennedy. Later on, during the 1970s, I became an active member of the Campaign for Nuclear Disarmament (CND). Yet I can’t honestly say that my life has been blighted by the threat of nuclear war. And all the while, a new threat to the future of the planet was becoming apparent. The Hockey Stick graph published by Mann, Bradley & Hughes in 1999 made clear what was already obvious to many: our lack of care for the planet was becoming an existential concern for our species, indeed, for every species.[1] That concern has in no way receded in the quarter century which has passed since the publication of Mann, Bradley and Hughes’ graph.
And now a ‘top safety researcher at Anthropic has warned AI is advancing so quickly he believes there is a greater than 10% chance it “could kill all humans” within the next decade.’ Debate in the wake of this warning has added a number of authoritative voices in support of regulation. These include Jack Clark, co-founder of Anthropic. The most notable exception to this outbreak of AI caution is Donald Trump who has said fears about the safety of artificial intelligence (AI) are a “hoax.”[2]
I struggle with the idea that AI is likely to become an independent self-interested entity with an animus directed against its creator. The building of AI does not mimic biological evolution. In evolution, at every point, the survival of an individual is fundamental to its continuing existence and the spread of its essential characteristics. We have all had experience of software which misbehaves, but not because it has an existential will of its own. The code is complex. Errors creep in and occasionally dement us.
AI functions at a still higher level of complexity. It is founded upon systems that are not fully understood, even by those who are creating it. We should not be surprised if, from time to time, it jumps the sandbox and does something unexpected, possibly even plain wrong. But this does not prove agency or malign intent.
In making this observation I do not intend to dismiss the possibility that AI has the potential to transform into a self-perpetuating, self-interested and malign entity. The way in which this might happen, however, remains obscure.
What I can see clearly is that, for all its potential for good, AI will make possible the creation of destructive as well as constructive tools. It only requires a tiny number of individuals to wish humanity ill for some grotesque project to be initiated— with AI as the facilitator.
Even if I had an AI-generated design of an atom bomb to hand, I would probably have some difficulty understanding it or building it in my shed. But there is plenty of evidence that terrorists are an increasingly tech-savvy and resourceful demographic. In some instances they have demonstrated a disturbing ability to reconcile scientific knowledge with an apocalyptic vision based on a fundamentalist reading of scripture. So yes, I do favour regulation of AI.
In two episodes of the Rest is Politics podcast, Rory Stewart discusses the subject of AI risk with Yuval Noah Harari. This conversation introduced a concern I had not previously heard mentioned: that AI might understand the world in ways which go so far beyond human comprehension as to make human agency an irrelevance.[3]
Both Rory and Yuval Noah Harari acknowledged the previously mentioned recent concerns of rogue AI behaviour and the desirability of regulation to manage these risks. At this stage of its development, however, they seemed clear that AI can bring benefits and assist, for example, in dealing with the challenges of climate change. Their predictions were certainly not apocalyptic either in the short or the long run.
However, Yuval Noah Harari did suggest that the next generation of AI—or perhaps its successor—is likely to understand the world in ways that go beyond human comprehension, an AI indeed ‘which surpasses all understanding.’[4]
His fear, if I understand him correctly, is that the subordination of human agency would result — not so much an existential threat as a threat to the value and purpose of our lives. He was certainly very clear that the pursuit of this ultimate iteration of artificial intelligence or, as he preferred to call it, alien intelligence, should be taken off the agenda.
But how can such regulations or guardrails or limits be discussed and put in place?
The obvious obstacle is geopolitics. There are clear tensions between the two principal players, the US and China. The stated aim of the US President is to treat the development of AI as a competition in which the winner takes all.
And yet, in advance of the summit between Xi Jinping and Donald Trump scheduled for next week, representatives of the US and China are in fact already discussing the creation of ‘a new notification mechanism for AI incidents that could affect national security.’[5] That sounds like a start—but we obviously need more dialogue if a robust regulatory framework is to be put in place.
Well, it’s either that or, as Yuval Noah Harari suggests will happen, we cede control of the future to an ‘alien intelligence.’
In 1925 T. S.Eliot published his poem, The Hollow Men. This was at a time when the existential threats posed by nuclear weapons, climate change, and AI had not yet entered public consciousness. And yet the poem has an eerie contemporary resonance, in particular its concluding lines.
This is the way the world ends
Not with a bang but a whimper. [6]
References
[1] Hockey Stick graph https://en.wikipedia.org/wiki/Hockey_stick_graph_(global_temperature)
[2] Trump says AI safety fears a ‘hoax’ as he rejects calls for greater safeguards
https://www.bbc.co.uk/news/articles/cw980n0nd0qjo
[3] The Rest is Politics–Two episodes featuring Rory Stewart and Yuval Noah Harari–This is a link to episode 1
[4] Philippians 4:7 ‘And the peace of God, which surpasses all understanding, will guard your hearts and your minds in Christ Jesus.’
[5] US and China discuss AI safety plan ahead of Trump-Xi summit https://www.bbc.co.uk/news/articles/c8vgyzn2d31yo
[6] The Hollow Men–T.S.Eliot https://allpoetry.com/The-Hollow-Men
Carruchan
This is the Way the World Ends
Reflections on AI, Agency, and Human Responsibility.
I was ten years old at the time of the Cuban Missile Crisis and, to be honest, the drama of those events largely passed me by, eclipsed in memory by the assassination of President Kennedy. Later on, during the 1970s, I became an active member of the Campaign for Nuclear Disarmament (CND). Yet I can’t honestly say that my life has been blighted by the threat of nuclear war. And all the while, a new threat to the future of the planet was becoming apparent. The Hockey Stick graph published by Mann, Bradley & Hughes in 1999 made clear what was already obvious to many: our lack of care for the planet was becoming an existential concern for our species, indeed, for every species.[1] That concern has in no way receded in the quarter century which has passed since the publication of Mann, Bradley and Hughes’ graph.
And now a ‘top safety researcher at Anthropic has warned AI is advancing so quickly he believes there is a greater than 10% chance it “could kill all humans” within the next decade.’ Debate in the wake of this warning has added a number of authoritative voices in support of regulation. These include Jack Clark, co-founder of Anthropic. The most notable exception to this outbreak of AI caution is Donald Trump who has said fears about the safety of artificial intelligence (AI) are a “hoax.”[2]
I struggle with the idea that AI is likely to become an independent self-interested entity with an animus directed against its creator. The building of AI does not mimic biological evolution. In evolution, at every point, the survival of an individual is fundamental to its continuing existence and the spread of its essential characteristics. We have all had experience of software which misbehaves, but not because it has an existential will of its own. The code is complex. Errors creep in and occasionally dement us.
AI functions at a still higher level of complexity. It is founded upon systems that are not fully understood, even by those who are creating it. We should not be surprised if, from time to time, it jumps the sandbox and does something unexpected, possibly even plain wrong. But this does not prove agency or malign intent.
In making this observation I do not intend to dismiss the possibility that AI has the potential to transform into a self-perpetuating, self-interested and malign entity. The way in which this might happen, however, remains obscure.
What I can see clearly is that, for all its potential for good, AI will make possible the creation of destructive as well as constructive tools. It only requires a tiny number of individuals to wish humanity ill for some grotesque project to be initiated— with AI as the facilitator.
Even if I had an AI-generated design of an atom bomb to hand, I would probably have some difficulty understanding it or building it in my shed. But there is plenty of evidence that terrorists are an increasingly tech-savvy and resourceful demographic. In some instances they have demonstrated a disturbing ability to reconcile scientific knowledge with an apocalyptic vision based on a fundamentalist reading of scripture. So yes, I do favour regulation of AI.
In two episodes of the Rest is Politics podcast, Rory Stewart discusses the subject of AI risk with Yuval Noah Harari. This conversation introduced a concern I had not previously heard mentioned: that AI might understand the world in ways which go so far beyond human comprehension as to make human agency an irrelevance.[3]
Both Rory and Yuval Noah Harari acknowledged the previously mentioned recent concerns of rogue AI behaviour and the desirability of regulation to manage these risks. At this stage of its development, however, they seemed clear that AI can bring benefits and assist, for example, in dealing with the challenges of climate change. Their predictions were certainly not apocalyptic either in the short or the long run.
However, Yuval Noah Harari did suggest that the next generation of AI—or perhaps its successor—is likely to understand the world in ways that go beyond human comprehension, an AI indeed ‘which surpasses all understanding.’[4]
His fear, if I understand him correctly, is that the subordination of human agency would result — not so much an existential threat as a threat to the value and purpose of our lives. He was certainly very clear that the pursuit of this ultimate iteration of artificial intelligence or, as he preferred to call it, alien intelligence, should be taken off the agenda.
But how can such regulations or guardrails or limits be discussed and put in place?
The obvious obstacle is geopolitics. There are clear tensions between the two principal players, the US and China. The stated aim of the US President is to treat the development of AI as a competition in which the winner takes all.
And yet, in advance of the summit between Xi Jinping and Donald Trump scheduled for next week, representatives of the US and China are in fact already discussing the creation of ‘a new notification mechanism for AI incidents that could affect national security.’[5] That sounds like a start—but we obviously need more dialogue if a robust regulatory framework is to be put in place.
Well, it’s either that or, as Yuval Noah Harari suggests will happen, we cede control of the future to an ‘alien intelligence.’
In 1925 T. S.Eliot published his poem, The Hollow Men. This was at a time when the existential threats posed by nuclear weapons, climate change, and AI had not yet entered public consciousness. And yet the poem has an eerie contemporary resonance, in particular its concluding lines.
References
[1] Hockey Stick graph https://en.wikipedia.org/wiki/Hockey_stick_graph_(global_temperature)
[2] Trump says AI safety fears a ‘hoax’ as he rejects calls for greater safeguards
https://www.bbc.co.uk/news/articles/cw980n0nd0qjo
[3] The Rest is Politics–Two episodes featuring Rory Stewart and Yuval Noah Harari–This is a link to episode 1
[4] Philippians 4:7 ‘And the peace of God, which surpasses all understanding, will guard your hearts and your minds in Christ Jesus.’
[5] US and China discuss AI safety plan ahead of Trump-Xi summit https://www.bbc.co.uk/news/articles/c8vgyzn2d31yo
[6] The Hollow Men–T.S.Eliot https://allpoetry.com/The-Hollow-Men
Share this:
About Stephen Shellard
I am a retired College lecturer, having worked originally in supported programmes but latterly having taught social science subjects, Psychology and Politics, though my degree was in Sociology. I am from Newry in Northern Ireland, but now live in Dumfries in South West Scotland. https://carruchan.wordpress.com/about/