Existential risk? Regulatory capture? AI for one and all? A look at what’s going on with AI in the UK

The promise and pitfall of artificial intelligence is a hot topic these days. Some say AI will save us: It’s already on the case to fix pernicious health problems, patch up digital divides in education, and do other good works. Others fret about the threats it poses in warfare, security, misinformation and more. It has also become a wildly popular diversion for ordinary people and an alarm bell in business.

AI is a lot, but it has not (yet) managed to replace the noise of rooms full of people chattering to each other. And this week, a host of academics, regulators, government heads, startups, Big Tech players and dozens of profit and non-profit organizations are converging in the U.K. to do just that as they talk and debate about AI.

Why the U.K.? Why now?

On Wednesday and Thursday, the U.K. is hosting what it has described as the first event of its kind, the “AI Safety Summit” at Bletchley Park, the historic site that was once home to the World War 2 Codebreakers and now houses the National Museum of Computing.

Months in the planning, the Summit aims to explore some of the long-term questions and risks AI poses. The objectives are idealistic rather than specific: “A shared understanding of the risks posed by frontier AI and the need for action,” “A forward process for international collaboration on frontier AI safety, including how best to support national and international frameworks,” “Appropriate measures which individual organisations should take to increase frontier AI safety,” and so on.

That high-level aspiration is also reflected in who is taking part: top-level government officials, captains of industry, and notable thinkers in the space are among those expected to attend. (Latest late entry: Elon Musk; latest no’s reportedly include President Biden, Justin Trudeau and Olaf Scholz.)

It sounds exclusive, and it is: “Golden tickets” (as Azeem Azhar, a London-based tech founder and writer, describes them) to the Summit are in scarce supply. Conversations will be small and mostly closed. So because nature abhors a vacuum, a whole raft of other events and news developments have sprung up around the Summit, looping in the many other issues and stakeholders at play. These have included talks at the Royal Society (the U.K.’s national academy of sciences); a big “AI Fringe” conference that’s being held across multiple cities all week; many announcements of task forces; and more.

“We’re going to play the summit we’ve been dealt,” Gina Neff, executive director of the Minderoo Centre for Technology and Democracy at the University of Cambridge, speaking at an evening panel last week on science and safety at the Royal Society. In other words, the event in Bletchley will do what it does, and whatever is not in the purview there becomes an opportunity for people to put their heads together to talk about the rest.

Neff’s panel was an apt example of that: In a packed hall at the Royal Society, she sat alongside a representative from Human Rights Watch, a national officer from the mega trade union Unite, the founder of the Tech Global Institute, a think tank focused on tech equity in the Global South, the public policy head from the startup Stability AI, and a computer scientist from Cambridge.

AI Fringe, meanwhile, you might say is fringe only in name. With the Bletchley Summit in the middle of the week and in one location, and with a very limited guest list and equally limited access to what’s being discussed, AI Fringe has quickly spilled into, and filled out, an agenda that has wrapped itself around Bletchley, literally and figuratively. Organized not by the government but by, interestingly, a well-connected PR firm called Milltown Partners that has represented companies like DeepMind, Stripe and the VC Atomico, it carries on through the whole week, in multiple locations in the country, free to attend in person for those who could snag tickets — many events sold out — and with streaming components for many parts of it.

Even with the profusion of events, and the goodwill that’s pervaded the events we’ve been at ourselves so far, it’s been a very sore point for people that discussion of AI, nascent as it is, remains so divided: one conference in the corridors of power (where most sessions will be closed only to invited guests) and the other for the rest of us.

Earlier today, a group of 100 trade unions and rights campaigners sent a letter to the prime minister saying that the government is “squeezing out” their voices in the conversation by not having them be a part of the Bletchley Park event. (They may not have gotten their golden tickets, but they were definitely canny how they objected: The group publicized its letter by sharing it with no less than the Financial Times, the most elite of economic publications in the country.)

And normal people are not the only ones who have been snubbed. “None of the people I know have been invited,” Carissa Véliz, a tutor in philosophy at the University of Oxford, said during one of the AI Fringe events today.

Some believe there is a merit in streamlining.

Marius Hobbhahn, an AI research scientist who is also the co-founder and head of Apollo Research, a startup building AI safety tools, believes that smaller numbers can also create more focus: “The more people you have in the room, the harder it will get to come to any conclusions, or to have effective discussions,” he said.

More broadly, the summit has become an anchor and only one part of the bigger conversation going on right now. Last week, U.K. prime minister Rishi Sunak outlined an intention to launch a new AI safety institute and a research network in the U.K. to put more time and thought into AI implications; a group of prominent academics, led by Yoshua Bengio and Geoffrey Hinton, published a paper called “Managing AI Risks in an Era of Rapid Progress” to put their collective oar into the the waters; and the UN announced its own task force to explore the implications of AI. Today, U.S. president Joe Biden issued the country’s own executive order to set standards for AI security and safety.

“Existential risk”

One of the biggest debates has been around whether the idea of AI posing “existential risk” has been overblown, perhaps even intentionally to remove scrutiny of more immediate AI activities.

One of the areas that gets cited a lot is misinformation, pointed out Matt Kelly, a professor of Mathematics of Systems at the University of Cambridge.

“Misinformation is not new. It’s not even new to this century or last century,” he said in an interview last week. “But that’s one of the areas where we think AI short and medium term has potential risks attached to it. And those risks have been slowly developing over time.” Kelly is a fellow of the Royal Society of Science, which — in the lead-up to the Summit — also ran a red/blue team exercise focusing specifically on misinformation in science, to see how large language models would just play out when they try to compete with one another, he said. “It’s an attempt to try and understand a little better what the risks are now.”

The U.K. government appears to be playing both sides of that debate. The harm element is spelled out no more plainly than the name of the event it’s holding, the AI Safety Summit.

“Right now, we don’t have a shared understanding of the risks that we face,” said Sunak in his speech last week. “And without that, we cannot hope to work together to address them. That’s why we will push hard to agree on the first ever international statement about the nature of these risks.”

But in setting up the summit in the first place, it’s positioning itself as a central player in setting the agenda for “what we talk about when we talk about AI,” and it certainly has an economic angle, too.

“By making the U.K. a global leader in safe AI, we will attract even more of the new jobs and investment that will come from this new wave of technology,” Sunak noted. (And other departments have gotten the memo, too: the Home Secretary today held an event with the Internet Watch Foundation and a number of large consumer app companies like TikTok and Snap to tackle the proliferation of AI-generated sex abuse images.)

Having Big Tech in the room might appear helpful in one regard, but critics often regularly see that as a problem, too. “Regulatory capture,” where the bigger power players in the industry take proactive steps toward discussing and framing risks and protections, has been another big theme in the brave new world of AI, and it’s looming large this week, too.

“Be very wary of AI technology leaders that throw up their hands and say, ‘regulate me, regulate me.’ Governments might be tempted to rush in and take them at their word,” Nigel Toon, the CEO of AI chipmaker Graphcore, astutely noted in his own essay about the summit coming up this week. (He’s not quite Fringe himself, though: He’ll be at the event himself.)

Meanwhile, there are many still debating whether existential risk is a useful thought exercise at this point.

“I think the way the frontier and AI have been used as rhetorical crutches over the past year has led us to a place where a lot of people are afraid of technology,” said Ben Brooks, the public policy lead of Stability AI, on a panel at the Royal Society, where he cited the “paperclip maximizer” thought experiment — where an AI set to create paperclips without any regard of human need or safety could feasibly destroy the world — as one example of that intentionally limiting approach. “They’re not thinking about the circumstances in which you can deploy AI. You can develop it safely. We hope that is one thing that everyone comes away with, the sense that this can be done and it can be done safely.”

Others are not so sure.

“To be fair, I think that existential risks are not that long term,” Hobbhahn at Apollo Research said. “Let’s just call them catastrophic risks.” Taking the rate of development that we’ve seen in recent years, which has brought large language models into mainstream use by way of generative AI applications, he believes the biggest concerns will remain bad actors using AI rather than AI running riot: using it in biowarfare, in national security situations and misinformation that can alter the course of democracy. All of these, he said, are areas where he believes AI may well play a catastrophic role.

“To have Turing Award winners worry a lot in public about the existential and the catastrophic risks . . . We should really think about this,” he added.

The business outlook

Grave risks to one side, the U.K. is also hoping that by playing host to the bigger conversations about AI, it will help establish the country as a natural home for AI business. Some analysts believe that the road for investing in it, however, might not be as smooth as some predict.

“I think reality is starting to set in and enterprises are beginning to understand how much time and money they need to allocate to generative AI projects in order to get reliable outputs that can indeed boost productivity and revenue,” said Avivah Litan, VP analyst at Gartner. “And even when they tune and engineer their projects repeatedly, they still need human supervision over operations and outputs. Simply put, GenAI outputs are not reliable enough yet and significant resources are required to make it reliable. Of course models are improving all the time, but this is the current state of the market. Still, at the same time, we do see more and more projects moving forward into production.”

She believes that AI investments “will certainly slow it down for the enterprises and government organizations that make use of them. Vendors are pushing their AI applications and products but the organizations can’t adopt them as quickly as they are being pushed to. In addition there are many risks associated with GenAI applications, for example democratized and easy access to confidential information even inside an organization.”

Just as “digital transformation” has been more of a slow-burn concept in reality, so too will AI investment strategies take more time for businesses. “Enterprises need time to lock down their structured and unstructured data sets and set permissions properly and effectively. There is too much oversharing in an enterprise that didn’t really matter much until now. Now anyone can access anyone’s files that are not sufficiently protected using simple native tongue, e.g., English, commands,” Litan added.

The fact that business interests of how to implement AI feel so far from the concerns of safety and risk that will be discussed at Bletchley Park speaks of the task ahead, but also tensions. Reportedly, late in the day, the Bletchley organizers have worked to expand the scope beyond high-level discussion of safety, down to where risks might actually come up, such as in healthcare, although that shift is not detailed in the current published agenda.

“There will be round tables with 100 or so experts, so it’s not very small groups, and they’re going to do this kind of horizon scanning. And I’m a critic, but that doesn’t sound like such a bad idea,” Neff, the Cambridge professor, said. “Now, is global regulation going to come up as a discussion? Absolutely not. Are we going to normalise East and West relations . . . and the second Cold War that is happening between the US and China over AI? Also, probably not. But we’re going to get the summit that we’ve got. And I think there are really interesting opportunities that can come out of this moment.”