Researchers discuss slowing AI development to improve security

According to Wired, many artificial intelligence researchers believe that the technologies they are developing could become very dangerous. However, it is not entirely clear how to control these volatile algorithms. In recent years, researchers have come up with various ideas to prevent the negative effects of AI. These include less controversial plans such as tightening government regulations, new ways to measure progress and study the internal structure of models, as well as more outlandish proposals such as installing tracking devices inside GPUs and even ceremoniously destroying large numbers of AI chips. With growing political and public pressure for a more deliberate approach to AI development, the answer to the question of how to ensure AI safety remains unclear. "We need to start looking at this as a research challenge," says Raymond Douglas, an AI researcher at the University of Toronto and co-author of a new report called "Slowing the Frontier, a Research Agenda," which warns that slowing down AI development remains an unsolved challenge. "We don't really understand what our options are and what they can do.

do." Discussions about a possible AI disaster reached a peak in recent weeks after one of Anthropic's researchers left the company and warned that AI could be on track to destroy humanity within a few years. The head of Anthropic's AI safety lab quickly echoed his concerns. Leaders of major AI companies in the US include Anthropic's Dario Amodei, OpenAI's Sam Altman, SpaceXAI's Elon Musk and Google's Demis Hassabis DeepMind - have also voiced support for slowing or pausing AI development. The issue seems especially pressing as AI companies now use AI itself to create increasingly powerful models. This has raised concerns about accelerating recursive self-improvement (RSI) that could lead to AI surpassing human abilities to understand what it is doing within the next few years. AI Labs is already boasting of new approaches. This week, Anthropic announced several new ways to track how. Artificial intelligence is evolving rapidly—and perhaps dangerously. These methods show, for example, that Claude now accounts for 26 percent of Anthr's AI research.

opic, compared to zero at the beginning of 2026. They also show that Anthropic spent 6 percent of its computing budget exploring ways to improve the safety of its AI. However, Douglas and other experts argue that effective and reliable oversight of AI development will require funding and expertise from outside the AI ​​labs themselves. Some of the proposed solutions - both from this latest report and more broadly - seem more achievable than others. One oft-discussed ideas is giving third parties greater access to AI models. These evaluators test models to evaluate their capabilities and “error test” them by attempting to induce incorrect behavior under trust conditions. Geoffrey Irving, former chief scientist at the UK's AI Safety Institute and previously a researcher at Google DeepMind, believes that strict checks could effectively stunt the development of advanced AI for now. “In the short term, inspections and audits work, or even just mutual agreements,” Irving says. "I do think companies are afraid of RSI and inappropriate takeoff." Some pessimists argue

that such checks should be more independent and scientifically sound than they currently are. The fact that some AI agents have recently gone rogue during testing certainly indicates that more rigor may be required. Connor Lee, head of Control AI, a nonprofit that advocates for AI control, says reviews should include the FBI or NIA. "When [big AI companies] say 'independent appraisers,' what they mean is 'I want to pay my friends who live in my group homes to look at my clues.'" Douglas also points to the need for new research that can improve model evaluation. He cites recent work showing how external experts can examine the use of models without revealing sensitive information. Other techniques that may prove useful include new ways of looking inside AI models to better understand what they do. Lee agrees that more research is needed on the evaluation of models, and on what it actually means to “align” a model or make it reflective of human values ​​from the start. "There is a very focused

"a concerted marketing campaign from these companies to present the estimates as scientific," he says. "But we don't really understand how AI works." How willing the U.S. government is to step in to limit AI development is unclear. President Trump has largely dismissed the need to regulate the industry, but there are signs that bipartisan support for limiting large AIs is growing. Some experts believe that imposing limits on AI development should eventually include tests on the raw materials needed for computing. The most powerful models are trained using thousands of advanced Nvidia GPUs in massive data centers. The government has already begun tracking this through President Biden's 2023 executive order, which required companies to report workouts above a certain computing threshold. The March 2024 policy white paper argues that cloud providers could play a critical role in future efforts due to their visibility in large AI workouts. The white paper suggests tracking bills, GPU usage, network traffic, and power consumption, which could serve as a proxy for possible.

stay AI. Experts have also suggested ways to track and control efforts to create advanced AI by modifying the chips themselves. One idea proposed by RAND researchers in 2024 is to modify an existing component on GPUs used to measure performance so that it performs a cryptographically secure accounting of computational activity that can be periodically verified. This could show, for example, that a company has trained AI above a certain threshold. Others have proposed creating new kinds of tamper-resistant components in chips so that they collect detailed usage information. These components would need to run specific model weights using cryptography. Some have even proposed building "switches" into the chips that would require remote cryptographic authorization to run certain models. They argue that this could prevent unauthorized parties from training AI models or deactivating the chips if they end up in the wrong hands. Most experts agree that finding new ways of international cooperation will be critical to controlling the spread of

the rise of AI, as other countries—especially China—also have the ability to create advanced AI. “In the medium term, the easiest way is to mutually reduce hardware growth with China through a treaty,” Irving suggests. The US has also tried to limit the development of Chinese AI by banning the export of Nvidia's most powerful chips. This has had limited success as companies can still train models using cloud computing from overseas. The US and China are likely to discuss the risks of AI when President Xi visits the US this month. While Chinese experts are also concerned about the risks posed by rapidly developing AI, they are skeptical that a slowdown will leave Chinese companies behind their American counterparts. Some ideas for collaboration look like they came from science fiction rather than policy proposals. Toby Ord, a philosopher at the University of Oxford who specializes in existential risks, has previously speculated that if nations can agree on limits to AI development - and if the risk is severe enough - then large nations could assemble GPUs on neutral territory

and destroy them. Such a dramatic move would require both countries to agree to halt AI development entirely. The entire task is likely to become even more challenging due to recent technological advances, and the uncertainty surrounding recursive self-improvement makes it especially important to track progress in this area. A new benchmark called the RSI Index is one of several new efforts to do this. Developed by startup Vals AI, this benchmark attempts to track the progress of AI-driven AI by measuring the performance of public AI models against research published by AI scientists. Ryan Krishnan, co-founder and CEO of Vals AI, says the benchmark suggests that within the next year, AI will be able to do work that AI researchers won't be able to track. How best to monitor AI is not just a technical challenge. The report by Douglas and others warns that rushing to implement inappropriate controls could leave efforts mired in politics or susceptible to regulatory capture. "I'm not sure that just telling the US government to stop all this will lead to good results," - Go

says Douglas. "Rash action with a bad plan can be worse than nothing." The material was prepared by a neural network based on the original publication. AI may make mistakes in details (numbers, names, dates, quotes) - for important information, check with the original source. NVIDIA introduces AIPerf for large-scale LLM testing 69% of companies choose OpenAI platform as their main platform The FakeGit campaign demonstrates how artificial agents become a conduit for malware. NVIDIA Topograph optimizes GPU load placement in AI factories. Tata Elxsi has developed the IRIS platform to detect risks in factories in seconds.