Sentinel minutes for week #37/2024

Status: green; forecasters did not anticipate events spiraling into global catastrophes this week. An attempted assassination attempt on Trump did not succeed.

Tech

OpenAI has released its new o1 models, based on their Strawberry/Q* reasoning technology. It seems to be substantially better than previous models at some complex tasks, though not better at every task. It scores 83% on the US Math Olympiad qualifying exam, compared to GPT-4o's 13%, putting it among the top 500 American students. 

The o1 models highlight scaling laws, where the more compute spent on inference (“thinking”), the more accurate the results produced are. Epoch, an AI trends analysis think tank, has some analysis of the tradeoffs here and here.

The new models sometimes "instrumentally faked alignment" during pre-deployment testing. This seems to bolster concerns about deceptive misalignment. 

The new models, without mitigation measures, also have an increased risk of being used to assist in the development of bioweapons, clearly outperforming experts on long-form biorisk questions in experiments described in their system card. They have rated o1 “medium” for Chemical, Biological, Radiological and Nuclear (CBRN) risks, the first model for which this is true.

Sixty countries endorsed a blueprint for military AI use, but China opted out of the non-binding document.

Chai Discovery released Chai-1, "a foundation model for molecular structure prediction that performs at the state-of-the-art across a variety of drug discovery tasks." It is an open-source model that seems to perform at least as well as AlphaFold3 and ESM3.

Over 100 current and former employees of top AI companies have written to Newsom—the governor of California, who has the power to veto bills—in support of California’s AI bill SB-1047, which would put some safety requirements on developers of the highest risk AI models. The SAG-AFTRA actors labor union and two women’s groups have also urged Governor Newsom to sign. Traders at Manifold give 51% to it becoming law this session. This site presents opinions on the bill in a somewhat quantitative way.

Anthropic has endorsed the AI Advancement and Reliability Act and the Future of AI Innovation Act.

Biorisk

H5N1

Officials are still trying to work out how an individual in Missouri contracted H5 bird flu. It has emerged that a close contact of the case was reportedly sick at the same time but wasn’t tested. We discussed the case last week. One bottleneck has been the fact that Missouri state officials haven't requested help from the CDC, which is required before the CDC can conduct an investigation. Forecasters think it’s highly likely that this case, at worst, involved limited (or short-chain) human-to-human transmission—otherwise we’d probably be seeing more cases already.

Mpox

The DRC is beginning its mpox vaccination campaign on October 2, almost a week earlier than previously planned. So far, 200,000 vaccine doses have been delivered to the country, a small fraction of the 3.5 million doses that its health minister has said will be needed to end the outbreak in the DRC. Multiple governments and organizations have promised 3-4M vaccine doses; most recently, Gavi, the Vaccine Alliance, has pledged $2.9M to support mpox vaccination in the DRC.

The Africa CDC is calling on rich nations like the US and Japan to donate a fraction (10 million) of their several hundred million vaccines by the end of 2025. Three vaccines are currently available, and they are all expensive and in short supply. All of these vaccines are smallpox vaccines that happen to work against mpox as well; while efficacy data are limited, a meta analysis of efficacy data found that one dose is 76% effective against infection, and two doses are 82% effective.

Bavarian Nordic now says it can supply 13 million doses of an mpox vaccine by the end of 2025. This is up from 10 million, and it's looking to expand its capacity further. 

Moderna is developing an mRNA mpox vaccine, and preliminary studies in animals have had encouraging results. China’s first domestic mpox vaccine has also been approved for clinical trials. It is not clear whether 10 million mpox vaccine doses will be available in African countries by the end of 2025, although production of that many doses is certainly possible; other control strategies may need to be relied on to greater extents if that goal is not met.

There was a suspected mpox case detected in India. Officials later confirmed that it was a Clade II, not Clade 1b, case. 

Other biological risks

Scientists are worried that Covid may usher in a surge in dementia cases. A member of our forecasting team thinks that it is highly likely that unless and until a highly effective universal Covid vaccine is developed and used widely.

The polio vaccination effort in Gaza has ended. More than 446,000 children have been vaccinated since September 1. Five healthcare facilities will continue to offer polio vaccines so that no child will be missed.

Geopolitics

United States

There was another serious assassination attempt on Trump. Details are still coming out, but online commentators point out that Trump getting killed would spark a legitimacy crisis in the US.

Europe

Vladimir Putin says that it would be considered an act of war if the West allows Ukraine to strike Russian territory using long-range missiles, one of the topics discussed by President Biden and UK Prime Minister Starmer. No announcement has yet been made, but forecasters did not take this statement particularly seriously as Russia has said similar things about other developments. In aggregate, forecasters believe that there's a ~0.2% chance (range: 0.05%-0.4%) that Russia will use a nuclear weapon by the end of 2024 (China, for one, would be unlikely to back their use) and a ~0.25% chance (range: 0.1%-0.3%) that there are more than 10 combatant deaths in a conflict between a NATO country and Russia by the end of 2024.

Meanwhile, Ukraine launched a major drone attack on Moscow, killing at least one person and causing flights to be grounded and diverted. Ukraine has also been using cluster munitions and thermite, which would make conflicts around the world more bloody if they spread.      

Iran is supplying short-range missiles to Russia to use in its war against Ukraine, despite months of US warnings to the Islamic Republic.

Russia conducted its largest-ever naval exercise, together with China, and a large number of military vessels and planes from Russia and NATO countries were active in and near the Baltic Sea on September 11. 

A cargo ship loaded with 20,000 tons of ammonium nitrate from Russia is on the move again, first to Andoya, Norway, where there is a military air base, and next to Klaipėda, Lithuania for repairs. One forecaster noted that an “accidental” explosion could further Russia’s strategic goals and that the passage of the ship could itself be diverting Western resources away from other locales. 

In Sweden, air traffic around Stockholm Arlanda Airport was disrupted by four drones overnight on September 8-9. The incident is being investigated as an act of sabotage.

England released around 1.7k prisoners who had served 40% of their sentences to deal with prison overcrowding. 

Middle East 

The US has shifted one or two aircraft carriers away from the Middle East, in a sign that the risk of an imminent Iranian attack on Israel has receded. 

A ProPublica investigation summarizes findings from a lawsuit of 9/11 families that shows that Saudi Arabia probably supported the hijackers.

Asia-Pacific 

A Chinese university is establishing a ‘Ryukyu research center’, which could signal that China plans to reverse its official position that Okinawa belongs to Japan. It could also be a warning to Japan not to interfere with China’s ambitions to control Taiwan.

Kim Jong-un has vowed to ‘exponentially’ boost North Korea’s nuclear arsenal, and it has recently diversified its nuclear delivery systems.