Is there a place readers can see the exact operationalizations of the questions your forecasters forecasted on? Or some indication of how many + what kind of forecasters forecast on these on average (e.g. 15 forecasters with xyz markers of good track records, or 5 with mixed track records), and how long they spend or what their track record on the forecasts for the newsletter are (e.g. in case they put less effort into these than their forecasts on Metaculus)?
E.g. for "forecasters think there’s a 60% (35% to 75%) chance that there will still be at least one federal or state regulation in force ...", that's somewhat lower than I'd have expected and so may be an important update to me. But I'm unsure what "in force" means (e.g. just "on the books" or also "vigorously enforced, defined in abc way"), and if the number + track records + time spent by the forecasters warrants a significant update from me.
Hi Michael: five forecasters, one to a few hours each. The operationalization is messy because the underlying reality is messy, we have a paragraph on it below:
> The operationalization of that question was a bit messy, given the four players involved in proposing, blocking or enabling such a regulation: Congress, the executive, states and the multiple levels of the judiciary. The forecasters’ consensus was that by the end of the year, there will be no federal law, an executive order aiming to restrict state regulations, and at least some state regulations. The executive order and state regulations will likely be embroiled in legal fights, but it’s unclear who will win it and on what merits. One forecaster noted that the Trump administration has not been skillful at wielding the Department of Justice to accomplish its goals, and suggests, with low confidence, that the federal administration will not get a federal judge to issue a preliminary injunction on state regulations. We think that this is an area where future forecasting efforts could be fruitful.
The full probabilities were: 60%, 65%, 75%, 65%, 35%.
I think to the extent you are making important decisions that depend on whether this is an 80% or a 60%, you should think about how much you value additional forecasting and maybe commission some?
That paragraph doesn't really seem like an operationalization of the question, more like further statements on what the forecasters think will/might happen. In particular I'm not sure if those % forecasts are for a law being at least technically in place, vigorously enforced, or something else. E.g. if SB53 is still in place and there's no injunction but the federal govt has made California afraid to enforce it and so companies are basically ignoring it, would that count.
Might be useful to default to having the operationalizations in a footnote after the forecasts, or like a link near the top to a doc listing all the operationalizations. And maybe noting near the top of each post (or something) that most Qs have ~5 forecasters spending ~1-3hr each per Q. (Still also sometimes people should commission things, but this may slightly boost how useful the forecasts+posts you're already helpfully providing are, hopefully with little added time cost for your team.)
Thanks for the suggestions Michael! We were forecasting on whether a law would technically still be in place without an injunction, but did not consider enforcement.
Thanks for the post!
Is there a place readers can see the exact operationalizations of the questions your forecasters forecasted on? Or some indication of how many + what kind of forecasters forecast on these on average (e.g. 15 forecasters with xyz markers of good track records, or 5 with mixed track records), and how long they spend or what their track record on the forecasts for the newsletter are (e.g. in case they put less effort into these than their forecasts on Metaculus)?
E.g. for "forecasters think there’s a 60% (35% to 75%) chance that there will still be at least one federal or state regulation in force ...", that's somewhat lower than I'd have expected and so may be an important update to me. But I'm unsure what "in force" means (e.g. just "on the books" or also "vigorously enforced, defined in abc way"), and if the number + track records + time spent by the forecasters warrants a significant update from me.
Hi Michael: five forecasters, one to a few hours each. The operationalization is messy because the underlying reality is messy, we have a paragraph on it below:
> The operationalization of that question was a bit messy, given the four players involved in proposing, blocking or enabling such a regulation: Congress, the executive, states and the multiple levels of the judiciary. The forecasters’ consensus was that by the end of the year, there will be no federal law, an executive order aiming to restrict state regulations, and at least some state regulations. The executive order and state regulations will likely be embroiled in legal fights, but it’s unclear who will win it and on what merits. One forecaster noted that the Trump administration has not been skillful at wielding the Department of Justice to accomplish its goals, and suggests, with low confidence, that the federal administration will not get a federal judge to issue a preliminary injunction on state regulations. We think that this is an area where future forecasting efforts could be fruitful.
The full probabilities were: 60%, 65%, 75%, 65%, 35%.
I think to the extent you are making important decisions that depend on whether this is an 80% or a 60%, you should think about how much you value additional forecasting and maybe commission some?
Thanks!
That paragraph doesn't really seem like an operationalization of the question, more like further statements on what the forecasters think will/might happen. In particular I'm not sure if those % forecasts are for a law being at least technically in place, vigorously enforced, or something else. E.g. if SB53 is still in place and there's no injunction but the federal govt has made California afraid to enforce it and so companies are basically ignoring it, would that count.
Might be useful to default to having the operationalizations in a footnote after the forecasts, or like a link near the top to a doc listing all the operationalizations. And maybe noting near the top of each post (or something) that most Qs have ~5 forecasters spending ~1-3hr each per Q. (Still also sometimes people should commission things, but this may slightly boost how useful the forecasts+posts you're already helpfully providing are, hopefully with little added time cost for your team.)
Thanks for the suggestions Michael! We were forecasting on whether a law would technically still be in place without an injunction, but did not consider enforcement.
> In China, a jet-powered drone capable of carrying drone swarm payloads was flown for the first time.
video-game-based forecasting undefeated https://x.com/rai_sur11/status/1959977177497084170