This form is an expression of interest for membership of the first round of the AI Resolution council.

Note that this round will last for ~1 year and likely be more experimental than future rounds, as we try to figure out what structures work best for this type of project.

________________________________________________________________________________________________________________________

BACKGROUND AND MOTIVATION

Getting well-calibrated forecasts about the future of AI is important to both policymakers, grantmakers and researchers in AI safety and policy.

Forecasting platforms such as www.metaculus.com have proven successful for this purpose, but they face an important bottleneck: questions must be clearly resolvable. This is hard for several important questions:

1. *Counterfactual questions* E.g. consider a question regarding an AI system successfully passing a specific capability test.  The test might be made very precise, but there is no guarantee the exact test described would actually be performed, so resolution would be ambiguous even if it were quite clear that a system that could pass the test exists.
2. *Similarity questions* E.g. If one benchmark becomes replaced by another, can we compare performance of algorithms across the two? If RL agents are trained on a new version of Dota, can we compare performance to OpenAI Five’s on Dota 2?
3. *Definition-of-terms questions* E.g. what counts as “hard-coded domain knowledge”? How much reward shaping can you add before the system no longer learns from “first principles”?
4. *Valuation questions* E.g. “How plausible does research agenda X seem?”, “How impressive was progress in field Y?”

Composing an expert council of senior AI researchers could serve as an “oracle” for these questions. The introduction of such a resolution mechanism would provide a rich expansion to the set of forecastable questions.

Moreover, the council would be an independent entity, providing open-source evaluations as a public good to be used by any forecasting project. (Current organisations with AI forecasting projects including Metaculus, the Open Philanthropy Project, OpenAI, and more.)

________________________________________________________________________________________________________________________

STRUCTURE OF COUNCIL

*Composition:* Initially >=5 members, in future possibly >=20. Members should:
---Have a technical background in AI (e.g. at least be PhD students)
---Represent a range of organisations
---Represent a range of technical fields (e.g. RL, unsupervised learning, NLP, safety, …)
---Have some understanding of forecasting (in particular, the importance of operationalizations and the possibility of divergence between “the letter and spirit of the law”)  

*Frequency of meetings:* once per question batch, which might occur roughly every quarter

*Estimated time-commitment per member & question batch:* 1 hour for initially evaluating questions, 2 hours for settling on the resolution of all questions that need resolving, and time-value of overhead cost ~2-6 hours

*Compensation:* members will be payed a symbolic honorarium, amount TBD

*Responsibility of members:* sign a public pledge to commit to answering questions to the best of their ability at a pre-specified date; then to attend the meeting and vote on the resolution (there are no requirements for justification)

*Support and management:* operations for the council will be handled by Parallel, LLC (Jacob Lagerros and Ben Goldhaber), supported by a grant from the Berkeley Existential Risk Initiative (http://existence.org/grants/). Parallel are also responsible for preparing pre-reading material for the meetings to inform decisions and save time; as well as filtering which requests are included in the evaluation batch

*Transparency:* at least membership of the council is public knowledge. Proceedings might follow Chatham House rules

*Opinion aggregation method:* For ordinary binary questions, majority vote (the only fair social choice function if we require a binary resolution)

*Deliverable:* Council verdicts are added to the AI Resolution dictionary to inform future cases, and this dictionary is made open-source available to any forecasting project

________________________________________________________________________________________________________________________

Please address any questions to: jacob@parallelforecast.com