The Brazilian Reproducibility Initiative is a systematic, multicenter project to replicate between 60-100 experiments from Brazilian life sciences literature. Each of the selected experiments will be replicated by 3 independent laboratories from a network of over 60 collaborating labs, with results due in 2021. It is the first national-level effort to evaluate scientific reproducibility rates, inspired by other systematic replication projects conducted in the fields of psychology, cancer biology and social sciences. The results will allow us to estimate the reproducibility rate of Brazilian biomedical science and to investigate aspects of published literature that can be used to predict it, as well as provide data in interlaboratory variability.

The first wave of experiments will involve three different laboratory techniques for outcome detection: MTT assay (a method to evaluate viability in cell cultures), RT-PCR (a method to amplify and detect specific mRNA sequences) and elevated plus maze (a behavioral task to measure anxiety in rodents). 20 experiments from Brazilian articles over the last 20 years have been selected for replication in at least 3 different laboratories, with replications expected to start in early 2020.

If you are interested in reading more about the project, you can find additional information at our OSF page (https://osf.io/6av7k/), website (https://www.reprodutibilidade.bio.br/home) or feature article at eLife (https://elifesciences.org/articles/41602), or contact reprodutibilidade.br@gmail.com

The aim of this project is to study whether researchers can accurately predict which of the experiments selected for the Brazilian Reproducibility Initiative will be successfully replicated. This will be done in two parts: a survey of beliefs and a prediction market. Separate surveys and markets will be held for each of the 3 methods in the first wave of experiments in the Brazilian Reproducibility Initiative (MTT assay, RT-PCR and elevated plus maze). You can choose to participate only in the survey or in both the survey and prediction markets. The study is being coordinated by Tiago Bortolini (Instituto D'Or de Pesquisa e Ensino - IDOR),  Anna Dreber (Stockholm School of Economics) and Olavo Amaral (Universidade Federal do Rio de Janeiro) and has been approved by the Research Ethics Comittee (CEP) at IDOR.

Survey of Beliefs
In the survey of beliefs, you will be asked for (a) the probability that each of the 20 studies using each method will be successfully replicated (defined by the finding of a statistically significant result, defined by p < 0.05, in the same direction as in the original study in a meta-analysis of the three replications) and (b) the expected effect size for the replication. This will be done via an online questionnaire collected through SurveyMonkey and should take between 1 and 2h to complete for each set of 20 experiments.

Prediction Markets
In the prediction market, participants will bet on the possibility that each study will replicate successfully (using the same criterion described above). All participants will get US$50 in credits on their prediction account to trade with. By trading on ‘stocks’ for each study (similar to those in a stock market), which will be paid only when a study replicates, participants can earn more or less money depending on how they bet on different hypotheses. On average participants, will earn about $50 as Amazon gift cards, and payments will be resolved once we finish obtaining replication results (due in 2021). Prediction markets will be held for 2 weeks via an online platform from Cultivate Labs (https://www.cultivatelabs.com). After finishing the survey, bets can be done in less than an hour and updated as desired throughout the 2 weeks.

If you are a researcher or have had previous experience with experimental research (including as a student) you are eligible to participate and can fill our preliminary registration below. Once the study begins - which is expected to happen in mid-November for the survey and early December for the markets - we will e-mail you a link to formally enroll in the study.

If you wish, you can also read more about surveys of beliefs and prediction markets in other reproducibility projects:  

-Using prediction markets to estimate the reproducibility of scientific research - Anna Dreber et al. 2015 https://doi.org/10.1073/pnas.1516179112

-Evaluating the replicability of social science experiments in Nature and Science between  2010 and 2015 - Camerer et al. 2018 https://doi.org/10.1038/s41562-018-0399-z

-Predicting replication outcomes in the Many Labs 2 study - Forsell et al. 2018 https://doi.org/10.1016/j.joep.2018.10.009

If you need further information, feel free to contact reprodutibilidade.br@gmail.com