The value of multi-lab replication studies as a means of evaluating psychological theories
According to Popper, we can never be sure that a theory is true. A good theory has to be supported by multiple studies, testing a variety of hypotheses derived from that theory.
What then are the implications, if one of the studies supporting the theory cannot be replicated? The basic viewpoint here is that the non-replication is just a research outcome, and as such not final, but open to the same criticism as the original study it failed to replicate. Various reasons for non-replication and the consequences of non-replication are discussed, illustrated by some concrete examples of research in the field of social psychology:
- What are the problems of exact versus conceptual replications?
- What are the consequences of methodological problems of the replicated study (e.g., lack of power, questionable research practices)?
- What are the consequences of methodological problems with the replication?
- Finding a black swan: Can non-replication of a single study that supported a theory falsify that theory?
- Can non-replication be used as an indication of questionable research practices?
- Is a successful replication of a finding an indication of the absence of scientific misconduct?
- When can a theory be considered as sufficiently falsified or supported?
- How useful is investing vast resources in multi-lab replications of single studies?