Skip to content
BackGOD PLAYS DICE™Big questions, honest answers

THE QUESTION LIBRARY HOW WE KNOW

How does science catch and correct its own mistakes?

How scientists check and correct mistakes, what failed replications can reveal, and why anomalies need evidence before they become discoveries.

THE SHORT ANSWER

Science catches mistakes when people challenge claims, inspect the work and try the tests again. Better evidence can lead to a revised explanation or a corrected research record. That makes correction possible; it does not guarantee that every error is found quickly, or that scientists always respond well.

  • Self-correction requires actual checking and revised public claims.
  • Honest error differs from misconduct; failed replication is not proof of fraud.
  • The 2015 psychology project is a bounded example, not a failure rate for all science.
  • Neptune illustrates a specific prediction followed by observation.
On this page

WHAT WE DON’T KNOW YET

These sources do not establish how many errors remain undiscovered across all scientific fields or how quickly any particular claim will be corrected. Failed replication alone does not distinguish fraud, error or differing conditions.

WHAT WOULD CHANGE THIS ANSWER

For a particular claim: new independent tests, a demonstrated analysis error or a substantiated investigation finding could change the assessment. No universal timetable or automatic correction guarantee.

Calling science self-correcting describes work people must do: expose a claim to checks, investigate what went wrong, and revise the public record when the evidence calls for it. The route matters more than the slogan.

THE LONG ANSWER

Free long answer 3 of 3 this month.

What does “self-correcting” actually mean?

It means a published claim can be checked and changed. Publication is a starting point for scrutiny, not protection from it. The National Academies’ 2019 review distinguishes two useful checks: rerunning calculations with the original data and code, and collecting new data to test the same question. The first checks the analysis; the second checks whether the finding holds again.

How are mistakes and fraud different?

A wrong answer is not automatically dishonest. In the 2000 U.S. federal research misconduct policy, making up results, misrepresenting research and taking others’ work without credit are distinguished from honest error or disagreement. That distinction matters: a failed test is not proof that someone cheated.

The policy describes inquiries, investigations and decisions about corrective action. It also includes correcting the research record among possible responses. These are concrete routes for investigating a concern, rather than asking readers to trust a promise that science will eventually fix itself. An accusation still needs evidence and a fair investigation.

What did the replication crisis reveal?

The phrase describes concern about published findings that later tests did not confirm. A prominent 2015 psychology project repeated 100 studies from three journals. Thirty-six percent of the repeat studies produced statistically significant results. That means their results crossed a statistical threshold; it is not a count of findings proved true or false.

The researchers used several measures of replication success and warned that no single measure was enough. Their results showed substantially weaker support for many original findings. They did not show that 64 percent of all science is false, or that every unsuccessful replication exposes fraud.

The National Academies’ review recommends clearer methods, uncertainty estimates and appropriate statistical practice. A failure to repeat can reflect an error, bias or an important difference between conditions. Finding out which requires further work. The lesson is to make claims easier to challenge, not to treat a single paper as the last word.

Can a strange anomaly become a discovery?

Yes, but an unexplained result is only the beginning. NASA describes how Neptune was found: Uranus did not move exactly as expected, and Urbain Le Verrier calculated where another planet might explain the difference. Johann Galle then found Neptune in 1846.

The important step was a prediction followed by observation. The puzzle did not establish the planet by itself. This example is a reason to investigate anomalies carefully, not a reason to assume every surprising measurement announces a revolution.

How should scientists change their minds?

A useful response asks what the new evidence changes. Does it undermine the result, narrow the conditions where it holds, or point to a different explanation? “We were wrong” and “this works only under these conditions” are different corrections. As a reader, ask to see the comparison: what was expected, what happened, and which claim was revised?

How confident should we be?

Confidence: strong that these checking and correction routes exist; no guarantee that they work promptly in every case. Unknowns: the sources here do not tell us how many errors remain undiscovered across all fields. What would change an assessment: for a particular claim, new independent tests, a demonstrated analysis error or an investigation finding can justify revising it. Ask which evidence bears on that claim.

Read alongside whether science proves things for certain and how to judge a scientific study.

Words worth knowing (3)
Reproducibility
Here, repeating calculations with the same data and code, following the National Academies’ 2019 terminology.
Replication
Testing the same question using newly collected data.
Anomaly
An observation that does not fit what was expected.

Sources & further reading

(4) · Checked Oct 5, 2026
  1. Reproducibility and Replicability in Science — Executive Summary ↗National Academies of Sciences, Engineering, and MedicineOfficial consensus report; definitions of same-data computational reproducibility versus new-data replication, reasons for failed replication, transparency and uncertainty recommendations. Checked 5 October 2026.
  2. Estimating the reproducibility of psychological science ↗Open Science Collaboration / Science; abstract hosted by U.S. National Library of MedicineOriginal research abstract: 100 studies from three psychology journals; 36% of replications statistically significant, with multiple success criteria. This is not a failure rate for all science. Checked October 5, 2026.
  3. Federal Research Misconduct Policy ↗U.S. Office of Science and Technology Policy; reproduced by Office of Research IntegrityDated official policy distinguishes fabrication/falsification/plagiarism from honest error and disagreement; describes inquiry, investigation, adjudication and correcting research record. Historical example of mechanisms, not comprehensive current legal advice. Checked 5 October 2026.
  4. Neptune: Facts ↗NASA ScienceDiscovery account: unexpected Uranus motion led to Le Verrier’s prediction and Galle’s observation of Neptune in 1846. Historical year verified; page checked October 5, 2026.

See a mistake? Report a problem. Corrections are made openly.

The Architect AI companion · All Access

Your next question can go further.

Included with All Access →Up to 5 questions a day while available.

Put God Plays Dice on your Home Screen

On iPhone: tap Share (on iOS 26, tap ••• first), then Add to Home Screen, then Add.

On Android: In Chrome, you can also open the ⋮ menu and choose Add to Home screen or Install.

The Home Screen version keeps its own memory on iPhone, so your nights and saved answers start fresh there.

Search the Question Library

149 answers to read · 0 questions awaiting answers Your search runs on your device.

Answers about death, grief and crisis are always free, in full. If a question is personal right now →