How can you tell good science from bad science?

What should a scientific claim let you check?

Begin with the claim itself. “This medicine helps some patients under these conditions” is different from “This medicine cures everyone.” “We saw an unusual signal” is different from “We discovered a new particle.” Good science states the question and the proposed answer clearly enough that a measurement could count for or against it. If the claim changes whenever a test fails, you cannot tell whether it has learned from evidence.

Next, ask what was observed. Were the measurements direct, or did researchers infer something from a pattern? Inference is an ordinary part of science. Geoscientists, for example, reconstruct past events from the record left in rocks. The issue is whether the inference rules out plausible alternatives. A study should describe instruments, samples, comparison groups where useful, choices made in analysis, and sources of uncertainty. This gives other people a chance to find mistakes or test the conclusion.

The National Academies' report on reproducibility says that when research is reported with a clear, specific and complete account of materials, methods, results and uncertainty, other scientists can know how to interpret the results. If crucial information is missing, confidence should be lower until it is supplied. Transparency is valuable even when another team cannot rerun the exact event, as with an earthquake or an epidemic. The same report says such studies rely on careful accounts of uncertainty, data from past events and checks of models against what happens.

Is there one scientific method?

No. School diagrams often show a tidy line from question to hypothesis to experiment to conclusion. That is a useful introduction to one style of inquiry, but actual work can loop back and branch. A National Academies report quotes the American Association for the Advancement of Science (AAAS): there is “no fixed set of steps that scientists always follow.” Researchers may observe first, build a model, discover a surprise, revise the question, design a better instrument, or compare competing explanations.

Methods must fit the subject. A lab may change one factor while holding others as steady as possible. A health study may compare groups of people instead of running a controlled trial; the National Academies notes that which kind of evidence carries the most weight depends on the question being asked. A geologist cannot recreate Earth's whole history but can test what different histories predict for rocks. An astronomer can use several kinds of light to test an idea about a distant galaxy. The strength lies in the fit between the question and the method, the care of the measurement, and the willingness to revise.

There are shared principles despite this variety: claims about the natural world need evidence; uncertainty matters; proposed explanations should face tests; and methods and results should be open to scrutiny. A method may be ingenious and still weak if it cannot distinguish the favored explanation from its rivals.

How do independent checks build confidence?

One research team can make an honest error or encounter an unusual sample. Another team using new data can test whether a result holds under similar conditions. The National Academies calls this replicability. Reproducing a computational result, in the report's terminology, means using the original data and analysis steps to see whether the same output appears. The words are used differently in some fields, so ask what was actually repeated.

Agreement between independent studies raises confidence. Agreement between different kinds of evidence can be even stronger: if a theory fits lab results, observations and new predictions, a single flaw is less likely to explain every line. Researchers also combine studies to see the whole pattern. A single failed repeat does not automatically prove the first finding false. Differences in conditions, random variation or a hidden factor may matter. The failure should prompt investigation rather than a slogan.

Uncertainty is part of an honest result. A number without a range or an account of possible error can look more precise than it is. Good scientists state limits and revise conclusions when better evidence appears. That willingness is a strength of the process, even if it can make headlines sound less satisfying.

What makes a source trustworthy?

Look for a named author with relevant expertise, a clear publisher, a date, and links to the evidence. Ask who pays for the work and whether the page is informing you, selling something, or arguing for a position. Read the linked study: a source can be accurately quoted while its result is stretched beyond what it tested.

The National Library of Medicine recommends checking ownership, purpose, funding, review processes, references and dates when evaluating health information. These questions are useful starting points for other science claims too. A familiar logo is a clue to investigate, not a substitute for the evidence. For health decisions, its guidance also recommends discussing what you find with your health professional.

Check independent sources, and look for corrections or later tests. Several websites repeating the same paper are still one line of evidence. Peer review adds specialist scrutiny; the evidence still has to hold up.

Does peer review settle the matter?

No. Peer review means other researchers examine a paper before publication, usually to assess its methods, reasoning and place in the field. The National Academies lists rigorous peer review among the practices that reduce avoidable errors. It still has limits. Richard Smith, a former editor of the medical journal BMJ, wrote that a review that asks for raw data and repeats the analyses is “vanishingly rare.” In BMJ studies where major errors were deliberately inserted into papers, he reports, most reviewers spotted only about a quarter of them. His essay is one critic’s view. The practical lesson for a reader is modest: publication is a starting point for checking, not the end of it. A published paper is a piece of evidence, not a final stamp of truth. A preprint is a paper shared before this journal review; it may still be useful, but its status should be clear.

The strongest objection to a simple “good versus bad” label is that research quality lies on a range. A preliminary study can be careful yet uncertain. A celebrated study can contain a flaw. A negative result can be valuable if the test had enough power to find the effect it sought. A surprising finding deserves interest and careful checking, not automatic belief or dismissal. Look at what the study actually did, and compare it with the claim made about it.

Watch for warning signs: a conclusion much larger than the data support; no clear method; selective use of results; no discussion of alternatives; or claims that criticism itself proves the idea right. These signs call for closer examination. They do not let you judge a person from a headline alone.

What do people often get wrong about uncertainty?

“Scientists disagree” can mean several things. They may have measured different populations, used different tools, or asked slightly different questions. Their results may overlap once uncertainty is considered. Sometimes they truly disagree, and new work is needed. The National Academies says debate and dissenting views are built into the process of science, and its public guide adds that when studies disagree, it is useful to find out why.

Another mistake is thinking that because some findings change, none deserve confidence. Evidence can become very strong even though future work may refine it. The National Academies describes knowledge as built through repeated tests and multiple lines of evidence. At the other extreme, certainty language can hide uncertainty that matters to a decision. Ask how firmly the specific claim is supported, not whether all of science is either perfect or broken.

How can you check this yourself?

Take a claim you saw in a headline and find the original paper or the research organization's account. Write down its exact question, who or what was studied, the main measurement, and the largest uncertainty. Check whether the headline describes the same claim. Then look for another group testing the issue, or a review that compares multiple studies. If the finding is new, it is fair to say “interesting, but early.” Our guide How to Check a Viral Science Claim in Five Minutes and the Library answer on extraordinary claims and evidence walk through the same habit.

The National Academies' public “Decoding Science” guide gives a simple path: wonder (ask questions), test, and confirm. Its longer report explains why the path is not a rigid recipe. Read both and notice the difference between a useful beginner's map and the actual variety of science. Finally, ask the question that keeps any explanation honest: what result would make me change my mind?