GED Science Scientific Practices Evaluating Evidence Sources Bias Replication

What Is This?

Evaluating Evidence: Sources, Bias, Replication is the systematic process of assessing the credibility, reliability, and relevance of scientific information. It involves critically examining the sources, identifying potential biases, and verifying the results through replication.

This topic appears in exams to test your ability to think critically, analyze complex information, and make informed decisions based on evidence. You can expect questions that require you to evaluate the credibility of sources, identify biases, and assess the validity of research findings.

Why It Matters

This topic is frequently tested in exams, such as the Advanced Placement (AP) Biology, Chemistry, and Physics exams, as well as in graduate-level exams like the Graduate Record Examination (GRE). It typically carries around 20-30% of the total marks and tests your ability to evaluate evidence, identify biases, and think critically.

Core Concepts

To tackle this topic, you need to own the following foundational ideas:

• Sources: Identify the credibility and reliability of sources, including primary and secondary sources, peer-reviewed journals, and online resources.
• Bias: Recognize and evaluate potential biases in research, including selection bias, confirmation bias, and publication bias.
• Replication: Understand the importance of replication in verifying research findings and the potential consequences of failing to replicate results.

Prerequisites

Before tackling this topic, you should already understand:

• The scientific method and the importance of evidence-based reasoning
• Basic statistical concepts, including mean, median, and standard deviation
• The principles of research design, including experimental and quasi-experimental designs

If you're missing these prerequisites, you may struggle to understand the underlying concepts and principles of evaluating evidence.

The Rule-Book (How It Works)

The primary rule of evaluating evidence is to critically examine the sources and identify potential biases. This involves:

• Evaluating the credibility and reliability of sources, including peer-reviewed journals and primary sources
• Recognizing and addressing potential biases, including selection bias and confirmation bias
• Verifying research findings through replication

Sub-rule 1: When evaluating sources, look for signal words such as "peer-reviewed," "primary source," and "experimental design."

Sub-rule 2: When identifying biases, consider the researcher's perspective and the study's limitations.

Sub-rule 3: When verifying research findings, look for replication studies and meta-analyses.

Exam / Job / Audit Weighting

Frequency: 20-30% Difficulty Rating: Intermediate Question Type or Real-World Task Type: Multiple-choice questions, short-answer questions, and case studies

Difficulty Level

Intermediate

Must-Know Rules, Formulas, Standards, or Principles

Here are the three most important rules for evaluating evidence:

1. The credibility rule: Evaluate the credibility and reliability of sources, including peer-reviewed journals and primary sources.
2. The bias rule: Recognize and address potential biases, including selection bias and confirmation bias.
3. The replication rule: Verify research findings through replication and meta-analyses.

Worked Examples (Step-by-Step)

Here are three solved examples that escalate in difficulty:

Easy Example
A study published in a peer-reviewed journal finds a significant correlation between exercise and weight loss. What is the credibility of the source?

Answer: The source is credible because it is a peer-reviewed journal.

Key rule applied: The credibility rule

Medium Example
A study finds a significant difference in the effects of two medications on blood pressure. However, the study was funded by one of the medication manufacturers. What is the potential bias in the study?

Answer: The potential bias is selection bias, as the study was funded by one of the medication manufacturers.

Key rule applied: The bias rule

Hard Example
A study finds a significant correlation between a new diet and weight loss. However, the study was not replicated, and the results were not verified through meta-analyses. What is the validity of the study?

Answer: The study is not valid because it was not replicated and the results were not verified through meta-analyses.

Key rule applied: The replication rule

Common Exam Traps & Mistakes

Here are four specific errors that cost marks in exams:

• Mistake 1: Failing to evaluate the credibility of sources
• Mistake 2: Ignoring potential biases in research
• Mistake 3: Failing to verify research findings through replication
• Mistake 4: Assuming that a single study is sufficient evidence for a claim

Trap 1: A study published in a non-peer-reviewed journal finds a significant correlation between exercise and weight loss. What is the credibility of the source?

Wrong answer: The source is credible because it is a study published in a journal.

Why it looks right: The study appears to be a legitimate source of information.

Correct approach: Evaluate the credibility of the source by looking for signal words such as "peer-reviewed" and "primary source."

Shortcut Strategies & Exam Hacks

Here are some practical techniques to solve questions faster or more accurately under time pressure:

• Memory aid: Use the PEER acronym to remember the credibility rule: Peer-reviewed, Experimental design, Evidence-based, and Reliability.
• Elimination strategy: Eliminate options that are clearly incorrect, such as studies that have not been peer-reviewed or have significant biases.
• Pattern recognition: Recognize patterns in research design, such as experimental and quasi-experimental designs.

Question-Type Taxonomy

Here are the three distinct question formats this topic appears in across different exams:

Practice Set (MCQs)

Here are five multiple-choice questions at mixed difficulty levels:

Question 1
What is the credibility of a study published in a non-peer-reviewed journal?

A) High B) Medium C) Low D) Uncertain

Correct answer: C) Low

Explanation: The study is not credible because it is not peer-reviewed.

Why the distractors are tempting:

• A) High: The study appears to be a legitimate source of information.
• B) Medium: The study may have some credibility, but it is not peer-reviewed.
• D) Uncertain: The credibility of the study is unknown.

Question 2
What is the potential bias in a study that finds a significant difference in the effects of two medications on blood pressure?

A) Selection bias B) Confirmation bias C) Publication bias D) None of the above

Correct answer: A) Selection bias

Explanation: The potential bias is selection bias, as the study was funded by one of the medication manufacturers.

Why the distractors are tempting:

• B) Confirmation bias: The study may have confirmed the expected results.
• C) Publication bias: The study may have been published because it had significant results.
• D) None of the above: The study may not have any biases.

Question 3
What is the validity of a study that finds a significant correlation between a new diet and weight loss?

A) High B) Medium C) Low D) Uncertain

Correct answer: C) Low

Explanation: The study is not valid because it was not replicated and the results were not verified through meta-analyses.

Why the distractors are tempting:

• A) High: The study appears to have significant results.
• B) Medium: The study may have some validity, but it was not replicated.
• D) Uncertain: The validity of the study is unknown.

Question 4
What is the credibility of a study published in a peer-reviewed journal?

A) High B) Medium C) Low D) Uncertain

Correct answer: A) High

Explanation: The study is credible because it is published in a peer-reviewed journal.

Why the distractors are tempting:

• B) Medium: The study may have some credibility, but it is not peer-reviewed.
• C) Low: The study may not be credible because it is not peer-reviewed.
• D) Uncertain: The credibility of the study is unknown.

Question 5
What is the potential bias in a study that finds a significant difference in the effects of two medications on blood pressure?

A) Selection bias B) Confirmation bias C) Publication bias D) None of the above

Correct answer: A) Selection bias

Explanation: The potential bias is selection bias, as the study was funded by one of the medication manufacturers.

Why the distractors are tempting:

• B) Confirmation bias: The study may have confirmed the expected results.
• C) Publication bias: The study may have been published because it had significant results.
• D) None of the above: The study may not have any biases.

30-Second Cheat Sheet

Here are the five things you need to remember walking into the exam hall:

• Credibility: Evaluate the credibility of sources by looking for signal words such as "peer-reviewed" and "primary source."
• Bias: Recognize and address potential biases, including selection bias and confirmation bias.
• Replication: Verify research findings through replication and meta-analyses.
• Signal words: Use signal words such as "peer-reviewed," "primary source," and "experimental design" to evaluate credibility.
• PEER: Use the PEER acronym to remember the credibility rule: Peer-reviewed, Experimental design, Evidence-based, and Reliability.

Learning Path

Here is a suggested study sequence to master this topic from scratch to exam-ready:

1. Beginner foundation: Understand the scientific method and basic statistical concepts.
2. Core rules: Learn the credibility rule, bias rule, and replication rule.
3. Practice: Practice evaluating sources, identifying biases, and verifying research findings through replication.
4. Timed drills: Practice timed drills to improve your speed and accuracy.
5. Mock tests: Take mock tests to assess your knowledge and identify areas for improvement.

Related Topics

Here are three closely connected topics that appear alongside this one in exams:

• Research design: Understand the principles of research design, including experimental and quasi-experimental designs.
• Statistical analysis: Understand basic statistical concepts, including mean, median, and standard deviation.
• Evidence-based reasoning: Understand the importance of evidence-based reasoning in scientific inquiry.