By Design

Planning Research on Higher Education

Richard J. Light

Judith D. Singer

John B. Willett

    1. Why Do Research On Higher Education?

    Many Questions, Many Options

    Our Philosophy of Research Design

    How This Book is Organized

    2. What Are Your Questions?

    Why Are Research Questions So Important?

    Getting Specific

    Building on the Work of Others

    Correlation versus Causation

    The Wheel of Science

    3. What Groups Do You Want to Study?

    Specifying the Target Population

    Where Should you Conduct the Study

    Selecting Your Sample

    More Than One Type of Respondent

    Nonresponse Bias

    4. What Predictors Do You Want to Study?

    Types of Predictors

    The Important Role of Variation

    Other Reasons for Selecting Predictors

    The Integrity of Your Treatment

    Choosing Which Predictors to Study

    5. Compared to What?

    Why Do You Need a Comparison Group?

    Randomized Control Groups: The Best Comparisons

    Requiring Informed Consent

    Volunteer Bias

    Comparison Groups without Random Assignment

    Retrospective Case-control Studies

    Design Effects Can Swamp Treatment Effects

    6. What Are Your Outcomes?

    Different Kinds of Outcomes

    Will You Measure Status or Development

    Short-term versus Long-term Effects

    Are Your Measures Valid?

    7. How Can You Improve Your Measures?

    What is Measurement Error?

    Reliability and Measurement Error

    Six Strategies for Improving Measurement Quality

    Looking at Measurement Quality

    8. How Many People Should You Study?

    Why Is Sample Size So Important?

    What Size Effect Do You Want to Detect?

    What Type of Analysis Will You Use?

    Instruments Precision and Sample Size

    What If Students Drop Out?

    9. Should You Try It out on a Small Scale?

    The Advantages of Pilot Studies

    Piloting Instruments

    Relational Studies

    Informal Small-scale Experiments

    Generalizing From a Small Study

    10. Where Should You Go From Here?

    Getting Started

    Lessons From Our Seminar

    Decisions You Must Make

    Planning a Longer-term Research Program

    Reference

    Index

  • Cover: By Design

Richard J. Light is Professor in the Graduate School of Education and the John F. Kennedy School of Government at Harvard University.

Judith D. Singeris an Associate Professor at the Graduate School of Education, Harvard University.

John B. Willett is an Associate Professor at the Graduate School of Education, Harvard University.