Publications

Scientific research
This category includes research reports that offer quantitative or qualitative evidence to support conclusions. These reports vary in terms of research rigor, scale of data collection, and scope of conclusions. Some are case studies that primarily describe a single program or the experiences of individuals. Other studies are more quantitative in nature and may use large-scale data collection and sophisticated sampling techniques.

Rigorous research
This category is exclusively for reports that meet the canons of high-quality research. These reports meet the majority of criteria used to determine the appropriateness and adequacy of the design, methods, and interpretation. Slightly different criteria were used for quantitative and qualitative research. Both sets of criteria are listed below.

Quantitative Research: Criteria for Rigor

Relevance:

  • The study provides insight into issues surrounding the topic in question.
  • The research questions posed are significant and can be investigated empirically.
  • The study was conducted relatively recently in the United States (e.g., in the last two decades).

Research Participants:

  • Population and sample are clearly defined.
  • Sample is representative of the population in clearly defined ways (appropriate/systematic scale).
  • Procedures for participant assignment or selection and any incentives are described.
  • Attrition is estimated, along with the procedures for preventing attrition and/or replacing participants (as necessary).

Research Apparatus:

  • Apparatus is clearly described.
  • Apparatus permits the most direct investigation of the research questions posed.
  • Multiple data collection methods are used.

Research Procedure:

  • Procedures are appropriate for answering the research questions posed.
  • Description of procedures permits replication.
  • Selection and measurement of variables was valid and reliable.
  • Data are thoroughly and systematically collected.
  • Data analytic treatment is appropriate to the problem and type of data collected.

Research Findings and Conclusions:

  • Appropriate statistics, including effect size or other indicators of meaningfulness, are presented.
  • Judgments about meaningfulness, validity, and reliability can be readily made from the information presented.
  • Interpretations and conclusions are defensible (given evidence), based on clear relationships among data, procedures, analysis to the research questions.
  • Similarities/differences between the current study's findings and others' are discussed.
  • Limitations and alternative explanations for the research are offered.
  • Implications for researchers, policymakers, and practitioners are provided.

Qualitative Research: Criteria for Rigor

Relevance:

  • The study provides insight into issues surrounding the topic in question.
  • The research questions posed are significant and can be investigated empirically.
  • The study was conducted relatively recently in the United States (e.g., in the last two decades).

Research Participants:

  • Procedures for participant selection and any incentives are described.
  • Attrition is estimated, along with the procedures for preventing attrition and/or replacing participants (as necessary).

Research Apparatus:

  • Apparatus is clearly described.
  • Apparatus permits the most direct investigation of the research questions posed.
  • Multiple data collection methods are used.

Research Procedure:

  • The method was justified because the study was characterized by a small sample size, data that were difficult to quantify, or phenomena for which no existing hypotheses had previously been developed.
  • Selection and measurement of variables was valid and reliable.
  • Data are thoroughly and systematically collected.
  • Data analytic treatment is appropriate to the problem and type of data collected.

Research Findings and Conclusions:

  • Interpretations and conclusions are defensible (given evidence), based on clear relationships among data, procedures, analysis to the research questions.
  • Similarities/differences between the current study's findings and others' are discussed.
  • Limitations and alternative explanations for the research are offered.
  • Implications for researchers, policymakers, and practitioners are provided.