Research Design & Stats
Critiquing Research Articles
Life care planners have a wealth of rehabilitation and medical information available, but must be able to discern which studies are of merit. Refer to research information from your own library and develop a critique form you can use to evaluate research articles.
Consider each of the primary sections within a research report (i.e., title, abstract, introduction, literature review, methods, results, discussion) and the questions which should be asked within each section. Create your document for use in your practice to guide you in consistently reviewing research articles. Here are some suggestions:
I. The Title
Did the title adequately describe the study?
Would appropriate keyword searches retrieve this article from an indexing system?
Was the title concise and free of extraneous words/phrases?
II. The Abstract
Did the abstract summarize the purpose, methodology, and findings of the study?
Did the abstract specify the independent and dependent variables?
Were all primary findings mentioned in the abstract?
Did the abstract provide enough information for a reader to determine whether the article was relevant to his/her research efforts?
What type of research most accurately describes this study?
III. The Introduction: The Research Problem
Was the problem clearly defined?
Is the problem significant enough to warrant investigation?
Was the problem logically deduced from a particular theory, or set of theories?
IV. The Literature Review
Does the literature review logically lead to the Methods section?
Are the hypotheses and research questions clearly stated and indicate the anticipated findings?
Identify the independent variable(s).
Identify the dependent variable(s).
V. The Method: Procedure
Are the procedures thoroughly described in chronological order?
Were the treatments and/or data collecting methods described so that you could replicate the study?
Were the treatments administered so that extraneous sources of error were either held constant for all treatments and control groups or randomized among subjects within groups?
VI. The Method: Design
Was the research design explicitly identified?
Was an appropriate research design selected in the analysis of the problem?
Was the population clearly defined?
Were the sampling methods clearly described?
Was a control or comparison group chosen in the same manner and from the same population as the sample?
Were the treatments randomly assigned to the groups?
VII. The Method: Sample
Is the sample clearly described in terms of size and demographics?
Is inclusion and exclusion criteria explicitly stated?
Was the sample appropriately identified by the researcher, based on the purpose of the study?
How were individuals assigned to treatment groups?
Were subjects offered incentives for participation?
VIII. The Method: Instrumentation
Are the instruments described in the study appropriate measures of the variables being investigated?
Did the author include the relevant psychometric properties (i.e., validity, reliability) of the instruments used throughout the study?
Are all of the materials listed and clearly described?
IX. The Results: Measurement
Is the Results section clearly written and logically organized?
Is the analytical procedure appropriate to the research design and hypotheses of the study?
Do the reported results address all aspects of the hypotheses/research questions?
Was any evidence of reliability of the measurements given?
Was any evidence of the validity of the measurement given?
Were tables and graphs easily interpreted and helpful to the reader in conceptualizing the results?
X. The Discussion
Were the conclusions consistent with the obtained results?
Were the generalizations confined to the population from which the sample was drawn?
Were the limitations of the study clearly discussed?
Are findings discussed in terms of the research problem, hypotheses, and purpose of the study?
Are suggestions for future research provided?
Are implications for the field of study discussed?
XI. The References
Are the citations sufficiently current?
Are “classic” studies within the body of literature of the discipline referenced?
Do the works cited represent a comprehensive review of the existing literature within the discipline?
General (Your Opinion)
Was the article well-written and well-organized?
What did you learn from the article?
Did the study address an important issue affecting individuals with disabilities or catastrophic injuries?
What are the strengths of this study?
How could this study have been improved?
Was the study important? Why?
These questions were adapted from:
Bellini, J. & Rumrill, P. (1999). Research in rehabilitation counseling: A guide to design, methodology, and utilization. Springfield, IL: Charles C. Thomas.
Farquar, W., & Krumboltz, J. (1959). A checklist evaluating experimental research in psychology and education. Journal of Educational Research, 52, 353-354.