Research Foundation
RBI is not derived from best-guess principles. It emerges from systematic synthesis of the empirical literature on rubric use.
Meta-Analysis 1
Camargo, Parra-Martínez, Chang, Maeda, & Traynor (2024) — Educational Psychology Review
1,777 records screened. After removal of two influential outliers, nine primary studies (n = 2,793 students) yielded 16 and 15 effect sizes respectively.
The self-regulation effect (g = 1.00) is large by any conventional benchmark. Studies with the highest effects combined rubric use with formative feedback and explicit classroom training.
Meta-Analysis 2
Camargo & Parra-Martínez (under review)
39 primary studies — 80 effect sizes — N = 8,968 students. 83% of estimates positive. Range: −1.65 to 2.18.
Mean Effect Size
SE = 0.09, 95% CI [0.28, 0.66]
t = 4.91, p < .001
Prediction interval: [−0.72, 1.66]
Moderator Analysis
No single moderator reached statistical significance, meaning the effect of rubrics is remarkably consistent. However, directional patterns point to actionable design and implementation factors.
Rubrics co-created with students showed the highest effects (g = 1.13 vs. 0.45), though not statistically significant given sample sizes.
Co-creation
Without
Studies combining rubric use with formative feedback outperformed rubric-only conditions (0.54 vs. 0.34).
With feedback
Without
Analytic rubrics (0.44) outperformed holistic rubrics (0.36). RBI exclusively recommends analytic design.
Analytic
Holistic
Qualitative Findings
A content analysis of 20 rubrics from the primary studies — categorized by effect size magnitude — identified specific linguistic and design features associated with large vs. negative effects.
Vague adjectives, task-focused rather than learning-focused, complex tasks without scaffolding.
Non-parallel descriptions, inconsistent verbs, unknown content, ambiguous quantifiers.
Action verbs, quantifiable metrics, clear adjectives, alignment with task and objectives.
Explicitly parallel descriptions, multi-source feedback, familiar tasks, socialized rubric language.
Presenting This Work
Rubricator, the AI-assisted rubric design tool built on this framework, is being presented as an Innovation Demonstration at the NCME AI in Measurement and Education (AIME) Conference in Pittsburgh, PA.
Rubricator: An AI Tool for Research-Aligned Rubric Design and Implementation