Research Foundation
RBI draws on a published meta-analysis of self-efficacy and self-regulation and a performance meta-analysis currently under review. Together, they synthesize evidence across three outcomes while preserving variation among studies.
Meta-Analysis 1
Camargo, Parra-Martínez, Chang, Maeda, & Traynor (2024) — Educational Psychology Review
1,777 records screened. After removal of two influential outliers, nine primary studies (n = 2,793 students) yielded 16 and 15 effect sizes respectively.
The self-regulation effect (g = 1.00) is large by any conventional benchmark. Studies with the highest effects combined rubric use with formative feedback and explicit classroom training.
Meta-Analysis 2
Camargo & Parra-Martínez (under review)
39 primary studies — 80 effect sizes — N = 8,968 students. 83% of estimates positive. Range: −1.65 to 2.18.
Mean Effect Size
SE = 0.09, 95% CI [0.28, 0.66]
t = 4.91, p < .001
Prediction interval: [−0.72, 1.66]
Moderator Analysis
No tested moderator reached statistical significance. The directional patterns therefore guide questions about design and implementation; they should not be read as confirmed subgroup effects.
Rubrics co-created with students showed the highest effects (g = 1.13 vs. 0.45), though not statistically significant given sample sizes.
Co-creation
Without
The estimated effect was larger in studies combining rubrics with formative feedback than in rubric-only conditions (0.54 vs. 0.34). This descriptive difference was not statistically significant.
With feedback
Without
Analytic rubrics had a larger estimated effect than holistic rubrics (0.44 vs. 0.36), although the difference was not statistically significant. RBI favors analytic design because it also supports criterion-specific feedback.
Analytic
Holistic
Qualitative Findings
A qualitative content analysis examined 20 rubrics grouped by effect-size segment. The patterns below describe features observed within those groups; they do not establish that any single feature caused the observed effects.
Vague adjectives, task-focused rather than learning-focused, complex tasks without scaffolding.
Non-parallel descriptions, inconsistent verbs, unknown content, ambiguous quantifiers.
Action verbs, quantifiable metrics, clear adjectives, alignment with task and objectives.
Explicitly parallel descriptions, multi-source feedback, familiar tasks, socialized rubric language.
Presenting This Work
Rubricator, the AI-assisted rubric design tool built on this framework, is being presented as an Innovation Demonstration at the NCME AI in Measurement and Education (AIME) Conference in Pittsburgh, PA.
Rubricator: An AI Tool for Research-Aligned Rubric Design and Implementation