CalSERVES Expanded Learning Impact Evaluation

The Napa County Office of Education’s CalSERVES AmeriCorps Program (CalSERVES) is a collective impact model of student support. Through the model, AmeriCorps members provide individualized tutoring and mentoring to more than 500 students in afterschool programs and schoolwide activities to improve the school climate.

Study Goals:

The CalSERVES Expanding Learning Program has previously fully implemented the collective impacts model and has tracked student academic and social-emotional improvement. The program was then interested in conducting an additional layer of evaluation to see if these changes were also being reflected in the California State Smarter Balanced Assessments.

Research Questions:

The study included both confirmatory evaluation and exploratory research questions. The research questions were:

• Evaluation Questions:
  • As measured by the California State Smarter Balanced Assessments, do CalSERVES participants who receive tutoring demonstrate improved academic performance when compared to similar students who do not receive tutoring?
  • As measured by the districtwide Walker Survey Instrument, do CalSERVES participants who receive mentoring services demonstrate improved social and behavioral skills when compared to similar students who do not receive mentoring?
• Exploratory Research Questions:
  • Do CalSERVES participants who receive high levels of tutoring or mentoring show greater benefits than those who receive less?
    • Do CalSERVES participants who receive 10 or more tutoring sessions demonstrate improved academic performance compared to participants who received less than 10 tutoring sessions?
    • Do CalSERVES participants who receive 10 or more mentoring sessions demonstrate improved social or behavioral skills compared to participants who received less than 10 mentoring sessions?
    • Does the number of CalSERVES tutoring or mentoring sessions received predict improved academic performance or social and behavioral skills?
  • Does the effect of CalSERVES program participation vary by grade?

Findings:

The study found the following:

• Students in Grade 4 who received STEM tutoring had significantly higher math percentile scores at post-test and were significantly more likely to have performance level improvements.
• There were no statistically significant differences between students who received any STEM tutoring and those who did not for Grades 5-7.
• There were no statistically significant differences between students who received literacy tutoring and those who did not for Grades 6-8.
• Overall, there were no statistically significant differences between the Walker Survey Instrument’s social emotional behavioral assessment test scores of students who participated in CalSERVES and those who did not.

For more information, download the full report.