EXPERIMENT IN SMALL-GROUP HOMEWORK TUTORING FOR REMEDIAL MATHEMATICS STUDENTS: PRELIMINARY RESULTS

Alice Welt Cunningham and Olen Dias with Nieves Angulo

INTRODUCTION

This paper presents the preliminary results of an 18-section experiment conducted during the Fall 2010 semester regarding the impact of small-group homework-completion tutoring on the performance of Hostos’ remedial mathematics students. The research in question was performed pursuant to a grant, Improving Undergraduate Mathematics Learning: The Effect of Small-Group Homework Tutoring on Remedial Mathematics Learning, from the CUNY Central Office of Academic Affairs. Permission from Hostos’ Institutional Review Board was granted for the conduct of the experiment and for the dissemination of the results.

The remedial mathematics classes in question are the two courses, Math 010 (Basic Math Skills) and Math 020 (Elementary Algebra), directed toward passing the two levels of the CUNY-wide COMPASS exit test (M1 and M2, respectively) which are necessary for graduation and college level work. Such courses meet four days per week, three times with an instructor and the fourth with a tutor. Traditionally, the tutor-led meeting, called the Math Lab, has followed one of two formats: (1) additional pencil-and-paper exercises related to that week’s class-work or

(2) tutoring using Pearson Publishing’s MathXL interactive online textbook-based homework assignments. Most of the full-time and many of the adjunct instructors of these courses now require the use of MathXL.

This experiment focused on the tutoring component of the two remedial mathematics courses. Prior research indicates that small-group homework- completion tutoring improves student performance not only in the current but in subsequent mathematics courses (e.g., Hagedorn, Sahger, Siadhat, 2000; Perrin, 2004; Harootounian Quinn, 2008). In addition, an earlier Hostos study showed the efficacy for mathematics performance of using MathXL homework (Menil Dias, 2008). Thus, we hypothesized that having additional tutors available in the weekly Math Lab meeting of each course to lower the tutor/student ratio would facilitate the students’ completion of their MathXL homework, improving their performance on both the COMPASS and in class. Moreover, because mathematics learning is cumulative (e.g., National Academy of Education, 2009; National Mathematics Advisory Panel, 2008; National Research Council, 2001), and requires active student problem-solving rather than passive note-taking (e.g., Menil Dias, 2008), we took the view that assistance in completing assigned homework as soon as possible after each class was particularly important. Thus, the tutoring component of the course constituted the independent variable, and COMPASS performance and final class grades the dependent variables. These issues are important, as difficulty completing mathematics courses constitutes a significant contributing factor to low graduation rates, both at CUNY (e.g., Hinds, 2009; Hostos Community College, 2008) and nationwide (Biswas). This paper reports on the COMPASS component of the study.

METHOD

Study Design

As each of the participating students self-selected the respective sections through online registration without previous knowledge of the experiment, the research reflects a quasi-experimental design (DePree, 1998). The research involved 18 sections, 9 for each of the two remedial courses. Of those nine, six (three experimental with five tutors per section and three control with one tutor per section), used the Math Labs for MathXL homework-completion tutoring. The remaining three of the nine sections for each course, again with one tutor each, used the Math Labs for the departmentally prepared pencil-and-paper exercises. Thus, for each of the two remedial courses, the Math Lab classes involved three experimental sections with multiple tutors using MathXL for the Math Lab classes (E), three control sections with the traditional single tutor using MathXL (C1), and three control sections with the traditional single tutor using pencil-and-paper exercises (C2).

Classes ranged in size from 27-30 students. For the Basic Math Skills course, the experimental sections had 89 students, the C1 sections had 90 students, and the C2 sections had 86 students, for a total of 265 students. For the Elementary Algebra course, the three experimental sections had 88 students, the C1 sections had 89 students, and the C2 sections had 87 students, for a total of 264 students. Thus, overall, the experiment involved 529 students, of whom 177 (or 33.5%) were in the two experimental cohorts.

Data were collected regarding the gender and ethnic background of the students, as well as the number of their previous attempts at passing the COMPASS exit exam. Approximately ten percent of the students were under the age of 18. As the Hostos IRB required documentation of student consent only for those students under the age of 18, no data were collected regarding the ages of the remaining students.

In order to control for teaching variations, each of the pairs of sections using MathXL homework completion tutoring (E and C1) was taught during early morning hours by the same full-time daytime instructor. The pencil-and-paper sections were taught, again during early morning hours, primarily by adjunct instructors. With one exception, all of the instructors had substantial previous experience in teaching their respective courses. Additional non-experimental sections not included in the original 18-section study design were taught by the instructors involved in both the C1 and C2 sections. Results from such sections may be taken into account as our analysis expands.

Study Effectuation

Due primarily to state budget constraints that delayed receipt of the grant funding, the research was not effectuated entirely in accordance with its design. While both sets of control sections (C1 and C2) had tutors available for the Math Lab sections by the third week of the 14-week semester, tutors for the experimental sections were not available until the sixth week of that semester. Therefore, the Math Lab tutors supplied to both sets of control sections came from a pool of pre-existing tutors, all with previous experience in teaching the respective courses, and many with higher-level credentials. By contrast, the new tutors were drawn primarily from Hostos students working towards their own degrees and therefore with no previous experience in teaching these or other courses. Finally, again because of the delay in hiring the experimental tutors, training by Pearson Publishing of the tutors in the use of MathXL was deferred from before the beginning of the semester until midway through the semester (the 7th of 14 weeks) and occurred just once rather than twice.

PRELIMINARY RESULTS

Initial results for each course (Basic Math Skills and Elementary Algebra) are attached as Appendices I and II at the end of this article.

Basic Math Skills

Table 1 reflects three comparisons for the experimental and two control groups: the COMPASS certification rate, the COMPASS pass rate of those students certified, and the COMPASS pass-rate of the whole class. (Instructors determine their students’ eligibility, or certification, to take the COMPASS based primarily on the students’ scores on a departmental midterm designed for this purpose.) In each instance, although the C1 group performed better than the experimental group (56% v. 51% certified; 81% v. 67% certified pass rate; and 43% v. 35% whole class pass rate), the experimental group uniformly outperformed the C2 (pencil- and-paper) group (51% v. 45% certified; 67% v. 52% certified pass rate; and 35% v. 24% whole-class pass rate). Thus, the data strongly support not only the efficacy of student problem-solving using the interactive MathXL homework software, but also the importance of beginning homework-completion assistance as early in the semester as possible.

Table 1

MAT 010: Average Percentages of certification rate and pass rate for each group.

Elementary Algebra

As shown by Table 2, the results for the upper level remedial course are less probative. Although the experimental group outperformed the C1 groups in terms of the COMPASS certification rate (53% v. 52%), both the COMPASS pass rates of those students certified (57% E v. 73% for C1 and 69% for C2) and of the whole class (31% E v. 38% for C1 and 43% for C2) were higher for the two control groups than for the experimental group.

Table 2

MAT 020: Average Percentages of certification rate and pass rate for each group.

ANALYSIS AND SUMMARY

Without taking into account the delay in the provision of tutors and the differential experience level of those tutors for the experimental cohorts for both courses, the C1 groups (using MathXL with a single tutor) performed better than the experimental groups (using MathXL with multiple tutors). For Basic Math Skills, the COMPASS certification rate for the experimental group was only slightly lower (9%) than for the C1 group (56% C1 cert rate minus 51% E cert rate = 5 percentage point difference; 5 p.p. difference/ 56% C1 cert rate = 9% lower E cert rate). However, the certified pass rate for the experimental group was 17% lower than for the C1 group (81% C1 pass rate minus 67% E pass rate = 14% p.p. difference; 14%

p.p. difference /81% C1 pass rate = 17% lower E pass rate). Similarly, the whole-class pass rate for the experimental group was 19% lower than for the C1 group (43% C1 pass rate minus 35% E pass rate = 8 p.p. difference; 8 p.p. difference/ 43% C1 pass rate = 19% lower E pass rate).

For Elementary Algebra, the results are substantially similar. The COMPASS certification rate was 2% higher for the experimental than for the C1 group (53% E cert rate minus 52% C1 cert rate = 1 percentage point difference; 1 p.p. difference

/52% C2 cert rate = 2% higher). However, the certified COMPASS pass-rate was substantially higher (22%) for the C1 group than for the experimental group (73% C1 pass rate minus 57% E pass rate = 16 p.p. difference; 16 p.p. difference/73% C1 pass rate = 22% lower E pass rate), as was the whole class COMPASS pass-rate (38% C1 pass rate minus 31% E pass rate = 7 p.p. difference; 7 p.p. difference/38% C2 pass rate = 18% lower E pass rate).

Most importantly from the standpoint of the importance of the homework completion tutoring being tested in this study, however, in Basic Math Skills the multiple-tutor MathXL groups outperformed the pencil-and-paper lab groups not- withstanding the later tutoring start date. As shown by the table below, this result occurred in all three categories: certification rate; certified COMPASS pass rate; and whole-class pass rate.

Specifically, the experimental group’s certification rate was 13% higher than the C2 certification rate (51% E cert rate minus 45 % C2 cert rate = 6 percentage points difference; 6 p.p. difference /45% C2 cert rate = 13% higher E cert rate). Similarly, the experimental group’s certified Compass pass rate was 29% higher than the C2 pass rate (67% E pass rate minus 52% C2 pass rate =15 p.p difference; 15 p.p. difference/52% C2 pass rate = 29% higher E pass rate). Again, the experimental group’s whole-class pass rate was close to 50% higher than the C2 pass rate (35% E pass rate minus 24% C2 pass rate = 11 p.p. difference; 11 p.p. difference/ 24% C2 pass rate = 46% higher E pass rate).

The same pattern did not hold true for Elementary Algebra, where both the MathXL and the pencil-and-paper control groups outperformed the experimental groups. Nevertheless, the Basic Math Skills result, showing across-the-board improved performance for the homework-tutored groups, both corroborates earlier research on this issue (Menil Dias, 2008) and strongly supports the importance to remedial mathematics performance of homework completion tutoring begun as early in the semester as possible.

CONCLUSION

For the most part, the preliminary results of our experiment are not robust. For both remedial courses, the single-tutor MathXL groups performed better than the multiple-tutor MathXL groups in both COMPASS pass-rate categories (certified and whole-class). These conclusions could be interpreted to suggest that a single tutor is more effective than multiple tutors for homework-completion assistance in the two remedial math courses’ Math Lab classes. Nevertheless, we regard our preliminary results as pointing in the other direction. As noted above, mathematics learning is cumulative. Due to the state budget constraints that led to the delay in the provision of tutors to the experimental group as compared to both control groups (6 weeks as compared to 3 weeks into the semester, or close to half-way through the 14-week semester), students in the experimental cohort were unable to obtain the full benefit of the Math Lab tutoring classes. Because the cohorts that received homework tutoring earlier outperformed those that received such tutoring only later in the semester, we believe that our results support the importance for remedial mathematics students of homework-completion tutoring, the earlier begun, the better.

ISSUES FOR FURTHER EXPLORATION

Issues Still Outstanding from the Current Study

Another measure of performance in addition to the COMPASS pass rate, the final class grades in each cohort, has yet to be analyzed. Other data available from the current study but not yet analyzed include the effects of gender and ethnic background on performance in the various cohorts. The same is true of withdrawal rates and lab attendance rates. The literature suggests that, in the case small-group support of the type provided in our experimental sections, even those students who clearly understood that they were not doing passing work decided to continue with the course (Hagedorn, et al., 2000). In our study in particular, one of the instructor’s paired MathXL classes reflected a substantially higher Math Lab attendance rate in the Experimental (multi-tutor) than in the C1 (single tutor) cohort, thus suggesting that her students valued the availability of more tutors to assist in home- work completion. In addition, it might be helpful to consider the characteristics of entering students. For example, although students self-selected their respective classes without knowledge of the experiment, one of the instructor’s paired classes showed that 35% of the students in the experimental class entered having passed neither the M1 nor the M2 level of the COMPASS, while only 13% of the C1 control group entered with that characteristic. Similarly, comparing the numbers in each cohort of students repeating the course with those taking the course for the first time might further elucidate the results.

Issues for Subsequent Research

The literature suggests that students benefiting from mathematics homework- completion assistance show improved performance not only during the semester in which such assistance occurs but in subsequent mathematics courses and other courses as well (Hagedorn, et al, 2000, pp. 134, 151)). However, because of the impact of financial constraints on the results of this study, additional research regarding these students may not be probative on this issue.

ACKNOWLEDGEMENTS

The conduct of this experiment involved burdensome reporting requirements on the part of the participating instructors. The Investigators wish to thank the following instructors for their willingness to take on this burden: Professors Nieves Angulo, Ross Flek, Henry Glover, Dae Hong, Mark Miller, Loreto Porte, Amrit Singh, Dionicio Taveras, and Nicholas West. In addition, the authors wish to thank Professor Loreto Porte for generously sharing her advice, her expertise, and her custom-enhanced computer MathXL homework with the Math 010 instructors involved in this experiment.

REFERENCES

Biswas, R. R. (2007, September). Accelerating remedial math education: How institutional innovation and state policy interact. Achieving the Dream policy brief for Jobs for the Future, 1-11.

DePree, J. (1998). Small-group instruction: Impact on basic algebra students.

Journal of Developmental Education, 22(1), 2-5.

Hagedorn, L. S., Sagher, Y., Siadat, M. V. (2000). Building study skills in a college mathematics classroom. The Journal of General Education, 49(2), 132-55.

Harootounian, J. A. Quinn, R. J. (2008). Identifying and describing tutor archetypes: The pragmatist, the architect, and the surveryor. Clearing House, 82(1), 15-20.

Hinds, S. (2009). More than rules: College transition math teaching for GED graduates at The City University of New York, 1-43. NY: City University of New York (Central Office of Academic Affairs).

Hostos Community College (2008, February). Report for President’s Executive Retreat. NY: Hostos Community College.

Menil, V. Dias, O. (2008). The effectiveness of the “Do Math” approaches—The bridge to close the cognitive gap between arithmetic and algebra. Mathematics Teaching-Research Journal Online, 2(2). Retrieved 1/14/10 from https://www. hotos.cuny.edu/departments/math/mtrj/archives/volume2/issue2full.pdf.

National Academy of Education (2009). Science and mathematics education: Education policy white paper. Retrieved November 5, 2009, from www. naeducation.org.

National Mathematics Advisory Panel Report (2008). Foundations for success: The final report of the National Mathematics Advisory Panel. Washington, D.C.: U.S. Department of Education.

National Research Council (2001). Adding it up: Helping children learn mathematics.

J. Kilpatrick, J. Swafford, B. Findell (Eds.), Mathematics Learning Study Committee, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, D.C.: National Academy Press.

Perrin, D. (2004). Remediation beyond developmental education: The use of learning assistance centers to increase academic preparedness in community colleges. Community College Journal of Research and Practice, 28, 559-582.