To Flip or Not to Flip

By Allison M. Franzese

Introduction

In the 2017-2018 academic year, I experimented with transitioning one of our chemistry courses to a “flipped classroom.” The flipped classroom describes a reversal of traditional teaching in which students gain first-exposure learning prior to class and focus on the processing part of learning (synthesizing, analyzing, problem-solving, etc.) in class (,). Recent studies of flipping the classroom have found significant academic improvement, a >50% reduction in DFW percentage, and improvements in students’ motivation and retention of STEM majors (,). A flipped classroom model for general chemistry has been successfully implemented at both Hunter and Lehman Colleges, and I worked with a postdoc who taught a flipped general chemistry class at Lehman to develop a curriculum here at Hostos.

Methodology

In Fall 2017, I piloted the flipped classroom model in CHE 210, the first semester of our General Chemistry course for science and engineering majors. I used the flipped classroom model in one section of CHE 210 (216A), while I taught my other section (216B) in the usual manner with lectures, inquiry-based activities, and homework that reinforced topics already covered in class. For Section 216A, I “flipped” 1-2 topics from each chapter, with the remaining topics being taught in the same manner as my other section. Both sections were given homework assignments via MasteringChemistry, an online learning platform associated with our general chemistry textbook. The students in the flipped section were given additional assignments, due before class began. These assignments were also completed on MasteringChemistry and included a video, tutorial, and/or simulation that introduced new material, followed by a short set of simple questions to check their understanding of the material. Each class began with a set of problems for the students to work on in groups, typically assigned through Learning Catalytics, a real-time delivery system offered in conjunction with MasteringChemistry. I would tailor these problem sets based on the students’ performance with the pre-lecture assignment; they typically included some questions that were similar (or identical) to those assigned in the pre-lecture homework, and some that were more complex. I walked around the room to provide feedback and assistance when needed. Before moving on to the day’s new material, students put the answers to these problems on the board and we discussed them, focusing on topics with which students were still struggling. At the end of the semester, I asked the students in my flipped section to fill out a survey about their perceptions of the flipped teaching style.

In Spring 2018, I continued using the flipped approach for CHE 210, with additional flipped modules, so that pre-lecture assignments were due every week on Monday morning. I modified the assignments based on students’ results during the previous term. I also replaced the longer in-class modules with short daily quizzes based on the pre-lecture assignments. Each quiz consisted of one to two questions that were similar (or identical) to those assigned in the pre-lecture homework. These quizzes were graded 50-50 on completion and accuracy. I also added a motivational tool that’s used at Lehman College, which offered a bonus point to the whole class each time 100% of the students got 100% on a quiz.

Results and Discussion

Fall 2017 Comparison: Flipped vs. Traditional

In Fall 2017, I taught two sections of CHE 210, piloting the flipped approach with one section. There was no significant difference in the mean grade for each section; they were both in the C+ range. The flipped section did have a lower percentage of students earning D, W, WU, or F but it also had a lower percentage of students earning grades of B+ or better (Fig. 1).

Figure 1: Final course grades for Fall 2017 sections as a percentage of students in each section. The flipped section is in blue; traditional section is red.

The traditional section serves as a pseudo control for the flipped section. Both lectures and recitations were taught by me, scheduled at the same time of day (9:30-10:45 AM), used the online learning platforms Blackboard and MasteringChemistry, and composed primarily of STEM students with ~75% seeking an A.S. degree (Fig 2A). There were, however, some demographic differences in terms of degree program and level of schooling that might make this a less than perfect comparison (Fig. 2).

Figure 2: Fall 2017 Student demographics as a percent of students in each section. The flipped section is shown in blue; traditional section is red. A. Degree program of students enrolled in each section. Total AS is the sum of Liberal Arts AS and Engineering. B. Level of students enrolled in each section.

For example, the flipped section had a higher proportion of new freshman and of upper sophomores compared to the traditional section.

Other differences may be more important: the traditional class may have had some advantage over the flipped section for several reasons. The “traditional” class was held in a smart classroom and had a dedicated peer leader who provided supplemental instruction (SI). The class size was also significantly smaller (13 students in the traditional class vs. 20 in the flipped class). These factors may explain the large percentage of high grades in the traditional class, and could also suppress the difference in %DWF.

Flipped vs. Traditional: Long-term Trends

For comparison purposes, I compiled final grade data for all sections of CHE 210 I have taught since Fall 2014. Aside from one outlier in Spring 2017, the mean grades for all sections were consistently in the C to C+ range, regardless of mode of instruction (Fig. 3, yellow).

Figure 3. Final course grades for all sections of CHE 210 I taught at Hostos. Top panel: Mean (yellow) and median (purple) course grades for each semester. Bottom panel: Final course grades as a percentage of students in each section. Sum of all D, W, WU, and F in blue; Sum of all B, B+, A-, and A in red. Left: Classes I taught in a more traditional, lecture-based method (but included inquiry-based techniques as I’ve described in previous years). Right: Classes I taught using a flipped approach.

This is also true for the median grades, with the Fall 2017 traditional section being the exception, with a very high median grade of B+ (Fig. 3, purple). It is clear from the data that the Fall 2017 traditional section was an outlier in the very high percentage of students earning B or higher, while the percent of students earning B or better in my Fall 2017 flipped class was the same as the average value for all the traditional sections I’ve taught (Fig. 3, red). Furthermore, the percent of students earning D, F, or withdrawing (either officially or unofficially) has been fairly constant for the CHE 210 classes I taught in the traditional manner, hovering near 50%. The < 40% DWF for the Fall 2017 flipped section is a significant improvement (Fig. 3, blue).

Based on the significant reduction in %DWF (one of the main goals for switching to a flipped learning style) and the positive results of the student surveys, I decided to use the flipped strategy for my Spring 2018 CHE 210 class. I made a few adjustments based on the experience I gained in the fall, and set the expectations early in the semester, hoping that the outcome would be the same or better than the previous semester. Unfortunately, this turned out not to be the case, as the data clearly shows; %DWF was roughly the same as for traditional classes and %B and above was the lowest of any section I’ve taught (Fig. 3).

Flipped vs. Flipped (or What went wrong in Spring 2018?)

In Spring 2018, I implemented a motivational tool to encourage students to complete their pre-lecture assignments (and to complete them on time). I gave short daily quizzes based on the pre-lecture homework that were graded 50% on completion, and I offered bonus points to the class each time they all scored 100% on a daily quiz. I looked at the rates at which students completed the pre-lecture assignments to assess the effectiveness of this reward system and evaluate the reasons for the difference in success of the two flipped sections. I hypothesized that the daily quizzes were not successful in motivating students to complete the pre-lecture assignments, and that lower completion rates led to the poorer student outcomes (i.e. higher %DWF and lower %B and above) in Spring 2018 compared to Fall 2017. The results do not support either of these hypotheses.

It appears that the daily quizzes I gave in Spring 2018 were effective in motivating students to complete the pre-lecture assignments. On average, a greater percentage of students completed the pre-lecture assignments in Spring 2018 than in Fall 2017, especially early in the term (Fig 4.).

Figure 4: Percentage of students who completed each pre-lecture assignment as a function of time. Red squares show data for all students enrolled past the add/drop period (F’17 n=18; Sp’18 n=22). Data in orange circles excludes students after they withdrew officially or unofficially (i.e. after they stopped attending class or submitting work). Green diamonds exclude all data from students who withdrew officially or unofficially (n=14, n=15). Blue triangles exclude data from all students who withdrew as well as those who failed the class (n=13, n=14).

Removing the data for students who withdrew from the class (blue and green curves), more than 60% of students completed each pre-lecture assignment during Spring 2018, and more than 75% of the class completed 9 out of 13 pre-lecture assignments (Fig. 4, right). These completion rates were much better than in Fall 2017, when only 7 out of 12 pre-lecture assignments were completed by more than 60% of the class, and only one third of the assignments were completed by more than 70% of students (Fig. 4, left). The poorer student outcomes in Spring 2018 were obviously not because fewer students completed the pre-lecture assignments.

Flipped vs. Traditional: The bottom line We can use the final grade data compilation (Fig. 3) along with the pre-lecture assignment completion rates (Fig. 4), to test the underlying hypothesis behind assigning pre-lecture assignments, which is that the flipped classroom model will lead to improvements in the success of our students.

Support for the flipped model:

Support for this hypothesis comes from the significant reduction in %DWF in the Fall 2017 flipped section compared to all traditional sections I taught (Fig. 3), but only if one assumes that there were other variables responsible for the differing outcomes of the Fall 2017 and Spring 2018 flipped classes. For example, there may have been important differences in student demographics or classroom atmosphere, or other variables for which I have not accounted.

Another piece of data that could be interpreted to support the idea that the pre-lecture assignments may significantly contribute to student success comes from comparing the pre-lecture assignment completion rates (Fig. 4) of students who completed the course (blue and green curves) with those who did not (red and orange). In both semesters, removing data for students who did not complete the course increases the percentage of students who completed the pre-lecture assignments. Thus, students who completed these assignments were more likely to complete the course.

Against the flipped model:

Some of my data does not support the flipped model as a positive intervention for our students in CHE 210. In general, more of my students completed the pre-lecture assignments in Spring 2018 than in Fall 2017 (Fig. 4), yet the Spring 2018 students earned a higher %DWF and lower %B and above than any other CHE 210 class I’ve taught (Fig. 3). When looking more closely at the rates at which students completed the pre-lecture assignments in each term, some interesting trends emerge. As noted above, throughout both terms, students who completed more of the pre-lecture assignments than students who ailed or withdrew (Fig. 4). Comparing the two terms, however, it is clear there were dramatic differences between the groups of students who completed the term compared to those who didn’t (Fig. 4). It may be that the additional homework assignments were an added hardship for students with competing commitments and/or poor time management skills, and the students who could not complete them felt that they were at a disadvantage and decided to drop the class. In other words, completing the pre-lecture assignments did not necessarily help students to pass the course; those students who completed them may have passed anyway. But, not being able to complete the pre-lecture assignments may have caused more students to withdraw, raising the %DWF for Spring 2018. So, including the data for Spring 2018, one could conclude that the flipped classroom model does not actually lead to improvements in the success of our students; it may actually hinder their progress.

Conclusions

The results of my experiment with the flipped classroom for CHE 210 do not clearly indicate whether the technique has a positive effect, a negative effect, or no significant effect on our students’ success. This is primarily due to a low sample size; more data would be needed to properly assess the effectiveness of a flipped classroom model in improving our students’ outcomes.

References

(1) C. Brame, Flipping the classroom. Vanderbilt University Center for Teaching. (2013). http://cft.vanderbilt.edu/guides-sub-pages/flipping-the-classroom/

(2) M. D. Ryan and S. A. Reid, J. Chem. Ed. 93 (1): 13–23 (2016). doi:10.1021/acs.jchemed.5b00717(3) L. Hibbard, S. Sung, B. Wells, J. Chem. Ed. 93 (1): 24–30 (2016). doi:10.1021/acs.jchemed.5b00592