Academics have known for a while that assigning textbook reading before coming to class increases the ability of students to learn in the classroom and will help them succeed in a course. The big question is, how can we get students to pre-read before coming to class?
Now I can show proof to my students. And that means I can tell my students with confidence, “If you do the pre-reading, you’re likely to do better and get a higher grade.”
I teach Organic Chemistry at the University of Waterloo, a 200-level course. Organic Chemistry requires very different skills than introductory chemistry, which is more math-based, but is still required as a prerequisite. Although there is some correlation between students’ performance in the introductory course and my OChem course, success in introductory chemistry is not a predictor of success in my course.
Making use of online resources
For the past 14 years, 2004–18, I’ve made use of an expanding universe of digital resources, and tracked their evolution and effectiveness:
Student Resources Used in Chem 266
Year |
’18 |
’17 |
’16 |
’15 |
’14 |
’13 |
’12 |
’11 |
’10 |
’09 |
’08 |
’07 |
’05 |
’04 |
Assignments |
||||||||||||||
In-class
|
||||||||||||||
Extra
|
||||||||||||||
Comprehensive
|
||||||||||||||
Online
|
||||||||||||||
Pre-reading |
||||||||||||||
Post
|
||||||||||||||
Final exam
|
66 | 62 | 59 | 60 | 63 | 59 | 57 | 57 | 56 | 54 | 62 | 54 | 56 | 54 |
Number of
|
676 | 654 | 720 | 674 | 740 | 719 | 708 | 702 | 645 | 626 | 681 | 592 | 473 | 684 |
As with all courses, we have assignments. In 2005 we switched from hand-written assignments to online assignments, which at the time was a big technological and philosophical change. We thought making students write assignments was better than clicking on a screen, but having the students write the assignment or do them online seemed to make no difference to their final exam grades.
To encourage class participation and an active learning experience in the classroom, iClicker, an in-class response system, was introduced in 2007. To accommodate the time needed for students to think and work out questions in the classroom we increased the amount of time spent doing questions to approximately 60–70 percent of the lecture time.
In 2018 we switched to the Top Hat in-class response system because students could bring their own devices and Top Hat provided a website where content could be uploaded. Top Hat helped me produce graphical demonstrations on their website and helped expand learning outside the classroom.
Over the years I continued to add content to the Top Hat platform, and by 2013 I had created four learning modules, which was the beginning of a textbook. I wanted the reading experience to be similar to the classroom experience. In the classroom, content is presented in small digestible chunks, then students are asked to complete a question on the content they had just heard me explain. With online content, they are asked a question on the paragraph they have just read. In some ways this is better because students can go back and reread the content.
Thus, I created learning modules that presented the topics in a dynamic and interactive way by combining text, diagrams, videos and in-chapter questions. I was hoping to take active learning from the classroom to an online format, where students actively read and are engaged in reading. With the help of contributing authors a complete interactive textbook was created. The textbook was written in the same style as the modules, which allows students to self-test as they read, by obtaining immediate feedback from the in-chapter questions.
In 2014 the complete interactive textbook was available for students to use. At the same time that I was completing the textbook my co-instructor, Julie Goll, rewrote the course notes and changed them from a pdf of our presentation slides to comprehensive course notes that had more in-depth explanations of the topics.
All my students can buy the course notes from the campus bookstore. Buying the online book is not a course requirement, though I encourage it because I believe it helps students succeed. Approximately 80 percent of the students buy the textbook.
In 2014, I first encouraged the class to read the course material before coming to class by assigning pre-lecture reading assignments. The final exam average jumped to its highest point yet. To encourage students to participate, five percent of their grade was based on completing the weekly in-chapter questions. If they completed 80 percent of the textbook reading assignments, they earn 100 percent of the final participation grade.
From surveys that are linked to student grades, I found out that students don’t like pre-reading assignments. Even the good students don’t like them (though they still see the benefit of it). So in 2015–16, I switched to post-lecture reading assignments. And the final exam average score slipped a little. I returned to pre-lecture reading assignments in 2017–18. The difference between an online textbook and a normal textbook is that instructors usually don’t know if the students are completing the readings, but with Top Hat’s ability to include assessment questions in reading content, I am able to both track and reward students’ reading.
Here’s a visual representation of the change:
Chem 266 Final Exam Grade Distribution by Availability of Online Textbook and Comprehensive Course Notes
Proof of pre-reading benefits
Not only did the average exam score rebound after moving back to pre-class readings, last year it hit its highest level ever. Even the weaker students benefited from pre-reading, and 33 percent of the class in 2018 scored 80/100 or higher on the final exam.
To be able to compare the final exam grades from different years I wanted to make sure the difficulty level of the exam had not changed significantly over the years. To determine this, the 2017 final exam was based on the 2012 final exam, where 95 percent of the questions were slightly modified or identical. The final exam average for 2012 and 2017 was 57 percent and 62 percent respectively. The 2017 class had both the online textbook and comprehensive course notes, the 2012 class did not.
I repeated this in 2018, where 70 percent of the questions were slightly modified or identical to the 2014 exam. The exam averages were 66 percent and 63 percent respectively. Both classes had the textbook and comprehensive course notes. Could the ongoing enrichment of the textbook, with additional homework questions, and even questions where students can draw molecules, be the reason for the better exam grades in 2018? There is no way of confirming this, but it will be interesting to see the grade distribution for the 2019 class.
The following density plot displays the effect of pre-reading on grade distribution. For students who participated in reading the material in the textbook before seeing it in the classroom, their distribution of grades were shifted to better grades than those who did not read the textbook. For students who read the textbook soon after the material was taught in class, their distribution of grades was also shifted to better grades, although not as significantly as pre-reading. Because we can follow textbook participation, we can determine the distribution of grades for students who only partially participated in pre- and post reading. Their grades were not shifted to better grades with respect to students from 2004 to 2013. The no pre- and post reading plots are students who did not buy the textbook but had the comprehensive course notes.
Chem 266 Final Exam Grade Distribution by Student Resources used
Number of students in each group
No online
|
No post reading |
No pre-reading |
Partial
|
Partial
|
Post reading |
Pre-reading |
5722 | 372 | 333 | 273 | 255 | 745 | 1481 |
The change in grade distribution suggests that the online textbook provides an incentive through participation grades to motivate students to read the textbook before coming to class and leads to better grades. It is my belief that professors who are using online course materials and texts are likely to see similar improvements in their students’ outcomes, if they incorporate extra credit pre-reading assignments into future course design decisions.
I gave a presentation at the Digital Pedagogy Institute conference at the University of Waterloo of my findings. My presentation, containing full regression analysis and more detailed graphs, can be found below.