On-Line Gateway Exams in Calculus
John Lindsay Orr and William J. Lewis, University of Nebraska - Lincoln
How do you teach a reform calculus course, with all the discussion and problem-solving work that it entails, and at the same time ensure that your students have a strong enough grasp of core algebraic computations to satisfy you, your colleagues, and your client disciplines? Group work, open-ended problems, and classroom discussionthe hallmarks of reform calculuscan be very time-consuming. The concern heard most about calculus reform is not that we shouldn't emphasize these aspects, but that the time they demand squeezes out time the students need to learn how to do basic computations of integrals and derivatives. The good news is that there is an alternative to using valuable class time for building these skills.
The Gateway Solution
The Gateway Exam meets head-on the legitimate concern that some students might pass a course with, say, a C grade based entirely on partial credit and group work, and without demonstrating a solid mastery of any of the skills of the course. In setting a Gateway Exam with a high standard, one must assume that many students will not pass initially, and will need an opportunity to retake the exam. The process of taking and retaking the Gateway Exam forces many students to assume responsibility for learning material which they might otherwise have neglectedin the belief that partial credit from other sections of the course would pull up their average.
At the University of NebraskaLincoln, we have taught reformed calculus using the first edition of the CCH materials since 1994. By 1996, we were hearing concerns expressed by colleagues that many students did not have adequate computational skills. That fall we introduced Gateway Exams to test our students' ability to find integrals and derivatives.
Implementation on the Web
The system we created offers students essentially unlimited retakes, together with unlimited opportunity to practice tests on their own. The tests are delivered over the Web and, in principle, can be taken from any computer with an Internet connection and a web-browser. In practice, there is a natural concern about honesty that makes us insist that when students want to take a test for credit they must come to our computer lab. Of course there is no such restriction on practice tests, and students are encouraged to work practice tests from home, the dorms, or computer labs around campus.
It is hard to overstate the importance of the fact that students can practice our Gateway Exams on their own. Students typically prepare at first as they would for any exam. We offer one chance to take the Gateway Exam in class, on paper, which the very good students pass at once. However a large number cannot meet the high standard; the second author found only 31% of his class passed the integration exam the first time it was given.
Once they realize the expectation is real and the standard is high, students start practicing the exam on the Web. Practice tests are drawn from the same database as the tests given for credit, so that students know they are practicing on a test which could have been the test they were given for credit. This seems to inspire them in a way that simply having a long list of problems and answers from a book does not. (In the example reported above, all but one of 88 students met the standard within 4 weeks.)
The Software Behind the Test
Tests include both multiple choice and free-form formats. Free-form questions ask students to enter a formula for the answer in the same format they use with their graphing calculators; the system then parses the expression so as to recognize when two formulas express the same function in different ways. As soon as a test is completed, the machine grades it, gives the student immediate feedback on his/her grade, and lists solutions to questions the student got wrong.
In order to make sure the exam is done closed-book and without the use of a sophisticated calculator, and that the person taking the exam is who they say they are, students take the on-line test for credit in our computer lab. After a student has finished a test and before it is graded, the student calls over the lab attendant, who serves as proctor for the exams. The attendant checks the student's ID and, assuming everything is satisfactory, the attendant authorizes the test for grading using his/her own secret password.
The lab we use for the tests has 20 computers, arranged in rows. One row of 7 computers is reserved exclusively for the tests, with students taking exams spilling over to the remaining machines if no one is using them for other classes. Since we give the first version of the test in class, the lab is used only for retakes, and since the lab is open over 50 hours each week, we have over 1000 machine-hours available per week. We have found that has been enough to serve classes as large as 850 (when both Calculus I and Precalculus classes have been taking tests at the same time) on a first-come-first-served basis.
Gateway Exam Results
In the authors' courses, 85% of the students have eventually passed the Gateway Exam. If we restrict our attention to those students who continue in the course to the point of taking the final exam, the pass rate on the Gateway Exam is 92%. We're also seeing that success on the exam is a very good predictor for success in the course; less than 1% of our students passed the Gateway and then failed the course. Also, only 3% passed the course without passing the Gateway Exam, usually with a D/D+ for a final grade. There's also evidence of the benefits of multiple retakes; the success rate among those who took the test four or more times was almost identical to the success rate of those who only needed between one and three retakes.
It's important to emphasize, though, that the Gateway Exams we give are one part of the assessment that takes place within the context of a reformed calculus course. Students in this course are also assessed on homework, quizzes, group writing projects, and written tests which explore conceptual and open-ended questions. Adding a Gateway Exam on core computational skills to our repertoire of testing has served our calculus program in two complementary ways. It enables us to simultaneously meet the concerns of those who worry about students losing grasp of algebraic skills, and at the same time, by allowing students to practice and improve their performance on their own over the Web, we protect the human interaction of the classroom for exploration of the conceptual aspects of calculus that are beyond the reach of the computer.
Starting this spring, our on-line testing system will be distributed by John Wiley & Sons, Inc. for use at other schools. Interested readers can explore the features of our testing software on the Web, or ask your local Wiley representative about "Wiley Web Tests for Calculus."
| Wiley College Home
| Modern Language
| Engineering/Computer Science | Science/Mathematics | Social/Behavioral Science |