Focus on Calculus
A Newsletter for the Calculus Consortium Based at Harvard University
Winter 1998, Issue No. 14

Mathematics in Engineering: Notes from a Foreign Correspondent
Brad Osgood

AP Calculus Success
Christine S. Lucas

International Conference on the Teaching of Mathematics
ODE Architect: Building Order Out of Chaos
Robert L. Borrelli and Courtney S. Coleman

Calculus Comes Alive... Electronically
Paul R. Wellin

From the Publisher
About this Newsletter

On-Line Gateway Exams in Calculus

John Lindsay Orr and William J. Lewis, University of Nebraska - Lincoln

How do you teach a reform calculus course, with all the discussion and problem-solving work that it entails, and at the same time ensure that your students have a strong enough grasp of core algebraic computations to satisfy you, your colleagues, and your client disciplines? Group work, open-ended problems, and classroom discussion—the hallmarks of reform calculus—can be very time-consuming. The concern heard most about calculus reform is not that we shouldn't emphasize these aspects, but that the time they demand squeezes out time the students need to learn how to do basic computations of integrals and derivatives. The good news is that there is an alternative to using valuable class time for building these skills.

The Gateway Solution
One partial solution a number of schools are adopting is to use a "Gateway Exam" to test computational skills. As we use the term, a Gateway Exam is a test that students are required to pass (at some specified level of proficiency) in order to pass the course and proceed to subsequent courses. Thus a Gateway Exam should focus on core competencies of the subject which no one should pass the course without knowing. Our model of a Gateway Exam is inspired by the tests used at the University of Michigan.

The Gateway Exam meets head-on the legitimate concern that some students might pass a course with, say, a C grade based entirely on partial credit and group work, and without demonstrating a solid mastery of any of the skills of the course. In setting a Gateway Exam with a high standard, one must assume that many students will not pass initially, and will need an opportunity to retake the exam. The process of taking and retaking the Gateway Exam forces many students to assume responsibility for learning material which they might otherwise have neglected—in the belief that partial credit from other sections of the course would pull up their average.

At the University of Nebraska–Lincoln, we have taught reformed calculus using the first edition of the CCH materials since 1994. By 1996, we were hearing concerns expressed by colleagues that many students did not have adequate computational skills. That fall we introduced Gateway Exams to test our students' ability to find integrals and derivatives.

Implementation on the Web
The argument for giving these tests by machine was clear: We first experimented with paper Gateway Exams for our Calculus II class, and found that the job of setting and grading multiple retakes of paper tests for a class of over 300 students placed a prohibitive burden on faculty and GTA time, and essentially monopolized our Math Resource Center throughout the period of the test. Each fall semester we teach over 600 students in Calculus I and at least 250 in Calculus II. In the spring the breakdown is different, but the total is always greater than 700 students. If we were to offer tests with multiple retakes to these classes, we had to have a way of setting and grading the tests automatically.

The system we created offers students essentially unlimited retakes, together with unlimited opportunity to practice tests on their own. The tests are delivered over the Web and, in principle, can be taken from any computer with an Internet connection and a web-browser. In practice, there is a natural concern about honesty that makes us insist that when students want to take a test for credit they must come to our computer lab. Of course there is no such restriction on practice tests, and students are encouraged to work practice tests from home, the dorms, or computer labs around campus.

It is hard to overstate the importance of the fact that students can practice our Gateway Exams on their own. Students typically prepare at first as they would for any exam. We offer one chance to take the Gateway Exam in class, on paper, which the very good students pass at once. However a large number cannot meet the high standard; the second author found only 31% of his class passed the integration exam the first time it was given.

Once they realize the expectation is real and the standard is high, students start practicing the exam on the Web. Practice tests are drawn from the same database as the tests given for credit, so that students know they are practicing on a test which could have been the test they were given for credit. This seems to inspire them in a way that simply having a long list of problems and answers from a book does not. (In the example reported above, all but one of 88 students met the standard within 4 weeks.)

The Software Behind the Test
The software that we use is a self-contained test-server written by the first author. It sets and grades tests which it creates from databases of questions and answers written by faculty and graduate students in our department. Indeed a feature of the design is that the questions and answers are text files which can be written in any text editor, in TeX, or in Microsoft Word without any need for involvement in software design.

Tests include both multiple choice and free-form formats. Free-form questions ask students to enter a formula for the answer in the same format they use with their graphing calculators; the system then parses the expression so as to recognize when two formulas express the same function in different ways. As soon as a test is completed, the machine grades it, gives the student immediate feedback on his/her grade, and lists solutions to questions the student got wrong.

In order to make sure the exam is done closed-book and without the use of a sophisticated calculator, and that the person taking the exam is who they say they are, students take the on-line test for credit in our computer lab. After a student has finished a test and before it is graded, the student calls over the lab attendant, who serves as proctor for the exams. The attendant checks the student's ID and, assuming everything is satisfactory, the attendant authorizes the test for grading using his/her own secret password.

The lab we use for the tests has 20 computers, arranged in rows. One row of 7 computers is reserved exclusively for the tests, with students taking exams spilling over to the remaining machines if no one is using them for other classes. Since we give the first version of the test in class, the lab is used only for retakes, and since the lab is open over 50 hours each week, we have over 1000 machine-hours available per week. We have found that has been enough to serve classes as large as 850 (when both Calculus I and Precalculus classes have been taking tests at the same time) on a first-come-first-served basis.

Gateway Exam Results
Having practice tests available over the Web and being able to offer multiple retakes enable us to hold the students to a very high standard of success. In all of our on-line tests, the passing score is between 80% and 85%, and students receive credit only for passing, with no partial credit on individual problems or for scoring below the passing grade. Most of our faculty have stopped just short of making our on-line tests quite as stringent as the Gateway model described above—success in the on-line tests is not an absolute requirement for success in the course, although our grading schemes count the test as worth approximately one letter grade. Typically, one or two students may prove they can pass the course (typically with a grade of D) without passing the Gateway Exam, but most students treat passing the Gateway Exam as a course requirement. However this approach appears more reasonable to students because it has an award for those who pass, rather than a punishment for those who do not pass.

In the authors' courses, 85% of the students have eventually passed the Gateway Exam. If we restrict our attention to those students who continue in the course to the point of taking the final exam, the pass rate on the Gateway Exam is 92%. We're also seeing that success on the exam is a very good predictor for success in the course; less than 1% of our students passed the Gateway and then failed the course. Also, only 3% passed the course without passing the Gateway Exam, usually with a D/D+ for a final grade. There's also evidence of the benefits of multiple retakes; the success rate among those who took the test four or more times was almost identical to the success rate of those who only needed between one and three retakes.

It's important to emphasize, though, that the Gateway Exams we give are one part of the assessment that takes place within the context of a reformed calculus course. Students in this course are also assessed on homework, quizzes, group writing projects, and written tests which explore conceptual and open-ended questions. Adding a Gateway Exam on core computational skills to our repertoire of testing has served our calculus program in two complementary ways. It enables us to simultaneously meet the concerns of those who worry about students losing grasp of algebraic skills, and at the same time, by allowing students to practice and improve their performance on their own over the Web, we protect the human interaction of the classroom for exploration of the conceptual aspects of calculus that are beyond the reach of the computer.

Starting this spring, our on-line testing system will be distributed by John Wiley & Sons, Inc. for use at other schools. Interested readers can explore the features of our testing software on the Web, or ask your local Wiley representative about "Wiley Web Tests for Calculus."

How to Order

| Wiley College Home | Modern Language | Business |
| Engineering/Computer Science | Science/Mathematics | Social/Behavioral Science |