CAPSI

Pear, J. J. (2004). Enhanced feedback using computer-aided personalized system of instruction. In W. Buskist, V. W. Hevern, B. K. Saville, & T. Zinn, (Eds.), Essays from e-xcellence in teaching, 2003 (Chap. 11).

http://teachpsych.lemoyne.edu/teachpsych/eit/eit2003/eit03-11.pdf)

--------------------------------------------------------------------------------

Enhanced Feedback Using Computer-Aided Personalized System of Instruction
Joseph J. Pear
University of Manitoba

(This essay originally appeared as the monthly "E-xcellence in Teaching" e-column in the PsychTeacher Electronic Discussion List for November 2003.)

Many university classes are too large to teach in an ideal manner. This greatly limits the amount of verbal (both oral and written) engagement students can have with the subject matter and the amount of quality feedback they receive. The result is that students may complete courses without being able to express coherently the course material, let alone incorporate it into higher-order thought processes.

There appears to be general agreement on a number of instructional features that help to promote the development of higher-order thinking, including frequent testing requiring expository writing, peer interaction, and student-generated questions (Graesser, Person, & Hu, 2002). Conner-Greene (2000) reported that students given short, frequent tests rather than longer, infrequent exams not only demonstrated increased higher-order thinking, but also obtained higher average scores in the course. Jackson (2000) noted that collaboration and group work increased thinking levels in students from every age group, ranging from elementary school children to graduate students. Finally, encouraging students to ask and even develop questions for testing can help promote higher-order thinking (Carroll, 2001). Students both asking questions of, and providing feedback to, their peers should thus be highly effective in fostering the development of higher-order thinking.

Empirical research has not yet established the delivery system most suited to provide the necessary features for developing and maximizing higher- order thinking. However, with increasing student numbers and diversity, traditional teaching methods may not always be the most feasible or efficient systems. This holds especially true in institutions of higher education, where the development of higher-order thinking is crucial. Online methods offer a promising way to deliver courses to larger populations of diverse students. One of the benefits of using computer technology is that it can be programmed to keep track of students' work and level of accomplishment. In addition, the program can be systematically changed to compare methods for enhancing learning and higher-order thinking.

In recent decades, several approaches to computer-mediated education have appeared in which assignments are completed through the Internet or a campus computer network (e.g., Hiltz, 1986). Although only a few forms of computer-assisted and computer-mediated instruction have been studied experimentally, research indicates that these courses can be as effective as, or more effective than, traditional methods (Kulik & Kulik, 1991). Apparent in all of these approaches is the fact that as computer and communication network technology changes, so does the role of the instructor (Kook, 1997). Instructors' roles will take on the features of information consultants, team collaborators, facilitators of critical and creative thinking, course developers, and academic advisors.

At least that is the promise of education on the Internet. Unfortunately, that promise has so far remained largely unfulfilled. Probably the most dramatic change computer technology has brought to higher education is the use of e-mail for communication of students with the instructor and with other students. On the whole, computers have served in this regard simply as a messaging system-an extremely convenient one both for instructors and students-but not a primary instructional tool. Other common uses of computer technology in higher education include providing an online catalogue of library material, conducting literature searches, and obtaining articles and other information posted on websites.

Direct applications of computer technology in higher education have largely been devoted to attempts at adapting standard classroom activities to a Web environment. A popular commercial program for facilitating instruction on the Web, for example, is called WebCT, which is short for "web course tools." As its name implies, this program provides instructors with "tools" that permit them to recreate an online version of standard course practices. Instructors using this program, or other course delivery programs, have posted lectures, conducted online discussions, and arranged for students to engage in group-work on the Internet. These activities often highlight problems that exist in many standard classroom activities. For example, in a standard classroom discussion, the fact that only a few students are participating may not be readily apparent to the instructor. Because the class discussion is auditory, the instructor knows that students are hearing the discussion, but may overlook the fact that they may not be listening. The problem of non-participation by large numbers of students is much more evident in an online course because the program logs every contribution that each student makes to the discussion. However, when contingencies are placed on contributing to an online discussion, it is almost impossible for an instructor to evaluate and provide effective feedback to the huge amount of verbal material thus generated.

It would appear that recreating standard course procedures on the Internet is probably not an efficient use of computer technology in higher education. Systematic educational procedures proven to be effective and that make maximal use of the potential of computers are required. Methods must therefore be implemented for monitoring the activities of students and providing them with feedback.

The method my colleagues and I use in some of our undergraduate courses is called computer-aided personalized system of instruction (CAPSI). In this method, which is based on Keller's (1968) personalized system of instruction, students proceed through the course material by completing unit assignments on study questions designed to initiate student inquiry. Students who are further advanced act as peer reviewers. To ensure high- quality student involvement, the program requires that the instructor, teaching assistant, or two peer reviewers evaluate or review a student's unit assignment and provide feedback to the student. Students must demonstrate mastery on a unit in order to proceed to the next one. In addition, all assignments and the feedback provided are recorded automatically for the instructor to sample. Students receive as many attempts as they need to demonstrate mastery of a unit, but at least one hour for restudy must elapse between successive attempts. There is also a built-in appeal process for arguing the validity of a given answer. The program is applicable to any course topic and any set of questions or problems. It has been used successfully over several decades in a number of psychology courses at the University of Manitoba (Kinsner & Pear, 1988; Pear & Crone-Todd, 1999; Pear & Kinsner, 1988; Pear & Novak, 1996).

In a course using CAPSI, students demonstrate mastery through unit assignments, midterms, and a supervised final exam. Students study the text independently and complete study questions selected by the program from a bank of essay-type questions on the material they have just learned. Students proceed at their own pace through the study units; and, as soon as a student has demonstrated mastery of a unit (defined as correctly answering all questions on the unit assignment), he or she may serve as a peer reviewer on that unit. The program selects peer reviewers for each completed assignment according to an algorithm that takes a number of factors into account, such as the student's current level in the course, the number of times the student has served as a peer reviewer, and the availability of the student to peer review a unit assignment in a timely fashion (i.e., 24 hours from the submission of the assignment). Peer reviewers receive a small amount of course credit each time they review an assignment.

Data on CAPSI show that the instructor and teaching assistant provide the majority of feedback on unit assignments during the first few weeks of the course (Pear & Crone-Todd, 2002). As the course continues, the instructor and teaching assistant continue to provide feedback to the first few students who complete each unit. However, peer reviewers then provide increasingly more of the feedback. Feedback is considered to be minimal if it consists only of a short statement such as "good answer." It is substantive if it includes a specific reference to the actual content of the answer. It is particularly noteworthy that with CAPSI the amount of substantive feedback that students give and receive appears to be much greater than could occur in a typical university course with comparable enrollment (Pear & Crone-Todd, 2002).

CAPSI is equally important as a tool for researching the education process. Although CAPSI differs significantly from traditional course procedures, CAPSI variables are traditional educational variables. Traditional courses contain textual material on which students are tested, and through discussions and other activities, students, as well as
instructors, provide information and feedback to other students. Some of the dependent variables that my students, my colleagues, and I are currently researching through CAPSI are methods for increasing higher- order thinking (Crone-Todd, 2002; Crone-Todd, Pear, & Read, 2000; Pear, Crone-Todd, Wirth, & Simister, 2002), improving the accuracy of peer reviewers (Martin, Pear, & Martin, 2002a), increasing the amount of substantive feedback peers provide (Pear & Crone-Todd, 2002), and using feedback to increase the compliance of students (Martin, Pear, & Martin, 2002b).

In conclusion, CAPSI is an effective solution for instructing and providing feedback to large numbers of students. Moreover, because the instructor deals with each student individually rather than as a group, CAPSI can be used to teach multiple courses at the same time. With a proven track record, CAPSI has shown promise for developing higher levels of student exposition and comprehension of course materials. Because of its success at the University of Manitoba, the Faculty of Arts and the Department of Psychology have recently budgeted funds to develop and upgrade further the CAPSI program. Further information about CAPSI may be obtained by visiting www.capsi.org.

References

Carroll, D. W. (2001). Using ignorance questions to promote thinking skills. Teaching of Psychology, 28, 98-100.

Conner-Greene, P. A. (2000). Assessing and promoting student learning: Blurring the line between teaching and testing. Teaching of Psychology, 27, 84-88.

Crone-Todd, D. E. (2002). Increasing the levels at which undergraduate students answer questions in a Computer-Aided Personalized System of Instruction course. Unpublished doctoral dissertation. University of Manitoba, Winnipeg, Manitoba, Canada.

Crone-Todd, D. E., Pear, J. J., & Read, C. N. (2000). Operational definitions for higher order thinking objectives at the post-secondary level. Academic Exchange Quarterly, 4(3), 99-106.

Graesser, A. C., Person, N. K., & Hu, X. (2002). Improving comprehension through discourse processing. In D. F. Halpern & M. D. Hakel (Eds.), Applying the science of learning to university teaching and beyond (pp. 33-44). New York, NY: Wiley.

Hiltz, S. R. (1986). The "virtual classroom": Using computer-mediated communication for university teaching. Journal of Communication, 36, 95-104.

Jackson, L. (2000). Increasing critical thinking skills to improve problem-solving ability in mathematics. Master's Action Research Project, Saint Xavier University.

Keller, F. S. (1968). "Good-bye teacher ..." Journal of Applied Behavior Analysis, 1, 79-89.

Kinsner, W., & Pear, J. J. (1988). Computer-aided personalized system of instruction for the virtual classroom. Canadian Journal of Educational Communication, 17, 21-36.

Kook, J. K. (1997). Computers and communication networks in educational settings in the twenty-first century: Preparation for educator's new roles. Educational Technology, 37(2), 56-60.

Kulik, C. L., C., & Kulik, J. A. (1991). Effectiveness of computer-based instruction: An updated analysis. Computers in Human Behavior, 7, 75-94.

Martin, T. L., Pear, J. J., & Martin, G. L. (2002a). Analysis of proctor marking accuracy in a computer-aided personalized system of instruction course. Journal of Applied Behavior Analysis, 35, 309-312.

Martin, T. L., Pear, J. J., & Martin, G. L. (2002b). Feedback and its effectiveness in a computer-aided personalized system of instruction course. Journal of Applied Behavior Analysis, 35, 427-430.

Pear, J. J., & Crone-Todd, D. E. (1999). Personalized system of instruction in cyberspace. Journal of Applied Behavior Analysis, 32, 205-209.

Pear, J. J., Crone-Todd, D. E., Wirth, K. M., & Simister, H. D. (2002) Assessment of thinking levels in students' answers. Academic Exchange Quarterly, 5(4), 94-98.

Pear, J. J., & Crone-Todd, D. E. (2002). A social constructivist approach to computer-mediated instruction. Computers & Education, 38, 221-231.

Pear, J. J., & Kinsner, W. (1988). Computer-aided personalized system of instruction: An effective and economical method for short- and long-distance education. Machine-Mediated Learning, 2, 213-237.

Pear, J. J., & Novak, M. (1996). Computer-aided personalized system of instruction: A program evaluation. Teaching of Psychology, 23, 119-123.

Copyright © 2003 Joseph J. Pear. http://teachpsych.lemoyne.edu/teachpsych/eit/index.html