Validation Panel Report

 

Student Evaluation Standards Project

 

 

 

prepared by

 

 

 

W. Todd Rogers, Chair

 

Placido Garcia, Member

 

Validation Panel

 

 

 

October 18, 1999

 

This report is based on

 

1.      the student evaluation standards produced by the Joint Committee’s Special Task Force at the meeting of the Task Force on August 16 and 17, 1999;

2.      the discussion among members of the Committee during the annual meeting of the Joint Committee held on September 30 to October 2, 1999;

3.      our interactions with individual members during the annual meeting; and

4.      a sample of the revised standards produced by the members of the Joint Committee during the annual meeting.

 

We first want to acknowledge the serious attention and strong effort given by

 

1.      the members of the Special Task Force who attended the August meeting to the task of preparing draft standards to be considered by members of the Joint Committee at the Committee’s annual meeting, and

2.      by the members of the Joint Committee who attended the annual meeting to the task of preparing the draft of the Student Evaluation Standards to be sent to members of the national and international review panels.

 

The balance of this report is organized in two parts. First, we comment on the actions that were taken based on the suggestions and recommendations we made following last year’s annual meeting of the Joint Committee. This is followed by our comments on what happened during the 1999 meeting.

 

I.

 

A.     The revision of the Personnel Evaluation Standards was postponed for one year. During this year’s meeting, Dr. Gerry Horn was elected as a member-at-large on the Joint Committee. Following his election, he agreed to manage the revision and validation of the Personnel Evaluation Standards. In our opinion these two actions on the part of the Joint Committee will address some of the concern we had last year. However, the concern still remains that, given the heavy workload associated with developing and validating the Student Evaluation Standards and the likely heavy workload associated with revising the Personnel Evaluation Standards, the quality of one or both may be compromised. The intensity of the review and revision process is high. Given both projects will be proceeding concurrently, it is imperative that a meeting schedule be established that allows Joint Committee members adequate time to attend to each project. The Joint Committee needs to seriously consider two separate meetings each year during the duration of time both projects will be active, with each project discussed at only one meeting.

 

B.     Dr. Gullickson, Chair of the Joint Committee, is now the principal investigator for both grants received from the Kellogg Foundation to develop and validate the Student Evaluation Standards and the Personnel Evaluation Standards.

 

C.     As foreshadowed above, the recommendations that the national and international reviews be delayed by one year and that the Chair of the Joint Committee establish a small team of writers with known expertise in student evaluation in the classroom to prepare draft standards were both implemented. The Special Task Force completed its work in August 1999.

 

While the quality of the draft standards was uneven, they were stronger than the draft standards the Joint Committee members worked with during the 1998 meeting. However, there were still some substantial differences in format, completeness, reading level, and consistency within standards.

 

D.     Students are now the objects of evaluation. However, there still appears to be some confusion among the Joint Committee members regarding

 

i.                     the definition of student evaluation,

ii.                   clarification of who constitutes a student evaluation, and

iii.                 who is (are) the audience(s) for the Student Evaluation Standards.

 

This confusion likely will be eliminated with the development of a Glossary for the Standards.

 

II.

 

A.     The main task of the Joint Committee at the 1999 annual meeting was to revise the draft standards prepared by the Special Task Force. The product was to be draft of the standards to be reviewed by the national and international review panels.

 

Subject to final editing by the project staff, and the need to complete the revision of a few standards by members once they return home (e.g., U4), in our opinion the draft prepared by the Joint Committee members at the 1999 meeting are suitable for the national and international reviews. Further, we concur with the decision of the Chair to distribute the standards to the Joint Committee members prior to the reviews for one last look and check. The timeline established for this look and check seems reasonable.

 

There was insufficient time to prepare the introduction and glossary for the Standards during the meeting. To be completed by project staff, these materials are also to be included with the standards sent to the Joint Committee members.

 

B.     It is clear that project staff are experiencing difficulty doing what needs to be done between meetings. For example, little work other than editing had been completed on the Special Task Force standards following the meeting of the Task Force and prior to the Joint Committee meeting. It is also clear that some of the final revisions made by the Task Force were not included in the standards presented to the Joint Committee. It is imperative that additional the project staff, dedicated to the Student Evaluation Standards project and, perhaps, the Personnel Evaluation Standards project, be hired. By so doing, material can be processed more quickly and in time to provide to the members of the Joint Committee prior to a meeting of the Committee.

 

C.     Closely related is our concern with the level of funding for the development and validation of the Student Evaluation Standards. We acknowledge that a 3.5 million-dollar, five year proposal has been submitted to the National Science Foundation. Student evaluation occupies a central and important role in teaching and learning. It is estimated that teachers spend up to a third of their instructional time on assessment and evaluation. Additional sources of funds need to be identified to help ensure that development and marketing of a credible and believable set of standards for student evaluation. The task is formidable. We believe that the Sponsoring Organizations can and must take a more active role in identifying and culturing needed additional sources of funds[1].

 

D.     It is not too early to begin to advertise and market the Student Evaluation Standards. It is therefore recommended that each of the Sponsoring Organizations include on their Web sites, or some other suitable venue, a two page summary describing the progress of the Standards and that individual standards may be viewed and commented on using the Web site maintained by the Evaluation Center.

 

E.      We noted that not all of the Sponsoring Organizations were represented at the 1999 annual meeting. Given the centrality of student evaluation in the teaching/learning process, it is imperative that all members of the Joint Committee be represented at the forthcoming meetings of the Joint Committee at which the Student Evaluation Standards will be discussed and worked on.

 

It was with regret that I submitted my resignation as Chair and member of the Validation Panel for the Student Evaluation Standards effective October 31, 1999. I crossed the line of impartiality when, while attending the meeting of the Special Task Force in August 1999, I took part in the revision of the draft standards produced by the Special Task Force. “Not only must justice be done, but also it must be seen to be done.” The Joint Committee graciously accepted by resignation at the 1999 annual meeting.

 

I want to take this opportunity to thank the members of the Joint Committee for the assistance they provided to the Validation Panel. I feel confident that a very meaningful, telling set of standards will be developed which will help to ensure the fair and equitable evaluation of all students.