Introduction
Background. The Joint Committee on Standards for Educational Evaluation (JCSEE) is currently developing the Student Evaluation Standards (SES) to guide the evaluations of students in a wide variety of situations, especially K-12. Evaluation is defined to include both formal evaluations (for example those related to grading ) and informal evaluations (teacher evaluations through oral responses to student questions, comments and the like). Upon completion of the development efforts, the standards will be certified by ANSI (the American National Standards Institute) as American National Standards.
Created in 1975, the JCSEE is a coalition of 16 major professional associations concerned with the quality of evaluation in education (see Table 1). The JCSEE has published two sets of evaluation standards that are now widely recognized: the Personnel Evaluation Standards, published by Corwin Press in 1988, and the Program Evaluation Standards (2nd edition), certified by ANSI as American National Standards, and published in 1994 by Sage Publications. In addition to their national impact, previous work of the JCSEE (resulting in the Program Evaluation Standards and the Personnel Evaluation Standards) has resulted in widespread translation and/or dissemination internationally, for use in Europe (especially Germany and Switzerland), Latin America and Canada.
The JCSEE has currently drafted and obtained national and international reviews for the Student Evaluation Standards, and revised its draft based on those reviews. These standards will be field tested in Spring 2001 and submitted to national hearings after completion of the field testing. Final revisions of the standards will incorporate findings from the field tests and hearings. Approval of these standards is expected either in 2001 or 2002. The revised SES (Field Test Draft) was available for review on the JCSEE Web site beginning in December 2000.
Preliminary Proposed Field Test Procedures
The development of the SES in their current form has relied on input from members of the JCSEE, on drafts from a panel of writers who have reflected the current scholarly literature on student evaluation and responded to the concerns and suggestions of the JCSEE, and on the comments and suggestions from national and international reviewers, following ANSI approved procedures. While these procedures are designed to result in technical accuracy and a high quality draft SES, it is important that these draft standards also undergo investigation in venues where they will ultimately be used and have an impact on the practices of student evaluation. To some extent, development to date has relied on experts. The field trials will test the standards in situations with the intended users and beneficiaries of the standards.
Specifically, the field tests are designed to evaluate the SES as educational products in keeping with the principles outlined in the Program Evaluation Standards (Guidelines for the evaluation of educational programs, projects and products, JCSEE, 1984). Four general areas of quality in the standards will guide the field tests:
The objectives for the field tests are:
Methodological Framework. The specific research questions to be addressed by the individual field tests as currently conceptualized will be in line with the general purpose of the field tests as described above, and will result from suggestions by JCSEE members and the field testers themselves, ultimately to be approved by the JCSEE or its Executive Committee acting as the agent of the JCSEE.
The evidence to be collected may be qualitative or quantitative in nature depending on the methods used at different field test sites. The evidence must demonstrate validity for purposes of investigating the quality of the standards, individually and collectively, as well as of the ancillary and supporting materials. The methodology must provide for quality assurance and quality determination of the evidence itself.
Field study designs at specific sites will differ in order to best address different research questions in specific situations. For example,
The expectation is that the field tests will provide representative national scope and some degree of international representation. The current plan calls for field tests involving volunteers representative of a wide range of schools and teacher training organizations across the US, including those that can best shed light on the fairness and equity of the applied SES for typically under-represented groups. Field testers and field tests sites will be solicited through advertisements in the professional journals and newsletters, listservs, and other vehicles typically used to disseminate information to members of all sixteen member organizations represented on the JCSEE.
Each field test site will be responsible for reporting results (draft reporting form attached, currently being revised by the Chair and Subcommittee). Each field test site will provide a full report, including a description of its research questions, information sources, methodology, analysis approaches, and information quality. These reports will be appended to the Summary Field Test Report, which will discuss the strengths and weaknesses of the SES and ancillary materials, both individually and collectively. Field tests will begin in January 2001 and continue until a sufficient number of field tests have been completed (at least 40 independent field tests sites). Our goal is to complete the field tests by June 2001.