EDUC 603: Evaluation of Student Learning
Credit Hours: (3)
Students improve their educational practice through learning and applying effective assessment and evaluation principles and procedures. Lectures, readings and exams address topics such as designing valid and reliable assessment items, interpreting assessment data, and deciphering standardized test scores. Additionally, real-world projects provide meaning and context to the course's instructional goals and objectives by providing opportunities for students to design assessment instruments for instructional programs and interpret data collected within educational research and evaluation studies.
Detailed Description and Course Syllabus
This lecture and project-based course is designed to address the following topics:
- Why Do Teachers Need to Know about Evaluation and Assessment?
- Estimates of Reliability
- Evidence of Validity
- Absence of Bias
- Why Do Teachers Need to Know about Evaluation and Assessment?
- Strategies for Effective Selected-Response Tests
- Strategies for Effective Constructed-Response Tests
- Performance Assessments and Effective Rubrics
- Portfolio Assessment Strategies
- Assessing Affect
- Strategies for Improving Teacher-Developed Assessments
- Instructionally Oriented Assessment
- Making Sense Out of Standardized Test Scores
- Continuous Improvement: Evaluating Teaching and Grading Students
Detailed Description of Conduct of Course
Course lectures and readings will serve as the primary means of introducing information, examples, practice and interactive feedback for most of the course content. Case studies will be used to introduce concepts and provide opportunities for meaningful practice. Written tests will be administered, and a real-world project addressing the application of assessment development and data analyses will be integrated into the second half of the course.
Goals and Objectives of the Course
In general, the goals, objectives, projects and assessments in this course address the following NCATE
Standards:
1b. Pedagogical content knowledge for teacher candidates
1c. Professional and pedagogical knowledge and skills for teacher candidates
1d. Student learning for teacher candidates
Upon completion of this course the student will be able to:
- Describe educational variables of interest that are important to be measured.
- Distinguish between definitions and examples of assessment versus evaluation.
- Describe some of the important reasons why educators should apply effective assessment and evaluation principles and procedures within their professional practice.
- Given a description of reliability measures applied to a specific set of data, determine whether stability (t-rt), alternate-form, or internal consistency reliable has been estimated.
- Given a description of test analysis need, determine the best type of reliability to estimate (stability, alternate-form, or internal consistency).
- Describe the purpose of the standard error of measurement.
- Use the standard error of measurement to help interpret the meaning of individual test scores.
- Classify given data sets as measured along a nominal, ordinal, interval, or ratio scales.
- Identify the definitions of the following basic statistical terms: Mean, Median, Mode, Range, Standard Deviation, Correlation Coefficient
- Distinguish between definitions and examples of reliability estimates and evidence of validity.
- Given descriptions of tests conducted to determine evidence of validity for specific assessment data, determine whether content-related, criterion-related or construct-related evidence is being estimated.
- Describe the relationship between validity and reliability.
- Given a description of an assessment scenario, identify which of the following threats to the assessment result's evidence of validity might be present: History, Maturation, Testing, Instrumentation, Selection, Mortality, Statistical Regression, Instability
- In an effort to address the characteristics and needs of diverse learners, identify where given assessment items might be biased and suggest strategies for improving the item.
- Describe major difficulties facing educators who wish to use empirical methods to identify and correct item bias within the tests they create themselves.
- Classify the outcomes presented in an instructional objective as an example of verbal information, an intellectual skill, a motor skill, or an attitude.
- Given the parameters of an instructional environment, suggest the most effective type of assessment item for specific types of outcomes (verbal information, intellectual skills, motor skills, attitudes).
- Edit poorly-worded assessment directions to make them well-written.
- Edit poorly-constructed selected response and constructed response assessment items to make them well-written.
- Given an instructional objective, write selected response and/or constructed response assessment items that appropriately measure the SKA indicated within the objectives under the conditions stated (if applicable).
- Construct a well-written rubric scoring guide for a given instructional goal or task.
- Describe the different roles portfolios CAN play within an instructional environment.
- Given a specific portfolio assessment scenario, critique how well the portfolio adheres to the seven key ingredients of portfolio assessment.
- Given a description of an educator’s efforts to assess learner attitudes, identify the strengths and weaknesses of the methods and materials employed.
- Analyze data collected from an assessment instrument designed to measure attitudes, determine which items are effective and which need to be fixed or discarded.
- Develop an effective assessment instrument designed to measure attitudes for a specific instructional experience.
- Differentiate between examples of judgmental and empirical assessment item improvement strategies.
- Given the results of an item (distracter) analysis for a multiple choice assessment along with a general description of the purpose of the test, determine whether or not specific test items should be included in subsequent versions of the test.
- Distinguish between examples of formative and summative assessments.
- Describe where in the instructional planning process assessment creation should take place.
- Describe the role assessment creation should play in the instructional planning process.
- Distinguish between examples of norm-referenced and criterion-referenced assessments.
- Indentify the primary characteristics and limitations of standardized test scores.
- Compare and contrast percentile, grade-equivalent, and scale test scores.
- Critique an educator’s assessment development and implementation practices.
- Describe specific ways in which professional educators can improve their practice through the application of effective formative and summative assessment and evaluation principles and procedures.
- Describe strategies for using technology to develop valid and reliable assessment items, implement assessments, collect assessment data, and analyze assessment data.
- Analyze and interpret existing standardized test score data to determine achievement trends among sub-populations, possible evidence of assessment bias, and other issues associated with high-stake test results.
Due to the specific nature of the assessment-oriented outcomes delineated above, they could not be individually aligned with corresponding professional organization standards. However, collectively these assessment-oriented outcomes are addressed in some way within the following professional organization standards:
|
Professional Organization |
Standards |
|
National Association for Gifted Children (NAGC)-Council for Exceptional Children (CEC) |
Standard 4: S4: Pre-assess the learning needs of individuals with gifts and talents in various domains and adjust instruction based on continual assessment. Standard 8: K2: Uses, limitations, and interpretation of multiple assessments in different domains for identifying individuals with exceptional learning needs, including those from diverse backgrounds. K3: Uses and limitations of assessments documenting academic growth of individuals with gifts and talents. S1: Use non-biased and equitable approaches for identifying individuals with gifts and talents, including those from diverse backgrounds. S2: Use technically adequate qualitative and quantitative assessments for identifying and placing individuals with gifts and talents. S3: Develop differentiated curriculum-based assessments for use in instructional planning and delivery for individuals with gifts and talents. S4: Use alternative assessments and technologies to evaluate learning of individuals with gifts and talents. |
|
National Association for the Education of Young Children-Advanced (NAEYC) |
Standard 3: 3a: Understanding the goals, benefits, and uses of assessment 3b: Knowing about and using observation, documentation, and other appropriate assessment tools and approaches 3c: Understanding and practicing responsible assessment |
|
Interstate School Leaders Licensure Consortium Standards (ISLLC) |
Standard 2 E: Develop assessment and accountability systems to monitor student progress. I: Monitor and evaluate the impact of the instructional program. |
|
Association for Childhood Education International (ACEI) |
4.0 Assessment for Instruction: Candidates know, understand, and use formal and informal assessment strategies to plan, evaluate and strengthen instruction that will promote continuous intellectual, social, emotional, and physical development of each elementary student. |
|
National Council of Teachers of English (NCTE) |
4.10: Integrate assessment consistently into instruction by establishing criteria and developing strategies for assessment that allow all students to understand what they know and can do in light of their instructional experiences; interpreting the individual and group results of any assessments and drawing upon a variety of information in these assessments to inform instruction; assisting all students in becoming monitors of their own work and growth in speaking, listening, writing, reading, enacting, and viewing; explaining to students, parents, and others concerned with education how students are assessed. |
|
National Council of Teachers of Mathematics (NCTM) |
8.3 Uses multiple strategies, including listening to and understanding the ways students 8.6 Demonstrates knowledge of research results in the teaching and learning of mathematics. |
|
National Middle Schools Association (NMSA) |
Standard 3: 4. Are knowledgeable about local, state, and national middle level curriculum standards and of ways to assess the student knowledge reflected in those standards. 7. Understand multiple assessment strategies that effectively measure student mastery of the curriculum. Standard 5: 2. Know a wide variety of teaching, learning, and assessment strategies, and when to implement them. 3. Know that teaching higher order thinking skills is an integral part of instruction and assessment. 4. Know how to select and develop formal, informal, and performance assessments based on their relative advantages and limitations. 6. Know how to evaluate the effectiveness of teaching strategies. |
|
National Science Teachers Association (NSTA) |
Standards 8: a. Use multiple assessment tools and strategies to achieve important goals for instruction that are aligned with methods of instruction and the needs of students. b. Use the results of multiple assessments to guide and modify instruction, the classroom environment, or the assessment process. |
|
National Council for the Social Studies (NCSS) |
Pedagogical Standards 7: ASSESSMENT Social studies teachers should possess the knowledge, capabilities, and dispositions to use formal and informal assessment strategies at the appropriate school levels to evaluate and ensure the continuous intellectual, social, and physical development of learners. They should be able to assess student learning using various assessment formats, including performance assessment, fixed response, open-ended questioning, and portfolio strategies. |
|
Virginia Department of Education Standards for the Professional Practice of Teachers (VA DOE) |
Standard Three: Planning, Delivery, and Assessment of Instruction Key Element 7: The teacher communicates specific performance expectations and uses a variety of assessment strategies to plan instruction and to monitor and document student progress. |
Assessment Measures
The following types of assessments will be used to evaluate student learning and achievement throughout the course:
- Individual case studies and course projects assessed using detailed analytic rubrics
- One-three written exams scored according to the nature of individual exam items.
- Class participation in discussions, small group activities, and case study reviews
Other Course Information
Approval and Subsequent Reviews
4/24/09 Course syllabus revised and submitted Greg Sherman