Teacher Leader Effectiveness Evaluation System |
Previous | 1 of 1 | Next |
|
small (250x250 max)
medium (500x500 max)
Large
Extra Large
large ( > 500x500)
Full Resolution
|
This page
All
|
1 Oklahoma Teacher and Leader Effectiveness Evaluation System (TLE) REPORT To the Oklahoma State Legislature and Governor on the Recommendations of the TLE Commission and the Adoption of the TLE by the Oklahoma State Board of Education OVERVIEW Oklahoma state law (70 O.S. § 6-101.16) established the Oklahoma Teacher and Leader Effectiveness Evaluation System (TLE), a new evaluation system for teachers and leaders designed to encourage continuous professional growth leading toward improved student achievement for all Oklahoma students. The law requires that the new system be comprised of multiple measures of teacher and administrator effectiveness: 50% Qualitative Measures (observable characteristics of teacher and leader performance that are correlated to student achievement) 35% Quantitative Measures of Student Academic Growth (based on multiple years of standardized test data) 15% Quantitative Measures of Other Academic Factors According to state law, all local board of education evaluation policies must align with the TLE by the 2013-2014 school year. This report has been developed pursuant to state statute requiring an annual report of the TLE Commission. The Commission shall issue a report by December 31 of each year and submit a copy of the report to the Governor, the Speaker of the House of Representatives and the President Pro Tempore of the Senate. (70 O.S. § 6-101.17I) ADOPTION PROCESS State law required the Oklahoma State Board of Education to adopt the TLE by December 15, 2011, and to receive advice from the TLE Commission as discussed later in this report. By December 15, 2011, the State Board of Education shall adopt a new statewide system of evaluation to be known as the Oklahoma Teacher and Leader Effectiveness Evaluation System (TLE). (70 O.S. § 6-101.16A) The Teacher and Leader Effectiveness Commission shall provide oversight and advise the State Board of Education on the development and implementation of the TLE. (70 O.S. § 6-101.16C) COMPONENTS OF THE SYSTEM The TLE shall include the following components: 1. A five-tier rating system as follows: a. superior, b. highly effective, 2 c. effective, d. needs improvement, and e. ineffective; 2. Annual evaluations that provide feedback to improve student learning and outcomes; 3. Comprehensive remediation plans and instructional coaching for all teachers rated as needs improvement or ineffective; 4. Quantitative and qualitative assessment components measured as follows: a. fifty percent (50%) of the ratings of teachers and leaders shall be based on quantitative components which shall be divided as follows: (1) thirty-five percentage points based on student academic growth using multiple years of standardized test data, as available, and (2) fifteen percentage points based on other academic measurements, and b. fifty percent (50%) of the rating of teachers and leaders shall be based on rigorous and fair qualitative assessment components; 5. An evidence-based qualitative assessment tool for the teacher qualitative portion of the TLE that will include observable and measurable characteristics of personnel and classroom practices that are correlated to student performance success, including, but not limited to: a. organizational and classroom management skills, b. ability to provide effective instruction, c. focus on continuous improvement and professional growth, d. interpersonal skills, and e. leadership skills; 6. An evidence-based qualitative assessment tool for the leader qualitative portion of the TLE that will include observable and measurable characteristics of personnel and site management practices that are correlated to student performance success, including, but not limited to: a. organizational and school management, including retention and development of effective teachers and dismissal of ineffective teachers, b. instructional leadership, c. professional growth and responsibility, d. interpersonal skills, e. leadership skills, and f. stakeholder perceptions; and 7. For those teachers in grades and subjects for which there is no state-mandated testing measure to create a quantitative assessment for the quantitative portion of the TLE, an assessment using objective measures of teacher effectiveness including student performance on unit or end-of-year tests. Emphasis shall be placed on the observed qualitative assessment as well as contribution to the overall school academic growth. (70 O.S. § 6-101.16B) TLE COMMISSION OVERVIEW The law established the TLE Commission to make recommendations about the design of the system and to oversee implementation of the system through June 30, 2016. The TLE Commission is comprised of various education and public sector stakeholders appointed by the Governor, President Pro Tempore of the Senate, and Speaker of the House, or as designated by statute. The TLE Commission met eight times from June 29 through December 5, 2011, to study the various components of the TLE, to evaluate a variety of qualitative measurement tools (frameworks) and quantitative measurement tools, to make formal design and implementation 3 recommendations to the State Board of Education, and to solicit public input on those recommendations. Five design and implementation recommendations were approved at the TLE Commission meeting on December 5, 2011, after receiving feedback from a wide representation of stakeholders, including teachers, administrators, students, parents, community members, and policymakers. See Attachment A. Details regarding the extensive study conducted by the TLE Commission as well as a summary of public comment received is provided in subsequent sections of this report and in the attachments. TLE COMMISSION ANALYSIS OF QUALITATIVE MEASURES The TLE Commission reviewed four teacher evaluation frameworks for the qualitative measure and four leader evaluation frameworks for the qualitative measure. These frameworks were: Teacher Frameworks Danielson’s Framework for Teaching (See Attachment C) Marzano’s Causal Teacher Evaluation Model (See Attachment D) McREL’s Teacher Evaluation System Tulsa’s TLE Observation and Evaluation System (See Attachment E) Leader Frameworks Marzano’s Leadership Evaluation System McREL’s Principal Evaluation System (See Attachment F) Reeves’s Leadership Performance Matrix (See Attachment G) Vanderbilt’s Assessment for Leadership in Education The TLE Commission preliminarily recommended six of these tools – three teacher frameworks and three leader frameworks – for consideration. The McREL Teacher Evaluation System was eliminated because the TLE Commission determined that the framework did not meet the needs of Oklahoma educators. In addition, the Marzano Leadership Evaluation System was eliminated because it was still in production at the time of final recommendations and the Vanderbilt Assessment for Leadership in Education was also eliminated because the TLE Commission determined that the framework did not meet the needs of Oklahoma educators. The rubrics used to compare each framework to requirements of state statute and national best practices are available as Attachment B. Recommendations #1c and #1f indicate that the TLE Commission recommended Danielson’s Framework for Teaching, Marzano’s Causal Teacher Evaluation Model, Tulsa’s TLE Observation and Evaluation System, McREL’s Principal Evaluation System, and Reeves’s Leadership Performance Matrix for district selection. See Attachment A. 4 TLE COMMISSION ANALYSIS OF QUANTITATIVE MEASURES The Commission examined a variety of possible ways to evaluate student growth for teachers who teach grades or subject areas where student growth data exists. One option the Commission reviewed was a Simple Growth Model. This model compares student performance at the end of instruction to performance prior to instruction. The Commission also reviewed Value Added Models. While this option also measures student growth, it measures that growth against the student’s predicted growth level for the school year. This prediction is determined through a complex series of calculations that factor in such variables as attendance, mobility, past achievement, ELL status, and/or number of subject-specific courses in which the student is enrolled. The focus of the variables can be based either on the student’s prior achievement (Covariate Model), or on the student’s propensity to achieve along with the durability of the teacher’s effect on the expected growth (Learning Path Model). In essence, a Value Added Model determines what value the teacher added to the student’s success. The Commission determined that utilizing a Value Added Model would best reflect Oklahoma’s need to take into account other student and school-level variables in order to have the most accurate evaluation system possible. Therefore, the Commission recommended adoption of a Value Added Model for teachers and leaders of buildings for which multiple years of standardized test data exist. See Attachment A: #3a, #3b. For teachers who teach in grades or subject areas in which no state-mandated testing exists, the quantitative component of the TLE shall involve an assessment using objective measures of teacher effectiveness including student performance on unit or end-of-year tests. The Commission has reviewed several ways to generate data for those grades and subjects where statewide student assessment data does not exist. These methods include developing additional state assessments, developing a list of content-specific appropriate measures of student achievement, using student growth data of “owned students” or all school-wide data, or using a combination of the above referenced methods. The Commission recommended conducting further research on the most appropriate measure(s) of teacher effectiveness for those teachers in non-tested grades and subjects and to take into consideration the input of representatives of those teacher groups. See Attachment A: #4. In addition, the Commission reviewed options for the quantitative measures identified as “Other Academic Measures.” The Commission recommended involving Oklahoma educators in development of a list of appropriate measures for teacher and supervisor selection based on findings from research regarding multiple measures of teacher effectiveness. See Attachment A: #5. TLE COMMISSION ANALYSIS OF IMPLEMENTATION PROCESSES In order to facilitate implementation of the TLE statewide, the TLE Commission made several recommendations regarding selection of default frameworks and funding for training. The Commission recommended selecting a default framework for the teacher qualitative assessment and a default framework for the leader qualitative assessment. See Attachment A: #1a, #1d. The 5 Commission selected their recommended default frameworks after much debate and consideration of each framework. Despite the public comments found in Attachment H, which indicate that the majority of responders favored Marzano’s Causal Teacher Evaluation Model, in a split vote, the TLE Commission recommended Tulsa’s TLE Observation and Evaluation System as the default teacher qualitative assessment tool. See Attachment A: #1b. Also in a split vote, the Commission recommended McREL��s Principal Evaluation System as the default leader qualitative assessment tool. See Attachment A: #1e. In addition, the TLE Commission made recommendations to reserve a portion of the available state funds designated for training and implementation for the frameworks not selected as the default. See Attachment A: #1c, #1f. Lastly, the TLE Commission recommended that any modifications made to the default framework or other approved frameworks must be approved by the Oklahoma State board of Education. See Attachment A: #2. STATE BOARD OF EDUCATION OVERVIEW OF APPROVED TLE SYSTEM On December 15, 2011, the State Board of Education approved a TLE System pursuant to 70 O.S. § 6-101.16A. The State Board named the Tulsa TLE Observation and Evaluation System as the presumptive default for teacher evaluations and the McREL Principal Evaluation System as the presumptive default for leader evaluations. See Attachment I. During the pilot year of implementation (2012-2013), districts will be allowed to choose from three teacher evaluation frameworks (Tulsa’s TLE Observation and Evaluation System, Marzano’s Causal Teacher Evaluation Model, and Danielson’s Framework for Teaching) and two leader evaluation frameworks (McREL’s Principal Evaluation System, and Reeves’s Leadership Performance Matrix). During this pilot period, districts are required to select and implement a teacher and leader framework from the list of approved frameworks. Throughout the pilot year, districts will be asked to provide input and feedback regarding the frameworks, and the data provided by districts will be reported by OSDE to the TLE Commission and State Board of Education for consideration. Teacher and leader evaluations obtained during the pilot year will not count against teachers or leaders during the 2012-2013 school year. However, the data obtained during the pilot year may be used by districts to establish baselines and offer guidance as Oklahoma schools move forward with permanent implementation during the 2013-2014 school year. Data and research obtained during the pilot year may be independently analyzed to determine evidence of measurement of effective teaching and leadership as well as the ability of each model to scale up for statewide implementation. This information will be used by the TLE Commission to make further recommendations to the State Board of Education. At the end of the pilot year, the State Board of Education will adopt default frameworks. 6 The allocation of funds will be supported by local funds or at the discretion of the Oklahoma Department of Education through a formula based on the districts’ Average Daily Attendance. See Attachment I. Qualitative Measures (50% of Total TLE) See Attachment I. Teacher For the teacher qualitative assessment, the Oklahoma State Board of Education (OSBE) has approved three frameworks from which districts may choose: o Danielson’s Framework for Teaching(pending licensing agreements), o Marzano’s Causal Teacher Evaluation Model(pending licensing agreements), and Tulsa’s TLE Observation and Evaluation System(pending licensing agreements). For the teacher qualitative assessment, the OSBE has approved Tulsa’s TLE Observation and Evaluation System framework to become the presumptive default statewide framework. Leader For the leader qualitative assessment, the OSBE has approved two frameworks from which districts may choose: McREL’s Principal Evaluation System (pending correlation to statutory criteria and licensing agreements), and o Reeves’s Leadership Performance Matrix (pending correlation to statutory criteria and licensing agreements). For the leader qualitative assessment, the OSBE has approved McREL’s Principal Evaluation System framework to become the presumptive default statewide framework. Teacher and Leader Any modifications to the default frameworks or other approved frameworks must be approved by the Oklahoma State Board of Education against a specific set of criteria, including all statutory requirements, based on impact to student learning. Quantitative Measures of Student Academic Growth (35% of Total TLE) See Attachment I. The OSDE will use a Value Added Model in calculating the thirty-five percentage points attributed to student academic growth using multiple years of standardized test data for those teachers in grades and subjects for which multiple years of standardized test data exist. The OSDE will use a Value Added Model in calculating the thirty-five percentage points attributed to student academic growth using multiple years of standardized test data for those leaders of buildings containing grades and subjects for which multiple years of standardized test data exist. In addressing those teachers and leaders in grades and subjects for which there is no state-mandated testing measure to create a Value Added Score, the OSDE will conduct more research to determine the appropriate measure(s) of student achievement taking into account a combination of multiple measures and including teacher, leader, and specialist 8 LIST OF ATTACHMENTS ATTACHMENT DESCRIPTION A Teacher and Leader Effectiveness Commission Permanent Recommendations B Criteria Checklists for All Frameworks Reviewed C Danielson’s Framework for Teaching D Marzano’s Causal Teacher Evaluation System E Tulsa’s TLE Observation and Evaluation System F McREL’s Principal Evaluation System G Reeves’ Leadership Performance Matrix H Summary of Public Comment I State Board of Education Adopted Policies ATTACHEMENT A Teacher and Leader Effectiveness Commission Permanent Recommendations ATTACHMENT A Teacher and Leader Effectiveness Commission Permanent Recommendations Pursuant to 70 O.S. § 6-101.17 December 5, 2011 Permanent Recommendation #1a: For the Teacher Evaluation System, the TLE Commission recommends that the Oklahoma State Board of Education name a default framework that is paid for by the state in terms of training and implementation requirements to serve as the qualitative assessment component that must comprise 50% of the total evaluation criteria required by 70 O.S. § 6-101.16. Permanent Recommendation #1b: The TLE Commission recommends that the Teacher Evaluation default framework be Tulsa’s TLE Observation and Evaluation System. Permanent Recommendation #1c: The TLE Commission recommends that the Oklahoma State Board of Education name a limited number of frameworks that meet specific criteria, including all statutory requirements, for district selection. Frameworks other than the default will be supported by local funds and twenty-five percent (25%) of available state training funds. The following frameworks should be included in the list of approved options: Danielson’s Framework for Teaching, Marzano’s Causal Teacher Evaluation Model, and Tulsa’s TLE Observation and Evaluation System. Permanent Recommendation #1d: For the Leader Evaluation System, the TLE Commission recommends that the Oklahoma State Board of Education name a default framework that is paid for by the state in terms of training and implementation requirements to serve as the qualitative assessment component that must comprise 50% of the total evaluation criteria required by 70 O.S. § 6-101.16. Permanent Recommendation #1e: The TLE Commission recommends that the Leader Evaluation default framework be Mc.REL’s Principal Evaluation System. Permanent Recommendation #1f: The TLE Commission recommends that the Oklahoma State Board of Education name a limited number of frameworks that meet specific criteria, including all statutory requirements for district selection. Frameworks other than the default will be supported by local funds or at the discretion of the Oklahoma State Department of Education through a formula based on the district’s Average Daily Attendance. The following frameworks should be included in the list of approved options: McREL’s Principal Evaluation System (pending correlation to statutory criteria) and Reeves’s Leadership Performance Matrix (pending correlation to statutory criteria). Permanent Recommendation #2: For both the Teacher Evaluation System and the Leader Evaluation System, the TLE Commission recommends that any modifications to the default framework or other approved frameworks must be approved by the Oklahoma State Board of Education against a specific set of criteria, including all statutory requirements, based on impact to student learning. ATTACHMENT A Permanent Recommendation #3a: In regards to the quantitative portion of the Teacher and Leader Evaluation System, the TLE Commission recommends using a Value Added Model in calculating the thirty-five percentage points attributed to student academic growth using multiple years of standardized test data for those teachers in grades and subjects for which multiple years of standardized test data exist. Permanent Recommendation #3b: In regards to the quantitative portion of the Teacher and Leader Evaluation System, the TLE Commission recommends using a Value Added Model in calculating the thirty-five percentage points attributed to student academic growth using multiple years of standardized test data for those leaders of buildings containing grades and subjects for which multiple years of standardized test data exist. Permanent Recommendation #4: In addressing those teachers and leaders in grades and subjects for which there is no state-mandated testing measure to create a quantitative assessment, the TLE Commission recommends conducting more research to determine the appropriate measure(s) of student achievement taking into account a combination of multiple measures and including teacher, leader, and specialist input. Permanent Recommendation #5: In regards to the fifteen percentage points based on other academic measures, the TLE Commission recommends conducting further study of best practices across the country as well as inviting Oklahoma educators to provide input to develop a list of appropriate measures for Oklahoma. ATTACHEMENT B Criteria Checklists for All Frameworks Reviewed Teacher Frameworks Danielson’s Framework for Teaching Marzano’s Causal Teacher Evaluation Model McREL’s Teacher Evaluation System Tulsa’s TLE Observation and Evaluation System Leader Frameworks Marzano’s Leadership Evaluation System McREL’s Principal Evaluation System Reeves’s Leadership Performance Matrix Vanderbilt’s Assessment for Leadership in Education Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the Oklahoma State Department of Education and submitted to the Commission for consideration. Selection Description Does not meet the criteria From the materials provided, does not appear to meet the criteria Meets the criteria From the materials provided, appears to meet the criteria Exceeds the criteria From the materials provided, appears to exceed the criteria Framework: Danielson’s Framework for Teaching (ASCD Teacher Effectiveness Suite) Required by Statute Teacher Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Includes a Five-Tier Rating System (qualitative measures) 1) Superior 2) Highly Effective 3) Effective 4) Needs Improvement 5) Ineffective Though none was provided, the framework uses averaging to calculate a score which can be translated into the five-tier rating system. Annual evaluations that provide feedback to improve student learning and outcomes The model includes individualized professional development plans for teachers to work on their practice. Comprehensive remediation plans and instructional coaching for all teachers rated as Needs Improvement or Ineffective The model may be used for further supports for struggling teachers. Be evidence-based (e.g. research and field proven) The model is widely adopted including variations in Oklahoma. Include observable and measurable characteristics of personnel and classroom practices (rubrics and evidences for each rubric) Rubrics are included for each element. An evidence-based tool that will include characteristics of personnel and classroom practices that are correlated to student performance success Be based on research-based national best practices and methodology (contemporary research and practices of expertise development and strategies/behaviors for raising student achievement) The framework was developed in the 1990s and revised periodically. It was developed upon a review of the research but does not reflect the most contemporary research on strategies, lesson segments, and deliberate practice. Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria Required by Statute Teacher Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Must contain minimally: a) Organizational and classroom management skills b) Ability to provide effective instruction c) Focus on continuous improvement and professional growth d) Interpersonal skills e) Leadership skills The framework exceeds the minimum areas in its 76 elements. Accounts for years of service since teacher expertise develops over time No tools are provided in the model to account for years of service. Granular enough with “thin slices” of instruction to support deliberate practice because teachers develop expertise through engaging in focused practice with focused feedback Of the 76 elements, 33 are observable to instruction. Danielson’s framework is broader to the behaviors and lacks the specificity of Marzano to research-based strategies. Identifies the instructional context or lesson type or segment for when it is instructionally appropriate to see certain research-based strategies No documentation in the model identifies when it is appropriate to see certain research-based strategies. Includes scales or rubrics to identify the level of implementation for the strategies All elements have a rubric. Reflects the elements for a research-based common language of instruction that accurately reflects the complexity of teaching The framework reflects the complexity of teaching across its 76 elements. Clearly defines and articulates teacher and student evidences for each rubric The framework lacks tools for specific teacher and student evidences but does include critical attributes to help provide clarity. Be correlated to student performance success (validation studies and causal links studies for instructional strategies/behaviors) The studies conducted this far on the Danielson framework only show a slight increase in student learning. There are no experimental and control studies to verify the specific elements raise student achievement. Validation studies (Specific studies on the model/framework to verify its ability to identify levels of teaching performance correlated to student achievement results) Validation studies do exist for the framework. Research studies verifying the specific classroom practices in the rubrics have a “causal link” to raising student achievement No experimental and control studies have been identified at the element level for the specific strategies represented in the framework. Weights/emphasizes classroom instruction Of the 76 elements in the framework, 33 or 46% are observable to classroom practice. To make this a sufficient weight, additional weighting will need to Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria be placed on Domains 2 and 3. Depth of supports provided for the framework (qualified capacity to support statewide efforts) Due to its wide adoption, there is sufficient capacity and depth of services to support Oklahoma districts. Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the Oklahoma State Department of Education and submitted to the Commission for consideration. Selection Description Does not meet the criteria From the materials provided, does not appear to meet the criteria Meets the criteria From the materials provided, appears to meet the criteria Exceeds the criteria From the materials provided, appears to exceed the criteria Framework: Marzano Causal Teacher Evaluation Model Required by Statute Teacher Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Includes a Five-Tier Rating System (qualitative measures) 1) Superior 2) Highly Effective 3) Effective 4) Needs Improvement 5) Ineffective Presentation included calculation to generate Oklahoma’s five-tier rating system and is currently in use within Oklahoma City as presented by Dr. Brian Staples Annual evaluations that provide feedback to improve student learning and outcomes In addition to the causal link research, the model also contains reflection questions, video examples, teacher and student evidences, etc. to provide teachers with annual evaluations that support their growth and development to raise student learning and outcomes. Comprehensive remediation plans and instructional coaching for all teachers rated as Needs Improvement or Ineffective All teachers are required to develop Professional Growth Plans and engage in deliberate practice in order to document improvements in their teaching. Processes include supports and tools for instructional coaches to engage with struggling teachers and supervision models for more support and observational feedback for struggling teachers. Be evidence-based (e.g. research and field proven) The Art and Science of Teaching, upon which the evaluation model was developed is widely used. The evaluation model is also being used in large scale including a pilot in Oklahoma City and the state of Florida. Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria Required by Statute Teacher Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Include observable and measurable characteristics of personnel and classroom practices (rubrics and evidences for each rubric) The model contains rubrics, teacher and student evidence for each rubric, and coaching supports for each rubric. An evidence-based tool that will include characteristics of personnel and classroom practices that are correlated to student performance success Be based on research-based national best practices and methodology (contemporary research and practices of expertise development and strategies/behaviors for raising student achievement) The model draws upon 35 years of research for what works for raising student achievement. The model also cites contemporary research for the development of expertise and incorporates national best practices for accounting for years of service and growth over time. Must contain minimally: a) Organizational and classroom management skills b) Ability to provide effective instruction c) Focus on continuous improvement and professional growth d) Interpersonal skills e) Leadership skills The model exceeds the minimum requirements. Domains 3-4 incorporate interpersonal skills and leadership skills. Accounts for years of service since teacher expertise develops over time Model includes for categories of teachers accounting for years of service with recommendations for 0-3, 3-9, and 10 plus years of service. Granular enough with “thin slices” of instruction to support deliberate practice because teachers develop expertise through engaging in focused practice with focused feedback Classroom strategies and behaviors (Domain 1) includes 41 elements that are granular enough to support deliberate practice. Identifies the instructional context or lesson type or segment for when it is instructionally appropriate to see certain research-based strategies 41 elements in Domain 1: Classroom Strategies and Behaviors are classified into lesson types or segments for both teachers and observers to identify when it is appropriate to see certain strategies. Includes scales or rubrics to identify the level of implementation for the strategies All elements in the framework include 5 point scales/rubrics identifying levels of implementation of the strategies. Reflects the elements for a research-based common language of instruction that accurately reflects the complexity of teaching The complexity of teaching is represented in the model and the model reflects a substantial research base drawn from 35 years of research and meta-analysis. Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria Required by Statute Teacher Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Clearly defines and articulates teacher and student evidences for each rubric Every scale/rubric includes examples of teacher and student evidence. Be correlated to student performance success (validation studies and causal links studies for instructional strategies/behaviors) The model has substantial research documentation for raising student achievement through the 41 classroom strategies and behaviors in Domain 1. Validation studies (Specific studies on the model/framework to verify its ability to identify levels of teaching performance correlated to student achievement results) Only model where validation studies conducted within Oklahoma were cited for the model Research studies verifying the specific classroom practices in the rubrics have a “causal link” to raising student achievement Over 300 individual experimental and control studies have been completed identifying the causal link for use of strategies cited in the model to increases in student learning. Weights/emphasizes classroom instruction Of the 60 total elements in the Marzano model, 41 or 68% represent classroom instruction. Depth of supports provided for the framework (qualified capacity to support statewide efforts) Given the statewide implementations currently underway with the model, there is both capacity to support Oklahoma districts and a depth of supports from trainings or certification for evaluators for accuracy for observations. Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the Oklahoma State Department of Education and submitted to the Commission for consideration. Selection Description Does not meet the criteria From the materials provided, does not appear to meet the criteria Meets the criteria From the materials provided, appears to meet the criteria Exceeds the criteria From the materials provided, appears to exceed the criteria Teacher Framework: McREL’s Teacher Evaluation System Required by Statute Teacher Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Includes a Five-Tier Rating System (qualitative measures) 1) Superior 2) Highly Effective 3) Effective 4) Needs Improvement 5) Ineffective Although none was provided the framework could translate a score into the five tiers Annual evaluations that provide feedback to improve student learning and outcomes Comprehensive remediation plans and instructional coaching for all teachers rated as Needs Improvement or Ineffective Appears to have a feedback cycle and professional development plan process that may be adapted to this requirement Be evidence-based (e.g. research and field proven) Research provided Include observable and measurable characteristics of personnel and classroom practices (rubrics and evidences for each rubric) An evidence-based tool that will include characteristics of personnel and classroom practices that are correlated to student performance success Be based on research-based national best practices and methodology (contemporary research and practices of expertise development and strategies/behaviors for raising student achievement) Minimally meets as the framework is very broad (25 elements), which is less than half of Danielson or Marzano frameworks Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria Required by Statute Teacher Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Must contain minimally: a) Organizational and classroom management skills b) Ability to provide effective instruction c) Focus on continuous improvement and professional growth d) Interpersonal skills e) Leadership skills Minimally meets as the framework is very broad (25 elements), which is less than half of Danielson or Marzano frameworks Accounts for years of service since teacher expertise develops over time Granular enough with “thin slices” of instruction to support deliberate practice because teachers develop expertise through engaging in focused practice with focused feedback The framework is intentionally designed broadly and lacks specificity and clarity around use of research-based strategies Identifies the instructional context or lesson type or segment for when it is instructionally appropriate to see certain research-based strategies Includes scales or rubrics to identify the level of implementation for the strategies Reflects the elements for a research-based common language of instruction that accurately reflects the complexity of teaching Minimally meets as the framework is very broad (25 elements), which is less than half of Danielson or Marzano frameworks Clearly defines and articulates teacher and student evidences for each rubric Uses a progressive checklist approach with minimal clarity at each level Be correlated to student performance success (validation studies and causal links studies for instructional strategies/behaviors) None were provided Validation studies (Specific studies on the model/framework to verify its ability to identify levels of teaching performance correlated to student achievement results) None were provided Research studies verifying the specific classroom practices in the rubrics have a “causal link” to raising student achievement None were provided Weights/emphasizes classroom instruction No weighting recommendations were provided, but appears weighting could be adjusted Depth of supports provided for the framework (qualified capacity to support statewide efforts) Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the Oklahoma State Department of Education and submitted to the Commission for consideration. Selection Description Does not meet the criteria From the materials provided, does not appear to meet the criteria Meets the criteria From the materials provided, appears to meet the criteria Exceeds the criteria From the materials provided, appears to exceed the criteria Teacher Framework: Tulsa Public Schools Teacher Leader Effectiveness Observation and Evaluation System Required by Statute Teacher Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Includes a Five-Tier Rating System (qualitative measures) 1) Superior 2) Highly Effective 3) Effective 4) Needs Improvement 5) Ineffective Annual evaluations that provide feedback to improve student learning and outcomes Comprehensive remediation plans and instructional coaching for all teachers rated as Needs Improvement or Ineffective Be evidence-based (e.g. research and field proven) The evidence base and field experience is one district Include observable and measurable characteristics of personnel and classroom practices (rubrics and evidences for each rubric) ��� An evidence-based tool that will include characteristics of personnel and classroom practices that are correlated to student performance success Be based on research-based national best practices and methodology (contemporary research and practices of expertise development and strategies/behaviors for raising student achievement) Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria Required by Statute Teacher Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Must contain minimally: a) Organizational and classroom management skills b) Ability to provide effective instruction c) Focus on continuous improvement and professional growth d) Interpersonal skills e) Leadership skills Accounts for years of service since teacher expertise develops over time Granular enough with “thin slices” of instruction to support deliberate practice because teachers develop expertise through engaging in focused practice with focused feedback Framework is the most broad of all with 20 elements Identifies the instructional context or lesson type or segment for when it is instructionally appropriate to see certain research-based strategies Includes scales or rubrics to identify the level of implementation for the strategies Reflects the elements for a research-based common language of instruction that accurately reflects the complexity of teaching Clearly defines and articulates teacher and student evidences for each rubric Be correlated to student performance success (validation studies and causal links studies for instructional strategies/behaviors) There is some encouraging evidence emerging in the district but not yet a research study Validation studies (Specific studies on the model/framework to verify its ability to identify levels of teaching performance correlated to student achievement results) This question was raised during the presentation and the framework has not yet been validated Research studies verifying the specific classroom practices in the rubrics have a “causal link” to raising student achievement No research studies were submitted Weights/emphasizes classroom instruction Depth of supports provided for the framework (qualified capacity to support statewide efforts) Given only one district is currently implementing, there would likely be capacity issues trying to scale it statewide Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the Oklahoma State Department of Education and submitted to the Commission for consideration. Selection Description Does not meet the criteria From the materials provided, does not appear to meet the criteria Meets the criteria From the materials provided, appears to meet the criteria Exceeds the criteria From the materials provided, appears to exceed the criteria Leadership Framework: Marzano Leadership Evaluation System Required by Statute Leader Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Includes a Five-Tier Rating System (qualitative measures) 1) Superior 2) Highly Effective 3) Effective 4) Needs Improvement 5) Ineffective Currently being developed for Oklahoma City Annual evaluations that provide feedback to improve student learning and outcomes Be evidence-based (e.g. research and field proven) Just starting pilot Include observable and measurable characteristics of personnel and site management practices (rubrics and evidences for each rubric) Be correlated to student performance success Just starting pilot Be based on research-based national best practices and methodology Include the following six domains: organizational and school management skills, including retention and development of effective teachers and dismissal of ineffective teachers; instructional leadership; professional growth and responsibility; interpersonal skills; leadership skills; and stakeholder perceptions. Includes scales or rubrics to identify the level of implementation for the leadership behaviors Clearly defines and articulates teacher and student evidences for each rubric Currently being developed Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria Required by Statute Leader Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Validation studies (Specific studies on the model/framework to verify its ability to identify levels of leadership performance correlated to student achievement results) Based upon numerous research studies Weights/emphasizes teacher growth and development Depth of supports provided for the framework (qualified capacity to support statewide efforts) Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the Oklahoma State Department of Education and submitted to the Commission for consideration. Selection Description Does not meet the criteria From the materials provided, does not appear to meet the criteria Meets the criteria From the materials provided, appears to meet the criteria Exceeds the criteria From the materials provided, appears to exceed the criteria Leadership Framework: McREL’s Principal Evaluation System Required by Statute Leader Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Includes a Five-Tier Rating System (qualitative measures) 1) Superior 2) Highly Effective 3) Effective 4) Needs Improvement 5) Ineffective Although none was provided the framework could translate a score into the five tiers Annual evaluations that provide feedback to improve student learning and outcomes Be evidence-based (e.g. research and field proven) Include observable and measurable characteristics of personnel and site management practices (rubrics and evidences for each rubric) Broad framework with 21 elements compared to 38 for Reeves framework Be correlated to student performance success Based upon research from which the framework is drawn Be based on research-based national best practices and methodology Include the following six domains: organizational and school management skills, including retention and development of effective teachers and dismissal of ineffective teachers; instructional leadership; professional growth and responsibility; interpersonal skills; leadership skills; and stakeholder perceptions. Minimally addresses Includes scales or rubrics to identify the level of implementation for the leadership behaviors Clearly defines and articulates teacher and student evidences for each rubric Uses a progressive checklist approach with minimal evidences Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria Required by Statute Leader Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Validation studies (Specific studies on the model/framework to verify its ability to identify levels of leadership performance correlated to student achievement results) The framework is drawn from many studies Weights/emphasizes teacher growth and development Although weighting could be applied to emphasize teacher growth, none was provided Depth of supports provided for the framework (qualified capacity to support statewide efforts) Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the Oklahoma State Department of Education and submitted to the Commission for consideration. Selection Description Does not meet the criteria From the materials provided, does not appear to meet the criteria Meets the criteria From the materials provided, appears to meet the criteria Exceeds the criteria From the materials provided, appears to exceed the criteria Framework: Reeves’ Leadership Performance Matrix Required by Statute Leader Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Includes a Five-Tier Rating System (qualitative measures) 1) Superior 2) Highly Effective 3) Effective 4) Needs Improvement 5) Ineffective A clear translation from the current four-tier rating system to a five-tier system can be easily performed. Annual evaluations that provide feedback to improve student learning and outcomes The overall purpose of the Reeves’ MLA System is to improve leadership and provide a clear path for each element of performance. The MLA system is used as a learning system directly correlated to teacher action and student learning. Be evidence-based (e.g. research and field proven) The Reeves’ MLA System is widely used by individual school districts as well as being competitively selected as the statewide model in both Florida and New York. Include observable and measurable characteristics of personnel and site management practices (rubrics and evidences for each rubric) Same response as for item #2 above. Be correlated to student performance success The MLA system provides not just a rearview look but rather a windshield approach linking the student data, teacher data, and leader data in real time. Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria Required by Statute Leader Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Be based on research-based national best practices and methodology Consists of leadership best practices that are well documented in two of the most recent studies on leadership (Wahlstrom, Louis, Leithwood, & Anderson, 2010; Hattie, 2009) Include the following six domains: organizational and school management skills, including retention and development of effective teachers and dismissal of ineffective teachers; instructional leadership; professional growth and responsibility; interpersonal skills; leadership skills; and stakeholder perceptions. Each of the state identified leadership domains are clearly reflected in the Reeves’ MLA Framework as well as additional domains that are linked to effective leadership and increased student achievement. Includes scales or rubrics to identify the level of implementation for the leadership behaviors Ten leadership domains with 38 subdomains are identified. A continuum of performance is clearly described for each domain. Clearly defines and articulates teacher and student evidences for each rubric Each of the continuum descriptions within the ten domains clearly articulates and defines the required teacher and student evidence. It is an evidenced-based rubric, not opinion. Validation studies (Specific studies on the model/framework to verify its ability to identify levels of leadership performance correlated to student achievement results) The MLA is validated by multiple studies and methodologies i.e. Marzano, Waters, McNulty, Hattie, Reeves. Weights/emphasizes teacher growth and development The Reeves’ MLA Framework emphasizes teacher growth and development (Domain 6.0 Faculty Development) in addition to three other key leadership practices (i.e., Student Achievement, Leadership Development, and Personal and Professional Learning) all of which enhance teacher growth and development. Depth of supports provided for the framework (qualified capacity to support statewide efforts) Utilizes the findings from The Center’s five-year Implementation Audit Study involving over 2,000 schools across the United States and Canada and over 1.5 million students that assist organizations in the deep implementation of initiatives. Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the Oklahoma State Department of Education and submitted to the Commission for consideration. Selection Description Does not meet the criteria From the materials provided, does not appear to meet the criteria Meets the criteria From the materials provided, appears to meet the criteria Exceeds the criteria From the materials provided, appears to exceed the criteria Leadership Framework: VAL-ED (Vanderbilt Assessment of Leadership in Education) Required by Statute Leader Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Includes a Five-Tier Rating System (qualitative measures) 1) Superior 2) Highly Effective 3) Effective 4) Needs Improvement 5) Ineffective Could be part of a leadership evaluation but is not a complete evaluation framework Annual evaluations that provide feedback to improve student learning and outcomes Is not an evaluation system Be evidence-based (e.g. research and field proven) Include observable and measurable characteristics of personnel and site management practices (rubrics and evidences for each rubric) Be correlated to student performance success Be based on research-based national best practices and methodology Include the following six domains: organizational and school management skills, including retention and development of effective teachers and dismissal of ineffective teachers; instructional leadership; professional growth and responsibility; interpersonal skills; leadership skills; and stakeholder perceptions. May be used as part of a leadership evaluation system but is missing required components Includes scales or rubrics to identify the level of implementation for the leadership behaviors Clearly defines and articulates teacher and student evidences for each rubric Teacher and Leader Effectiveness Commission Teacher Evaluation Framework/Model Selection Criteria Required by Statute Leader Evaluation Framework/Model Selection Criteria for the Qualitative Measures Does not meet the criteria Meets the criteria Exceeds the criteria Reviewers’ Notes Validation studies (Specific studies on the model/framework to verify its ability to identify levels of leadership performance correlated to student achievement results) Weights/emphasizes teacher growth and development Depth of supports provided for the framework (qualified capacity to support statewide efforts) ATTACHEMENTS C-G Attachments C-G are the full, proprietary frameworks of Danielson, Marzano, Tulsa, McREL, and Reeves. These attachments were provided to the Governor, required members of the Legislature, and the Oklahoma State Board of Education for review and approval purposes only. Districts will be provided access to the full frameworks during TLE Training. ATTACHEMENT H Summary of Public Comment Attachment H Summary of Public Comment As a brief overview, on September 13, 2011, the State Department of Education created a public survey to gather input regarding the Commission’s first two preliminary recommendations. To date, this survey remains active; however, the following results reflect the data from September 13, 2011 through December 2, 2011. Accessibility to this electronic survey was made available through the Department’s website, and email listservs. In addition, several organizations throughout the state added links to the survey onto their websites. The following is a snapshot of some of the more pertinent data generated from this survey. As illustrated in the pie chart, the vast majority of the 1,158 survey responders were teachers, totaling 76.5% of the total responders. Building administrators accounted for 9.7% of the responders and district administrators accounted for 6.4% of the responders. In total, nearly 93% of the responders to the public comment survey were people who have direct, daily involvement with education. Teacher 77% Building Administrator 10% District Administrator 6% Other 7% Please select the role that best describes you 1,158 Total Responders Most responders agreed that each of the three Frameworks should be included as an option for district selection. When asked which Framework should be selected as the default, the Marzano’s Framework received the highest approval rating at 22.3%. Tulsa’s Framework received a 12% approval rating and Danielson’s Framework received a 7.5% approval rating. Most notably, when asked which Framework should not be included as an option, Tulsa’s Framework received the highest rating at 41.2%. For this same question, Marzano’s Framework received a 25.3% rating and Danielson Framework received a 36.9% rating. 55.4 49.2 58.8 25.2 41.3 36.8 22.3 12 7.5 Marzano Tulsa Danielson Which Teacher Frameworks do you believe should be included in an approved list for district selection and which should be named as the default? Include as an option Do not include as an option Name as a default It is important to note that when this question was originally posed to the public, the Marzano’s Leadership Evaluation System was presented as an option. However, as of the Commission’s December 5, 2011 meeting, the Marzano Leadership System was not fully developed. As a result, the Commission only considered the McREL and Reeves Frameworks in its selection. Interestingly, Marzano’s Leadership System received the highest “Name as Default” rating at 21.2%. 53.9 54.3 53.9 38.8 39.5 26.6 8.1 6.4 21.2 McREL Reeves' Marzano (Not considered) Please indicate which Leader frameworks you believe should be included in an approved list for district selection and which one should be named as the default Include as an option Do not inlcude as an option Name as default After the Commission made preliminary recommendations 3-5 at its November 7, 2011 meeting, the public was asked to respond, via email, to the newest recommendations as well as provide overall input regarding the TLE process. Twenty-eight emails and letters were received, the majority of which were generic comments and concerns regarding the TLE process. Two emails were specifically in favor of adopting the Marzano Framework. There were no emails received that favored either the Tulsa or the Danielson Framework. Below is a direct copy of one of the emails in support of the Marzano Framework. After much thought about which Teacher Leader Effectiveness Framework would make the greatest impact on Teaching and Learning in my district, I have come full circle on my preference! I first thought the Tulsa model would be good because it was the least amount of change, and thus would be easier to "sell" to anyone who is reluctant about change. I even sent Comments on TLE earlier that leaned in favor of the Tulsa model. After studying Robert Marzano's The Art and Science of Teaching, I now see the impact his framework could make on instruction, and THAT (improved instruction) is what will make a difference for our students in Mid-Del. We have caring teachers who prepare and teach well, but many do not employ a framework to design their instructional lessons and to organize their instructional strategies. That is the strength of Marzano's Framework! To further benefit and add to the professional development of educators using the protocol, Marzano's online observation tool contains video clips that relate directly to elements/ indicators in the observation protocol. So when I identify an area that needs to be strengthened in a teacher's toolkit of procedures and strategies, I can simply click to direct the teacher to a master teacher modeling that particular strategy. In Marzano's work, teaching<learning<evaluation of teaching and learning - - all is blended together with common language. It blends perfectly with the style of instruction required to teach Common Core effectively. Finally professional development would be directly tied to research and to the evaluation, and everyone would have a clear path and a purpose leading to improvement as we hone our skills as educators. In my 35 years as an educator, these are the most exciting times I've experienced! We have such an opportunity to truly impact the way teachers teach, and the way students learn! In Mid-Del, we are bringing Phil Warrick, from the Marzano Research group, to guide our principals in professional development using the framework The Art and Science of Teaching. I would invite any of the Commission members or State Department staff who would like to hear more and see the training unfold to join us in Mid-Del on November 30 during Dr. Warrick's presentation. Please share my thoughts with the TLE Commission and any others at the State Department who might want to hear my thoughts. Thank you! Kathy Dunn Executive Director of Teaching & Learning (405) 737-4461 x1225 Kdunn@mid-del.net Mid-Del Schools ATTACHEMENT I State Board of Education Adopted Policies Oklahoma State Board of Education Adopted Policies Pursuant to 70 O.S. § 6-101.16A December 15, 2011 The TLE Commission has approved certain frameworks for district selection both for Teacher and Leader Evaluation. A pilot program will be conducted over the next year (2012-2013) using the approved frameworks that are selected by each district. At the end of the pilot program, both the TLE Commission and the State Board of Education will be better able to evaluate each framework. Based on the TLE Commission’s recommendations, the State Board of Education named the Tulsa TLE Observation and Evaluation System for the Teacher Training Evaluation and the McREL Principal Evaluation System for the Leadership Training Evaluation as the presumptive default frameworks. During the pilot program, the allocation of funds between approved frameworks will be supported by local funds or at the discretion of the Oklahoma Department of Education through a formula based on the districts’ Average Daily Attendance. At the end of the pilot program, in one year, after further study and recommendations by the TLE Commission, the State Board of Education will adopt default frameworks. For the Teacher Evaluation System, the Oklahoma State Board of Education has named a limited number of frameworks that meet specific criteria, including all statutory requirements, for district selection. The following frameworks are included in the list of approved options: Danielson’s Framework for Teaching, Marzano’s Causal Teacher Evaluation Model, and Tulsa’s TLE Observation and Evaluation System. For the Leader Evaluation System, The Oklahoma State Board of Education has named a limited number of frameworks that meet specific criteria, including all statutory requirements for district selection. The following frameworks are included in the list of approved options: McREL’s Principal Evaluation System (pending correlation to statutory criteria) and Reeves’s Leadership Performance Matrix (pending correlation to statutory criteria). For both the Teacher Evaluation System and the Leader Evaluation System, any modifications to the default framework or other approved frameworks must be approved by the Oklahoma State Board of Education against a specific set of criteria, including all statutory requirements, based on impact to student learning. In regards to the quantitative portion of the Teacher and Leader Evaluation System, the Oklahoma State Board of Education approves using a Value Added Model in calculating the thirty-five percentage points attributed to student academic growth using multiple years of standardized test data for those teachers in grades and subjects for which multiple years of standardized test data exist. In regards to the quantitative portion of the Teacher and Leader Evaluation System, the Oklahoma State Board of Education approves using a Value Added Model in calculating the thirty-five percentage points attributed to student academic growth using multiple years of standardized test data for those leaders of buildings containing grades and subjects for which multiple years of standardized test data exist. In addressing those teachers and leaders in grades and subjects for which there is no state-mandated testing measure to create a quantitative assessment, the Oklahoma State Board of Education approves conducting more research to determine the appropriate measure(s) of student achievement taking into account a combination of multiple measures and including teacher, leader, and specialist input. In regards to the fifteen percentage points based on other academic measures, the Oklahoma State Board of Education approves conducting further study of best practices across the country as well as inviting Oklahoma educators to provide input to develop a list of appropriate measures for Oklahoma.
Object Description
Description
Title | Teacher Leader Effectiveness Evaluation System |
OkDocs Class# | E1800.3 T253l/r 2011 |
Digital Format | PDF, Adobe Reader required |
ODL electronic copy | Downloaded from agency website: http://www.sde.state.ok.us/Teacher/Commission/pdf/ReportLeg.pdf |
Rights and Permissions | This Oklahoma state government publication is provided for educational purposes under U.S. copyright law. Other usage requires permission of copyright holders. |
Language | English |
Full text |
1
Oklahoma Teacher and Leader Effectiveness Evaluation System (TLE)
REPORT
To the Oklahoma State Legislature and Governor on the Recommendations of the TLE
Commission and the Adoption of the TLE by the Oklahoma State Board of Education
OVERVIEW
Oklahoma state law (70 O.S. § 6-101.16) established the Oklahoma Teacher and Leader
Effectiveness Evaluation System (TLE), a new evaluation system for teachers and leaders
designed to encourage continuous professional growth leading toward improved student
achievement for all Oklahoma students. The law requires that the new system be comprised of
multiple measures of teacher and administrator effectiveness:
50% Qualitative Measures (observable characteristics of teacher and leader performance
that are correlated to student achievement)
35% Quantitative Measures of Student Academic Growth (based on multiple years of
standardized test data)
15% Quantitative Measures of Other Academic Factors
According to state law, all local board of education evaluation policies must align with the TLE
by the 2013-2014 school year.
This report has been developed pursuant to state statute requiring an annual report of the TLE
Commission.
The Commission shall issue a report by December 31 of each year and submit a copy of the report to the Governor,
the Speaker of the House of Representatives and the President Pro Tempore of the Senate. (70 O.S. § 6-101.17I)
ADOPTION PROCESS
State law required the Oklahoma State Board of Education to adopt the TLE by December 15,
2011, and to receive advice from the TLE Commission as discussed later in this report.
By December 15, 2011, the State Board of Education shall adopt a new statewide system of evaluation to be known
as the Oklahoma Teacher and Leader Effectiveness Evaluation System (TLE). (70 O.S. § 6-101.16A)
The Teacher and Leader Effectiveness Commission shall provide oversight and advise the State Board of Education
on the development and implementation of the TLE. (70 O.S. § 6-101.16C)
COMPONENTS OF THE SYSTEM
The TLE shall include the following components:
1. A five-tier rating system as follows:
a. superior,
b. highly effective,
2
c. effective,
d. needs improvement, and
e. ineffective;
2. Annual evaluations that provide feedback to improve student learning and outcomes;
3. Comprehensive remediation plans and instructional coaching for all teachers rated as needs improvement or
ineffective;
4. Quantitative and qualitative assessment components measured as follows:
a. fifty percent (50%) of the ratings of teachers and leaders shall be based on quantitative components
which shall be divided as follows:
(1) thirty-five percentage points based on student academic growth using multiple years of
standardized test data, as available, and
(2) fifteen percentage points based on other academic measurements, and
b. fifty percent (50%) of the rating of teachers and leaders shall be based on rigorous and fair qualitative
assessment components;
5. An evidence-based qualitative assessment tool for the teacher qualitative portion of the TLE that will include
observable and measurable characteristics of personnel and classroom practices that are correlated to student
performance success, including, but not limited to:
a. organizational and classroom management skills,
b. ability to provide effective instruction,
c. focus on continuous improvement and professional growth,
d. interpersonal skills, and
e. leadership skills;
6. An evidence-based qualitative assessment tool for the leader qualitative portion of the TLE that will include
observable and measurable characteristics of personnel and site management practices that are correlated to
student performance success, including, but not limited to:
a. organizational and school management, including retention and development of effective teachers and
dismissal of ineffective teachers,
b. instructional leadership,
c. professional growth and responsibility,
d. interpersonal skills,
e. leadership skills, and
f. stakeholder perceptions; and
7. For those teachers in grades and subjects for which there is no state-mandated testing measure to create a
quantitative assessment for the quantitative portion of the TLE, an assessment using objective measures of teacher
effectiveness including student performance on unit or end-of-year tests. Emphasis shall be placed on the observed
qualitative assessment as well as contribution to the overall school academic growth. (70 O.S. § 6-101.16B)
TLE COMMISSION
OVERVIEW
The law established the TLE Commission to make recommendations about the design of the
system and to oversee implementation of the system through June 30, 2016. The TLE
Commission is comprised of various education and public sector stakeholders appointed by the
Governor, President Pro Tempore of the Senate, and Speaker of the House, or as designated by
statute.
The TLE Commission met eight times from June 29 through December 5, 2011, to study the
various components of the TLE, to evaluate a variety of qualitative measurement tools
(frameworks) and quantitative measurement tools, to make formal design and implementation
3
recommendations to the State Board of Education, and to solicit public input on those
recommendations.
Five design and implementation recommendations were approved at the TLE Commission
meeting on December 5, 2011, after receiving feedback from a wide representation of
stakeholders, including teachers, administrators, students, parents, community members, and
policymakers. See Attachment A.
Details regarding the extensive study conducted by the TLE Commission as well as a summary
of public comment received is provided in subsequent sections of this report and in the
attachments.
TLE COMMISSION ANALYSIS OF QUALITATIVE MEASURES
The TLE Commission reviewed four teacher evaluation frameworks for the qualitative measure
and four leader evaluation frameworks for the qualitative measure. These frameworks were:
Teacher Frameworks
Danielson’s Framework for Teaching (See Attachment C)
Marzano’s Causal Teacher Evaluation Model (See Attachment D)
McREL’s Teacher Evaluation System
Tulsa’s TLE Observation and Evaluation System (See Attachment E)
Leader Frameworks
Marzano’s Leadership Evaluation System
McREL’s Principal Evaluation System (See Attachment F)
Reeves’s Leadership Performance Matrix (See Attachment G)
Vanderbilt’s Assessment for Leadership in Education
The TLE Commission preliminarily recommended six of these tools – three teacher frameworks
and three leader frameworks – for consideration. The McREL Teacher Evaluation System was
eliminated because the TLE Commission determined that the framework did not meet the needs
of Oklahoma educators. In addition, the Marzano Leadership Evaluation System was eliminated
because it was still in production at the time of final recommendations and the Vanderbilt
Assessment for Leadership in Education was also eliminated because the TLE Commission
determined that the framework did not meet the needs of Oklahoma educators.
The rubrics used to compare each framework to requirements of state statute and national best
practices are available as Attachment B.
Recommendations #1c and #1f indicate that the TLE Commission recommended Danielson’s
Framework for Teaching, Marzano’s Causal Teacher Evaluation Model, Tulsa’s TLE
Observation and Evaluation System, McREL’s Principal Evaluation System, and Reeves’s
Leadership Performance Matrix for district selection. See Attachment A.
4
TLE COMMISSION ANALYSIS OF QUANTITATIVE MEASURES
The Commission examined a variety of possible ways to evaluate student growth for teachers
who teach grades or subject areas where student growth data exists. One option the Commission
reviewed was a Simple Growth Model. This model compares student performance at the end of
instruction to performance prior to instruction. The Commission also reviewed Value Added
Models. While this option also measures student growth, it measures that growth against the
student’s predicted growth level for the school year. This prediction is determined through a
complex series of calculations that factor in such variables as attendance, mobility, past
achievement, ELL status, and/or number of subject-specific courses in which the student is
enrolled. The focus of the variables can be based either on the student’s prior achievement
(Covariate Model), or on the student’s propensity to achieve along with the durability of the
teacher’s effect on the expected growth (Learning Path Model). In essence, a Value Added
Model determines what value the teacher added to the student’s success.
The Commission determined that utilizing a Value Added Model would best reflect Oklahoma’s
need to take into account other student and school-level variables in order to have the most
accurate evaluation system possible. Therefore, the Commission recommended adoption of a
Value Added Model for teachers and leaders of buildings for which multiple years of
standardized test data exist. See Attachment A: #3a, #3b.
For teachers who teach in grades or subject areas in which no state-mandated testing exists, the
quantitative component of the TLE shall involve an assessment using objective measures of
teacher effectiveness including student performance on unit or end-of-year tests. The
Commission has reviewed several ways to generate data for those grades and subjects where
statewide student assessment data does not exist. These methods include developing additional
state assessments, developing a list of content-specific appropriate measures of student
achievement, using student growth data of “owned students” or all school-wide data, or using a
combination of the above referenced methods.
The Commission recommended conducting further research on the most appropriate measure(s)
of teacher effectiveness for those teachers in non-tested grades and subjects and to take into
consideration the input of representatives of those teacher groups. See Attachment A: #4.
In addition, the Commission reviewed options for the quantitative measures identified as “Other
Academic Measures.” The Commission recommended involving Oklahoma educators in
development of a list of appropriate measures for teacher and supervisor selection based on
findings from research regarding multiple measures of teacher effectiveness. See Attachment A:
#5.
TLE COMMISSION ANALYSIS OF IMPLEMENTATION PROCESSES
In order to facilitate implementation of the TLE statewide, the TLE Commission made several
recommendations regarding selection of default frameworks and funding for training. The
Commission recommended selecting a default framework for the teacher qualitative assessment
and a default framework for the leader qualitative assessment. See Attachment A: #1a, #1d. The
5
Commission selected their recommended default frameworks after much debate and
consideration of each framework.
Despite the public comments found in Attachment H, which indicate that the majority of
responders favored Marzano’s Causal Teacher Evaluation Model, in a split vote, the TLE
Commission recommended Tulsa’s TLE Observation and Evaluation System as the default
teacher qualitative assessment tool. See Attachment A: #1b. Also in a split vote, the
Commission recommended McREL��s Principal Evaluation System as the default leader
qualitative assessment tool. See Attachment A: #1e.
In addition, the TLE Commission made recommendations to reserve a portion of the available
state funds designated for training and implementation for the frameworks not selected as the
default. See Attachment A: #1c, #1f.
Lastly, the TLE Commission recommended that any modifications made to the default
framework or other approved frameworks must be approved by the Oklahoma State board of
Education. See Attachment A: #2.
STATE BOARD OF EDUCATION
OVERVIEW OF APPROVED TLE SYSTEM
On December 15, 2011, the State Board of Education approved a TLE System pursuant to 70
O.S. § 6-101.16A. The State Board named the Tulsa TLE Observation and Evaluation System as
the presumptive default for teacher evaluations and the McREL Principal Evaluation System as
the presumptive default for leader evaluations. See Attachment I.
During the pilot year of implementation (2012-2013), districts will be allowed to choose from
three teacher evaluation frameworks (Tulsa’s TLE Observation and Evaluation System,
Marzano’s Causal Teacher Evaluation Model, and Danielson’s Framework for Teaching) and
two leader evaluation frameworks (McREL’s Principal Evaluation System, and Reeves’s
Leadership Performance Matrix). During this pilot period, districts are required to select and
implement a teacher and leader framework from the list of approved frameworks.
Throughout the pilot year, districts will be asked to provide input and feedback regarding the
frameworks, and the data provided by districts will be reported by OSDE to the TLE
Commission and State Board of Education for consideration. Teacher and leader evaluations
obtained during the pilot year will not count against teachers or leaders during the 2012-2013
school year. However, the data obtained during the pilot year may be used by districts to
establish baselines and offer guidance as Oklahoma schools move forward with permanent
implementation during the 2013-2014 school year. Data and research obtained during the pilot
year may be independently analyzed to determine evidence of measurement of effective teaching
and leadership as well as the ability of each model to scale up for statewide implementation.
This information will be used by the TLE Commission to make further recommendations to the
State Board of Education. At the end of the pilot year, the State Board of Education will adopt
default frameworks.
6
The allocation of funds will be supported by local funds or at the discretion of the Oklahoma
Department of Education through a formula based on the districts’ Average Daily Attendance.
See Attachment I.
Qualitative Measures (50% of Total TLE) See Attachment I.
Teacher
For the teacher qualitative assessment, the Oklahoma State Board of Education (OSBE)
has approved three frameworks from which districts may choose:
o Danielson’s Framework for Teaching(pending licensing agreements),
o Marzano’s Causal Teacher Evaluation Model(pending licensing agreements), and
Tulsa’s TLE Observation and Evaluation System(pending licensing agreements).
For the teacher qualitative assessment, the OSBE has approved Tulsa’s TLE Observation
and Evaluation System framework to become the presumptive default statewide
framework.
Leader
For the leader qualitative assessment, the OSBE has approved two frameworks from
which districts may choose:
McREL’s Principal Evaluation System (pending correlation to statutory criteria and
licensing agreements), and
o Reeves’s Leadership Performance Matrix (pending correlation to statutory criteria
and licensing agreements).
For the leader qualitative assessment, the OSBE has approved McREL’s Principal Evaluation
System framework to become the presumptive default statewide framework.
Teacher and Leader
Any modifications to the default frameworks or other approved frameworks must be
approved by the Oklahoma State Board of Education against a specific set of criteria,
including all statutory requirements, based on impact to student learning.
Quantitative Measures of Student Academic Growth (35% of Total TLE) See Attachment I.
The OSDE will use a Value Added Model in calculating the thirty-five percentage points
attributed to student academic growth using multiple years of standardized test data for
those teachers in grades and subjects for which multiple years of standardized test data
exist.
The OSDE will use a Value Added Model in calculating the thirty-five percentage points
attributed to student academic growth using multiple years of standardized test data for
those leaders of buildings containing grades and subjects for which multiple years of
standardized test data exist.
In addressing those teachers and leaders in grades and subjects for which there is no
state-mandated testing measure to create a Value Added Score, the OSDE will conduct
more research to determine the appropriate measure(s) of student achievement taking into
account a combination of multiple measures and including teacher, leader, and specialist
8
LIST OF ATTACHMENTS
ATTACHMENT DESCRIPTION
A Teacher and Leader Effectiveness Commission
Permanent Recommendations
B Criteria Checklists for All Frameworks Reviewed
C Danielson’s Framework for Teaching
D Marzano’s Causal Teacher Evaluation System
E Tulsa’s TLE Observation and Evaluation System
F McREL’s Principal Evaluation System
G Reeves’ Leadership Performance Matrix
H Summary of Public Comment
I State Board of Education Adopted Policies
ATTACHEMENT A
Teacher and Leader Effectiveness Commission Permanent Recommendations
ATTACHMENT A
Teacher and Leader Effectiveness Commission Permanent Recommendations
Pursuant to 70 O.S. § 6-101.17
December 5, 2011
Permanent Recommendation #1a: For the Teacher Evaluation System, the TLE
Commission recommends that the Oklahoma State Board of Education name a default
framework that is paid for by the state in terms of training and implementation
requirements to serve as the qualitative assessment component that must comprise 50%
of the total evaluation criteria required by 70 O.S. § 6-101.16.
Permanent Recommendation #1b: The TLE Commission recommends that the Teacher
Evaluation default framework be Tulsa’s TLE Observation and Evaluation System.
Permanent Recommendation #1c: The TLE Commission recommends that the
Oklahoma State Board of Education name a limited number of frameworks that meet
specific criteria, including all statutory requirements, for district selection. Frameworks
other than the default will be supported by local funds and twenty-five percent (25%) of
available state training funds. The following frameworks should be included in the list of
approved options: Danielson’s Framework for Teaching, Marzano’s Causal Teacher
Evaluation Model, and Tulsa’s TLE Observation and Evaluation System.
Permanent Recommendation #1d: For the Leader Evaluation System, the TLE
Commission recommends that the Oklahoma State Board of Education name a default
framework that is paid for by the state in terms of training and implementation
requirements to serve as the qualitative assessment component that must comprise 50%
of the total evaluation criteria required by 70 O.S. § 6-101.16.
Permanent Recommendation #1e: The TLE Commission recommends that the Leader
Evaluation default framework be Mc.REL’s Principal Evaluation System.
Permanent Recommendation #1f: The TLE Commission recommends that the
Oklahoma State Board of Education name a limited number of frameworks that meet
specific criteria, including all statutory requirements for district selection. Frameworks
other than the default will be supported by local funds or at the discretion of the
Oklahoma State Department of Education through a formula based on the district’s
Average Daily Attendance. The following frameworks should be included in the list of
approved options: McREL’s Principal Evaluation System (pending correlation to
statutory criteria) and Reeves’s Leadership Performance Matrix (pending correlation to
statutory criteria).
Permanent Recommendation #2: For both the Teacher Evaluation System and the
Leader Evaluation System, the TLE Commission recommends that any modifications to
the default framework or other approved frameworks must be approved by the Oklahoma
State Board of Education against a specific set of criteria, including all statutory
requirements, based on impact to student learning.
ATTACHMENT A
Permanent Recommendation #3a: In regards to the quantitative portion of the Teacher
and Leader Evaluation System, the TLE Commission recommends using a Value Added
Model in calculating the thirty-five percentage points attributed to student academic
growth using multiple years of standardized test data for those teachers in grades and
subjects for which multiple years of standardized test data exist.
Permanent Recommendation #3b: In regards to the quantitative portion of the Teacher
and Leader Evaluation System, the TLE Commission recommends using a Value Added
Model in calculating the thirty-five percentage points attributed to student academic
growth using multiple years of standardized test data for those leaders of buildings
containing grades and subjects for which multiple years of standardized test data exist.
Permanent Recommendation #4: In addressing those teachers and leaders in grades and
subjects for which there is no state-mandated testing measure to create a quantitative
assessment, the TLE Commission recommends conducting more research to determine
the appropriate measure(s) of student achievement taking into account a combination of
multiple measures and including teacher, leader, and specialist input.
Permanent Recommendation #5: In regards to the fifteen percentage points based on
other academic measures, the TLE Commission recommends conducting further study of
best practices across the country as well as inviting Oklahoma educators to provide input
to develop a list of appropriate measures for Oklahoma.
ATTACHEMENT B
Criteria Checklists for All Frameworks Reviewed
Teacher Frameworks
Danielson’s Framework for Teaching
Marzano’s Causal Teacher Evaluation Model
McREL’s Teacher Evaluation System
Tulsa’s TLE Observation and Evaluation System
Leader Frameworks
Marzano’s Leadership Evaluation System
McREL’s Principal Evaluation System
Reeves’s Leadership Performance Matrix
Vanderbilt’s Assessment for Leadership in Education
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the
Oklahoma State Department of Education and submitted to the Commission for consideration.
Selection Description
Does not meet the criteria From the materials provided, does not appear to meet the criteria
Meets the criteria From the materials provided, appears to meet the criteria
Exceeds the criteria From the materials provided, appears to exceed the criteria
Framework: Danielson’s Framework for Teaching (ASCD Teacher Effectiveness Suite)
Required
by Statute
Teacher Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Includes a Five-Tier Rating System (qualitative
measures)
1) Superior
2) Highly Effective
3) Effective
4) Needs Improvement
5) Ineffective
Though none was provided, the
framework uses averaging to calculate a
score which can be translated into the
five-tier rating system.
Annual evaluations that provide feedback to
improve student learning and outcomes
The model includes individualized
professional development plans for
teachers to work on their practice.
Comprehensive remediation plans and
instructional coaching for all teachers rated as
Needs Improvement or Ineffective
The model may be used for further
supports for struggling teachers.
Be evidence-based (e.g. research and field
proven)
The model is widely adopted including
variations in Oklahoma.
Include observable and measurable
characteristics of personnel and classroom
practices (rubrics and evidences for each rubric)
Rubrics are included for each element.
An evidence-based tool that will include
characteristics of personnel and classroom
practices that are correlated to student
performance success
Be based on research-based national best
practices and methodology (contemporary
research and practices of expertise
development and strategies/behaviors for
raising student achievement)
The framework was developed in the
1990s and revised periodically. It was
developed upon a review of the research
but does not reflect the most
contemporary research on strategies,
lesson segments, and deliberate
practice.
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
Required
by Statute
Teacher Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Must contain minimally:
a) Organizational and classroom
management skills
b) Ability to provide effective instruction
c) Focus on continuous improvement and
professional growth
d) Interpersonal skills
e) Leadership skills
The framework exceeds the minimum
areas in its 76 elements.
Accounts for years of service since teacher
expertise develops over time
No tools are provided in the model to
account for years of service.
Granular enough with “thin slices” of
instruction to support deliberate practice
because teachers develop expertise through
engaging in focused practice with focused
feedback
Of the 76 elements, 33 are observable to
instruction. Danielson’s framework is
broader to the behaviors and lacks the
specificity of Marzano to research-based
strategies.
Identifies the instructional context or lesson
type or segment for when it is instructionally
appropriate to see certain research-based
strategies
No documentation in the model
identifies when it is appropriate to see
certain research-based strategies.
Includes scales or rubrics to identify the level of
implementation for the strategies
All elements have a rubric.
Reflects the elements for a research-based
common language of instruction that
accurately reflects the complexity of teaching
The framework reflects the complexity of
teaching across its 76 elements.
Clearly defines and articulates teacher and
student evidences for each rubric
The framework lacks tools for specific
teacher and student evidences but does
include critical attributes to help provide
clarity.
Be correlated to student performance success
(validation studies and causal links studies for
instructional strategies/behaviors)
The studies conducted this far on the
Danielson framework only show a slight
increase in student learning. There are
no experimental and control studies to
verify the specific elements raise student
achievement.
Validation studies (Specific studies on the
model/framework to verify its ability to identify
levels of teaching performance correlated to
student achievement results)
Validation studies do exist for the
framework.
Research studies verifying the specific
classroom practices in the rubrics have a
“causal link” to raising student achievement
No experimental and control studies
have been identified at the element level
for the specific strategies represented in
the framework.
Weights/emphasizes classroom instruction
Of the 76 elements in the framework, 33
or 46% are observable to classroom
practice. To make this a sufficient
weight, additional weighting will need to
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
be placed on Domains 2 and 3.
Depth of supports provided for the framework
(qualified capacity to support statewide efforts)
Due to its wide adoption, there is
sufficient capacity and depth of services
to support Oklahoma districts.
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the
Oklahoma State Department of Education and submitted to the Commission for consideration.
Selection Description
Does not meet the criteria From the materials provided, does not appear to meet the criteria
Meets the criteria From the materials provided, appears to meet the criteria
Exceeds the criteria From the materials provided, appears to exceed the criteria
Framework: Marzano Causal Teacher Evaluation Model
Required
by Statute
Teacher Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Includes a Five-Tier Rating System (qualitative
measures)
1) Superior
2) Highly Effective
3) Effective
4) Needs Improvement
5) Ineffective
Presentation included calculation to
generate Oklahoma’s five-tier rating
system and is currently in use within
Oklahoma City as presented by Dr. Brian
Staples
Annual evaluations that provide feedback to
improve student learning and outcomes
In addition to the causal link research,
the model also contains reflection
questions, video examples, teacher and
student evidences, etc. to provide
teachers with annual evaluations that
support their growth and development
to raise student learning and outcomes.
Comprehensive remediation plans and
instructional coaching for all teachers rated as
Needs Improvement or Ineffective
All teachers are required to develop
Professional Growth Plans and engage in
deliberate practice in order to document
improvements in their teaching.
Processes include supports and tools for
instructional coaches to engage with
struggling teachers and supervision
models for more support and
observational feedback for struggling
teachers.
Be evidence-based (e.g. research and field
proven)
The Art and Science of Teaching, upon
which the evaluation model was
developed is widely used. The evaluation
model is also being used in large scale
including a pilot in Oklahoma City and
the state of Florida.
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
Required
by Statute
Teacher Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Include observable and measurable
characteristics of personnel and classroom
practices (rubrics and evidences for each
rubric)
The model contains rubrics, teacher and
student evidence for each rubric, and
coaching supports for each rubric.
An evidence-based tool that will include
characteristics of personnel and classroom
practices that are correlated to student
performance success
Be based on research-based national best
practices and methodology (contemporary
research and practices of expertise
development and strategies/behaviors for
raising student achievement)
The model draws upon 35 years of
research for what works for raising
student achievement. The model also
cites contemporary research for the
development of expertise and
incorporates national best practices for
accounting for years of service and
growth over time.
Must contain minimally:
a) Organizational and classroom
management skills
b) Ability to provide effective instruction
c) Focus on continuous improvement
and professional growth
d) Interpersonal skills
e) Leadership skills
The model exceeds the minimum
requirements. Domains 3-4 incorporate
interpersonal skills and leadership skills.
Accounts for years of service since teacher
expertise develops over time
Model includes for categories of teachers
accounting for years of service with
recommendations for 0-3, 3-9, and 10
plus years of service.
Granular enough with “thin slices” of
instruction to support deliberate practice
because teachers develop expertise through
engaging in focused practice with focused
feedback
Classroom strategies and behaviors
(Domain 1) includes 41 elements that are
granular enough to support deliberate
practice.
Identifies the instructional context or lesson
type or segment for when it is instructionally
appropriate to see certain research-based
strategies
41 elements in Domain 1: Classroom
Strategies and Behaviors are classified
into lesson types or segments for both
teachers and observers to identify when
it is appropriate to see certain strategies.
Includes scales or rubrics to identify the level of
implementation for the strategies
All elements in the framework include 5
point scales/rubrics identifying levels of
implementation of the strategies.
Reflects the elements for a research-based
common language of instruction that
accurately reflects the complexity of teaching
The complexity of teaching is
represented in the model and the model
reflects a substantial research base
drawn from 35 years of research and
meta-analysis.
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
Required
by Statute
Teacher Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Clearly defines and articulates teacher and
student evidences for each rubric
Every scale/rubric includes examples of
teacher and student evidence.
Be correlated to student performance success
(validation studies and causal links studies for
instructional strategies/behaviors)
The model has substantial research
documentation for raising student
achievement through the 41 classroom
strategies and behaviors in Domain 1.
Validation studies (Specific studies on the
model/framework to verify its ability to identify
levels of teaching performance correlated to
student achievement results)
Only model where validation studies
conducted within Oklahoma were cited
for the model
Research studies verifying the specific
classroom practices in the rubrics have a
“causal link” to raising student achievement
Over 300 individual experimental and
control studies have been completed
identifying the causal link for use of
strategies cited in the model to increases
in student learning.
Weights/emphasizes classroom instruction
Of the 60 total elements in the Marzano
model, 41 or 68% represent classroom
instruction.
Depth of supports provided for the framework
(qualified capacity to support statewide efforts)
Given the statewide implementations
currently underway with the model,
there is both capacity to support
Oklahoma districts and a depth of
supports from trainings or certification
for evaluators for accuracy for
observations.
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the
Oklahoma State Department of Education and submitted to the Commission for consideration.
Selection Description
Does not meet the criteria From the materials provided, does not appear to meet the criteria
Meets the criteria From the materials provided, appears to meet the criteria
Exceeds the criteria From the materials provided, appears to exceed the criteria
Teacher Framework: McREL’s Teacher Evaluation System
Required
by Statute
Teacher Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Includes a Five-Tier Rating System (qualitative
measures)
1) Superior
2) Highly Effective
3) Effective
4) Needs Improvement
5) Ineffective
Although none was provided the
framework could translate a score into
the five tiers
Annual evaluations that provide feedback to
improve student learning and outcomes
Comprehensive remediation plans and
instructional coaching for all teachers rated as
Needs Improvement or Ineffective
Appears to have a feedback cycle and
professional development plan process
that may be adapted to this requirement
Be evidence-based (e.g. research and field
proven)
Research provided
Include observable and measurable
characteristics of personnel and classroom
practices (rubrics and evidences for each rubric)
An evidence-based tool that will include
characteristics of personnel and classroom
practices that are correlated to student
performance success
Be based on research-based national best
practices and methodology (contemporary
research and practices of expertise development
and strategies/behaviors for raising student
achievement)
Minimally meets as the framework is
very broad (25 elements), which is less
than half of Danielson or Marzano
frameworks
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
Required
by Statute
Teacher Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Must contain minimally:
a) Organizational and classroom
management skills
b) Ability to provide effective instruction
c) Focus on continuous improvement and
professional growth
d) Interpersonal skills
e) Leadership skills
Minimally meets as the framework is
very broad (25 elements), which is less
than half of Danielson or Marzano
frameworks
Accounts for years of service since teacher
expertise develops over time
Granular enough with “thin slices” of instruction
to support deliberate practice because teachers
develop expertise through engaging in focused
practice with focused feedback
The framework is intentionally designed
broadly and lacks specificity and clarity
around use of research-based strategies
Identifies the instructional context or lesson type
or segment for when it is instructionally
appropriate to see certain research-based
strategies
Includes scales or rubrics to identify the level of
implementation for the strategies
Reflects the elements for a research-based
common language of instruction that accurately
reflects the complexity of teaching
Minimally meets as the framework is
very broad (25 elements), which is less
than half of Danielson or Marzano
frameworks
Clearly defines and articulates teacher and
student evidences for each rubric
Uses a progressive checklist approach
with minimal clarity at each level
Be correlated to student performance success
(validation studies and causal links studies for
instructional strategies/behaviors)
None were provided
Validation studies (Specific studies on the
model/framework to verify its ability to identify
levels of teaching performance correlated to
student achievement results)
None were provided
Research studies verifying the specific classroom
practices in the rubrics have a “causal link” to
raising student achievement
None were provided
Weights/emphasizes classroom instruction
No weighting recommendations were
provided, but appears weighting could
be adjusted
Depth of supports provided for the framework
(qualified capacity to support statewide efforts)
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the
Oklahoma State Department of Education and submitted to the Commission for consideration.
Selection Description
Does not meet the criteria From the materials provided, does not appear to meet the criteria
Meets the criteria From the materials provided, appears to meet the criteria
Exceeds the criteria From the materials provided, appears to exceed the criteria
Teacher Framework: Tulsa Public Schools Teacher Leader Effectiveness Observation and Evaluation System
Required
by Statute
Teacher Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Includes a Five-Tier Rating System (qualitative
measures)
1) Superior
2) Highly Effective
3) Effective
4) Needs Improvement
5) Ineffective
Annual evaluations that provide feedback to
improve student learning and outcomes
Comprehensive remediation plans and
instructional coaching for all teachers rated as
Needs Improvement or Ineffective
Be evidence-based (e.g. research and field
proven)
The evidence base and field experience
is one district
Include observable and measurable
characteristics of personnel and classroom
practices (rubrics and evidences for each rubric)
���
An evidence-based tool that will include
characteristics of personnel and classroom
practices that are correlated to student
performance success
Be based on research-based national best
practices and methodology (contemporary
research and practices of expertise development
and strategies/behaviors for raising student
achievement)
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
Required
by Statute
Teacher Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Must contain minimally:
a) Organizational and classroom
management skills
b) Ability to provide effective instruction
c) Focus on continuous improvement and
professional growth
d) Interpersonal skills
e) Leadership skills
Accounts for years of service since teacher
expertise develops over time
Granular enough with “thin slices” of instruction
to support deliberate practice because teachers
develop expertise through engaging in focused
practice with focused feedback
Framework is the most broad of all with
20 elements
Identifies the instructional context or lesson type
or segment for when it is instructionally
appropriate to see certain research-based
strategies
Includes scales or rubrics to identify the level of
implementation for the strategies
Reflects the elements for a research-based
common language of instruction that accurately
reflects the complexity of teaching
Clearly defines and articulates teacher and
student evidences for each rubric
Be correlated to student performance success
(validation studies and causal links studies for
instructional strategies/behaviors)
There is some encouraging evidence
emerging in the district but not yet a
research study
Validation studies (Specific studies on the
model/framework to verify its ability to identify
levels of teaching performance correlated to
student achievement results)
This question was raised during the
presentation and the framework has not
yet been validated
Research studies verifying the specific classroom
practices in the rubrics have a “causal link” to
raising student achievement
No research studies were submitted
Weights/emphasizes classroom instruction
Depth of supports provided for the framework
(qualified capacity to support statewide efforts)
Given only one district is currently
implementing, there would likely be
capacity issues trying to scale it
statewide
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the
Oklahoma State Department of Education and submitted to the Commission for consideration.
Selection Description
Does not meet the criteria From the materials provided, does not appear to meet the criteria
Meets the criteria From the materials provided, appears to meet the criteria
Exceeds the criteria From the materials provided, appears to exceed the criteria
Leadership Framework: Marzano Leadership Evaluation System
Required
by Statute
Leader Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Includes a Five-Tier Rating System (qualitative
measures)
1) Superior
2) Highly Effective
3) Effective
4) Needs Improvement
5) Ineffective
Currently being developed for Oklahoma
City
Annual evaluations that provide feedback to
improve student learning and outcomes
Be evidence-based (e.g. research and field
proven)
Just starting pilot
Include observable and measurable
characteristics of personnel and site
management practices (rubrics and evidences
for each rubric)
Be correlated to student performance success Just starting pilot
Be based on research-based national best
practices and methodology
Include the following six domains: organizational
and school management skills, including
retention and development of effective teachers
and dismissal of ineffective teachers;
instructional leadership; professional growth
and responsibility; interpersonal skills;
leadership skills; and stakeholder perceptions.
Includes scales or rubrics to identify the level of
implementation for the leadership behaviors
Clearly defines and articulates teacher and
student evidences for each rubric
Currently being developed
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
Required
by Statute
Leader Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Validation studies (Specific studies on the
model/framework to verify its ability to identify
levels of leadership performance correlated to
student achievement results)
Based upon numerous research studies
Weights/emphasizes teacher growth and
development
Depth of supports provided for the framework
(qualified capacity to support statewide efforts)
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the
Oklahoma State Department of Education and submitted to the Commission for consideration.
Selection Description
Does not meet the criteria From the materials provided, does not appear to meet the criteria
Meets the criteria From the materials provided, appears to meet the criteria
Exceeds the criteria From the materials provided, appears to exceed the criteria
Leadership Framework: McREL’s Principal Evaluation System
Required
by Statute
Leader Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Includes a Five-Tier Rating System (qualitative
measures)
1) Superior
2) Highly Effective
3) Effective
4) Needs Improvement
5) Ineffective
Although none was provided the
framework could translate a score into
the five tiers
Annual evaluations that provide feedback to
improve student learning and outcomes
Be evidence-based (e.g. research and field
proven)
Include observable and measurable
characteristics of personnel and site
management practices (rubrics and evidences
for each rubric)
Broad framework with 21 elements
compared to 38 for Reeves framework
Be correlated to student performance success Based upon research from which the
framework is drawn
Be based on research-based national best
practices and methodology
Include the following six domains: organizational
and school management skills, including
retention and development of effective teachers
and dismissal of ineffective teachers;
instructional leadership; professional growth
and responsibility; interpersonal skills;
leadership skills; and stakeholder perceptions.
Minimally addresses
Includes scales or rubrics to identify the level of
implementation for the leadership behaviors
Clearly defines and articulates teacher and
student evidences for each rubric
Uses a progressive checklist approach
with minimal evidences
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
Required
by Statute
Leader Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Validation studies (Specific studies on the
model/framework to verify its ability to identify
levels of leadership performance correlated to
student achievement results)
The framework is drawn from many
studies
Weights/emphasizes teacher growth and
development
Although weighting could be applied to
emphasize teacher growth, none was
provided
Depth of supports provided for the framework
(qualified capacity to support statewide efforts)
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the
Oklahoma State Department of Education and submitted to the Commission for consideration.
Selection Description
Does not meet the criteria From the materials provided, does not appear to meet the criteria
Meets the criteria From the materials provided, appears to meet the criteria
Exceeds the criteria From the materials provided, appears to exceed the criteria
Framework: Reeves’ Leadership Performance Matrix
Required
by Statute
Leader Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Includes a Five-Tier Rating System (qualitative
measures)
1) Superior
2) Highly Effective
3) Effective
4) Needs Improvement
5) Ineffective
A clear translation from the current
four-tier rating system to a five-tier
system can be easily performed.
Annual evaluations that provide feedback to
improve student learning and outcomes
The overall purpose of the Reeves’
MLA System is to improve leadership
and provide a clear path for each
element of performance. The MLA
system is used as a learning system
directly correlated to teacher action
and student learning.
Be evidence-based (e.g. research and field
proven)
The Reeves’ MLA System is widely
used by individual school districts as
well as being competitively selected
as the statewide model in both
Florida and New York.
Include observable and measurable
characteristics of personnel and site
management practices (rubrics and evidences
for each rubric)
Same response as for item #2 above.
Be correlated to student performance success
The MLA system provides not just a
rearview look but rather a
windshield approach linking the
student data, teacher data, and
leader data in real time.
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
Required
by Statute
Leader Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Be based on research-based national best
practices and methodology
Consists of leadership best practices
that are well documented in two of
the most recent studies on
leadership (Wahlstrom, Louis,
Leithwood, & Anderson, 2010;
Hattie, 2009)
Include the following six domains: organizational
and school management skills, including
retention and development of effective teachers
and dismissal of ineffective teachers;
instructional leadership; professional growth
and responsibility; interpersonal skills;
leadership skills; and stakeholder perceptions.
Each of the state identified
leadership domains are clearly
reflected in the Reeves’ MLA
Framework as well as additional
domains that are linked to effective
leadership and increased student
achievement.
Includes scales or rubrics to identify the level of
implementation for the leadership behaviors
Ten leadership domains with 38
subdomains are identified. A
continuum of performance is clearly
described for each domain.
Clearly defines and articulates teacher and
student evidences for each rubric
Each of the continuum descriptions
within the ten domains clearly
articulates and defines the required
teacher and student evidence. It is
an evidenced-based rubric, not
opinion.
Validation studies (Specific studies on the
model/framework to verify its ability to identify
levels of leadership performance correlated to
student achievement results)
The MLA is validated by multiple
studies and methodologies i.e.
Marzano, Waters, McNulty, Hattie,
Reeves.
Weights/emphasizes teacher growth and
development
The Reeves’ MLA Framework
emphasizes teacher growth and
development (Domain 6.0 Faculty
Development) in addition to three
other key leadership practices (i.e.,
Student Achievement, Leadership
Development, and Personal and
Professional Learning) all of which
enhance teacher growth and
development.
Depth of supports provided for the framework
(qualified capacity to support statewide efforts)
Utilizes the findings from The
Center’s five-year Implementation
Audit Study involving over 2,000
schools across the United States and
Canada and over 1.5 million students
that assist organizations in the deep
implementation of initiatives.
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
The review following the criteria set forth by the Teacher and Leader Effectiveness Commission will be conducted by the
Oklahoma State Department of Education and submitted to the Commission for consideration.
Selection Description
Does not meet the criteria From the materials provided, does not appear to meet the criteria
Meets the criteria From the materials provided, appears to meet the criteria
Exceeds the criteria From the materials provided, appears to exceed the criteria
Leadership Framework: VAL-ED (Vanderbilt Assessment of Leadership in Education)
Required
by Statute
Leader Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Includes a Five-Tier Rating System (qualitative
measures)
1) Superior
2) Highly Effective
3) Effective
4) Needs Improvement
5) Ineffective
Could be part of a leadership evaluation
but is not a complete evaluation
framework
Annual evaluations that provide feedback to
improve student learning and outcomes
Is not an evaluation system
Be evidence-based (e.g. research and field
proven)
Include observable and measurable
characteristics of personnel and site
management practices (rubrics and evidences
for each rubric)
Be correlated to student performance success
Be based on research-based national best
practices and methodology
Include the following six domains: organizational
and school management skills, including
retention and development of effective teachers
and dismissal of ineffective teachers;
instructional leadership; professional growth
and responsibility; interpersonal skills;
leadership skills; and stakeholder perceptions.
May be used as part of a leadership
evaluation system but is missing
required components
Includes scales or rubrics to identify the level of
implementation for the leadership behaviors
Clearly defines and articulates teacher and
student evidences for each rubric
Teacher and Leader Effectiveness Commission Teacher
Evaluation Framework/Model Selection Criteria
Required
by Statute
Leader Evaluation Framework/Model
Selection Criteria for the Qualitative
Measures
Does not
meet the
criteria
Meets the
criteria
Exceeds
the criteria
Reviewers’ Notes
Validation studies (Specific studies on the
model/framework to verify its ability to identify
levels of leadership performance correlated to
student achievement results)
Weights/emphasizes teacher growth and
development
Depth of supports provided for the framework
(qualified capacity to support statewide efforts)
ATTACHEMENTS C-G
Attachments C-G are the full, proprietary frameworks of Danielson, Marzano, Tulsa, McREL,
and Reeves. These attachments were provided to the Governor, required members of the
Legislature, and the Oklahoma State Board of Education for review and approval purposes only.
Districts will be provided access to the full frameworks during TLE Training.
ATTACHEMENT H
Summary of Public Comment
Attachment H
Summary of Public Comment
As a brief overview, on September 13, 2011, the State Department of Education created a
public survey to gather input regarding the Commission’s first two preliminary
recommendations. To date, this survey remains active; however, the following results reflect
the data from September 13, 2011 through December 2, 2011. Accessibility to this electronic
survey was made available through the Department’s website, and email listservs. In addition,
several organizations throughout the state added links to the survey onto their websites. The
following is a snapshot of some of the more pertinent data generated from this survey.
As illustrated in the pie chart, the vast majority of the 1,158 survey responders were
teachers, totaling 76.5% of the total responders. Building administrators accounted for 9.7% of
the responders and district administrators accounted for 6.4% of the responders. In total,
nearly 93% of the responders to the public comment survey were people who have direct, daily
involvement with education.
Teacher
77%
Building Administrator
10%
District Administrator
6% Other
7%
Please select the role that best describes you
1,158 Total
Responders
Most responders agreed that each of the three Frameworks should be included as an option for
district selection. When asked which Framework should be selected as the default, the
Marzano’s Framework received the highest approval rating at 22.3%. Tulsa’s Framework
received a 12% approval rating and Danielson’s Framework received a 7.5% approval rating.
Most notably, when asked which Framework should not be included as an option, Tulsa’s
Framework received the highest rating at 41.2%. For this same question, Marzano’s Framework
received a 25.3% rating and Danielson Framework received a 36.9% rating.
55.4
49.2
58.8
25.2
41.3
36.8
22.3
12
7.5
Marzano Tulsa Danielson
Which Teacher Frameworks do you believe
should be included in an approved list for
district selection and which should be named
as the default?
Include as an option Do not include as an option Name as a default
It is important to note that when this question was originally posed to the public, the Marzano’s
Leadership Evaluation System was presented as an option. However, as of the Commission’s December
5, 2011 meeting, the Marzano Leadership System was not fully developed. As a result, the Commission
only considered the McREL and Reeves Frameworks in its selection. Interestingly, Marzano’s Leadership
System received the highest “Name as Default” rating at 21.2%.
53.9 54.3 53.9
38.8 39.5
26.6
8.1
6.4
21.2
McREL Reeves' Marzano (Not considered)
Please indicate which Leader frameworks you
believe should be included in an approved list for
district selection and which one should be named
as the default
Include as an option Do not inlcude as an option Name as default
After the Commission made preliminary recommendations 3-5 at its November 7, 2011 meeting, the
public was asked to respond, via email, to the newest recommendations as well as provide overall input
regarding the TLE process. Twenty-eight emails and letters were received, the majority of which were
generic comments and concerns regarding the TLE process. Two emails were specifically in favor of
adopting the Marzano Framework. There were no emails received that favored either the Tulsa or the
Danielson Framework. Below is a direct copy of one of the emails in support of the Marzano
Framework.
After much thought about which Teacher Leader Effectiveness Framework would make the greatest
impact on Teaching and Learning in my district, I have come full circle on my preference! I first
thought the Tulsa model would be good because it was the least amount of change, and thus would
be easier to "sell" to anyone who is reluctant about change. I even sent Comments on TLE earlier that
leaned in favor of the Tulsa model.
After studying Robert Marzano's The Art and Science of Teaching, I now see the impact his framework
could make on instruction, and THAT (improved instruction) is what will make a difference for our
students in Mid-Del. We have caring teachers who prepare and teach well, but many do not employ a
framework to design their instructional lessons and to organize their instructional strategies. That is
the strength of Marzano's Framework! To further benefit and add to the professional development of
educators using the protocol, Marzano's online observation tool contains video clips that relate
directly to elements/ indicators in the observation protocol. So when I identify an area that needs to
be strengthened in a teacher's toolkit of procedures and strategies, I can simply click to direct the
teacher to a master teacher modeling that particular strategy.
In Marzano's work, teaching |
Date created | 2012-01-13 |
Date modified | 2012-01-13 |
Tags
Add tags for Teacher Leader Effectiveness Evaluation System