Intelligent Inspection – Towards research-informed practise

Keynote I Wednesday 10:00

Daniel Muijs

Head of research at Ofsted
(Office for Standards in Education, Children’s Services and Skills)

Inspection is a part of many accountability systems across Europe, though the extent to which inspections are high stakes differs (Ehren et al, 2013). However, both the impact of inspection in terms of school improvement and its reliability and validity have been questioned (Altricher & Kemethofer, 2015; Gaertner et al, 2014). In terms of impact, we can draw on a number of useful models, such as Ehren et al (2015) which posit the impact of inspection as not necessarily resulting directly from inspection leading to change in the inspected school, but as much or more in the ways they frame behaviours by setting up parameters which define good practise. In terms of reliability and validity, some inspection systems internationally have attempted to increase reliability by drawing up detailed frameworks and minimising judgement, while others have continued to emphasise the primacy of judgement (Richards, 2001; Dedering, 2015; Earley et al, 2017). While it would seem at face value that the first approach is the one to be preferred if we are looking to develop a reliable system in high stakes contexts, this can easily lead to unintended consequences such as the reification of the framework and a box ticking approach to accountability that can be detrimental to developing a thoughtful and self-improving system, which we would see as a key aim of an intelligent accountability system. Such a system needs to be self-evaluative and corrective, which suggests that research has a key role in its development and maintenance. In this presentation I will describe a conceptual model for an intelligent inspection system, and illustrate the role of research using the examples of two major research projects focussing on key elements of inspection in England, curriculum and classroom observation. We will discuss both how as an inspectorate we can conduct research, and what the issues are, and how that research can then inform practise.

Designing Rigorous Evaluations of Educational Interventions

Keynote II Wednesday 16:00

Jessaca Spybrook, PhD

Western Michigan University

Cluster randomized trials have become increasingly common designs to assess the impact of educational interventions. The process of planning and implementing a rigorous cluster randomized trial involves multiple steps. This talk will focus on three of those steps:
1) strategies for selecting the appropriate design,
2) given the specific design, steps for conducting power calculations to answer the What Works and For Whom or Under What Conditions questions, and
3) steps for increasing the credibility and transparency of the impact study through pre-registration of the design and analysis plan.
I will discuss the design considerations and power calculations using real examples from impact studies of educational interventions. I will demonstrate the features of pre-registration using the Registry of Efficacy and Effectiveness Studies in Education (REES), a registry specifically targeted at impact studies in education. REES was developed with support from the Institute of Education Sciences, the research branch of the U.S. Department of Education, and is set to launch in 2018.

Success for All: Schoolwide Improvement for Disadvantaged Students

Keynote III Thursday 11:30

Robert E. Slavin

Director, Center for Research and Reform in Education
Johns Hopkins School of Education

Success for All is a whole-school approach designed to use all school staff and resources in an integrated way to ensure that all students in elementary and middle schools succeed, especially in reading. Begun in 1987 in Baltimore, it is now used widely in the US and UK, and on a smaller scale in the Netherlands. The main components of the approach include cooperative learning, extensive professional development, frequent assessment, tutoring for struggling students, parent involvement, and a whole-school leadership structure. Numerous evaluations, mostly by third parties, have found positive effects of Success for All on reading. This presentation will explain the development and current elements of Success for All, the research, the dissemination, the way this program leads to sustainable school improvement, and the place of this strategy in the evidence based reform movement in the US and UK.

25 years of school leagues tables, accountability and choice: Lessons from England

Keynote IV Friday 12:00

George Leckie

Centre for Multilevel Modelling
Graduate School of Education
University of Bristol

Education systems around the world increasingly use school performance measures derived from value-added statistical models to estimate school and teacher effects on student test scores and to reward and penalise schools and teachers accordingly. England has a 25-year history in this regard, additionally publishing schools’ performances in high profile ‘league tables’ to facilitate parental choice. In this talk, I will describe the evolution of school performance measures and statistical models brought about by successive English governments. I will then discuss the statistical challenges England has encountered when attempting to estimate school effects, including: measuring their stability and consistency, quantifying and communicating their statistical uncertainty, accounting for student mobility and missing data, and monitoring their influence on narrowing student achievement gaps. I will conclude with thoughts and recommendations for other education systems embarking on data-driven school accountability systems.