You are in: Wiki-Summaries >> Monitor,Report, Evaluate Improve

Monitor, Report, Evaluate to Improve Programs & Approaches (MREI) - Introduction

This page introduces the topics covered in this sub-section of wiki-summaries on monitoring, reporting, evaluating and improving (MREI) programs and approaches. Click on any of the items listed in the navigation bar on the right hand side of this page to access the various summaries. These topics have been identified and are being developed through the previous and current activities of the International Discussion Group on MREI systems.

The wiki-summaries on each topic will eventually include Glossary Terms (1-3 paragraphs), Encyclopedia Entries (Two pages), Handbook Sections (Book Chapters) and Bibliographies/Toolboxes (Extensive lists updated periodically) as well as Additional Resources.   

We welcome your comments and suggestions which you can post on the pages linked to this introduction or send them directly to info@internationalschoolhealth.org

Key Messages/Lessons Learned:

Here are some of the key messages or things that we have learned thus far in these discussions:

• The topics related to MREI systems are extensive and will likely seem extremely challenging to even consider. This is obviously because of the limited time and funding allocated to building adequate MREI systems. However, this list of topics may also seem overwhelming because we usually are concerned only with a limited number of issues and programs. Consequently, truly monitoring if and how schools, agencies and ministries are addressing the needs of the whole child can seem overwhelming.

• We often try to use monitoring and reporting activities to garner attention and support for the issues and programs which we are already addressing. And, policy-makers and administrators are often reluctant to ask questions about issues they already know are not being addressed. As a consequence, the MREI activities in many countries and states are characterized by disjointed, narrow, one time or short-lived studies rather than coherent, regular monitoring and reporting.
• Further, we often see advocacy organizations and UN agencies using monitoring, reporting and evaluating as a substitute for inspiring leadership and good management. The reports and monitoring activities are often distorted to identify "what is going wrong" rather than enabling the employees, teams in and across their systems to identify and implement improvements in their practices, policies and programs. We see one sector anxious to report on the deficient actions of the other systems. We see constant competition to have "our" favourite policies and programs monitored and reported.
• When working across several systems, each of which are comprised of professionals often working in loosely coupled, complex, bureaucracies which are designed to be stable and whose concerns, boundaries and timelines for change will need to be negotiated at every level, these narrowly conceived and competitive MREI activities are doomed to be ineffective at best and counter productive at their worst.
• This collection of topics has one other premise related to school-based and school-linked beyond the need to address the needs of the whole child. That premise is that we cannot transfer the burden of systems change and management to the front-line workers and individual schools. Schools, local health, social and safety agencies and some workers have some autonomy but their work is funded, governed, directed and guided by their respective agencies and ministries. Consequently, the unit of analysis and action has to be at or begin with the national or state/provincial level.

Despite the complex and contested insights noted above, there are a number of better practices that can be identified and promoted. These include:

• a first step for countries and states wanting to strengthen their MREI efforts should be to collate and review the existing data sources, reporting and surveys within their jurisdictions to identify where thay can be linked and aligned
• the MREI processes should be systematic and ongoing, implemented and maintained across several systems and levels within those systems
• the MREI processes (monitoring, reporting, evaluation and improvement) should be inter-linked and firmly positioned within ministry, agency and school improvement planning processes
• a variety of data sources (administrative, survey, program evaluations, consultations etc.) should be used, collated and shared widely
• specific aspects of health or social development can and should be examined as well as policies and programs to address them. However, this should always be done within whole child and over the life course approaches so that results and improvements are always reported coherently and not in a competitive manner
  
• intrinsic motivation to do well, rather than external authority aimed at avoiding mistakes should underlie the MREI efforts
  
• inter-dependence and shared/mutual accountability across systems, agencies and professionals for progress within school-based and school-linked approaches and programs should be reflected in the design of investigations, reporting of results and dissemination/discussions of subsequent improvements within a shared vision and shared purposes frameworks  
  

Section: Monitor, Report, Evaluate, Improve

For updates and reader comments on this section, go to our Mini-Blog on Monitor-Report-Evaluate-Improve (MREI)

The number of summaries completed or drafted in this section are listed below)

- Overview

• MREI of What: Student Learning? Youth Behaviours? Health & Development? Status of Policies and Programs?
• Evidence, Reports & Data about the Impact of MREI Systems
• Examples & Models of MREI Systems

- Key Definitions/Descriptions

• Monitoring
  - Out of School/Dropout Data
  - Administrative Data (EMIS)
  - Child & Adolescent Health & Development Status
  - Child & Adolescent Attitudes, Behaviours & Development
  - Teacher Norm-based Assessments of Student Development
  - School Physical Conditions & Resources
  - School Psycho-Socail Environment & Supports
  - Student Incidents, Suspensions & Injury Data
  - Student knowledge, skills, learning (Instruction & Extended Education)
  - Health, Development & Employment of Young Adults
  - Teacher Wellness, Work Lives, Supply, Qualifications, Development, Concerns
  - Education Faculty Programs/Capacity
  - Use, Training of Other Professionals
  
• Reporting
  - Different Reporting Formats (Progress Reports, Success Stories, Practice Stories, Case & Cohort Studies, Inventories & Intervention Mapping, RCTs and Reviews)
  - Global, Regional, National Updates
  - School Accreditation/Award Programs
  - Policy/Program Surveys
  - Practitioner, Student,Parent or  Community Satisfaction Surveys
  - Consolidated Reports on Children, Adolescents & Young Adults
• Evaluating/Assessing/Analyzing
  - Scheduled Program Evaluations
  - Planned or Post-Hoc Case Studies
  - School Inspections
  - Access, Reach, Response Times of School-based and School-Linked Services
  - Access, Reach, Breadth of Extended Education Activities
  - Student Knowledge, Skills & Attitudes in Health & Life Skills Education (tests and Surveys)
  - Health, Safety & Environmental Conditions in and Near Schools
• Improving
  - Self-Assessment Tools
  - School Recognition Programs
  - Using Student, Parent, Neighbourhood & Teacher Satisfaction Surveys
  - Using School Improvement Planning
  - Using Joint Sector Reviews
  - Using Health, Social, Other Sector Quality Management Systems

- Key Concepts/Insights/Issues

• Competition among Monitors
• Accountability-Purposes, Politics, People, Shared
• Indicators: Valid, Relevant or Convenient?
• Continuous Improvement
• Logic Models: Linear or Complex?
• Data-driven decision-making
• Intrinsic or Extrinsic Motivators within Systems

- How to Implement, Maintain, Scale Up, Sustain, Build Capacity, Adapt to Different Contexts, Leverage to Support Core Components

• Data Sharing Across & Within Systems
• Coordinated Data Collection & Analysis
• Segmenting & Comparing Data by Country & Community Contexts
• Building MREI Capacity in Countries

How to integrate within Education Systems

• Educator Experiences with High-Stakes, Standardized Testing
• Time Frames for Education Change vs Other Sector Programs
• Teacher, Administrator & Other Adopter Concerns

- Implications of Ecological/ Systems Approach

• Shared MREI in Multiple, Complex, Loosely Coupled Systems
• Within Child/Youth Reporting Systems in other Sectors
• Links with Early Childhood Education
• Links with Post-Secondary Education

- Research Questions/Knowledge Development

• Research Programs/Knowledge Centres
• Impact on Student Learning
• Cost Effectiveness, Cost Benefits

• Research Methods & Data Collection Issues