Evaluating a leadership training programme
Kaleidoscope carried out a midline evaluation of the Versus Arthritis MSK Champions programme. We assessed the programme design, the building of leadership and influencing skills, the growth of professional networks, and the impact of the programme on MSK services across the UK.
The challenge
The MSK Champions programme was designed to bridge the gap between Versus Arthritis and people working on the frontline of musculoskeletal (MSK) care. It aims to upskill and develop local and national leaders in MSK care, and foster a national expert community of multi-disciplinary learners, leaders and influencers of change in services. The ultimate aim is to improve population musculoskeletal health.
The aims and objectives of the programme are to:
- develop local and national leaders and influencers of change in the MSK health context
- cultivate systemic leadership skills for a highly complex environment
- facilitate and enhance leadership, connection and action
- inspire and support local projects
- foster a community that lives and gives.
Our approach
Our collaborative approach to evaluation design began with a design workshop with Versus Arthritis to define the domains of enquiry, draft evaluation questions, and set out the priority audiences and participants for the evaluation.
It was important for this evaluation to assess the validity of the programme design and the evidence base used to formulate the programme concept, as well as to explore evidence of effect in terms of building leadership, fostering professional networks, and delivering changes in MSK care.
Our methods included:
- a background documentation and literature review to assess the structure and process of the original programme design, how this was translated into implementation, and existing evidence of impact at the midpoint of the programme
- a survey of priority key informant groups to gather qualitative data to shape our interview guides and build an understanding of the variety of experiences the Champions had
- in-depth key informant interviews with 32 people from five priority stakeholder groups
- producing six case studies to explore elements of programme design, and Champions’ experiences, successes and challenges in participating in the programme
- social network analyses of six Champions to present visual maps of the professional networks that had been fostered and the communities of practice emerging – these maps helped to describe the geographic range, diversity in professional backgrounds, and the type of support provided by the many stakeholders included in the Champions’ professional networks.
Results
Our work culminated with an interactive digital workshop to transform the evidence into impactful solutions for programme adaptation, strengthening of results, and wider systems change.
Working with Kaleidoscope on our programme evaluation has been a true exercise in partnership – their unique approach to codesign has resulted in a high quality piece of work, using a process that has been both fun and constructively challenging. We have been encouraged to think differently, creatively and have enjoyed working with every member of the team to produce a high quality, credible evaluation.
James Dunphy, Senior Programme Lead ‑ MSK Clinical Champions, Versus Arthritis
