Monitoring and evaluation
Chapter 4.5
High-quality data enable the most effective interventions to be put in place or expanded. This is as true in the drug-use field as in any other part of health and social care.
High-quality data ensures those who need support get the best possible intervention to meet their needs. It also helps to ensure the efficacy of public spending.
An honest appraisal needs to be made when an intervention is not effective. The ability to learn from experience and change practice accordingly is valuable. Similarly, lessons should be learned and shared from examples of good practice.
National evaluation
Audit Scotland recognised that a number of improvement initiatives have been established in the sector. It reflected, however, that it is “too early to assess their effectiveness”.
Audit Scotland also highlighted that it is still difficult to track spending and how it is being distributed and monitored, and criticised the transparency of the National Mission. This is also true of the approach to evaluation.
What needs to change
It is not clear to us how the National Mission is being evaluated. We do not yet know if evaluation will provide an opportunity to assess the impact of funded initiatives at
pace and make improvements to best target finite resources.
Action 113.
The Scottish Government must publish a detailed evaluation plan for the National Mission as part of the national outcomes framework and strategy.
Service evaluations
Individuals need varying levels of support. No two patient journeys are the same, and positive outcomes may differ for every individual. Pathways must be flexible and
open to change. The level of support needs to vary depending on need and be monitored and evaluated to ensure it is delivering what is required.
What needs to change
At present, it is unclear whether the effectiveness or quality of services is being measured or reported in any way. National monitoring of ADPs is light-touch and does not directly consider individual services. Monitoring and evaluation at local level appears limited and inconsistent.
All services should have a monitoring and evaluation plan in place. Services should evolve based on direct experience of delivering the service and embed a cycle of continuous quality improvement. Relevant stakeholders, including people with lived and living experience, their families and people operating the service, should be included in the ongoing evaluations.
Plans should be based on the national outcomes framework we have called for in Action 108. Plans should clearly demonstrate how services are supporting the outcomes of the National Mission. ADPs should support services in developing these plans and coordinating the sharing of evidence for lessons learned.
As part of the development of inspection for the alcohol and drugs sector, the inspectorate should ensure that appropriate plans are in place for monitoring and evaluation and the learning from quality improvement is being implemented effectively.
Action 114.
All services should develop a monitoring and evaluation plan by the end of the year. The plan should embed a quality improvement approach to ensure the best service for people who use drugs.
Sharing lessons learned
Many of our recommendations and actions support multidisciplinary working and facilitate effective monitoring and evaluation. They include actions on drug-related
death review groups, multidisciplinary teams for developing holistic care plans and service-level evaluations.
What needs to change
It is important that the learning from these forums is captured and fed into the ADP strategic plan to inform chief officers of the changes needed to save lives. This
learning should also regularly be shared between ADPs to encourage continuous improvement nationally.
Action 115.
If not already doing so, ADPs should develop formal mechanisms for capturing lessons learned through service delivery, partnership working, and monitoring and evaluation. They should actively share this learning and
quality improvement activity with other ADPs and the Scottish Government through the existing engagement structure.
Monitoring implementation
It will be crucial to monitor progress towards implementation of our recommendations and actions clearly and transparently.
What needs to change
Monitoring of implementation of the MAT standards provides a useful example of how this can be done. Consideration should be given to whether it can be expanded to monitor the implementation of other interventions. NFO pathways, for example, could be monitored as part of MAT Standard 3.
Live monitoring data and indicators on the services provided in each area should be available. The data should be publicly accessible so people can see if their local area provides the services they need and is delivering on key objectives. This will also support monitoring and the strategic overview of national coverage of different services.
As we outlined in section 4.2.2 (Accountability), chief officers should be held to account by the Scottish Government for the provision of this information in their local area and for delivery against key outcomes.
Action 116.
The Scottish Government and chief officers should ensure that transparent public monitoring information is available for the services delivered in local areas. This should include monitoring the implementation of the Taskforce recommendations and actions and delivery against the outcomes of the national outcomes framework.
Our final recommendations
Our final report was broken down into 4 chapters.
Search our archive
Didn't find what you are looking for? search our archive or view all of our reports and reference documents.