How are we doing?
Self-evaluation and external inspection of services to protect children will focus on the experiences of children and young people and their families and the impact services have on their lives.
In order to know how well we are doing, we need to know:
How well are children and young people protected and their needs met?
To answer this overarching question and have a full understanding of the impact of services to protect children and young people, we need to gather information to answer the following key questions:
Professionals have always reflected on the quality of the services they provide. By working together to evaluate the impact of their services on children, young people and families, professionals can come to a shared view of how well they are doing and how they can make things better.
Making such evaluations is dependent on a shared understanding within and across service providers and stakeholders of what constitutes high quality processes and outcomes. The quality indicators in this document set out to support the development of a shared understanding for all those concerned with delivering or evaluating services to protect children and young people. The indicators are organised round the key questions above and taken together answer the overarching question, How well are children and young people protected and their needs met? Answering this question requires us to summarise the impact of services on children, young people and families and the quality of outcomes for them. To ensure services are maintained and improved, we also need to evaluate the effectiveness of how services are delivered and managed and the effectiveness of planning for improvement.
However, it is not always helpful or manageable to try to evaluate every aspect of practice in depth at the same time. We can answer How are we doing? by asking questions we choose such as:
We could also focus on specific themes by asking such questions as:
By encouraging professionals to structure their discussion and reflection on the work they are doing, the quality indicator framework can support improvement at the level of individuals and very small teams as well as at a range of operational and managerial levels.
Self-evaluation involves:
Practitioners within a particular service or in multi-disciplinary groups such as the local CPC can form a broad view by scanning across all the quality indicators. Using evidence which has been gathered in the normal course of work, they can identify broad strengths and weaknesses. This will give an immediate impression of areas of major strength, or areas where more improvement is required.
In order to have a greater understanding of the effectiveness of particular aspects of practice, managers and practitioners can also take a closer look at aspects of the service. The stimulus to take a closer look could derive from a range of issues. The following are some examples:
Links between self-evaluation and inspection
It is always useful to have an external measure of how we are doing. HMIE, while carrying out school inspections, evaluates the quality of care and protection within schools. The Care Commission inspects a range of services for children, including residential care, early education and childcare. Within these inspections the Care Standard related to Child Protection is considered. Information from these inspections can form part of the evidence for local self-evaluation. The Services for Children Unit (SFCU) within HMIE has a responsibility to evaluate how well children and young people are protected and their needs met across all areas of Scotland. Inspection by HMIE (SFCU) covers the same ground as local self-evaluation. Inspectors will gather evidence, make evaluations using the quality indicators and answer the key questions outlined above. They will write a report which will address the questions outlined, based on evaluations of what they see and summarise the key strengths and areas requiring further development in an area. Any self-evaluation which has recently been carried out locally will contribute to the inspection process.
How do we know?
We have a number of sources of evidence which can tell us how well we are protecting children and young people and meeting their needs, or to answer questions we choose as a starting point for self-evaluation.
Performance outcomes
We can see how we are doing by comparing our effectiveness with the vision, values and aims expressed through the Integrated Children’s Services Plan and objectives set out in the plan. In addition, the CPC will have a set of objectives for improving and maintaining the standards of service they wish children, young people and families to experience in their area. Individual service development or improvement plans such as the local health improvement plan, the council’s community plan and the local policing plan may also refer to specific objectives related to protecting children and young people.
Some of these objectives may include such things as:
Information on progress towards objectives such as these can be collated as part of the self-evaluation process and will give a clear indication of progress being made. Any difficulties in achieving targets set can be followed up by taking a closer look to find out what the barriers to achieving the objectives are.
Other objectives will refer to improving the quality of the services and will not lend themselves easily to the collation of performance data. The quality indicators will help us evaluate progress in these areas.
Information gathered from stakeholders
This would be information primarily gathered from children, young people and their families along with any safe-guarders or advocates on behalf of children and young people. It may also include information from surveys of the general public or voluntary organisations working in the area.
Information can be collected systematically when children, young people and families are using a service, or when they end their involvement with a service. We may also, from time to time, specifically survey service users or meet groups of service users to find out their views.
Whatever approach we decide to use, gathering information from users is an essential part of the self-evaluation process. Without it, it is very difficult to understand the impact of our work on children, young people and families and to know if we are doing the right things. It is almost impossible to answer the key questions effectively without including the views of service users.
Using the quality indicators in Part 2
Quality indicators help us evaluate the quality of service experienced by children, young people and their families in our area.
The Framework for Standards and the Children’s Charter describe the expectations of the service each child or young person should receive and the practice and processes required to meet those expectations. The quality indicators will provide valuable tools in deciding whether these expectations are being met.
The quality indicators are organised around the set of key questions:
Each quality indicator contains illustrations which describe practice which is very good and practice which is weak. The very good illustrations are largely based on the advice in the Framework for Standards. Illustrations are intended to provide examples, not to be fully comprehensive. The illustrations help us make a judgement about the strengths and weaknesses in that aspect of practice. The process of discussing our practice and relating it to the illustrations is key when using the indicators for self-evaluation. Identifying and describing the strengths and weaknesses will help us focus on what we need to do.
The indicators can also be used to help us to form a view on the level of effectiveness in a particular aspect of practice. During inspection, HMIE (SFCU) evaluate quality indicators against six levels of effectiveness. Awarding levels is a professional skill rather than a technical process and there are many ways in which provision can merit a particular evaluation. However, the following describe key characteristics at each of six levels.
Excellent
An evaluation of excellent will apply to performance which is a model of its type. The outcomes for children, young people and their families along with their experience of services will be of a very high quality. An evaluation of excellent will represent an outstanding standard of performance, which will exemplify very best practice and will be worth disseminating beyond the service or area. It will imply these very high levels of performance are sustainable and will be maintained.
Very Good
An evaluation of very good will apply to performance characterised by major strengths. There will be very few areas for improvement and any that do exist will not significantly diminish the experience of children, young people and their families. While an evaluation of very good will represent a high standard of performance, it is a standard that should be achievable by all. It will imply that it is fully appropriate to continue the delivery of service without significant adjustment. However, there will be an expectation that professionals will take opportunities to improve and strive to raise performance to excellent.
Good
An evaluation of good will apply to performance characterised by important strengths, which taken together clearly outweigh any areas for improvement. An evaluation of good will represent a standard of performance in which the strengths have a significant positive impact. However, the quality of outcomes and experiences of children, young people and their families will be diminished in some way by aspects where improvement is required. It will imply that the services should seek to improve further the areas of important strength, but take action to address the areas for improvement.
Adequate
An evaluation of adequate will apply to performance characterised by strengths, which just outweigh weaknesses. An evaluation of adequate will indicate that children, young people and their families have access to a basic level of service. It represents a standard where the strengths have a positive impact on the experiences of children, young people and their families. However, while the weaknesses will not be important enough to have a substantially adverse impact, they will constrain the overall quality of outcomes and experiences. It will imply that professionals should take action to address areas of weakness while building on its strengths.
Weak
An evaluation of weak will apply to performance, which has some strengths, but where there will be important weaknesses. In general, an evaluation of weak may be arrived at in a number of circumstances. While there may be some strengths, the important weaknesses, either individually or collectively, are sufficient to diminish the experiences of children young people and their families in substantial ways. It may imply that some children and young people may be left at risk or their needs not met unless action is taken. It will imply the need for structured and planned action on the part of the agencies involved.
Unsatisfactory
An evaluation of unsatisfactory will apply when there are major weaknesses in performance in critical aspects requiring immediate remedial action. The outcomes and experiences of children, young people and their families will be at risk in significant respects. In almost all cases, professionals responsible for provision evaluated as unsatisfactory will require support from senior managers in planning and carrying out the necessary actions to effect improvement. This may involve working alongside other staff or agencies. Urgent action will be required to ensure that children and young people are protected and their needs met.
The illustrations provided in the indicators describe two of these levels, namely, very good and weak. We can identify whether our practice fits best with one of these levels. If it has several of the features of very good practice and some weaknesses, provision is likely to be good.
Evaluations against the quality indicators are not an end in themselves. They indicate aspects of good or weak practice, which need to be looked at more closely. Evaluations should not be aggregated across different indicators or turned into percentages. In such cases, percentages can be at best meaningless and at worst deceptive. We are not trying to add up evaluations and give our area a "score" or a clean bill of health.
Although all the areas of practice described in the indicators are important, there are some that are more crucial than others. For example, finding that our practice in risk assessment or meeting children’s and young people’s needs is weak may alert us to possible very serious consequences for children, young people and families.
It can be very useful to look at patterns of evaluations across particular indicators, particularly if we are using the quality indicator to evaluate across services. This can help identify whether a particular strength or weakness is within one service, or one locality or replicated across the area. This can help focus on the underlying causes of weaknesses and identify appropriate development priorities, or resource allocations. Over time, trends in evaluations can indicate whether the implementation of particular processes or new guidance has brought about the improvements sought.
What are we going to do now?
At whatever level we are using the quality indicators to look at our practice, we should now be in a position to plan for improvement or report on how well we are doing. It is important when undertaking any form of self-evaluation to keep focused on the end purpose, which is to make things better for vulnerable children and young people and to reduce the possibility of them experiencing harm. The answer to What are we going to do now? must therefore always be action which will make a difference for children, young people and their families.
Reporting on self-evaluation
Whether and how the outcomes of self-evaluation are reported or written depends on the purpose and the level at which the work was done. The purpose of reporting is primarily to help decision-making about how to make improvements, but in some cases may also be necessary to inform stakeholders, such as children, young people and families, the CPC, Children’s Panel Advisory Committee, the public, elected members of councils, and police and health boards.
The following types of reports may be the outcome of a self-evaluation activity.
The priorities identified are then fed into the Integrated Children’s Services Plan.
Whoever the report is addressed to it should:
Improving services
As protecting children and young people and meeting their needs is "everyone’s job"1, developments to make things better for children and young people can involve a wide range of professionals in a range of organisations, including voluntary organisations. It is important therefore that the outcomes of self-evaluation and plans for improvement feed into the appropriate planning structures and are shared across services. Depending on the work undertaken the outcomes may result in priorities being included in a number of planning structures. The following list gives some examples.
Whatever the planning structure(s) for taking forward improvement, it will help if we select a manageable number of priorities for which we can identify specific, achievable targets.
There may be occasions when, for some aspects, the answer to the question What are we are going to do now? may be "nothing at present". Even where we can see how improvements can be made across a number of aspects, we may wish to focus on those of greater concern. It will make more impact on services if a manageable number of priorities are taken forward, with a notable impact on the experiences of children, young people and their families. In some cases small changes in practice identified by a group of practitioners in an area can have a significant positive impact. It is not always necessary for self-evaluation to result in major changes or reviews of practice.
1 See It’s everyone’s job to make sure I’m alright, (Scottish Executive 2002)