[Previous] [Contents] [Next]

Evaluating Inclusiveness - A Guide for Scotland's Colleges

SECTION 2: SELF-EVALUATION AND IMPROVEMENT IN PRACTICE

Colleges routinely carry out self-evaluation using the Quality Framework which is used by HMIE in conducting college reviews on behalf of the Scottish Funding Council. Nine quality elements are used to look at subject areas, and seven quality elements to look at cross-college activity. Each element contains two or more quality indicators and a set of key prompts that suggest areas to look at when evaluating against that indicator.

In Section 3 we take each quality element along with its associated quality indicators and key prompts. We provide an overall statement about what inclusiveness means in each quality element. Then, for each quality indicator (QI) we provide statements, in the central column, of what an inclusive college will be doing against a group of key prompts. These statements show how the QI might be applied inclusively.

Self-evaluation for inclusiveness is the process of looking at these statements and seeking evidence that shows to what extent the college is acting in accordance with them. Some examples are provided of evidence that could be used to support the statements. In practice colleges should be able to supply a wide range of evidence.

Inclusiveness involves constant exploration of the needs and circumstances of potential learners in each college’s area of operation, so the sets of statements in this guide are not definitive. Colleges should treat them as indicative, and be prepared to write and use additional statements which they find helpful.

Examples of how this guide might be used

Colleges are accustomed to using the Quality Framework to evaluate their work at college level and at programme team level. Sometimes they use it comprehensively, to cover all areas of their work. Sometimes they use parts of the framework selectively, to target specific areas of work. In either case, they identify strengths and weaknesses and set action points and targets to address them. This guide can be used in a similarly flexible way.

Example 1

College A is undertaking a review of curriculum in preparation for moving to a new building. Senior managers want to ensure that the curriculum portfolio in each section is inclusive. The Director of Curriculum asks each programme team to evaluate programmes against Quality Element A1 of the Framework and in doing so to use the A1 part of this guide to identify strengths and weaknesses in terms of inclusiveness. He asks the teams to identify action points for action.

Taking it further, senior managers ask department heads to identify any further support needed by staff in using inclusive approaches in programme design, and refer this to the college’s coordinator for continuous professional development for action. This provides evidence for addressing inclusiveness under Quality Element B5: Staff.

Also, senior managers ask the quality manager to build some of the evidence statements into the college’s course approval process in the form of criteria to ensure that any new programmes meet the standards the college wants to set for inclusive practice. This provides evidence for addressing inclusiveness under Quality Element B6: Quality Assurance.

Example 2

College B has appointed a new section head in Computing and Engineering. She has visited some classes and looked at teaching materials and, while appreciating the high standards of teaching and learning, wants to improve the ways in which her staff respond to less motivated learners and to an increasing proportion of minority ethnic learners. She asks programme teams to evaluate inclusiveness against A3 (Equipment and materials), A5 (Learning and teaching process), A7 (Learner progress and outcomes) and A8 (Guidance and support) with the help of a well-informed member of the student services team. Subsequently, she meets her team leaders to discuss the results of the evaluation. They agree a prioritised set of action points which include targets for reflecting diversity in teaching materials and for working with learners on individual goal setting in the guidance process.

Taking it further, at the same meeting, they discuss what further advice and professional development they need as a section, and agree to request staff seminars from an external race relations organisation and from the college’s learner support services.

Example 3

College C has had major changes in its senior management team and has restructured its departments and support functions. The college serves a diverse community. It has extensive outreach provision and a high proportion of learners with additional support needs. It works in partnership with local regeneration agencies to promote employability and enable learners to access employment. The college is well respected in this work, but the new senior management team want to ensure that its good practice in inclusion is maintained and enhanced. The quality manager devises a broad-brush evaluation exercise based on this guide.

Taking it further, the senior managers consider the quality manager’s report. They identify a few areas of concern and ask for more detailed evaluation of inclusiveness to be carried out for introductory level programmes (SCQF levels 3 and 4) and for B5 (Staff).

Gathering evidence towards evaluation

The starting point for evaluation is the set of statements shown in the middle column. These are not in themselves evidence. The first task is to identify the activities and outcomes that back up the statements. There are many sources of evidence and it is important to use several. In many cases the most important pieces of evidence are those which show that the diverse range of learners have a positive experience and achieve success in their learning and in their personal development. The presence of procedures is less significant than the actual impact on learners.

The main sources of evidence are as follows.

People, for example:

• asking learners and staff
• using approaches such as individual and group interviews, focus groups, feedback from student representatives and customer survey questionnaires
• consulting external partners and users of the college’s services.

Direct observation, for example:

• the physical environment including access routes, publicity material, signage
• behaviours of staff and learners in public space around the college, including libraries and resources centres
• classroom observation in main buildings and in outreach locations.

Measurable outcomes, for example:

• analysis of monitoring data, for example on ethnicity, linked with attainment data
• learner achievement and attainment data for special programmes
• analysis of progression data.

Documents, records and resources, for example:

• policies, procedures and related internal audits
• CPD records and evaluation reports noting impact on staff practices
• content and presentation of paper-based and online learning materials.

Identifying and prioritising areas for action

Once the evidence has been gathered and recorded, the next step is to assess whether it supports the statement. This is often best done through discussion by the team involved, but it is helpful to have one or two people external to the team to provide objectivity. These might include people from external organisations depending on the area being evaluated. Questions to be asked and discussed might include the following.

• Are there any gaps in what we do?
• Has our practice resulted in good learning experiences for all our learners?
• Are we providing a poorer service for some learners than for others?
• Have we let any of our learners down in any way? If so, what lessons have we learned and how have we improved our practice?
• Is there any good practice that we should extend or pass on to others?

The discussion is likely to produce a number of action points. The team involved in the evaluation may be able to take immediate action on some points, while other points may be longer term or heavier on resources. Action points should be prioritised and recorded.

Planning action, setting targets, making improvements

Action points that cannot be dealt with immediately should be built into forward planning, for example in section or college operational plans. Senior managers accountable for the implementation of inclusiveness policy would wish to be aware of action required at all levels of the college. They might, for example, decide that action proposed in one team would be beneficial across the college as a whole and should be incorporated into the college operational plan, in support of college aims and objectives regarding inclusiveness. Senior managers would also need to approve any proposals that had significant resource requirements, and would wish to be apprised of good practice with a view to disseminating it.

Setting targets and recording them in planning documents means that good ideas emanating from evaluation are not lost. It is important that colleges monitor the achievement of targets and that the effect of the action taken is assessed, always with a view to making a positive impact on learners.

Colleges often recognise the need to seek external advice, for example on approaches to and support for particular groups of learners. Two of the key national sources, both funded by the Scottish Funding Council (SFC), are the Scottish Further Education Unit (SFEU) and BRITE (Beattie Resources for Inclusiveness in Technology and Education). SFEU resources are outlined in Appendix 2. SFC is, at the time of publication, establishing an Equalities Unit to support equality in both FE and HE. Colleges often also seek advice from national or local organisations, for example those representing or providing services for particular groups of people.

Example of subject area evaluation. A4.1 Staff: Qualifications and experience

Example of college area evaluation. B4.4 Resources and services to support the learner:

Facilities to support access and inclusion

[Previous] [Contents] [Next]