This section describes the main features of the new HMIE external review methodology to be applied in Scotland’s colleges over the period 1 August 2008 to 31 July 2012. The new review methodology and the quality framework around which it is structured take account of new thinking about the role of external review in the public sector and place more emphasis on high-level strategic issues such as the organisational capacity for and commitment to internal review and self-evaluation, a focus on performance outcomes and impact on users, and effective leadership and management. There is greater emphasis on institutional ownership of quality and recognition of greater diversity of practice in internal review and self-evaluation. SFC has already reminded colleges of this in its Circular SFC/33/2008: Council guidance to colleges on quality.7 Opportunities for learner engagement in external review are strengthened in the new review methodology. Moreover, HMIE reviews will be more explicitly informed by considerations of risk8 and proportionality.
External reviews will be designed to:
In realising the above aims, the breadth and depth of the external review will be based on, and proportionate to, the intelligence held by SFC and HMIE on a college’s capacity and commitment to deliver High quality learning, Learner engagement, and Quality culture.
Scotland’s colleges will continue to be reviewed over a four-year period. The scope of reviews in 2008-12 will be determined from intelligence established and updated through the annual engagement process. Adjustments may be made to the sequencing of reviews as against previous four-year cycles to reflect changing information about the context and effectiveness of each college.
External reviews will be structured according to the quality framework HMIE quality framework, September 2008: for reference during the external review of Scotland’s colleges. The reviews will be proportionate in nature. Colleges having a strong track record and where the conclusions of annual engagement are consistent with college effectiveness will normally receive the minimum form of review. This form of external review will be based on the reference quality indicators within the four elements, that enable HMIE to evaluate the college against the three key principles of High quality learning, Learner engagement, and Quality culture. Other colleges may be reviewed with varyingly greater depth of investigation and/or wider samples of provision. In these circumstances, review teams will use other quality indicators in the framework in a proportionate way to form evaluations where other significant factors are identified.
The external review will include a generic review of learning and teaching. HMIE will agree with each college the sampling approach to be used on a particular review. The review will aim to provide an independent judgement of the quality of learning and teaching in the college and evidence on the effectiveness of college-led internal review and self-evaluation of learning and teaching. However, where intelligence and risk analysis signal concerns in specific subject areas, the review of a college may be supplemented by the additional subject review of learning and teaching in these specific areas.
The generic approach will allow reviewers to be less constrained by defined subject areas and to sample across all curriculum areas, levels and modes of learning. It is intended to broaden and promote professional dialogue about learning and teaching and pedagogy beyond the confines and expectations of specific subject areas. The generic approach also aims to encourage the dissemination of good practice in learning and teaching across curriculum areas.
Reviewers will continue to report the outcome of external review in terms of strengths and areas for further development. They will use confidence statements to summarise their evaluations of the college.
The phasing of external reviews of colleges in academic year 2008-09 will be over the period February to April 2009. Thereafter, reviews will generally commence in late October and continue through to May in each academic year. Approximately eight weeks before the review, HMIE will brief the college principal, the chair of the Board of Management, managers and staff, the student association and learner representatives about the external review process. At this stage, the managing inspector (MI) will confirm the scope of the review, which will be based on the outcomes of the annual engagement process, updated by latest intelligence on the college.
In the pre-review period, the managing inspector will liaise with the college and explain the scope of the review including the nature and extent of any additional activity necessary beyond the minimum form of review. The college will be invited to provide current information for the review team. The managing inspector will agree with the college the sampling approach to be used for the generic review of learning and teaching.
The external review will normally last for three days, usually over the period Monday to Wednesday. On Monday morning, the review team will meet with the principal, senior management team and student association representative. Over the three-day period, the review team will:
On Friday, a sub-group of the review team will return to the college to provide oral feedback on the findings of the external review, including the provisional confidence statements. In the third working week following the review, HMIE will convey in writing the findings of the review and the confirmed confidence statements. A report on the review will normally be published within 13 working weeks following the completion of an external review.
Where annual engagement activity signals college effectiveness, the review team will involve typically a student team member and five reviewers. There will be a managing inspector and four team members, usually two HMI and two associate assessors. The number of reviewers will be adjusted to take account of the size and complexity of the college. For example, where colleges deliver in excess of 100,000 SUMs9 there are likely to be at least six reviewers. Where colleges deliver 10,000 SUMs and below the number of reviewers is likely to be reduced to four. Sometimes, HMIE may make arrangements with colleges for an international or other observer to join the external review team.
Where risks or weaknesses are evident through the annual engagement process, additional reviewers will be appointed to extend or enhance coverage of subject and/or cross-college themes. These additional proportionate review activities will normally be carried out at the same time as the activities in the minimum form of review. At the planning stage, HMIE will give due consideration to requests from colleges for additional engagement by HMIE to undertake additional review activities.
Before the review, HMIE will discuss and agree with each college the proposed nature and levels of reviewer engagement with managers and staff. HMIE will encourage college senior managers to work in partnership with reviewers to identify appropriate evidence.
HMIE will continue with the practice whereby colleges identify a senior manager with a comprehensive strategic overview to fulfil the role of review co-ordinator. This senior manager will play a key role in supporting the smooth operation of the review. The role of review co-ordinator is to:
Colleges that believe reviewers have not received all available relevant evidence before arriving at an evaluation or confidence statement should submit any additional evidence in writing to the MI within five working days of the oral feedback by the review team. The MI, consulting members of the review team as necessary, will consider any additional evidence and reply in writing, with a decision on any changes to be made.
College managers who have a concern or complaint about the conduct or the outcomes of a review should raise it in the first instance with the MI. If the matter is not resolved to the satisfaction of the college, it should be raised with HM Assistant Chief Inspector responsible for the college review programme, who will seek to resolve the issue.
If the concern or complaint is not satisfactorily resolved at this stage the college should pursue it through HMIE’s published complaints procedure. Colleges will receive printed copies of the procedure in advance of the review, and it is also available at www.hmie.gov.uk.
Colleges will be required to address areas for development and main points for action identified during external review. SFC will consider the external review report and write to the college indicating the scope and nature of further engagement by the college with SFC and HMIE. In some circumstances, SFC may request HMIE to conduct formal follow-up reviews of colleges. The scope of such reviews will be customised to address the specific context of each college.