[Previous] [Contents] [Next]

Analysis of HMIE Reviews of Quality and Standards in Further Education Academic Year 2004/05

8. College staff evaluations of the review process

HMIE invite and encourage colleges to provide feedback about the review process and its outcomes through the use of questionnaires and a discussion with the principal after both phases of the reviews are complete. Some comments taken from staff questionnaire returns are detailed later in this chapter. All comments from college staff are considered by HMIE in adjusting and in developing procedures for future reviews.

8.1 Evaluations from review phase one

The pie charts below summarise the feedback received by HMIE on college perceptions of the suitability of methods employed by reviewers during reviews, the quality of the feedback given and the overall helpfulness of the review process to the college. The first set of pie charts deals with perceptions from staff responsible for subject areas reviewed during phase one:

Eighty-five percent of the respondents felt suitability of methods, deployment of reviewers and procedures employed were good or better. Ninety one percent of respondents rated the quality of feedback as good or better. Of those who rated feedback as unsatisfactory, some disagreed with review team evaluations, and others indicated that they had not been involved in the groups of staff to whom reviewers fed back.

Reflecting the new focus as against reviews in previous years, staff were asked how they rated the effectiveness of the focus on A5 and A7 during the review. The responses are shown in the pie charts below.

Eighty-three percent of respondents rated the effectiveness of the focus on A5 as good or better. A similar pattern emerged for the response to the effectiveness of the focus on A7, with 85% rating it good or better.

Eight-eight per cent of staff involved in the subject review process felt that the helpfulness of the review process to the college was good or better.

The pie charts below consider the responses from staff responsible for areas reviewed in college review to questions similar to those addressed in the previous section.

Respondents rated the suitability of methods, deployment of reviewers and procedures as good or better in 94% of cases. Ninety-two percent of respondents rated the quality of feedback as good or better, including 53% of respondents rating it very good.

In terms of helpfulness to the college, 91% of respondents rated the review process as good or better.

The following collection of views from college managers and subject leaders has been chosen to reflect the range of perceptions of the review process on specific aspects of the quality of provision in colleges:

• The AA demonstrated genuine interest in all aspects of the School and took time to sample learning and teaching.
• The initial meeting between the Head of School and the reviewer was very productive and helpful.
• Some subject areas were reviewed quite heavily (and had been reviewed in the previous inspection visit) whilst other subject areas were not reviewed and had not been reviewed previously.
• Verbal feedback session to the School staff team was excellent.
• HMI were open to discussion and demonstrated a determination to be clear about the methodologies, resources and decisions made at the college.
• The 'end of day' feedback was a very good approach.
• HMI and AAs made the review process work well and opportunities to comment or discuss elements were well focused and well balanced.
• A wider range of Outreach Centres could have been visited by reviewers to gain knowledge and understanding about the diversity of our provision.
• A thorough review which was helpful in confirming what we are getting right, informing action planning and providing encouragement and motivation to staff.
• The number of classroom observations in one subject area reviewed could not review the breadth and depth of the subject area.
• Written feedback is a massive leap forward, as staff can digest comments and reflect upon them.
• The written feedback is undoubtedly useful and given the depth and breadth of the review it is extremely useful to have been given verbal feedback to clarify the written reports.
• It proved difficult to schedule all the interviews for the College Review simultaneously with the Subject Review. It is extremely difficult to get groups of student from across the College together, especially at short notice, and staff spent a great deal of time and effort organising this.
• The process appears to be more rigorous in some respects when compared with the previous model particularly in relation to the interaction between subject and college reviews and how this supports evaluation of the B elements.
• Greater focus on the learning and teaching was seen as an improvement by teaching staff involved.

8.2 Evaluations from the follow-through phase

The pie charts below summarise the feedback received on the suitability of methods employed by reviewers in the follow-through phase during reviews, the quality of the feedback given and the overall helpfulness of this phase to the college. The first set of pie charts deals with perceptions from staff responsible for the subject areas reviewed during the follow-through phase.

Eighty-one percent of the respondents felt suitability of methods, deployment of reviewers and procedures employed were good or better. Ninety-one per cent of respondents rated the quality of feedback as good or better.

Eighty-four per cent of staff involved in the subject areas reviewed during the follow-through phase felt that the helpfulness of the review process to the college was good or better. In some cases, respondents considered that the follow-through had not added much to phase one findings. In other cases, they expressed dissatisfaction with the scope of follow-through activity as defined in the national briefing document.

The second set of pie charts deals with perceptions from staff responsible for the cross-college areas reviewed during the follow-through phase.

Respondents rated the suitability of methods, deployment of reviewers and procedures in the cross-college follow-through phase as good or better in 81% of cases. Ninety-six percent of respondents rated the quality of feedback as good or better, including 53% of respondents rating it very good.

In terms of helpfulness to the college, 94% of cross-college respondents rated the follow—through phase of the review process as good or better.

An additional question for both the subject and college staff involved in the follow-through phase was how they rated this phase of the review in terms of the exploration of good practice. The pie charts are shown below.

In the subject review follow-through phase, 93% of respondents rated the exploration of good practice as good or better. Ninety percent of respondents in the college review follow-through phase rated it good or better.

The following collection of views from college managers and subject leaders has been chosen to reflect the range of perceptions of the follow-through phase of the review process:

• We were given ample opportunity for discussion and review of phase one.
• It has provided us with an opportunity to discuss in some detail good practice elements in the context of the whole college.
• Some written feedback would be helpful even at this early stage as there was a tendency to miss valuable points during feedback.
• The level of engagement felt appropriate in relation to the outcomes from phase one, with an appropriate balance between the follow through on weaknesses identified and good practice.
• I found the HMI extremely approachable and supportive.
• The feedback was good, helpful suggestions for development where applicable and clear information on issues.
• Phase two was much more developmental and relaxed with opportunities for discussion.
• It might be helpful for HMIE to ensure the phase two reviewer who is considering good practice liaises with the phase one reviewer who proposed it.
• I, like other colleagues, welcomed the opportunity to take advantage of the consultancy aspect of the review which was very helpful and non-threatening.
• Tended to be directed at senior management so consideration should be given to a wider audience.
• The HMIE team worked methodically with minimum disruption to staff.

There are examples above of comments from college managers and staff that included some reservations about the process and criticisms about operational aspects. Often, these comments were the views of individuals. However, there were two recurring themes, to which HMIE has responded as noted below.

• More than a few respondents felt that the smaller number of lesson observations compared with the previous cycle either led to rushed processes or did not provide a sound underpinning of the grades awarded for learning and teaching process.
  
  HMIE decided that the number of lesson observations normally to be undertaken for 2005/06 reviews would be increased from six to eight or nine.

• It was evident that in more than a few colleges, the selection of subjects for review in 2004/05 meant that other subjects had not been reviewed over a long period of time.
  
  In selecting subjects for reviews in 2005/06, HMIE has aimed to provide wider coverage of subjects by paying greater attention to the coverage of subjects in previous reviews and inspections of the colleges involved.

[Previous] [Contents] [Next]