Colleges used questionnaires as a means of collecting client satisfaction data and for getting more detailed feedback on particular topics. This process was well established in all colleges, although different approaches to collecting and analysing the information were in place. Some colleges favoured online questionnaires as a more efficient way of collecting the data and as a way of encouraging more students to participate. Others encouraged students to complete questionnaires by issuing and collecting them during class time. In some cases, samples of students were selected while in other cases questionnaires were given to all students.
A typical college student satisfaction questionnaire designed to evaluate the student experience dealt with facilities and resources, guidance, aspects of learning and teaching, equal opportunities and assessment. Where several questionnaires were issued during the year, staff might concentrate their questions on different aspects of college provision at different times of the year. For example, the quality of induction and initial guidance would be covered in the autumn, with other topics dealt with later in the year.
Results of college-level questionnaires were usually collected centrally, analysed and then distributed to departments and programme committees for consideration. Comments that were specific to one aspect of provision were usually given to the responsible individuals but not shared more widely. Data collected on the same basis over a period of time could provide trend information that was helpful in evaluating the impact of quality improvement activities and in highlighting any causes for concern. Overall results were often published in college annual reviews. One college, with a very comprehensive website, made available its student satisfaction data online. Some colleges used this data well. The HMIE college review report on Aberdeen College in 2004 observed that:
It (the college) produced a comprehensive set of quantitative data from student and client surveys and used this information effectively to inform quality improvement in teaching sections and support services.
Teaching departments in colleges sometimes used questionnaires for their own programmes. A few teaching staff, in turn, used questionnaires to get feedback on the units they were teaching. These unit evaluations, where these were used, were found by staff to be more useful for evaluating learning and teaching. However, questionnaires at unit level were not used consistently across the sector or used to the same extent by different teaching departments within the majority of colleges. Few staff used these evaluations to initiate detailed discussions on learning and teaching with their students. College support staff also used questionnaires widely to get feedback on specific aspects of provision, for example, guidance and library services.
Students’ views on questionnaires were mixed. They valued the fact that their views were sought, but sometimes found the questions too superficial and were doubtful that fellow students always answered the questions in a thoughtful way.
While all questionnaires were recognised as having shortcomings, they gave an overview of student attitudes to college provision, allowed trend information to be captured and provided the means to find out the views of students who might not normally be able or wish to participate in other arrangements for representation.
Student feedback based on questionnaires was an essential part of college quality systems and, in their various forms, questionnaires provided college managers and staff with valuable information on the strengths and weaknesses of the student experience. However, the effectiveness of any method of evaluating the student experience depended on the attitude of students. If they regarded questionnaires as inconvenient and superficial tick-lists that they had to complete, they were unlikely to give informed and reflective answers to the questions.
More than a few colleges used focus groups to get students’ views on a range of issues. They were valuable in providing a more detailed, although narrowly-based, insight into student views than could be gained from questionnaires. Colleges arranged thematic focus groups to discuss issues as diverse as induction, cultural awareness and childcare provision. In one college, focus group discussion on progression to higher education had led the college to reconsider its approach to learner independence and learning styles. In addition to thematic focus groups, one college had arrangements for regular discussion group sessions between students and senior managers and similar meetings between students and department heads. Topics for these discussion groups covered general student experience issues. A number of colleges involved in new campus developments had made special arrangements for collecting students’ views on these developments. For example, one college had had extensive consultations with student representatives that resulted in a "shopping list" for the new campus. Students and college managers valued this consultation. In a few colleges, students were involved in some of the colleges’ audit processes to help evaluate aspects of the colleges’ provision.
The HMIE review report on Cumbernauld College in 2004 reported under the Quality improvement element that:
Staff used a variety of approaches to evaluate learning and teaching. These approaches included peer observation and sharing of practice through team teaching, and student focus groups where students assessed critically their learning experiences.