Die LdV-Peer Review Projekte: Qualitätsentwicklung in der beruflichen Erstausbildung durch externe Evaluation
Workshop „Wirkung von Peer Review“ am 14.3.2011 an der HTL Steyr
Bericht "Peer Review Wirkungsanalyse"
Ein Service des
A - B - C - D - E - F - G - H - I - J - K -L - M - N - O - P - Q - R - S - T - U - V - W - X - Y - Z
Common Quality Assurance Framework (CQAF)
The Common Quality Assurance Framework (CQAF) has been developed by the Technical Working Group on Quality, a high-level European working group, following a mandate from the European Commission.
The CQAF constitutes a European reference framework to ensure and develop quality in VET, building on the key principles of the most relevant, existing, quality assurance models.
The CQAF comprises:
• a model (4 common core criteria)
• a methodology for assessment and review of systems: the emphasis has been given to selfassessment, combined with external evaluation;
• a monitoring system: to be identified as appropriate at national or regional level, and possibly combined with voluntary peer review at European level;
• a measurement tool: a set of reference indicators aiming at facilitating monitoring and evaluation by member states of their own systems at national or regional levels.
The model presents the 4 common core criteria for quality assurance: Planning, Implementation, Evaluation and Assessment, and Review (feedback and procedures for change).
Communicative validation is used in qualitative social research to enhance the validity of results: feedback on findings is systematically solicited from different stakeholders to challenge the data collected as well as its interpretation. A communicative validation can be carried out whenever necessary in the Peer Review process; in most cases it will be used in the final stages of the Visit, e.g. shortly before, during or after the feedback session with the VET provider.
Coordinating Body for Peer Review
If a suitable structure and sufficient funding is available, the coordination of the Peer Review network can be carried out by a competent organisation/unit. For the purpose of this Manual, this support structure will be called the "coordinating body". Establishing such a body is recommended for the management of complex (transnational) Peer Review networks.
The coordinating body can be central to the coordination and organisation of Peer Reviews. It can be either a coordination unit set up by a network of VET providers, the staff of a (publicly funded) pilot project on Peer Review or a (more or less) independent Peer Review agency established by education authorities.
The degree of influence and the scope of the tasks of the coordinating body may vary, depending on its set-up: it may process applications from Peers, select the Peers according to a predefined profile, match the VET providers with suitable Peers, draw up a timetable for the Reviews, collect and forward information, organise Peer training and provide consultation for the VET providers throughout the whole process.
Synonym of "Peers".
The Evaluation Expert is a Peer with additional knowledge and expertise in evaluation. In addition to the activities of a Peer, s/he will support the Peer Team in preparing interview questions for the Peer Visit, s/he will moderate the internal discussion sessions of the Peer Team during the Visit and also the communicative validation session with representatives of the VET provider at the end of the Visit. S/he may also coach/assist the Peer Coordinator in the writing of the Peer Review Report.
Formative Evaluation is an ongoing evaluation that serves the purpose of improving ("forming") the evaluation object, which may be, for example, a Quality Area, an entire organisation, a programme, a project, a product, an intervention, a policy or a person. In the case of the European Peer Review, a formative evaluation is carried out of certain areas or departments of VET providers/institutions.
The main focus of a formative evaluation is to support further improvement and sustainable development (whereas a summative evaluation is geared towards quality assurance and control). It can be used to exchange and share information and to provide feedback to staff, students, participants and other persons involved. In the European Peer Review, the results of the formative review are addressed primarily to the reviewed institution, to be used for internal quality development.
Management of a VET Provider
Person(s) responsible for managing the institution: these can be the directors, principals, general managers, etc. plus department heads and other managers (i.e. financial, quality managers, etc.).
Peers are mostly colleagues from other VET providers/institutions (teachers, managers, counsellors, other staff). They are external but work in a similar environment and have specific professional expertise and knowledge of the evaluated subject. They are independent and "persons of equal standing" with the persons whose performance is being reviewed.
Peers are sometimes also called "critical friends".
Peer Review is a form of external evaluation with the aim of supporting the reviewed educational institution in its quality assurance and quality development efforts.
An external group of experts, known as Peers, is invited to judge the quality of different fields of the institution, such as the quality of education and training provision of individual departments or of the entire organisation. During the evaluation process, the Peers usually visit the reviewed institution.
Peer Review Facilitator
The Peer Review Facilitator is the person responsible for the organisation and the smooth running of the Peer Review at the VET provider/institution. S/he will see to it that the Peers are selected and invited in due time, that the Self-Report is ready and forwarded to the Peers and that the Peer Visit is prepared. S/he will be also the primary contact person for the Peers during the whole Peer Review procedure.
Peer Review Report
The Peer Review Report is a written documentation of the Peer Review. It is drawn up by the Peers. Usually the Peer Coordinator, with the help of the Evaluation Expert, will write the report on the basis of notes taken by the Peers, internal discussions among the Peers and the outcomes of the communicative validation. All Peers will contribute to the report and the Peer Team as a whole is responsible for the Peer Review Report.
Peer Tandems are pairs of Peers. For all activities concerning data collection it is recommended that two Peers be present at any given time. This is an important precondition for a fair and equitable process since, with two peers involved, the probability of subjective and arbitrary judgements can be reduced substantially (principle of dual control). Two persons will also be able to take in more than one person. In practice, this means that the Peer Team splits up into pairs "Peer Tandems" and carries out different activities at the same time, thus making the process more efficient.
Peer Review Network
Peer Reviews are very often carried out in networks of VET providers/institutions. This network may have been established for the purpose of conducting Peer Reviews or, alternatively, an existing network may have decided to carry out Peer Reviews. Peer Review Networks can prove a valuable means of exchanging good practice and working jointly on the improvement of the whole sector of VET.
Provider/Institution of VET
In the Peer Review Manual, the term "VET provider/institution" is used to encompass the institutions who are responsible for quality assurance and development primarily at the school/college level but also at the level of the maintaining institution if this is where quality assurance and development are coordinated. Throughout the Manual, the terms "VET provider" and "VET institution" are used synonymously.
Quality of Vocational Education and Training
"Quality" is a generic and context-dependent term. It can be equated with the fulfilment of goals. In other words, quality is the experienced reality measured against expectations (goals). For the European Peer Review procedure, important Quality Areas have been defined to give an indication of what quality in initial vocational education and training is about.
Self-Evaluation of a VET Provider
Self-evaluation is an evaluation carried out by the VET providers themselves. It is an important approach for fostering quality assurance and quality development at an institutional level throughout Europe. For a Peer Review to take place, a self-evaluation must first have been carried out. Results of the self-evaluation are an important basis for the Peer Review. They are usually documented in a Self-Report.
The Self-Report comprises the findings of the self-evaluation of the VET provider carried out prior to the Peer Review. It is the basic document for the Peer Review.
Summative evaluation aims at arriving at final conclusions concerning quality and usefulness of the evaluation object, which may be, for example, a Quality Area, an entire organisation, a programme, a project, a product, an intervention, a policy or a person. Summative evaluation is geared towards quality control and external accountability. It often uses quantitative and comparative information to make recommendations on possible actions, such as retaining, enlarging or reducing the evaluation object. Summative evaluations thus also support the process of decision-making by political authorities and funding bodies.
Stakeholders (in VET)
Stakeholders in VET are
• staff (managers, teachers, counsellors and administrative staff),
• enterprises (as cooperation partners in the provision of VET, as prospective employers),
• educational institutions leading to VET (institutions of compulsory education) or taking in graduates from VET (post-secondary/secondary sector of education),
• social partners,
• education authorities, etc.
The inclusion of various relevant groups of stakeholders in the whole review process is highly recommended. First of all, high-quality evaluation calls for the involvement of stakeholders in the process. Secondly, the importance of stakeholders in quality assurance and development has repeatedly been emphasised as an important aspect of European and national VET policy.
Students (of VET)
The term "students" is used to denote the participants in initial vocational education and training.
Synonyms are: pupils, learners.
In social research, the approach of including different methods and sources is called triangulation. Using different methods and different sources of information in the collection of data contributes to the quality of the evaluation in terms of objectivity, reliability and validity. Soliciting diverse points of view from different stakeholders during the Peer Visit will enable the Peers to gain a more accurate and complete picture.
VET is the acronym for "Vocational Education and Training".