Home College Regulations & Guidelines Conduct of Pre-Fellowship Oral Examinations
Conduct of Pre-Fellowship Oral Examinations Print E-mail
REGULATION

Conduct of Pre-Fellowship Oral Examinations
Approval Date: October 2016
Review Date: October 2019



1. Purpose
The purpose of this Regulation is to outline the standard rules and procedures for the conduct of the final oral examinations of the Fellowship Training Program of the College.


2. Scope
This regulation applies to all Candidates sitting the Pre-Fellowship oral examinations and outlines the assessment criteria and format of the examination. Information about the dates and exam process can be viewed on RACMA website.


3. Responsibilities
Board of Censors: The Board of Censors (BOC) comprises the Censors of the College and is chaired by the Censor in Chief. Members of BOC consist of Censors involved in the assessment of formative work, summative work, Research Training Program requirements and RACMA Oral examination

Censor in Chief: (CIC) is Chair of the Board of Censors (BOC) and makes recommendations on the College assessment processes, and assessment related policy matters.

The Censor in Chief will
  • Ensure the RACMA Assessment Framework for the Fellowship Training Program is current and appropriate to the medical speciality;
  • Lead the BOC in the processes involved in formative and summative assessment activities of the Fellowship Training Program
  • Approve proposed schedules and timetables for the examinations
  • Engage in the review of requests for Special Consideration for summative assessments
  • Engage in the process of appeals in accordance with the College Policy for Reconsideration, Review and Appeals of decisions of College Officers
  • Make recommendations to the BOC, Education and Training Committee and RACMA Board for the recruitment and training of College Censors
  • Provide performance feedback to Censors as part of their Professional Development and Performance as a RACMA Censor
  • Provide post-exam feedback to the Candidates seeking feedback after unsuccessful attempt at the Oral Examination.
  • Develop and provide reports on the examinations outcomes and process, including pass/fail results to the Education and Training Committee (ETC) and present to the RACMA Board.
  • Engage in the progression review of Candidates as required and recommended by the RACMA Progression Panel
Censor for Exam Questions: The Censor for Exam Questions will co-ordinate the drafting, evaluation and finalisation of questions for the Pre-Fellowship Oral Examinations, in consultation with the selected team of Censors nominated for exam question development. The Censor for Exam Questions is to ensure that the process is timely, confidential, well-coordinated, and is well mapped to the RACMA Leadership and Management Curriculum. The Censor for Exam Questions sends the final draft of the Oral Examination to the Moderator during the pre-issue period, either as the questions are developed or at the end, in time for the Moderator to assess the details of the consistency, including the choice of scenario for potential supplementary examination.

Censor for Examinations/Moderator: The Censor for Examinations is the Moderator for the exams, who reports to the Censor in Chief and is responsible for the conduct of College examinations and advice to Candidates about their results. The Moderator is also a censor leading the Moderating process with the examining Censors. The Moderator on the day is responsible for monitoring the recording of the results as the examination proceeds and ensuring that no circumstances in the examiner pairings or in their scoring is disadvantaging candidates.


4. Role of Moderator
In the RACMA Assessment Framework moderation is described as the process that ensures the consistency of marking of summative assessment tasks in terms of:
  • Alignment of assessment tasks with intended learning outcomes;
  • Appropriateness of content in terms of curriculum coverage;
  • Standardisation of level of challenge; and
  • Fairness to Candidates.
Moderation takes place in the 'pre-issue' phase and at the time of the examination, be it oral or written. Moderation involves the "Use of a template for setting an examination or written or oral task that clearly articulates the aim of the task ie to demonstrate, outline, report upon......"


5. Knowledge, management skills and communication behaviours:
  • Checking the details of the question setting for accuracy and consistency;
  • Preparing a marking scheme that is fit for purpose;
  • Training the examiners in the conduct of the assessment task and in the scoring;
  • Mentoring new examiners;
  • Supervising the implementation;
  • Possibly re-marking;
  • Analysing the results;
  • Assessing the impact of 'extenuating circumstances' or misadventure in an oral exam;
  • Identifying borderline cases;
  • Recommending scaling where necessary; ;
  • Determining appropriate methodology for acquiring more information (offering supplementary scenarios in oral examination, allowing for enhanced submission in a written task);
  • Preparing a report of activities.
Moderation can be conducted internally within or externally of, the panel preparing or executing the assessment task.

Education & Training Committee: The Education and Training Committee receives recommendations about examination and assessment policy and endorses these as recommendations to the RACMA Board for final approval consistent with its delegations.

Jurisdictional Coordinator of Training: The Jurisdictional Coordinator of Training (JCT) provides advice to Candidates, the National Office, the Chair of the Progression Panel, and the Censor in Chief on Candidates' progression, training issues and readiness to sit the Pre-Fellowship Examination. The JCT will also liaise with the Candidate's Preceptor/Coach and/or Supervisor to appraise the Candidate's progression issues, remediation and readiness to sit the Pre-Fellowship Examination.

Preceptor/Coach: The Preceptor or Executive Coach will assess their Candidate's progression against the role competencies defined in the RACMA Curriculum and assist the Candidates develop skills, knowledge and fulfil training gaps to ensure that he/she is progressing satisfactorily through the Fellowship Training Program and is ready for the Pre-Fellowship Examination. The Preceptor or Coach should also consult with and advise the Jurisdictional Coordinator of Training and the National Office on the Candidate's progression and readiness to sit the examination.


6. Moderating at the Oral Examination
After the examining period has ended the scores are revealed to the examining panel, for final moderation and commentary. The Moderator declares the outcomes for those who have passed, those who have been unsuccessful and those who are to be offered an additional exam station (supplementary examination). There is a guide and an assessment rubric for identifying those who are clearly successful or unsuccessful, and for identifying the borderline candidate. After discussion the Moderator identifies those candidates from whom more assessment information is required, and these Candidates are offered a 5th exam station or a supplementary examination on the day. Ordinarily those examiners who have not examined the candidate and have been quarantined from the Exam and moderation process on the day conduct the 5th exam station. The Moderator will not examine on the day.

The Moderator prepares a report of the conduct of the Oral Examination for the Censor-in-Chief/Board of Censors, noting any issues which have arisen with respect to consistency. The Censor-in-Chief reports on the release of results to the specially-convened teleconference meeting of the Education and Training Committee for recommendation to the Board and endorsement of Candidates eligible for election to Fellowship.


7. Examination Assessment Criteria
Candidates will be assessed on their skills, abilities, knowledge, attitude and depth of experience as medical administrators, using case study material. The areas of assessment of these role competencies are outlined in the RACMA Medical Leadership and Management Curriculum, and most, if not all, of the following areas will be examined and assessed by Censors.
  • general management principles;
  • current health policy initiatives;
  • medico-legal issues in health services management;
  • financial management of health services;
  • psycho-social issues in health services management;
  • human resource management in health services;
  • planning of health services, including epidemiological studies;
  • recent advances in health care;
  • analytical and presentation skills;
  • personal attributes of leadership;
  • culture and Indigenous health; and
  • leadership

8. Pairing of Censors for Pre-Fellowship Oral Examination
To maintain the integrity of RACMA examination process, Censors must meet the following selection criteria for the pairing of Censors at the RACMA Pre-Fellowship Oral Exams.
  • At least one of the examining Censors within the pair must have private and public hospital knowledge or experience in medical administration
  • Questions relating to hospital setting must involve Censors with the above hospital experience
  • New Censors, who have completed the required training prior to being eligible to examine, must be paired with a senior and more experienced Censor with at least 5 years of RACMA examination experience
  • All Censors must declare any conflicts of interest with Candidate and pairing Censors - examples of these should include:
    • previous knowledge of Candidates (e.g Being on job interview panel, previously working together etc)
    • having been Candidates' Preceptor/Executive Coach or Supervisor, and/or
    • Direct report, member of Board etc.

  • The Censor in Chief and/or Moderator must to the best of their knowledge and ability:
    • Avoid pairing together Censors with history for hard marking
    • Pair Censors to create:
      • Gender balance
      • Sufficient NZ context representation
      • New and experienced Censor pairing
      • Censors with knowledge in rural and metropolitan health service
      • Alignment of expertise to the themes of the exam case scenarios

  • When pairing Censors examining a compulsory question, Censors must demonstrate the appropriate expertise and knowledge in the compulsory question themes i.e. finance/data, clinical governance etc. New Censors will not be allocated to examining compulsory questions in their first year of appointment as a Censor. Censors who were involved in the development of compulsory case scenarios should be nominated to examine these questions, and provide training to their pairs if required.

9. Examination Format and Scoring Process
RACMA uses a series of 4 sequential scenario based VIVA examination interviews as per following:
  • Each Candidate will be assessed independently by four (or five if a 5th station is offered as a supplementary examination) pairs of Censors.
  • Candidates will have studied the scenario based case material for 20 minutes prior to the presentation to the Censors.
  • The performance during presentation and discussions emanating from the scenario based case material is assessed by the Censors based on the exam assessment rubric
  • Case studies will be mapped to the RACMA Curriculum and Candidates will be expected to identify the key potential issues from the case scenario, respond to questions and present possible strategies for management of these issues/questions to the Censors.
  • There is a choice of a minimum of 2 scenarios for 2 of the choice questions, and two questions are compulsory
  • The examining Censors (examining pair) will independently score and comment on Candidate's performance, using the College developed exam rubric-based assessment score sheet. This occurs after the Candidate has completed his/her presentation of the case-scenario and has left the exam station. The Censors will moderate the scores between themselves after the original scores are recorded on the score sheet. The moderated score is also recorded. If the Candidate has failed, detailed comments will be completed by the Censors to assist in providing feedback to the Candidate for post-exam feedback sessions. The score sheets are given to the Moderator for review and recording to determine the outcome of exam performance of Candidates. Moderation of scores will be conducted in the event of borderline performance outcomes or if there is a great level of disparity between the examining pair's scores.

10. 5th Exam station - Supplementary Question
Candidates who are assessed at moderation as borderline are offered an opportunity to present to a 5th (supplementary) exam station. Presentation to the 5th station is undertaken immediately after the initial set of four examination sessions and Candidates are advised immediately after the conclusion and moderation of all 4 exam sessions. The 5th exam station aims to gather further information on Candidates' competency, to inform the Board of Censors' "pass" or "fail" decision for borderline Candidates.

The pair of Censors assigned to the 5th (supplementary) session will be quarantined from all other examination stations and the moderation discussions at any of the examination stations. They will not be part of any discussions with other Censors and should be independent from all other deliberations.

The supplementary exam question for the 5th exam station should be generic and broad in medical administration competency to ensure that gaps that may have been demonstrated by borderline Candidates in previous exam stations are well covered and enable candidates to showcase their skills/knowledge in other competency areas. They should also be of the same standard as other questions.


11. Conclusion of the Exams
The exam period concludes once all Candidates have been assessed and the Board of Censors has their post examination Moderation and 'wrap-up' meeting.


12. Examination Results
Candidates will be notified of their results no later than 72 hours after the conclusion of the examination period. A report of exam results and recommendations for election to Fellowship is presented to the Education and Training Committee (ETC) from the Censor in Chief for endorsement. Results are distributed via electronic notification to Candidates' previously nominated and confirmed email addresses. Candidates are informed on how to access their results prior to the examination.


13. Special Consideration
Candidates should not be disadvantaged unnecessarily as a result of events or circumstances outside their control. Where an issue arises which may not be covered in the Regulation for Conduct of Exams or in the Guidelines for Examinations or other policies related to examination process, Candidates can seek special consideration for their performance at the Oral Exams in accordance with the Policy for Special Examination. The requests for special consideration must be forwarded to the Censor in Chief, who will set up a consultation review panel, comprising of members of the Board of Censors, to advise on the application for special consideration and the decision making. On the basis of information provided, the following will be included for consideration:
  • The severity of the circumstances and factors involved
  • The likely impact on the Candidate and his/her performance during the Exam
  • History of previous applications for special considerations
Applications for special consideration must be submitted prior to the post-exam feedback sessions scheduled by the College and within the time-frame as specified in the Policy for Special Consideration.


14. Feedback
For those Candidates who are unsuccessful at the oral examination, the College will provide a formal verbal feedback session within 3 - 4 weeks of the oral exam involving the Moderator/Censor in Chief, Candidate, the Candidate's Preceptor (if agreeable to the Candidate) and a member of the Board of Censors who was involved in the examination of that Candidate - particularly in relation to the questions that Candidate has failed. Candidates who wish to participate in a verbal feedback session are required to contact the College within 1 week of the examination to request this session.


15. Previous Examination Questions
Examples of previous case studies are available to Candidates, Preceptors and Supervisors on the RACMA Website. These examples allow Candidates to familiarise themselves with the scope, format and style of examination questions enabling Candidates to practise past exam questions with their Preceptors and Supervisors and identify where they may have training or knowledge gaps.


16. Bernard Nicholson Prize (Meritorious award)
The Bernard Nicholson Prize is awarded to the Candidate with the best/outstanding performance in the oral examination. The recommended winner is determined at the moderation meeting of the Board of Censors held directly after the examinations. The awarding of this prize is discretionary and the College may decline to make the award if, in the opinion of the Board of Censors, no Candidate has achieved a sufficiently outstanding result at the examination. On the recommendation of the Board of Censors, the Education and Training Committee will endorse the award of the Bernard Nicholson Prize to the Board for final approval.

Note: 'The Bernard Nicholson Prize was donated by and is awarded in memory of Dr Bernard Nicholson. Dr Nicholson was a past President of the College and played a major role in its establishment.'


17. Observer
There may be an observer in the examination room. By electing to sit the examinations, Candidates agree to have a Censor-in-training or another observer in the examination room.

The observer can be:
  • A Censor-in-training who will observe other Censors’ behaviour, technique etc.
  • a delegate from another College to observe RACMA exam
  • Censor in Chief
  • AMC representative
  • National Office Staff

18. Rules for Examination Day
This procedure provides clear documented guidance for Candidates, Censors and RACMA staff. Rules for Examination outline what Candidates are and are not permitted to take with them into the exam station and what is the step by step protocol for the examination process.


19. Appeals
Candidates can request to have a decision by the College Officer reconsidered and reviewed and can follow a College Appeals process in accordance with the Policy for Reconsideration, Review and Appeal of Decisions of the College Officers


20. Code of Ethics
All Censors and Candidates sitting the examination are expected to act at all times ethically, responsibly and in the best interest of the College. All Censors and Candidates will adhere to the RACMA Officer's Code of Conduct.


21. Related Documents:
  • College Handbook
  • RACMA Medical Leadership & Management Curriculum
  • Regulation: Eligibility to sit the RACMA Pre-Fellowship Oral Examination
  • Terms of Reference for the Board of Censors
  • Terms of Reference for the RACMA Progression Panel
  • Censor Position Description
  • Position Description of the Jurisdictional Coordinator of Training
  • Rules for the Examination Day
  • Policy for Appointment and Training of Censors
  • Privacy Policy
  • Policy for Special Consideration for the Oral Examination of Candidates requiring consideration for illness, accident, disability or compassionate grounds
  • Policy for Reconsideration, Review and Appeal of Decisions of the College Committees and Officers
Last Updated on Monday, 10 October 2016 08:49