Home College Regulations & Guidelines Conduct of the Oral Examination
Conduct of the Oral Examination Print E-mail

Regulation on the Conduct of the Oral Examination
Recommended by the Board of Censors

Endorsed by the Education and Training Committee

Approved by Board: December 2017
Next Review Date: December 2018

Purpose
The purpose of this Regulation is to outline the standard rules and procedures for the conduct of the Oral Examinations of the Fellowship Training Program of the College.


Scope

This regulation is relevant to all Candidates sitting the Oral Examination in 2017 and beyond; and is published on the RACMA website for Policies and Regulations. The dates for the Oral Examinations are published on the website and in the Annual Training Calendar. Candidates are also advised in the RACMA Notes and EdNews. 


Policy statement

Fellowship Training is delivered as four learning and assessment programs:

·       Health System Science
·       Medical Management Practice
·       Scholarship
·       Personal and Professional Leadership Development

The Oral Examination is to be retained as an exit ‘pre-fellowship’ Examination in 2017, 2018 and 2019. From 2020 it will be a method of summative assessment in the Medical Management Practice Program.

The Oral Examination is designed as an assessment tool that aligns with the learning objectives of the curriculum overall, and with the competencies (skills) identified in the graduate outcomes of Medical Expert, Medical Manager and Communicator, in particular. 

The standard expected of this performance is that of a candidate capable of satisfactory practice as an independent specialist medical administrator in Australia or New Zealand.

Business rules for the Oral Examination

The Oral Examination is held annually. In 2017 and 2018 it will be held in November or December. From 2019 it will be held in the middle of the academic year i.e. July or August.

The Oral Examination consists of four (4) examination stations, and the candidates present at four stations. Each station has a written scenario and accompanying questions. There are two stations with no choice of questions and two stations with a choice from 2 scenarios in 2017.  The Oral Examination is generally held over two days, depending on presenting numbers of Candidates. On Day 1, all the candidates are offered the same questions and a different set of scenarios are prepared for the candidates for Day 2 (and Day 3 if the numbers of Candidates presenting, require it).

The scenario and questions are presented to the Candidates and they have 20 minutes to prepare their responses in an allocated reading room. Following the preparation time, Candidates present their responses to two Censors in a twenty-minute period. Approximately 10 minutes is allowed for presentation and in the second ten minutes the Censors will ask questions prompted by the candidates’ presentations. The candidate’s ability to verbally identify the knowledge, skills and attitudes/approach required to address the scenario is marked against a calibrated marking rubric.

If a Candidate is identified as marginal in his/her overall performance at the Oral Examination, as determined in a moderation process involving group Censor discussion, he/she will be offered a 5th station, with a 7th prepared scenario, and an additional Censor pair who have been isolated from the previous moderation process or discussion about candidates. The additional station offers the opportunity for further exploration of Candidate knowledge and competency in medical management practice before the final result of Pass or Fail is determined.

Candidates are notified of their Oral Examination outcomes after endorsement of their results by the Education and Training Committee (within 48 hours of the completion of the Examination).

 
Oral Examination Topics
Candidates will be assessed on their ability to convey to the Censors that they have the requisite knowledge, skills and attitudes/approach to satisfactorily deal with the examination scenarios. The topics for assessment in this format are outlined in the RACMA Medical Leadership and Management Curriculum in the learning objectives of the role competencies of Medical Expert, Medical Manager and Communicator and cover:

·       planning in health systems, including epidemiological studies;
·       managing current health policy initiatives;
·       general management principles;
·       human resource management in health services;
·       managing clinical governance;
·       financial management of health services
·       medico-legal issues in health services management;
·       principles of disaster management
·       culture and Indigenous health

The scenario settings include critical care, acute care and sub-acute care; public and private hospitals; and government and non-government health services.


Eligibility to sit the ‘Pre-Fellowship’ Oral Examination

Candidates are eligible to sit the Oral Examination in 2017, 2018 and 2019, if, at the time of the examination:

·       They are within 6 months of experiencing a minimum of three full time equivalent years of approved supervised medical management practice (or are within six 6 months of completion);
·       They are within six (6) months of completion of their Master’s study units;
·       They are within 6 months of submission of the Research written paper;
(Or within 12 months of research written submission if undertaking this under the University auspice and this is required to fit in with RACMA-approved university academic timeframes.)
·       They have participated in a formatively assessed National Trial Oral Examination within the prior two years;
·       They have been working in an accredited medical management training post within the prior twelve months;
·       They have satisfactorily completed mandatory Fellowship Training Program written tasks;
·       Any application for Special Consideration has been granted;
·       They are in financial good standing with their RACMA membership status and have paid the relevant College and training fees, including the examination fee; and
·       Their Preceptors/Executive Coaches have advised on the Candidates’ readiness for the examination, and have signed the application form to indicate that they have provided advice.
 

Eligibility to sit the MMPP Oral Examination

Candidates will be eligible to sit the Oral Examination in 2020, if at the time of the examination:

·       They have satisfactorily participated in a minimum of two and a half full time equivalent years of programmed supervised medical management practice in accredited positions;
·       They have participated in a formatively assessed National Trial Oral Examination within the prior three years;
·       They have been working in an accredited medical management training post within the prior twelve months (with the exception of Candidates in the Medical Executive Pathway);
·       They are in financial good standing with their RACMA membership status and have paid the relevant College and training fees, including the examination fee; and
·       Their Preceptors/Executive Coaches have advised on the Candidates’ readiness for the examination, and have signed the application form to indicate that they have provided advice.


Moderation of Examination outcomes

In the RACMA Assessment Framework moderation is described as the process that ensures the consistency of preparation and marking of summative assessment tasks in terms of:

·       Alignment of assessment tasks with intended learning outcomes;
·       Appropriateness of assessment content in terms of curriculum coverage;
·       Standardisation of level of challenge; and
·       Fairness to candidates of processes.

Moderation takes place in the pre-event preparatory activities and during the marking of an examination and involves:

·       Design
·       Standard setting and calibration
·       Delivery
·       Agreement on outcomes

Design

·       Identification of appropriately trained participants in the examination setting. Moderation can be conducted internally within or externally of, the panel preparing or executing the assessment task.
·       Use of a template for setting an examination covering an oral task that clearly articulates the aim of the assessment method;
·       Checking the details of the scenario setting and questions for accuracy and consistency; and
·       Preparing a marking scheme that is fit for purpose.

Standard setting and calibration

·       Setting the standard that will be expected for a candidate to have reached a satisfactory result;
·       Determining appropriate methodology for acquiring more information (offering supplementary scenarios in the Oral examination) if needed;
·       Circulating business rules within adequate timeframes, to relevant stakeholders.
·       Training the Censors in the conduct of the assessment task and in the scoring; and
·       Mentoring new Censors.

Delivery of the Examination

·       Appointed person overseeing feedback from the Candidates;
·       Chief Censor analysing the results;
·       Assessing the impact of ‘extenuating circumstances’ or misadventure;
·       Identifying borderline scores;
·       Recommending scaling or re-marking where necessary;

Agreement on outcomes

·       Preparing a report of activities;
·       Evaluating processes for future design.

Responsibilities of the Censor-in-Chief and the Board of Censors
The Board of Censors (BOC), under the guidance of the Censor-in-Chief is responsible for the conduct of the Oral Examination and assessment of other programs of formative and summative activities. The BOC is comprised of selected Fellows with appropriate training and experience who have been approved by the Board to be admitted to the BOC. They form the Panels for the purpose of assessment at the Oral Examination. The Panel on the day is made up of the Censor pairs.

The Censor-in-Chief is the RACMA-Board-Appointed Chair of the Board of Censors (BOC). The Censor in Chief will, in conjunction with National Office staff:

·       Address and manage potential conflicts of interest as declared by Censors and Candidates;
·       Approve proposed schedules and timetables for the Oral Examinations;
·       Endorse the eligibility of Candidates for presentation to the Examination;
·       Engage in the review of requests for Special Consideration for the Oral Examination;
·       Make recommendations to the BOC, Education and Training Committee and RACMA Board for the recruitment and training of College Censors;
·       Appoint the Lead Censor for Oral Examination questions and the Questions Panel;
·       Be available for moderating on the day of the examination, or appoint a Censor to undertake that role;
·       Present the individual results to the Education and Training Committee (ETC) for their endorsement prior to their release to the Candidates, Supervisors, Preceptors and  Jurisdictional Coordinators of Training;
·       Develop and provide reports to the RACMA Board as required on the examinations processes and outcomes;
·       Lead the post-examination feedback to the Candidates after attempts at the Oral Examination;
·       Engage in the process of appeals in accordance with the College Policy for Reconsideration, Review and Appeals of decisions of College Officers and Committees, if required; and
·       Facilitate peer review and provide performance feedback to Censors as part of their Professional Development and Performance as a RACMA Censor.

Panel for Examination Questions

The Censor for Examination Questions co-ordinates the drafting, evaluation, calibration and finalisation of questions for the Oral Examinations, in consultation with the selected panel of Censors nominated for examination question development.  

The Panel is responsible for preparation of the marking rubric for each scenario – which is based on the agreed ‘general’ rubric which is published on the RACMA website each year.

The Panel members ensure the standard for the writing of the questions and the wording of the rubrics by internal circulation; or may consult other Fellows (new Fellows or Supervisors) to have ‘tested’ the time needed to address the scenarios and the validity of the rubrics, during the preparation period.

The Censor for Examination Questions sends the final draft of the Oral Examination (six questions for each day plus one ‘5th station’ for each day) to the Censor-in-Chief, Dean of Education and an independent Censor, during the pre-event period, either as the questions are developed or at completion, to enable them to assess and verify the details of consistency, including the choice of scenario for potential 5th station examination.

The nominated Censor Panel meets the day before the Oral Examination to further calibrate the expected responses, and identify any previously undeclared conflicts of interest. The Panel then presents the finalised questions and any recommendations to the Censor-in-Chief.


Pairing of Censors for the Oral Examination

To maintain the integrity of the RACMA examination process, Censors will, as far as possible, meet the following selection criteria for the pairing of Censors at the RACMA Oral Examination.  The process of agreed pairing is overseen by the Censor-in-Chief.

·       At least one of the examining Censors within the pair must have private or public hospital medical administration knowledge or experience;
·       Questions specifically relating to a hospital setting will involve Censors with recent hospital experience;
·       New Censors, who have completed the required training prior to being eligible to examine, will be paired with a senior and more experienced Censor with at least 5 years of RACMA examination experience;
·       Attempts will be made to create a mix of gender, rurality experience, international medical graduate familiarity; and recency of practice of the Censor pair in medical administration.
·       All Censors must declare, to the best of their knowledge any interests with Candidates and/or paired Censors that may prejudice/bias the Censor in the assessment of examination performance – examples of these could include:

o   personal or family relationships

o   previous knowledge of Candidates (such as being on job interview panel, previously working together, direct report etc.) which may lead to perceived bias; and

o   having been Candidates’ Preceptor/Executive Coach or Supervisor.

Note: Candidates are also able to nominate Censors with whom they believe there may be a conflict, for exemption from examination by them.

Observers
Candidates will be advised when registering to sit the exams that there may be approved Observers in or viewing the examination rooms. Candidates and Censors are made aware of cameras viewing and (with permission) videoing the processes streaming to the Control Room at the AMC National Test Centre. 

The Observer may be:

·       Other Censors / Censors-in-training who will observe other Censors’ behaviour, technique etc.;
·       An invited delegate from another College or a regulatory body to observe RACMA processes;
·       Censor in Chief;
·       National Office Staff, or
·       Other person approved by the Censor-in-Chief for the purpose of Examination integrity.

Marking

The Rubric

The generic marking rubric endorsed each year by the Board of Censors has three dimensions – Knowledge, Skills and Attitudes/Approach. Each dimension is scored out of ‘5’ and the scores are summed to give the Censor’s score out of 15. After discussion the Censors’ scores are summed to give the Censor Pair score out of 30 for the station. The pass score for each station is 18/30. Success at the Examination is achieved with passing four stations, hence the minimum score overall is 72/120 (60%).

 
Marking the candidates at each station

The examining Censors (examining pair) independently score and write comments on Candidates’ performance, using the calibrated rubric-based score sheet. This occurs after the Candidate has completed his/her presentation of the case-scenario and has left the examination station.

The Censors moderate the scores between themselves after the original scores are recorded on the score sheet. Each Censor’s moderated score is also recorded (out of 15). Detailed comments are provided by the Censors to assist in Candidate post-examination feedback sessions. The score sheets are given to the relevant personnel for review and recording of paired Censors’ summed scores (out of 30) to determine the outcome of the performance of Candidates.

Immediately after the examining period has ended, the moderated scores (with candidate de-identification) are revealed to the examining panel of censors. Further moderation of scores may be conducted in the event of borderline performance outcomes or if there is a great level of disparity between the examining pair’s scores. 

The Censor-in-Chief (or moderating Censor on the day) declares the outcomes for those who have passed, those who have been unsuccessful and those who are to be offered an additional examination station.  The Censor-in-Chief allows those Candidates whose results have been satisfactory, and those who have been unsuccessful, to leave; and offers the opportunity to sit another station to those Candidates identified as having marginal performances and from whom more assessment information is required.

 
Marginal Candidates

A Candidate has been successful if s/he has a summed score of 18+/30 in four stations.
A Candidate has been unsuccessful if the summed Censors’ score is below 15/30 in two or more stations.


5th Examination station

Candidates who are marginal because their scores are assessed as borderline may be offered an opportunity to present to a 5th examination station, on the same day.  The examination question for the 5th examination station will be of the same standard as other questions in the examination set.

The pair of Censors assigned to the 5th stations will have been assigned to be available for supplementary scenarios. They have been isolated from prior moderation to ensure independence from bias in relation to assessment.  

Conclusion of the Exams

The examination session concludes once all Candidates in that session have been discharged. The Examination concludes once all sessions have been completed.

Examination Results
The Censor-in-Chief prepares a report on the conduct of the Oral Examination for the Education and Training Committee and makes recommendations for the release of results to Candidates at a specially-convened teleconference within 48 hours of the finish of the last session. Results are distributed via electronic notification to Candidates (who have submitted previously their nominated contact details and confirmed email addresses).

Special Consideration

As a principle, Candidates shall not be disadvantaged unnecessarily as a result of events or circumstances outside their control. Once registered, Candidates are sent information about the Examination processes.  They are invited to identify personal issues for which they may seek Special Consideration, and they may indicate if they have any declared interests or perceived conflicts of interest with known examining Censors.

Where an issue arises, which may not be covered in the Regulation for Conduct of Exams or in the Guidelines for Examinations or other policies related to examination processes, candidates may seek Special Consideration for their performance at the Oral Examinations prior to the examination in accordance with the Policy for Special Consideration. This may be related to disability, chronic illness or personal events that may need special circumstances – accommodation for special aids in the examination rooms, extra reading time etc.

They may also apply for Special Consideration after the examination, if on the day, some misadventure arises that they believe may have influenced their performance negatively. Applications for special consideration must be submitted prior to the post-exam feedback sessions scheduled by the College and within the time-frame as specified in the Policy for Special Consideration. Any requests for Special Consideration must be forwarded to the Chief Executive, who will ask the Censor-in-Chief to set up a review panel, comprising members of the Board of Censors, to advise on the application for Special Consideration and the decision making that should ensue.

On the basis of information provided, the following will be included for consideration:

·       The severity of the circumstances and factors involved;
·       The likely impact on the Candidate and his/her performance during the Exam; or
·       History of previous applications for special considerations.

Post-examination Feedback

For those Candidates who are unsuccessful at the Oral Examination, the College will provide a formal verbal feedback session within 3 – 4 weeks of the Oral Examination involving the Censor in Chief on the day, the Candidate, the Candidate’s Preceptor and/or Supervisor (if agreeable to the Candidate) and a member of the Board of Censors who was involved in the examination of that Candidate – particularly in relation to the questions that a Candidate has failed. Candidates who wish to participate in a verbal feedback session are required to contact the College within 1 week of the examination to request this session.


Previous Examination Questions

Examples of previous station scenarios are available to Candidates, Preceptors and Supervisors on the RACMA Website. These examples allow Candidates to familiarise themselves with the scope, format, style and acceptable responses in examination questions. Candidates practising past examination questions with their Preceptors and Supervisors will be able to identify where they may have training or knowledge gaps. Many candidates prepare by participating in Examination Practice learning sets in their jurisdictions.

Bernard Nicholson Prize (Meritorious award)
The Bernard Nicholson Prize is awarded to the Candidate with an outstanding performance in the Oral Examination. The recommended winner is determined at the moderation meeting of the Board of Censors held directly after the set of examination sessions. The awarding of this prize is discretionary and the College may decline to make the award if, in the opinion of the Board of Censors, no Candidate has achieved an outstanding result during the examination.

On the recommendation of the Board of Censors, the Education and Training Committee will endorse the award of the Bernard Nicholson Prize to the Board of RACMA for final approval.

Note: ‘The Bernard Nicholson Prize was donated by and is awarded in memory of Dr Bernard Nicholson. Dr Nicholson was a past President of the College and played a major role in its establishment.’


Appeals

Candidates can request to have a decision by a College Officer/Committee, or in this case, the Board of Censors, reconsidered and reviewed and can follow the College Appeals process in accordance with the Policy for Reconsideration, Review and Appeal of Decisions of the College Officers and Committees.

 
Code of Conduct

All Censors and Candidates sitting the examination are expected to act at all times ethically, responsibly and in the best interest of the College. All Censors and Candidates will adhere to the RACMA Officer’s Code of Conduct.



Related Documents

·       College Handbook
·       RACMA Medical Leadership & Management Curriculum
·       Research Training Program Handbook
·       Terms of Reference for the Board of Censors
·       Terms of Reference for the RACMA Progression Panel
·       Censor Position Description 
·       Position Description of the Jurisdictional Coordinator of Training
·       Rules for the Examination Day
·       Policy for Appointment and Training of Censors
·       RACMA Privacy Policy
·       Policy for Special Consideration for the Oral Examination of Candidates requiring consideration for illness, accident, disability or compassionate grounds
·       Policy for Reconsideration, Review and Appeal of Decisions of the College Committees and Officers
·       Policy for the Requests for Extension of Assessment Tasks in the Fellowship Training Program
·       Policy for the declaration of Conflicts of Interest
 

Last Updated on Tuesday, 30 January 2018 13:26