Based around a core curricula, every medical school will run their medical program slightly differently recognising their individual context, characteristics, student entry point and criteria, and teaching methods.
There is therefore a need to enable schools to assess the performance of their students and graduates relative to other schools to ensure they remain at the required high standard and to enable them to evaluate changes to their student selection criteria, curricula and teaching methods.
Assessment benchmarking is an initiative run by Medical Deans in collaboration with the Australian Medical Council (AMC) that is underway in 19 of our medical schools to run coordinated benchmarking tests.
It involves utilising the AMC’s item bank of over 6,000+ scored multiple choice items covering the major medical disciplines, each fully calibrated with supporting Item Response Theory (IRT) data. Participating medical schools are provided access to a secure AMC portal to view and select a number of calibrated multiple choice items to be included in their school examinations. Data is then collated and analysed, and each medical school receives a de-identified confidential benchmarking report on how their graduates are performing relative to other schools.
Feedback is also provided to the AMC on the level of item difficulty that supports any necessary re-calibration of the standard of their multiple choice examinations to Australian and New Zealand medical graduate standards.
The assessment of the 2014 pilot project methodology indicated that it was possible to equate performance to a common standard using the IRT calibrated questions – this is being further tested in the 2018 project.
Medical Deans Clinical Discipline Benchmarking Project
In 2014, the Medical Deans lead a pilot project (funded by HWA) to develop a national approach to demonstrate quality graduate outcomes, with the specific aim of supporting medical schools in benchmarking their graduates’ performance in specialty-specific areas relative to other medical schools across Australia and New Zealand.
After the completion of the funded pilot project, the Medical Deans continued the work, in the following disciplines:
- 2018 – Obstetrics & Gynaecology
- 2017 – Psychiatry
- 2016 – Surgery
- 2015 – Paediatrics
- 2014 – Internal Medicine
For more details on this work or if you have any queries, please contact the Medical Deans’ office on firstname.lastname@example.org