Research Methods in Medicine

Research Methods in Medicine

What was the main aim of this project/program?

In 2020, Monash University graduated its first cohort of approximately 500 Doctor of Medicine (MD) students across Monash Australian and Malaysian campuses. A key element of transitioning from a Bachelor of Medicine, Bachelor of Surgery (MBBS(Hons)) degree to a vertical double Bachelor of Medical Science/MD degree was the introduction of the Year 3/B module called Research Methods in Medicine (RMM).  

RMM aims to enhance students’ research knowledge and skills, specifically in biostatistics and research methods. It teaches complex, high-level statistical knowledge, building on foundational knowledge gained from Years 1 and 2, whilst integrating with the current Year 3 and 4 evidenced-based medicine clinical practice content, and preparing students for their Year 5 Scholarly Intensive Placement (SIP) projects.  

A number of challenges were identified during the design and development stage of RMM: 

• Starting in 2018, there was an annual cohort of approximately 500 students  
• Passing this unit was a hurdle requirement for entry into 5th year medicine for all students 
• Authentic assessment and comprehensive assessment strategies were required for failing students 
• It had to be an accessible and consistent learning experience for metropolitan, rural and Malaysian student cohorts 
• It had to be flexible, with students taught whilst on clinical rotations at various sites 
• It had to demonstrate to the Australian Medical Council (AMC) how this work supported the transition for AQF 9 [1].

To address these challenges, RMM was designed using connectivism and a cognitive constructivist approach.  

How was the program implemented?

RMM was the first completely online component in the Monash medical degree, comprising an 11-week program with six modules developed as interactive e-books.  

For the most part students in RMM work on their projects autonomously, with the project tasks building on the learning modules. However, as students are randomly allocated to unique datasets, they can connect online and support each other through the scientific process, whilst not having the same answer. Further, the online discussion boards are structured with weekly prompts that lead directly into the weekly learning objectives and explore key topics. This encourages students to connect online and has stimulated opportunities for robust discussion (e.g. a discussion of the variable gender). 

Beginning in mid-February, all Year 3B medical students are enrolled into the unit and introduced to the program via a series of introductory videos aimed at putting the unit into context with the rest of their learning.  Students are allocated two self-directed hours per week to complete a module and an additional two hours per week to work on the corresponding section of the assessment. 

The six modules are: 1) The role of statistics in health, 2) Graphical and chart representation, 3) Quantifying uncertainty in data analysis, 4) Evaluation of statistical significance 5) Additional statistical methods: chi-square & non-parametric methods 6) Introduction to qualitative methods.  

These modules have been designed as bespoke activities and are structured so that each module starts with a learning objective which directly links to the Check Your Learning activity at the end. Each module is further supported with explanations and worked examples that are relevant to clinical practice and research. Although the examples differ, there are some key concepts that link the modules and provide an opportunity for cascading depth learning activities. In these instances, a worked example with a concurrent student activity will bridge multiple modules to scaffold the learning and build on previous concepts. For example, the same activity can use the same data variables where they are defined, graphed, tabled and then analysed.  

Other innovative elements of learning resources are the bespoke educational videos with H5P questions embedded, hands on application with the drawing of graphs, gamification of key concepts in form of consolidation crosswords, activities that require application and data manipulation, and interactive infographics. Student feedback of the resource has been overwhelming positive but also informative, as it has allowed for the program’s continued improvement. A snapshot of student feedback highlights how the design of RMM has enhanced student learning: 

‘The research methods in medicine content was really clearly presented and comprehensive. This definitely helped fill in those holes and aided me in learning and understanding a lot more of this content. Thanks for delivering this module helping us all get our head around the stats!’  

 ‘was very easy to use. The questions following each section allowed for excellent content revision and assisted with information retention’  

‘Helpful in consolidating concepts. Life-saver in understanding tricky concepts especially in an online learning environment where it’s more difficult to reach out for assistance. Enjoyed the extra questions to reinforce concepts!’ 

One of the prized outcomes of RMM is that this student learning experience was scalable, robust, applicable and flexible in nature. These characteristics allowed the program to be adapted to other programs, such as being incorporated into the coursework of the honours year of Bachelor of Biomedical Science (BMS). Further, the COVID-19 pandemic saw a dramatic increased usage of these syllabi into a number of other units at both a postgraduate and undergraduate level and across the Faculty including BMS, Bachelor of Health Sciences, Bachelor of Public Health, other years of the MD, Master of Public Health and Bachelor of Physiotherapy.  

This is evidenced by the data analytics and increased uptake of all modules since 2021, with the ebooks increasing in student views by 48-58% (see Table 1). As an example, The role of statistics in health was viewed by 221,649 students from the 2017 pilot phase to December 2020. The learning analytics show students returning to the materials throughout their studies and to support their learning throughout their MD and other studies.

Table 1. Research methods in medicine data analytics

This increase in usage is consistent with the positive feedback received from colleagues who have eagerly adopted these resources into their programs: 

‘I wanted to get in touch as I am using your modules from “Research Methods in Medicine” in the X this year to support the student’s understanding of biostatistics. The modules themselves look really clear and engaging and I am particularly impressed by the functionality to display the content in a dyslexia friendly font. Thank you for making this resource and I will let you know how the students go as they work through the modules this semester.’  

Assessment – Application, interpretation and working with data

The RMM assessment strategy was aligned and developed alongside the learning resources, allowing students to work on it progressively as they completed each module.  

The major assessment task is called The Sunnyville Report, which consists of 43 questions that require students to use the different skills they have developed, such as data description, an understanding of sampling, simple to complex calculations, graphing, use of statistical software and interpretation of the findings, all built into an online quiz platform to allow for automatic marking (see Appendix 1: Example of assessment questions).  

Sections of the report are complete with bespoke videos that link back to the learning resource as a lesson plan, and the questions directly link to the learning objectives of the modules. The learning outcomes of the assessment ensure students will be able to: 

• demonstrate an ability to present data clearly and interpret its results 
• conduct and show workings, where appropriate, for various statistical tests 
• describe the epidemiological foundations for obtaining the data 

• generate and interpret results based on various statistical tests and public health tasks. 

Students are randomly allocated to one of ten different datasets but have the same question set. Different datasets ensure that students need to be confident when choosing, creating/applying and interpreting their results. It also allows the students to support each other, as can been seen with high-level questions being asked on the discussion boards on the Moodle site.  

The Sunnyville Report is a form of experiential learning, as gamification is used with a strong narrative to ensure that the research task is exciting, engaging and is a problem that must solved. The assignment takes approximately 10-12 hours to complete and is approximately 2,500-word equivalent in length, with a focus on data analysis, calculations and reporting of findings. It is designed in a manner that allows students to receive their overall mark and feedback on each question, as per the marking criteria provided on the learning management system Moodle. 

How has the impact of this program been evaluated?

RMM was supported by the Faculty of Medicine, Nursing and Health Sciences and received funding through a competitive internal learning and teaching grant entitled Evaluating the development of research knowledge and skills.  

The aim of this project was to 1) develop the content material and then 2) evaluate the RMM program by assessing research knowledge and skills among the new MD student cohort. Students were asked to complete a before and after questionnaire with thirteen sections covering domains beyond the RMM program, to identify gaps in student learning. The research team is currently validating the questionnaire, however preliminary results have shown that skills in research methods require specific learning, with students benefiting from the structure of the program and activities of the RMM. 

RMM is well suited for the asynchronous learning environment, as it is fully online with a quiz-based assessment which is feasible for large student cohorts in different locations while completing their clinical placements. Importantly, student engagement with the module has been demonstrated by students returning to the resources throughout their studies. 

APPENDIX

Appendix 1. Example of assessment questions

Reference

[1] Australian Qualifications Framework Council, Australian Qualifications Framework, 2nd Edition, 2013. p. 59.