Using team-based learning to teach evidence-based medicine – University of Sydney

Using team-based learning to teach evidence-based medicine – University of Sydney

Prior to introducing its new research framework in 2020, the University of Sydney Medical School taught evidence-based medicine as a one-week intensive program in Year 1. Its new research framework uses clinical integration of research training, and a student–centred, team-based approach, to maximise student competency. 

Why was the program undertaken?  

Prior to introducing its new research framework in 2020, the University of Sydney Medical School taught evidence-based medicine (EBM) as a one-week intensive program scheduled late in Year 1. Content was delivered through mandatory face-to-face lectures and small group practical workshops. However, feedback suggested that students could not engage with the large amount of content delivered in traditional lectures and, while the critical appraisal workshops were well received, students had not retained content from the lectures and were not ready to apply it. Many students reported that they did not see the practical and clinical relevance of EBM. 

EBM and research methods are core components in medical education curricula globally. Best methods for effective teaching of EBM have not been definitively determined, however two key strategies have emerged for optimising curriculum delivery. First, clinical integration of EBM teaching is likely to bring about changes in student skills, attitudes and behaviour [1]. Second, blended interactive approaches, when compared with traditional didactic methods, have been shown to be more effective in improving student attitudes towards, and increasing self-reported use of, EBM in clinical practice [2]. Additionally, the blended learning approach has associated efficiencies, such as freeing up in-class time for student participation in active learning, and cost-effectiveness of delivery, which leads to improved value for institutions with repeated iterations [3].  

Team-based learning (TBL) has gained popularity in medical education [4], in line with the expansion of student-centred learning strategies, including flipped learning approaches and peer-assisted learning. TBL uses active, collaborative learning strategies to provide an authentic experience of approaching a clinical problem. TBL classes have been shown to provide an enhanced learning environment where students can build on basic knowledge and apply what has been learnt [5, 6]. The classic TBL framework is illustrated in Figure 1. [5] 

Figure 1: TBL framework 

In 2020, The University of Sydney Medical School launched a new curriculum and course structure for its four-year graduate MD program, with Research, Evidence and Informatics established as a vertical theme. The aims of the revised EBM curriculum were to ensure student competency in EBM learning outcomes, emphasise the clinical relevance of EBM, and improve student engagement and application of EBM teaching. The new program shifted EBM teaching from didactic face-to-face lectures to interactive, student-centred and clinically integrated modules.  

We used a modified TBL framework to develop Rapid Evidence for Practice (REP) modules that would provide students with EBM knowledge and skills, and prepare them to apply evidence within short timeframes – simulating real world clinical contexts. 

How was the program implemented?  

In the revised curriculum, EBM teaching is integrated into the Back to Basics (B2B) block, a nine-week term in Year 2, which introduces students to a range of clinical specialties, from paediatrics to aged care, following the human lifespan. This formal coursework program also provides foundational knowledge of research methods prior to students undertaking projects in a dedicated 14-week MD Research Project Block in Year 3.  

EBM content is informed by the Australian Medical Council (AMC) guidelines and a set of core competencies developed for health professionals [7]. The course design allows for a rapid change to online only format, if necessary, due to the uncertainty created by the Covid-19 pandemic regarding face-to-face teaching options. A logic model is used as a guide in developing, implementing and evaluating REP modules (Table 1).  

Table 1: REP modules Logic Model 

Four REP modules were developed, with each module focussing on a specific EBM study design in the following sequence: Randomised Controlled Trials, Systematic Reviews, Observational studies, Diagnostic Test Evaluations. An REP module development guide was created for each module (see Appendix, Table 2), detailing learning objectives, content suggestions, activities and assessment tasks, and ensuring consistency in structure.  

The REP module structure (Figure 2) comprises online informational content, an individual readiness quiz, a forum featuring a guided critical appraisal, and a student team-based rapid appraisal and application of a published article to a clinical scenario.  

Based on our previous experience of students not recognising the clinical importance of EBM and requiring considerable additional support to complete application tasks, we expanded the readiness assurance phase of REP modules to include an additional live forum. The forum features a panel session with real world clinician researcher experiences, a guided critical appraisal task led by our teaching team, and an opportunity for questions and discussion providing students with targeted appropriate knowledge and skills to apply in the subsequent REP activity.  

Figure 2: REP module structure 

In REP activities, students are given a clinical scenario and a relevant journal article, and they work through a series of questions with facilitator clarification of challenging questions. A summary of the session is then provided to the cohort. After the REP activity, students finalise and submit as a team a Decision Map (see Appendix, Figure 3), which illustrates how evidence, clinical knowledge and patient values intersect. General feedback on the Decision Map is provided to the cohort and direct feedback is provided to teams if a rubric element does not meet the expected standard. 

What is the program achieving?  

Evaluation methods 

In May 2021, Year 2 medical students completed a REP module on randomised controlled trials (RCTs). A mixed methods approach to evaluation included:  

1. Curriculum document analysis to assess content coverage of core EBM competencies. 
2. Student acceptance assessed using a questionnaire distributed immediately following the REP activity.  
3. Review of student participation and task completion using learning management system analytics (see Appendix).  

Document review  

Online videos, forum critical appraisal PowerPoint presentations and REP activity tasks were reviewed to determine the extent of their alignment with learning objectives specific to the RCT module and EBM competencies. The document review indicated that learning objectives specific to the RCT module were covered adequately. EBM competencies – including asking a focussed clinical question, applying the PICO framework to research questions, critical appraisal of a journal article and the application of research results to a clinical problem – were met. While acquiring appropriate evidence could be included within the REP module framework, we did not do so, due to time allocation constraints within the Back to Basics Block. Students have a separate lecture and access to library modules on literature searching.  

Student experience of the REP module.  

In total, 55/258 (21%) of participants completed the post-module questionnaire. Student responses to closed items regarding their experience of REP modules are presented in Figure 3 (see Appendix). The overall response was positive, with 81% of students agreeing or strongly agreeing that they were satisfied with this method of teaching. Responses to items 1–2 indicate that the preparation stage worked well, with 91% agreeing that short online videos were their preferred method of acquiring basic knowledge and 83% completing the pre-readings. Responses to items related to the readiness assurance phase were similarly positive, with 79% agreeing that the RQ improved understanding of concepts and 83% agreeing that the critical appraisal walk-through prepared them for the REP activity. Notably, 91% of students agreed that REP helped them develop a way to use evidence to approach clinical questions. 

Students’ responses to open-ended questions are illustrated in Table 3 (what students valued) and Table 4 (what could be improved) in the Appendix. Students valued the overall structure of REP modules due to the staged transition from knowledge acquisition to practice to application: “The stepwise approach to learning. The process has been very methodical and logical”. In addition, each component was valued for its role in the process. Short online videos were perceived as an effective method of delivering basic content: “The online videos are really good, easy to listen to and understand and not too long. They give a really good foundation of concepts that get reiterated in the activities”. The readiness assurance phase was valued as a practice task and REP activities were valued as real-life simulations: “It is good to have the chance to appraise an article and put the theory into practice in a manner very similar to how it would be in real life.” In addition, REP modules were perceived by students as being useful for their future career: “Very useful for me as I engage in the latter years of my training.”  

Suggested improvements included avoidance of repetition of basic content in the readiness assurance phase and changes to the clinician researcher presentations, which some students did not perceive as relevant: “The lectures with research examples do not benefit my learning. I feel like the information they give only apply to that one research project and nothing else”. Students also suggested expanded Q&A sessions, more practice activities, shorter duration of classes (two hours) and avoidance of replication of administrative tasks. 

Task completion  

All student groups submitted their Decision Maps and only one student from the cohort of 258 did not complete the RQ. 

Discussion 

We evaluated the REP module on RCTs embedded in Year 2 based on content, student participation and acceptance. Students found the TBL structure effective for their learning, enabling application of new knowledge to clinically relevant problems that may be encountered in their future careers. Student participation in the RCT REP module activities was high, with 100% completion of the outcome assessments.  

Some challenges we faced related to the introduction of a new teaching intervention, including the need for clarity regarding new terminology and scheduling of tasks. A welcome video explaining REP modules was created, all activities were included in students’ calendars and the assessment page detailed a timetable for the release of learning materials and due dates. There was some duplication of administrative tasks for students, which will be improved. Staff found it challenging to identify journal articles of an appropriate difficulty level for medical students that allowed them to focus on appraisal and application of EBM concepts and skills. Students have different levels of prior experience in critical appraisal, suggesting the need for additional targeted training in future iterations.   

The REP framework has successfully shifted our EBM and research methods teaching from didactic face-to-face lectures to interactive and clinically integrated modules. Students build on basic knowledge with practice activities and gain real-world experience in the rapid appraisal and application of EBM to clinical problems. The REP activity is an authentic task which promotes student collaboration, reflection and communication, with support from expert input. It provides students with the experience of small group learning in a large group setting, reducing the number of teaching sessions and facilitators required. In addition, REP modules can be run completely online, which provided a safeguard during the Covid-19 pandemic. 

References

[1] Khan KS, Coomarasamy A. A hierarchy of effective teaching and learning to acquire competence in evidenced-based medicine. BMC Medical Education. 2006;6(1):59.

[2] Ilic D, Nordin RB, Glasziou P,et al. A randomised controlled trial of a blended learning education intervention for teaching evidence-based medicine. BMC Med Educ. 2015;15:39. 

[3] Maloney S, Nicklen P, Rivers G, et al. A Cost-Effectiveness Analysis of Blended Versus Face-to-Face Delivery of EBMto Medical Students. J Med Internet Res. 2015;17(7):e182. 

[4] Burgess AW, McGregor DM, Mellis CM. Applying established guidelines to team-based learning programs in medical schools: a systematic review. Academic medicine : journal of the Association of American Medical Colleges. 2014;89(4):678-88.

[5] Burgess A, Bleasel J, Haq I, et al. Team-based learning (TBL) in the medical curriculum: better than PBL? BMC Med Educ. 2017;17(1):243.

[6] Burgess A, Haq I, Bleasel J, et al. Team-based learning (TBL): a community of practice. BMC Med Educ. 2019;19(1):369.

[7] Albarqouni L, Hoffmann T, Straus S, et al.Core Competencies in Evidence-Based Practice for Health Professionals: Consensus Statement Based on a Systematic Review and Delphi Survey. JAMA netw. 2018;1(2):e180281. 

APPENDIX 

Table 2: RCT Module development guide 

Figure 3: Decision Map outline

Figure 4: Student experience of the REP module

Table 3: Valued aspects of REP modules

Table 4: Improvements suggested by students for REP modules.

PDF of case studies available here: Research in the Medical Curriculum, Volume 1 – A window on innovation and good practice 2022