"Variability in Data Presentation in the Medical Student Performance Evaluation (MSPE)" and "A Generative Case System That Reflects Our Patient Population" - September 29

12:00-12:30pm ET - Variability in Data Presentation in the Medical Student Performance Evaluation (MSPE)

Every year, medical schools across the country spend a significant amount of time composing the Medical Student Performance Evaluation (MSPE) for graduating students. The MSPE contains data on student performance across the first three years and at times, the early fourth year of medical education. It is one of many important items residency program directors use when selecting incoming interns. 

Over the past few years, prompted by the 2016 recommendations of the AAMC MSPE Task Force, the medical education community has witnessed a change in the content of the MSPE. As reported by Giang, the majority of US medical schools have adopted the task force guidelines. While this adoption is occurring, wide variability still exists in the actual content and display of data within the MSPE. 

To better understand the variability we have created a 23-item data abstraction tool to code key MSPE data components. We have already used this tool to evaluate MSPEs written for the 2019 graduates of over 95% of US medical schools. We are now in the process of continuing this analysis for the 2020 graduates. We will present the results of the coding process, focusing on the variations in data presentation in the following areas: 

  1. Clerkship grades, including comparative performance;
  2. Overall comparative performance, including summary adjectives; and
  3. Competency-level performance data. 

12:30-1:00pm ET - A Generative Case System That Reflects Our Patient Population

After cataloging the patient demographics in our cardiac and respiratory pre-clerkship course, we recognized a bias towards male patients, and an absence of ethnic descriptors unless clinically relevant. Utilizing a patient case generation system in use at Cumming School of Medicine, we  further developed a mechanism that generated pulmonary function tests that were based on patient height, weight and gender. This allowed us to present students with a limitless supply of pulmonary function test results across a patient population that was reflective of Statscan data for the Canadian population. This intervention has generated 2316 patient cases and is open access.

This system, cards, in addition to the above material has now generated over 1 million cases. By using a system to manage these variables whenever they are not clinically relevant, we are able to tackle a long standing issue in our curriculum.

This webinar will discuss how we recognized the bias in our patient case content, and recognize how components of patient case content that can be freed of inherent gender and ethnicity bias. Attendees will be able to see a mechanism for generating patient case content that is based on population statistics. 

“MedBiquitous Community Connection” is a series of free one-hour webinars featuring health professions educators and technology innovators from around the globe. “MedBiquitous Community Connection” provides a forum for community experts to share how digital technologies are promoting improvement and better outcomes across the continuum of health professions education. A portion of each webinar will reserve time for a brief Q&A session, allowing attendees the opportunity to dialogue with guest speakers.

Judith Brenner, MD

Associate Dean of Curricular Integration and Assessment, Donald and Barbara Zucker SOM at Hofstra/Northwell

Judith Brenner, MD is an internist and the Associate Dean for Curricular Integration and Assessment at the Zucker School of Medicine at Hofstra/Northwell in New York where she oversees the development and implementation of a rigorous assessment and evaluation system throughout the educational program. She is responsible for the collection and visualization of data relevant to the mission of the school. This includes data used to improve the learning environment, address compliance and regulatory issues, support continuous quality improvement, and promote educational scholarship. She is interested in identifying best practices in assessment in undergraduate medical education and in identifying students at risk for poor performance with the hope of defining prevention strategies. The MSPE is a particular area of interest because of its role in communicating assessment data across the continuum of medical education.

Karen Friedman, MD, MS-HPPL, FACP

Vice Chair for Education, Professor of Medicine, Donald and Barbara Zucker SOM at Hofstra/Northwell

Dr. Karen Friedman is the Vice Chair for Education and Residency Program Director for the Zucker SOM program in Internal Medicine at North Shore University Hospital and Long Island Jewish Hospital. She received her MD degree from SUNY Downstate and her Masters in Health Professions Pedagogy and Leadership from Hofstra University. She has been on faculty in the Department of Medicine at North Shore Hospital as a Hospitalist since 2005 and is a Professor of Medicine. Dr. Friedman’s career and research has been dedicated to medical education. She has published in and presents nationally on novel residency curricula and assessment methods in competency based medical education. She has studied both the use of milestones to assess trainees competency as well as the use of Dashboards in medical educations. She is the author of The Informed Patient- published in 2017 by Cornell University Press. 

Jason Brenner

Research Associate and College Student, University of Michigan

Jason Brenner is a rising senior pre-medical student at the University of Michigan (UM) and a volunteer research assistant at the Zucker SOM. He is studying Biopsychology, Cognition, and Neuroscience at UM and has a minor in Business from the Ross School of Business. Jason has been involved with research at UM since his freshman year and is excited to contribute to this project with his efforts in data abstraction and analysis.

Mike Paget

Manager, Academic Technologies, University of Calgary, Cumming School of Medicine

Mike Paget is the manager of the Academic Technologies team at the Cumming School of Medicine. As a researcher, his focus is on clinical problem solving, assessment and cognitive load management. The Academic Technologies team focuses on system interventions that intersect student, faculty and staff needs specific to the MD program.

Scott Steil

Developer, University of Calgary, Cumming School of Medicine

Scott Steil has long been a champion of online software. After getting his degree in Computer Science, he focused his career working with web technologies believing its accessible, distributed, and inter-connected nature would be foundational in all bleeding-edge software. As a lead developer on the Academic Technologies team at the Cumming School of Medicine, he is proud to bring this passion advancing meaningful interventions to medical education.

Components visible upon registration.