Back to Journal

Journal of General Medicine

A Deep Neural Network Model of the Residency Rank Order List with and without Step 1 Scores

[ ISSN : 3068-0840 ]

Abstract Citation Background Methods Results Conclusion Limitations References
Details

Received: 22-Apr-2020

Accepted: 28-May-2020

Published: 01-Jun-2020

Jeffrey A. Summers*

Department of Internal Medicine, Quillen College of Medicine, East Tennessee State University, Johnson City, TN USA

Corresponding Author:

Jeffrey A. Summers, Department of Internal Medicine,

Quillen College of Medicine, East Tennessee State University,

Johnson City, TN USA,

Keywords

USMLE Step 1; Residency; Match Rank Order; Deep neural network

Abstract

Concerns have been raised in the academic community about the impact of the USMLE changing the Step 1 exam to pass/fail. This study used a deep neural network (DNN) to model this year’s rank order list at our program and determine the effect of disregarding the Step 1 scores. Correlation with and without the Step 1 scores was very high suggesting that once an applicant has reached the interview and ranking stage, Step 1 did not have a high impact on the rank order

Citation

Summers JA (2020) A Deep Neural Network Model of the Residency Rank Order List with and without Step 1 Scores. J Gen Med 3: 2.

Background

The United States Medical Licensing Examination (USMLE) Step exams are designed to assess a physician’s abilities to apply knowledge, concepts, and principles important to the provision of patient care [1]. As such, they are used by state medical boards in deciding the disposition of license applications.

These standardized tests have also often been used as part of the resident selection process. The USMLE, and others, have been concerned about the over reliance of programs on tests not intended for this purpose. The USMLE has then recently announced that they will be changing the reporting of Step 1 scores to Pass/Fail.

The question is then whether this will affect the rank order for residency applicants.

Methods

The resident selection committee for Internal Medicine at East Tennessee State University met as usual and selected a rank order for the residents for the incoming class of 2020. This procedure was unchanged from prior years. Information available to the ranking committee included Step 1 and Step 2 scores, interview scores (compiled from the interviews with categories such as fit, academic abilities, letters of reference, etc), scale (from the interviews, range from rank Top 10% to Do Not Rank), sex, medical school, and graduation date. For students who had taken both Comlex and USLME exams, only the USMLE results were considered. For those who only took the Comlex, conversion to a comparable USMLE score was performed [2]. Approximately 1 month later the announcement was made that Step 1 will be moving to Pass/Fail.

A deep neural network was created to model the rank order list. Categorical items such as sex or medical school location were converted to a series of attributes through one hot encoding. This model was trained on the actual rank order list. Using this model a rank order was created that was close to the original. This model was then used to create a rank order eliminating the effect of Step 1 by changing the value of each applicant’s step 1 score to the mean. The results were then compared to the predicted rank list using step 1 scores.

A separate multiple linear regression was performed to estimate the impact of each feature (attribute) on the rank order.

Results

The neural network was trained on the current list and reached a mean accuracy error of 4. Comparing the match order lists with and without Step 1 scores yielded a correlation coefficient of 0.983 with a p value of 1.64x10^-85, showing a very high degree of correlation between the lists with and without Step 1.

The linear regression showed the top four attributes by impact on the rank order were Scale (estimated rank by interviewers), Interview Score, Step 2 score, and Graduation Year (preference given to more recent graduates). Step 1 score had the lowest independent impact of any of the 15 attributes.

Conclusion

Based on the model results, it appears that USMLE Step 1 scores do not have a significant effect on the rank order of applicants to our Internal Medicine categorical residency program. Other factors appear to have a greater impact.

Limitations

Limitations to this study are numerous. The model applies to a single year in a single Internal Medicine program. Before a person can be ranked, they must be interviewed, and the decision to interview likely includes Step 1 scores. Interviewers also are aware of Step 1 scores and this may influence the Interview Score and Scale scores.

References

1. United States Medical Licensing Examination ®

2. Slocum PC, Louder JS How to predict USMLE scores from COMLEX USA scores: a guide for directors of ACGME-accredited residency programs. J Am Osteopath Assoc 2006 Sep; 106(9): 568-569.

Other Articles

Article Image 1

Screening for Depression, Anxiety and Stress among Patients Attending a Regional Rehabilitation Clinic in South Western, New South Wales, Australia

Rehabilitation medicine offers an individualized patient-centered service to optimize function and maximize quality of life of patients. Psychological distress can impact rehabilitation process and its outcomes [1]. In regional and rural areas, rehabilitation services are different from the ones in metropolitan areas. There are limited medical specialists and allied health services in those areas.

Sacred Heart Rehabilitation Service at St Vincent’s Hospital in Sydney Australia has been providing outreach rehabilitation services to Griffith Base Hospital (GBH), New South Wales (570 km away from Sydney). A rehabilitation clinic at GBH run by an outreach rehabilitation physician is well established with approximately 100 new referrals per year [2]. Although rehabilitation patients often have psychosocial issues requiring multidisciplinary input, there is limited study on the presence and extent of depression, anxiety and stress symptoms among patients attending an outreach rehabilitation clinic. This audit study aimed to screen the levels of negative emotional status who attended a regional rehabilitation clinic.

Yuriko Watanabe*


Article Image 1

Using Technology to Improve Adherence to HIV Medications in Transitional Age Youth: Research Reviewed, Methods Tried, Lessons Learned

In transitional age youth living with HIV or AIDS, non-adherence (<80%) to anti-retroviral medication is associated with viral resistance, disease progression, and an increased risk of death. This feasibility study investigated the Maya MedMinder electronic pillbox and cell phone texting with personalized motivational interviewing strategies to improve medication adherence in non-adherent youth. Twenty patients out of 30 identified as non-adherent by the Pediatric HIV team at the Medical University of South Carolina were approached, and 15 were recruited (Ages 12 to 20; 13.3% male, 86.7% female; 100% African-American). Following baseline MedMinder monitoring, subjects were randomized to intervention groups with reminder signals on or off. The time medications were taken was collected by the MedMinder, resulting in adherence scores. All were interviewed for readiness to change utilizing the Motivational Interviewing (MI) Stages of Change scores. Viral load and CD4 labs were scheduled every 6 weeks. Despite monetary incentives and personalized support, recruitment and adherence to the protocol was a challenge. Only 6/15 subjects completed the entire study scheduled for 6 months .Stages of change scores revealed that those that transitioned to making changes had higher CD4 percentages midway through the study. Challenges included missed appointments and labs despite efforts by text and phone to schedule convenient appointment times with participants. Device challenges included the large size of the MedMinder and faulty electronic signaling, especially from rural areas. The methodology was feasible with these patients. This small feasibility study highlights that technological tools to promote adherence and motivational enhancement strategies in teens and young adults who are non-adherent to HIV medication regimens can enhance biomarker outcomes associated with medication adherence.

 

Spratt ES1 , Papa CE1 , Mueller M2 , Patel S3 , Killeen T4 , Maher E5 , Drayton C1 , Dixon

TC1 , Fowler SL1 and Treiber F2


Article Image 1

Indications and Clinical Utility of Sternal Aspirates in Patients with Multiple Myeloma and Other Plasma Cell Dyscrasias

Background: A Bone Marrow (BM) evaluation is often required in patients with plasma cell dyscrasias,at the time of initial presentation for diagnostic purposes and risk stratification, and during the follow up for an accurate assessment of the response to therapy. In the vast majority of cases, the preferred site for obtaining a BM specimen is the posterior superior iliac crest. The role of sternal aspirates has fallen out of favor in 21st century medical practice. However, in certain clinical situations it appears to be the easiest site for specimen collection. Our study was designed to answer a set of basic clinical questions such as whether sternal Bone Marrow Aspirate (BMA) can provide reliable and sufficient specimen for morphologic, immunophenotypic and molecular evaluation of patients with clinical suspicion of plasma cell dyscrasias.

Methods: We reviewed indications, performed BM biopsies and obtained BMA from sternum in 51 patients with Multiple Myeloma (MM) and other plasma cell dyscrasias.

Results: No significant complications were observed. The most common indication for the sternal aspirates were: inability to reach the pelvic bone due to morbid obesity (65% of cases), followed by other factors, such as tetraplegia/immobility, pelvic fractures, infections, or radiotherapy. The concordance with the disease status, as defined by the presence or absence of a detectable paraprotein, was excellent, observed in 91.7% of samples.

Conclusions: Sternal aspirates provided satisfactory samples not only for morphologic evaluation, but also for ancillary studies, such as flow cytometry, metaphase cytogenetics, and Fluorescence In Situ Hybridization (FISH) studies.

Jozef Malysz1*, Nicole Leeper2 , Cinda M Boyer3 , Joseph J Drabick4 and Giampaolo Talamo5


Article Image 1

Is There a Correlation between Sleep Disordered Breathing and Maxillary Expansion? A Retrospective Study Based on Cephalometric Assessment and Questionnaires

Aim: This study aimed to document the effect of Maxillary Expansion (ME) during childhood on Sleep Disordered Breathing (SDB) symptoms in adults. The secondary aim is to try to find a screening tool for daily use in the orthodontic/dental office in children who are not yet diagnosed with OSA. We try to develop a tool that could help us in deciding which children should be referred for OSA screening, possibly including polysomnography, based on the cephalometric radiograph and the symptoms they report.

Methods: This is a retrospective study (S) focusing on cephalometric measurements performed on 27 Children (C), which had received maxillary expansion (RCS group) and as Adults (A) attended a post-treatment follow-up on average 21.1 (±7.24) years later (RAS group). A cephalometric radiograph before treatment and a cephalometric radiograph at post-treatment follow-up were traced. These were compared with untreated control (Co) groups of 50 subjects each (RCCo group and RACo group). Questionnaires related to SDB symptoms were administered in the RAS and RACo groups.

Results: Small changes in cephalometric measurements were seen comparing patients with (RCS group and RAS group) and without (RCCo group and RACo group) maxillary expansion. Questionnaires were answered similarly by the study (RAS group) and control group (RACo group).

Conclusions: Small cephalometric changes were seen between groups.

Current knowledge: The favorable effect of expansion of the maxilla on SDB symptoms has been demonstrated in several studies, the aim of this research was to document if patients treated with maxillary expansion during childhood can benefit from this intervention concerning Sleep Disordered Breathing (SDB) at adult age.

Study impact: Maxillary expansion during childhood might improve SDB symptoms at adult age.

Detailleur Valentine1 , Van Dyck Julie1 , Cadenas de Llano-Pérula Maria1 , Buyse

Bertien2 , Fieuws Steffen3 , Verdonck Anna1 , Politis Constantinus4 and Willems

Guy1*


Article Image 1

What’s Love Got to Do with It? The Relationship of Marriage to Health

Background: Traditional thinking and scholarship has indicated that marriage is a life saver - extends life. Marriage’s functionality contributes to the reduction of poor physiological health outcomes. Since women are not homogeneous in making marital decisions or social experiences, it was time to revisit the issue of the relationship of marriage and health.

Methods: From the 2015 National Health interview survey, we extracted a sample of women who were a parent of one or more minor children (n=4,899); experienced psychological distress and chronic conditions; by marital status.

Significant Data and Major Findings: The overall prevalence of psychological distress was 3.5%. Women with disruptive marriage had double (OR=2.18, 95% CI=1.24, 3.86) the likelihood of having psychological distress compared to married women, adjusting for socio-demographics (age, race/ethnicity, work status, family income, number of children and number of elderly in the household).There was significant interaction effect of marital status and race/ethnicity on the risks of having psychological distress. The difference between marriedwomen and those with disrupted marriage in the risk of having psychological distress was greater among Whites than that for African Americans and Latinas. Although 34.3% of the women had at least one chronic condition, there was no significant association between marital status and the likelihood of having chronic conditions after adjusting for socio-demographics.

Conclusion: Our findings indicate that White women experience the most psychological distress when their marriages are disrupted. On the contrary, African American and Hispanic women fared better psychologically when they experienced disrupted marriages, although they reported more socioeconomic hardships. More research is needed on disrupted marriages and women’s health as well as the role of reliance.

Alai Tan1 , Timiya S Nolan2 , Darryl B Hood3 and Karen Patricia Williams4*


Article Image 1

Cupping Glass Massage and Acupuncture for Chronic Low Back Pain - A Randomized Non-Inferiority Trial with Female Inpatients in Naturopathy

Study Background: The efficacy of Cupping Glass Massage (CGM) in patients with back pain has not yet been sufficiently proven [1,2]. In view of the increasing incidence and high prevalence of this disease, research into treatment options is of great importance. In the Clinic for True Naturopathy in Hattingen, Germany, cupping glass massage is subjectively successfully applied in patients with back pain. A randomized, controlled non-inferiority study was conducted to objectify the treatment successes.

Methods: The efficacy of CGM (n = 66) was compared with acupuncture therapy (ACU, n = 70) in in-patients with chronic non-specific low back pain. Primary objective was the non-inferiority of CGM compared to ACU with regard to functional ability in everyday life, operationalized by the Hannover-Functional-Ability-Questionnaire (HFAQ).

Results: In the per-protocol-Analysis the CGM responder-rate of 71,4 % is significantly higher than the ACU of 44,4 % (? = 27%; 95% : 7,3-46, 6%; p = 0,008). In the Intention-to-Treat analysis CGM is not inferior to ACU.

Conclusion: Results show that CGM is at least not inferior to ACU.

André-Michael Beer1 , Gordon Röser2 and Karl Rüdiger Wiebelitz3*


Article Image 1

Incidence of Cardiovascular Diseases in Type 2 Diabetes Mellitus Patients

Type 2 Diabetes Mellitus is a medical condition characterized by an elevation of blood glucose level, this metabolic disorder will taken place as a result of either insulin resistance and/or insulin deficiency. It is the most prevalent chronic metabolic disorder worldwide. Type 2 Diabetes Mellitus is the significant cause of premature morbidity and mortality imposing enormous socioeconomic burden globally. As per the current prevalence and trend of T2DM, International Diabetes Federation (IDF) predicted 592 million people will have T2DM by 2035 worldwide. Prevalence of T2DM is escalating at rapid pace in India due to westernization of lifestyle. As per IDF report, the prevalence of T2DM will increase to 101.2 million by 2030 among Indians. Type 2 diabetes mellitus is typically a chronic disease associated with a ten-year-shorter life expectancy. This is partly due to a number of complications with which it is associated, including two to four times the risk of cardiovascular disease, including ischemic heart disease and stroke; a 20-fold increase in lower limb amputations, and increased rates of hospitalizations. In the developed world, and increasingly elsewhere, type 2 diabetes mellitus is the largest cause of non-traumatic blindness and kidney failure. It has also been associated with an increased risk of cognitive dysfunction and dementia through disease processes such as Alzheimer’s disease and vascular dementia. The contemporary associations of type 2 diabetes mellitus with a wide range of incident cardiovascular diseases have been compared in this study. Results showed that Type 2 diabetes mellitus was positively associated with peripheral arterial disease, ischaemic stroke, heart failure, and non-fatal myocardial infarction, but was inversely associated with abdominal aortic aneurysm and subarachnoid haemorrhage, and not associated with arrhythmia or sudden cardiac death. Type 2 DM is a metabolic disease that can be prevented through lifestyle modification, diet control, and control of overweight and obesity. Novel drugs are being developed, yet no cure is available in sight for the disease, despite new insight into the pathophysiology of the disease. Management should be tailored to improve the quality of life of individuals with type 2 DM.

Ather Pasha and Rindha Venepally*


Article Image 1

Community Intervention- Teaching Cardiopulmonary Resuscitation in Two Schools in Madrid

Background: Coronary heart disease is the most important cause of death in the world. In Europe, cardiovascular disease represent 40% of total deaths among people aged less than 75 years and sudden cardiac arrest 60% of death in adults with coronary heart disease. Immediate cardiopulmonary resuscitation can double or even triple the survival of cardiac arrest.

Objectives: The main objective of the study was to increase knowledge of first aid among school students. This study also aimed to establish how much influence has variables like sex, parents’ educational background, social and economical factors over learning.

Methods: Two schools, one public in a disadvantaged neighborhood and one private in one of the richest areas of Madrid, Spain were selected. CPR training consisted of theoretical lesson followed by practice on manikins. Multiple choice questionnaires were provided before and after the training. The results were processed using central and dispersion-tendency statistics.

Results: In total, 85 school students aged between 14 and 19 year-old completed the training. Only 10.6 % of the students received previous training. Pre-test score was higher among public school students, but post-test evaluation showed better results among private school students. The parent’s educational background didn’t influence the outcomes.

Cristina Sicorschi Gutu*, Maria Jose Alarcon Gallardo and Marisela Roure Vasquez


Article Image 1

Quality Indicators in Home-Based Care: A Systematic Review

Introduction: Even though many quality indicators of health care have been announced, those which specify in home care services are limited in that none of them describe the stroke patients’ condition.

Aim: To determine through systematic literature review what methods can be used to assess the quality of home care that patients received and to identify what components can be used as a determinant of the quality of home care services.

Methodology: Google Scholar, EBSCO, ProQuest and PubMed database websites were searched for articles and information.

Results: The method that was used is a qualitative study using a literature review with quantitative analysis of a previously accepted research instrument with a questionnaire that has been widely available and considered reliable. The researchers identified thematic differentiation in grouping the quality indicators used by those articles. The first article stated 23 quality indicators, which are distributed as the following: functional (n=8), clinical (n=10), social and treatment (n=5). The second article discussed two groups of quality indicators based on 21 items: prevalence (n=15), and incidence (n=6), while the last article mentioned 16 quality indicators without any category. Overall quality indicators in home care that are used by the three articles are based on the Home Care Quality Indicators Instrument (HCQIs).

Conclusion: Several studies discussed home care quality indicators but no articles specifically analyze home care provision for stroke patients. Further research is needed to clarify the components indicators for stroke patients and more importantly, these indicators should be valid, and reliable.

Nur Chayati M Kep1,2*, Christantie Effendy3 and Ismail Setyopranoto4


Article Image 1

Intractable Chronic Migraine in Adolescent: Multidisciplinary Approach

Chronic migraine is a severe neurological disorder characterized by the presence of headache for 15 or more days/ month, for more than three months. Pain, on at least eight days/month, has the features of migraine. Pain is often intense, disabling and resistant to the usual treatments. Other disorders such as phonophotophobia, nausea-vomiting, diarrhea, sleep and mood disorders can be found in combination with chronic migraine pain. The long-lasting migraine pain may be favored by the presence of factors such as hormonal changes in the menstrual period, or states of anxiety, stress, mood deflection, or overuse of symptomatic drugs with rebound effect.

We report the case of a 14 year-old female patient, with positive familiarity for migraine, which was brought to our observation for the presence of chronic headache with daily frequency migraine-like attacks, highly disabling and resistant to pharmacotherapy. During the hospitalization, a wash-out of the pharmacotherapy was performed, associated with the autogenous training, muscular relaxation exercises, psychological support and introduction of Lamotrigine for prophylactic therapy. Our patient showed a considerable amelioration with this multidisciplinary approach.

Luca Maria Messina1,2*, Luigi Vetri1,2, Lucia Rocchitelli1,2, Flavia Drago1,2, Laura Silvestri1,2, Antonina D’Amico1,2, Giovanni Grillo1,2, Francesca Vanadia2 , Vincenzo Raieli2*