Advertisement

Drivers of Application Inflation: A National Survey of Internal Medicine Residents

Published:January 17, 2018DOI:https://doi.org/10.1016/j.amjmed.2018.01.002

      Keywords

      Perspectives Viewpoints
      • Residents identified several factors that influenced the number of applications submitted.
      • Advice from peers and near-peers was noted to be more influential than advice from faculty.
      • Fellowship placement and average USMLE Step 1 scores were identified as factors that could potentially mitigate application inflation.
      • Significant differences were noted between the responses of US graduates and international medical graduates.
      • Solutions to application inflation must consider the heterogeneous applicant pool.

      Introduction

      Application inflation describes the phenomenon of increasing applications to residency programs despite the lack of evidence that this practice confers a clear advantage for securing a residency position.
      • Association of American Medical Colleges, Careers in Medicine
      Apply smart in internal medicine.
      • Gliatto P.
      • Karani R.
      Viewpoint from 2 undergraduate medical education deans—the residency application process: working well, needs fixing, or broken beyond repair?.
      • Weissbart S.J.
      • Kim S.J.
      • Feinn R.S.
      • Stock J.A.
      Relationship between the number of residency applications and the yearly match rate: time to start thinking about an application limit?.
      Although some specialties such as family medicine and psychiatry are experiencing a greater percentage increase in applications submitted per applicant than internal medicine, internal medicine programs continue to receive the largest total number of applications.
      • Association of American Medical Colleges
      Electronic Residency Application Service data tables.
      During the 2017 recruitment season, preliminary data tables show that internal medicine categoric programs, on average, received 595 applications from US and Canadian graduates and 2581 applications from international medical graduates (IMGs), a 19% increase in total from 2013.
      • Association of American Medical Colleges
      Electronic Residency Application Service data tables.
      • National Resident Matching Program
      Results and Data: 2017 Main Residency Match®.
      Despite a steady increase in the number of residency applications, the Match rate for US senior applicants who rank any internal medicine program first on their rank order list (internal medicine preferring) has remained stable: between 96.5% and 98.6% from 1992 to 2016 (Figure 1).
      • National Resident Matching Program
      Results and Data: 2017 Main Residency Match®.
      Furthermore, these rates from US seniors and from other applicant groups likely underestimate final placement rates because they do not include Supplemental Offer and Acceptance Program and other post-Match activity statistics. In light of these stable Match results, factors unrelated to the Match appear to be driving application inflation, particularly for internal medicine–preferring US seniors.
      Figure 1
      Figure 1*Previous graduates of US medical schools, students/graduates of Canadian medical schools, and Fifth Pathway graduates. IM = internal medicine; IMG = international medical graduate; NRMP = National Resident Matching Program.
      Internal medicine–preferring US citizens who graduate from international medical schools and “other” applicant types—including previous graduates of US or Canadian medical schools and Fifth Pathway applicants—also have stable match rates (Figure 1). From 2013 to 2016, the Match rate for US citizens who graduate from international medical schools has varied between 50.4% and 52.2% and for “other” applicants between 74.9% and 79.7%. The match rates for internal medicine–preferring non–US citizen IMGs have also been stable from 2013 to 2016 (47.1%-53.2%), coinciding with the introduction of the National Resident Matching Program all-in policy.
      • National Resident Matching Program
      Results and Data: 2017 Main Residency Match®.
      • National Resident Matching Program
      All-In Policy.
      Despite these stable—even favorable—odds of matching, the number of applications has continued to climb each year since at least the 2012 Electronic Residency Application Service (ERAS) cycle. The average number of applications submitted by each US or Canadian medical school graduate (USMG) in categoric internal medicine has increased from 23.4 to 35.5 (51.7%) from 2012 to 2017.
      • Association of American Medical Colleges
      Electronic Residency Application Service data tables.
      Although IMGs generally submit a higher number of applications on average than USMGs, the percentage increase in number of applications submitted by IMGs, although still substantial, is less pronounced during the same period (from 60.0 to 77.7 or 28.3%). Even highly competitive applicants, such as Alpha Omega Alpha honor society members, have increased their average number of applications considerably (43.65% from 2012-2017), consistent with overall trends.
      • Association of American Medical Colleges
      Electronic Residency Application Service data tables.
      The discrepancy between Match outcomes and application patterns has led to numerous hypotheses for application inflation both at the level of the individual (eg, fear of not matching, influence of faculty, and peer advice) and the system level (eg, lack of tools to assess competitiveness for a given program).
      • Pereira A.G.
      • Appelbaum J.S.
      • Chheda S.G.
      • et al.
      Application inflation for internal medicine applicants in the match: drivers, consequences, and potential solutions.
      • Katsufrakis P.J.
      • Uhler T.A.
      • Jones L.D.
      The residency application process: pursuing improved outcomes through better understanding of the issues.
      To our knowledge, there has been no national survey of internal medicine residents to identify what influenced their decisions to submit an increasing number of applications. The aim of our study was to explore drivers of application inflation and to gather feedback on potential approaches to mitigating application inflation by surveying current internal medicine residents.

      Methods

      The authors developed questions about application inflation for inclusion on the resident survey that accompanied the 2016 Internal Medicine In-Training Examination (IM-ITE). Questions were aimed at ascertaining the number of programs applied to, sources of information that most influenced that number, and the most helpful information that would have retrospectively decreased the number of applications submitted. Feedback on question formulation and clarity was solicited, and the questions were pilot tested by a group of postgraduate year 4 internal medicine residents.
      Upon finishing IM-ITE, examinees were asked to complete the survey. They were informed that deidentified data may be used in a scholarly publication and presentations and were asked to consent to allow their responses to be used in such a manner. The University of Connecticut Internal Review Board determined in July 2017 that the resident survey did not involve human subjects' research as defined in the Department of Health and Human Services regulations; therefore, this survey did not require Institutional Review Board approval.
      Results are reported as percentages, using only responses from consenting examinees. Kendall's tau and Pearson chi-square were used to estimate statistically significant differences between IMGs and USMGs for factors most helpful in decreasing number of applications and influential on the number of submitted applications.

      Results

      Of the 26,021 postgraduate year 1 to 3 internal medicine residents who took IM-ITE, 24,104 (92.6%) completed the IM-ITE resident survey; 21,213 (81.5% of all IM-ITE takers and 88% of survey completers) consented to have their survey responses used for research. Respondents were evenly distributed among postgraduate year of training. Some 56% of respondents were identified as allopathic or osteopathic USMGs, and 44% were identified as IMGs.
      Overall, differences in application patterns emerge when stratifying residents by IMG/USMG status and postgraduate year level. IMGs are concentrated at the lower (<15) and upper ranges (>75) in the number of submitted applications (Figure 1). One quarter of IMGs reported applying to fewer than 15 programs. Approximately one half reported submitting more than 75 applications, with an increasing percentage of each subsequent postgraduate year cohort reporting in this range. The overall percentage of each postgraduate year 1 and postgraduate year 2 cohort who submitted between 15 and 75 applications decreased relative to the previous class. Application patterns of USMGs differ from IMGs, with approximately 9 of 10 USMGs submitting fewer than 50 applications (Figure 2). However, the percentage of USMGs submitting fewer than 20 applications has decreased over time, whereas there has been an increase in the percentage of applications submitted in all other ranges (Figure 2).
      Figure 2
      Figure 2Number of programs applied to by postgraduate year and applicant type. PGY = postgraduate.
      When asked to assess the influence of various factors on deciding the total number of residency applications to submit, respondents most frequently cited talking with peers (39%), talking with recent graduates of the same medical school (35%), and fear of not matching (34%) as “very influential” (Table 1). Factors most often rated “not at all influential” included advice from a local internal medicine program director (24%), advice from the dean's office (24%), and advice from the medicine clerkship director (23%). Statistically significant differences were noted in all responses based on applicant type (USMG vs IMG). Fear of not matching, other social media, and use of website data were “very influential” factors in which responses of IMGs and USMGs differed the most (15%, 14%, and 13% difference, respectively) and which were more influential for IMGs. Differences also appeared between IMG and USMG responses for factors ranked as “not at all influential.” These differences include advice from the dean's office (Δ18%), other social media (Δ16%), advice from an advisor (Δ13%), and advice from a clerkship director (Δ12%). With the exception of social media, IMGs ranked these factors as “not at all influential” at a higher percentage than USMGs.
      Table 1Sources of Information Most Influential to Residents in Determining the Number of Applications to Submit
      For each source of information, P < .001 for the difference between responses from USMGs and IMGs based on Pearson chi-square test.
      Table thumbnail ajm14466-fig-5001
      AAMC = Association of American Medical Colleges; ERAS = Electronic Residency Application Service; FREIDA = Fellowship and Residency Electronic Interactive Database; IM = internal medicine; IMG = international medical school graduate; USMG = US or Canadian medical school graduate.
      * For each source of information, P < .001 for the difference between responses from USMGs and IMGs based on Pearson chi-square test.
      From a list of 11 items, respondents were asked to select a single top factor that might have narrowed the number of programs to which they applied. Overall, subspecialty fellowship match results (24%), average US Medical Licensure Examination (USMLE) Step 1 score of current interns (23%), information about the training program workload and schedules (14%), and identities of current residents and the medical schools from which they graduated (13%) were most frequently chosen (Table 2).
      Table 2Factors Most Influential to Residents in Narrowing Down the Number of Residency Applications
      For each factor, P < .001 for the difference between responses from USMGs and IMGs in percentage of respondents rating each factor as most influential based on Pearson chi-square test.
      Table thumbnail ajm14466-fig-5002
      ABIM = American Board of Internal Medicine; IMG = international medical school graduate; USMG = US or Canadian medical school graduate; USMLE = US Medical Licensing Examination.
      * For each factor, P < .001 for the difference between responses from USMGs and IMGs in percentage of respondents rating each factor as most influential based on Pearson chi-square test.
      Although the frequency distribution of each of the top 4 factors differed between USMGs and IMGs, both groups cited these same factors as the most influential. USMGs placed greater weight on the knowledge of subspecialty match results and training program workload, whereas USMLE step scores and identities of current residents and where they graduated from medical school had greater importance to IMGs. Overall for this question, differences noted between responses from IMGs and USMGs for each factor were statistically significant.

      Discussion

      To our knowledge, our study represents the first national survey of current residents on factors influencing the number of applications submitted per applicant for internal medicine residency. As discussed, ERAS data indicated an increasing number of applications submitted per applicant to internal medicine residency programs since at least the 2012 ERAS cycle.
      • Association of American Medical Colleges
      Electronic Residency Application Service data tables.
      Our survey results are consistent with ERAS statistics with respect to overall increasing applications to internal medicine residency programs and the higher rate of applications by IMGs compared with USMGs. However, our analysis differs from ERAS reporting given our stratification based on postgraduate year level, applicant type, and application range, revealing application patterns that have not been previously discussed in the literature. For example, little is known about the approximately 20% of IMGs who apply to fewer than 15 programs, a subgroup that challenges our assumptions about all IMGs over-applying. Further research is needed to understand this sizable segment of the IMG pool. Likewise for USMGs, stratification suggests that there may be important subgroups to consider, which aggregate data may mask. Although only representing 5% to 8% of the cohort, a small group of current USMG internal medicine residents applied to more than 75 programs. Were these students at risk of not matching? Are they repeat applicants? If any, what are their shared academic or nonacademic attributes? Were they advised to do so by their deans or peers? Such questions are as of yet unanswered.
      Compared with faculty (student advisors, deans, internal medicine program directors, and clerkship directors), peers and recent graduates exert a greater influence over decisions on the number of applications to submit, suggesting they function as primary drivers of application inflation. Additional research is needed to understand why applicants value peer advice over other sources and how to assess and alter the quality and opportunities of customary advising sources to meet applicant needs. Based on differing responses by applicant type, subgroup analysis should be prioritized in any needs assessment or research endeavor because uniformity of need among all applicants is unlikely. In our study, no single factor was “very influential” or “not at all influential” for a majority of respondents, even when compared between USMGs and IMGs. It may be that other factors not appearing on our survey influence applicants on the number of applications to submit.
      Likewise, no single factor for narrowing the number of residency programs to which applicants applied garnered a majority. Although knowledge of the average USMLE Step 1 score and subspecialty fellowship Match results combined would have been the most helpful for 50% of respondents, underlying trends and drivers based on subgroup characteristics are likely masked. For example, subspecialty Match results would appear to be more pertinent for individuals who have defined career goals to enter fellowship than for learners who are undecided or have chosen not to pursue a fellowship.
      Additionally, in observing the increased interest of IMGs in average USMLE Step 1 scores and identities of current residents and medical schools from which they graduated compared with USMGs, it is worth exploring whether providing these data aggregated at a program level would impact the sizable IMG cohort submitting more than 75 applications. Further research is also needed to understand whether other factors (career choice, research interests, USMLE performance, type and competitiveness of medical school, geographic preferences, and marital or relationship status, ie, the couples match) affect the number of applications submitted.

      Limitations

      Our study has several limitations. First, survey data relied on recall of respondents and thus can be subject to recall bias. However, results on the numbers of applications submitted are consistent with national-level (ERAS) data, suggesting that the potential impact of recall bias is not likely substantial. Second, in our survey question about factors influencing the total number of submitted applications, the scale was not optimal in that the difference between “somewhat influential” and “not too influential” is not distinct. “Mostly influential” would have helped with balance—equal distance between values. Thus, we do not conduct quantitative analysis across values. Rather, we limit our discussion on applicant differences based on the end points, “very influential” and “not too influential.” Third, concerning factors that would have been helpful to narrow down the number of applications submitted, only a preselected menu of items was given with no option for “other.” As discussed, other factors not previously identified may wield greater influence and would have been likely captured with a comments sections. Likewise, because our data are limited to only applicants who secured residency placement in internal medicine, the patterns discussed may differ for individuals who did not successfully match or who applied to other specialties.

      Conclusions

      Application inflation continues despite national data that the Match is not becoming more competitive, given the stable match rates for every applicant type. This study sheds light on some of the factors that are driving application inflation and provides suggestions as to how we can collectively address the issue. No one single solution seems likely to stem application inflation as each unique subgroup of applicants is influenced by different factors, highlighting the need for further research and implementation of strategies that account for the needs of a heterogeneous applicant pool.

      Acknowledgments

      The authors thank Mona Signer and the National Resident Matching Program for providing data and for thoughtful review of this article.

      References

        • Association of American Medical Colleges, Careers in Medicine
        Apply smart in internal medicine.
        (Available at:)
        https://www.aamc.org/cim/480044/applysmartim.html
        Date: 2017
        Date accessed: September 1, 2017
        • Gliatto P.
        • Karani R.
        Viewpoint from 2 undergraduate medical education deans—the residency application process: working well, needs fixing, or broken beyond repair?.
        J Grad Med Educ. 2016; 8: 307-310
        • Weissbart S.J.
        • Kim S.J.
        • Feinn R.S.
        • Stock J.A.
        Relationship between the number of residency applications and the yearly match rate: time to start thinking about an application limit?.
        J Grad Med Educ. 2015; 7: 81-85
        • Association of American Medical Colleges
        Electronic Residency Application Service data tables.
        (Available at:) (and September 25, 2017)
        • National Resident Matching Program
        Results and Data: 2017 Main Residency Match®.
        (National Resident Matching Program; Available at:)
        • National Resident Matching Program
        Results and Data: 2017 Main Residency Match®.
        (National Resident Matching Program, Washington, DC; Available at:)
        • National Resident Matching Program
        All-In Policy.
        (Available at:)
        http://www.nrmp.org/all-in-policy/main-residency-match/
        Date: 2013
        Date accessed: July 5, 2017
        • Pereira A.G.
        • Appelbaum J.S.
        • Chheda S.G.
        • et al.
        Application inflation for internal medicine applicants in the match: drivers, consequences, and potential solutions.
        Am J Med. 2016; 129: 885-891
        • Katsufrakis P.J.
        • Uhler T.A.
        • Jones L.D.
        The residency application process: pursuing improved outcomes through better understanding of the issues.
        Acad Med. 2016; 91: 1483-1487