Advertisement

Resident Recruitment Costs: A National Survey of Internal Medicine Program Directors

      Interest in Internal Medicine is declining nationally
      • Schwartz M.D.
      • Durning S.
      • Linzer M.
      • Hauer K.E.
      Changes in medical students' views of internal medicine careers from 1990 to 2007.
      • Phillips Jr., R.L.
      • Dodoo M.S.
      • Petterson S.
      • et al.
      Specialty and geographic distribution of the physician workforce: what influences medical student & residency choices?.
      • Hauer K.E.
      • Durning S.J.
      • Kernan W.N.
      • et al.
      Factors associated with medical students' career choices regarding internal medicine.
      and government funding to graduate medical education may be cut by health care reforms.
      • Iglehart J.K.
      The uncertain future of Medicare and graduate medical education.
      Furthermore, new regulations requiring residency programs to either place all of their positions within the National Resident Matching Program (NRMP) or withdraw from the match entirely

      National Resident Matching Program. NRMP policy change: all positions in the match. Available at: http://www.nrmp.org/allinnewrulesept11.pdf. Accessed November 20, 2012.

      have raised concerns about increasing resources and effort necessary to recruit Internal Medicine residents, particularly for programs that have traditionally filled a portion of positions outside the match.
      • Adams M.
      • Morrison T.B.
      • Call S.
      • et al.
      Changes in the national residency matching program policy: are internal medicine program directors “all in”?.
      Additionally, new duty hour restrictions,
      • Nasca T.J.
      • Day S.H.
      • Amis Jr., E.S.
      ACGME Duty Hour Task Force
      The new recommendations on duty hours from the ACGME task force.
      increasing resident workload,
      • Arora V.M.
      • Georgitis E.
      • Siddique J.
      • et al.
      Association of workload of on-call medical interns with on-call sleep duration, shift duration, and participation in educational activities.
      • Ong M.
      • Bostrom A.
      • Vidyarthi A.
      • McCulloch C.
      • Auerbach A.
      House staff team workload and organization effects on patient outcomes in an academic general internal medicine inpatient service.
      and the growing number of US medical school graduates

      Liaison Committee on Medical Education (LCME). Developing medical education programs. Available at: http://www.lcme.org/newschoolprocess.htm. Accessed November 20, 2012.

      may push some programs to expand their resident complement and allocate additional resources to resident recruitment.
      • Median total cost of recruitment per program was $148,000.
      • Largest determinate of cost was program size.
      • Programs considering expanding can anticipate spending at least $6000 per additional postgraduate year 1 (PGY1) position.
      • About 25% of PGY1 positions were filled outside the match.
      • If all 362 Internal Medicine programs went “all-in,” this would add an estimated $8.9 million to recruitment cost nationwide.
      In this era of declining resources and increasing regulation in graduate medical education, it is important to understand the annual cost associated with recruiting residents.
      • Iglehart J.K.
      The uncertain future of Medicare and graduate medical education.
      This study provides a national “snapshot” of interview day characteristics and estimates costs associated with recruitment among US Internal Medicine residency programs. By examining national recruitment practices, we hope to inform programs so that they can assess their recruitment strategies and make necessary adjustments in response to new policy changes.

      National Resident Matching Program. NRMP policy change: all positions in the match. Available at: http://www.nrmp.org/allinnewrulesept11.pdf. Accessed November 20, 2012.

      • Nasca T.J.
      • Day S.H.
      • Amis Jr., E.S.
      ACGME Duty Hour Task Force
      The new recommendations on duty hours from the ACGME task force.

      Methods

       Survey of Program Directors

      The Association of Program Directors in Internal Medicine surveyed 366 directors of its member programs in August 2009, which represented 96.3% of the 380 categorical Internal Medicine residency programs in the US and its territories.

      American Board of Internal Medicine. Resident and fellow workforce data. Available at: http://www.abim.org/about/examInfo/data-workforce.aspx. Accessed November 20, 2012.

      Nine military-based and 4 unincorporated territory programs were excluded due to substantive differences in recruitment practices, leaving 353 programs eligible for this study. The survey was conducted electronically using a program-specific hyperlink to access the survey online. Nonresponders received subsequent e-mails with the hyperlink in September and October 2009. In November of 2009, the survey was closed.
      The survey contained questions addressing recruitment processes, interview day components, and estimates of recruitment costs. First, program directors were asked to report the number of applications received, number of US medical school graduate applicants, number of applicants invited to interview, and number of applicants interviewed. Second, program directors were asked to state the number of interview days, to describe the components of the interview day (tour, meals, conferences, rounds, etc.), and to define the interview process (faculty panel, group interview, one-on-one interviews, program director interview). Thirdly, to ascertain an estimate of recruitment costs, program directors reported the amount of money spent during the 2008-2009 recruitment year on food, printing and supplies, hotel accommodations, and any other costs. Lastly, program directors estimated the percentage effort spent on recruitment by residency program personnel including the program director, associate program directors, chief residents, department of medicine chair, program administrators, and secretaries.

       Estimation of Recruitment Costs

      The total cost of recruitment was estimated as the sum of the 2008-2009 recruitment budget (spent on food, printing and supplies, hotels, etc.) and the percentage effort dedicated to recruitment by residency program personnel multiplied by national salary data for those personnel by job category. Program director salary was reported in the survey by program directors either as exact annual salary or annual salary category. If no salary data were reported, the median program director salary of the study sample was used. Associate program director salary was calculated using 2008 American Medical Group Association compensation and financial survey data for Internal Medicine physicians by region.

      AMGA Medical Group Compensation and Financial Survey 2008. Salary by specialty and region. Available at: http://www.morganmdsearch.com/resources/AMGACompensationFinancial%20Survey2008.pdf. Accessed November 20, 2012.

      Regions within the US were defined using US Census Bureau definitions.

      U.S. Census Bureau. Census regions and division of the United States. Available at: http://www.census.gov/geo/www/us_regdiv.pdf. Accessed November 20, 2012.

      Chief resident salary was calculated using program-specific postgraduate year (PGY)4 resident salary as reported by the American Medical Association Fellowship and Residency Electronic Interactive Database Access System Online (FREIDA).

      American Medical Association. Graduate medical education: Fellowship and Residency Electronic Interactive Database Access system online (FRIEDA). Available at: https://freida.ama-assn.org/Freida/user/viewProgramSearch.do. Accessed November 20, 2012.

      If PGY4 salary was not available for a specific residency program, the mean PGY4 salary by US Census Bureau region was used. Department of Medicine Chair salary was taken from the median salary reported by the 2008 Administrators of Internal Medicine and Association of Professors of Medicine survey.

      Clayton CP. AIM-APM 2008 survey. Salary survey highlights from 2008. Available at: http://www.im.org/Publications/Insight/Archives/Documents/Vol6Issue2/AIMAPMSurvey.pdf. Accessed November 20, 2012.

      Program administrator salary was taken from the mean reported by the Associate Program Directors in Internal Medicine (APDIM) Program Administrators 2009 regional survey reports.

      Associate Program Directors in Internal Medicine (APDIM) Program Administrators 2009 Survey. Regional survey reports. Available at: http://www.im.org/toolbox/surveys/APDIMSurveyData/Pages/default.aspx. Accessed November 20, 2012.

      Secretarial staff salary was calculated using the Bureau of Labor Statistics annual mean medical secretary salary data by region.

      Bureau of Labor Statistics. 2010 annual mean medical secretary salary data by state. Available at: http://www.bls.gov/oes/2010/may/oes436013.htm. Accessed November 20, 2012.

      Additional program variables were collected using publicly available data. Programs were categorized as community based, university based, or community based university affiliated according to the FREIDA database.

      American Medical Association. Graduate medical education: Fellowship and Residency Electronic Interactive Database Access system online (FRIEDA). Available at: https://freida.ama-assn.org/Freida/user/viewProgramSearch.do. Accessed November 20, 2012.

      Board certification examination pass rate was obtained from the American Board of Internal Medicine (ABIM) 3-year program-specific running pass rate.

      American Board of Internal Medicine. Residency program pass rates 2006-2009. Available at: http://www.abim.org/pdf/pass-rates/residency-program-pass-rates.pdf. Accessed December 1, 2009.

      Accreditation cycle length, government affiliations, number of approved positions, and number of filled positions were obtained from the Accreditation Council for Graduate Medical Education (ACGME) Web site.

      Accreditation Council for Graduate Medical Education. Accredited programs – internal medicine. Available at: http://www.acgme.org/adspublic/. Accessed December 1, 2009.

      Program fill rate was calculated using NRMP 2009 match data by subtracting quota from matched positions.

      National Residency Match Program (NRMP). Program results 2009 main residency match. Available at: http://www.nrmp.org/data/resultsanddata2009.pdf. Accessed November 20, 2012.

      Number of positions accepted outside the match was calculated by subtracting approved ACGME positions from the NRMP quota positions.
      This study was approved by the Mayo Clinic Institutional Review Board.

       Statistical Analyses

      Cost data were summarized using median and interquartile range (IQR), while other variables were tabulated using count and percentage or mean and SD, as appropriate. Differences in interview day composition by program affiliation were assessed with odds ratios obtained from logistic regression models. Relationships between recruitment costs and program variables were examined using bivariate and multivariate quantile regression models for median total cost. Multivariate models were adjusted for size, region, program type, accreditation cycle length, government affiliation, presence of unfilled positions, ABIM rolling pass rate, percent volunteer faculty, and number of categorical PGY1 positions filled outside the NRMP match. A conservative alpha level of .01 was used to account for multiple comparisons. Analyses were conducted using SAS statistical software (version 9.3; SAS Institute Inc, Cary, NC).

      Results

      Surveys were completed by 270 of 353 (76.5%) program directors. The geographic distribution, program type, accreditation cycle length, government affiliation, and presence of unfilled positions are shown in Table 1 and are similar to those seen nationally.

      U.S. Census Bureau. Census regions and division of the United States. Available at: http://www.census.gov/geo/www/us_regdiv.pdf. Accessed November 20, 2012.

      American Medical Association. Graduate medical education: Fellowship and Residency Electronic Interactive Database Access system online (FRIEDA). Available at: https://freida.ama-assn.org/Freida/user/viewProgramSearch.do. Accessed November 20, 2012.

      American Board of Internal Medicine. Residency program pass rates 2006-2009. Available at: http://www.abim.org/pdf/pass-rates/residency-program-pass-rates.pdf. Accessed December 1, 2009.

      Accreditation Council for Graduate Medical Education. Accredited programs – internal medicine. Available at: http://www.acgme.org/adspublic/. Accessed December 1, 2009.

      Table 1Internal Medicine Residency Program Characteristics (n = 270)
      Characteristicn (%)
      Census region
       Northeast104 (38.5%)
       South68 (25.2%)
       Midwest63 (23.3%)
       West35 (13.0%)
      Program description
       Community-based, university-affiliated142 (52.6%)
       University-based91 (33.7%)
       Community-based37 (13.7%)
      Accreditation cycle length, y
       <326 (9.6%)
       ≥3-<572 (26.7%)
       ≥5172 (63.7%)
      Government affiliation
       Yes62 (23.0%)
       No208 (77.0%)
      Presence of unfilled positions
       >392 (34.1%)
       ≤3178 (65.9%)

       Recruitment Characteristics and Interview Day

      Programs received a mean (SD) of 2314 (1579) applications during the 2008-2009 recruitment year, of which 84% were from international medical graduates (Table 2). On average, programs invited 225 applicants to interview and conducted 173 interviews, with 10 interviews conducted per PGY1 position. University-based programs received fewer applications per approved PGY1 categorical position, yet a higher number of these applications were from US medical graduates, compared with other program types (P = .002 and P = .0006, respectively, Table 2). Candidates underwent an average of 3 individual interviews during the interview day at each residency program.
      Table 2Recruitment and Interview Day Characteristics of 270 US Internal Medicine Programs
      All Programs (n = 270)University Based (n = 91)Community Based University Affiliated (n = 142)Community Based (n = 37)P Value
      Recruitment, mean (SD)
       Total applications received2314 (1579)2270 (1107)2473 (1865)1804 (1191)
       Total USMG applications362 (467)715 (609)195 (242)177 (199)
       Total interviews extended225 (142)306 (137)193 (134)156 (97)
       Total interviews conducted173 (112)242 (113)143 (99)123 (76)
       Applications received per PGY1 position159 (208)94 (58)197 (267)162 (118).002
       USMG applications per PGY1 position18 (19)24 (18)14 (19)16 (16).0006
       Interviews extended per PGY1 position13 (6)12 (4)14 (7)13 (6).02
       Interviews conducted per PGY1 position10 (4)9 (3)10 (5)10 (5).07
       Interviews conducted by program director57 (72)55 (76)61 (73)45 (56).45
      Interview day components, n (%)
       Tour264 (98%)90 (99%)137 (97%)37 (100%).98
       Lunch249 (92%)89 (98%)125 (88%)35 (95%).56
       Program director overview/slideshow228 (84%)85 (93%)115 (81%)28 (76%).04
       Breakfast189 (70%)78 (86%)89 (63%)22 (60%).04
       Morning report187 (69%)72 (79%)92 (65%)23 (62%).02
       Greeting by Chair of Medicine142 (53%)76 (84%)58 (41%)8 (22%).0005
       Noon conference137 (51%)29 (32%)81 (57%)27 (73%)<.0001
       Dinner the night before interview112 (42%)66 (73%)39 (28%)7 (19%)<.0001
       Attending rounds53 (20%)17 (19%)30 (21%)6 (16%)<.0001
       Patient care rounds with residents25 (9%)11 (12%)11 (8%)3 (8%).77
       Faculty panel interview24 (9%)7 (8%)10 (7%)7 (19%)0.52
       Applicants interviewed as a group7 (3%)1 (1%)4 (3%)2 (5%)0.16
       Other
      Resident discussion panel, chief resident presentation, city tour, simulation center tour, meeting with fellowship directors.
      30 (11%)11 (12%)18 (13%)1 (3%)0.10
      PGY1 = post-graduate year 1; USMG = US medical graduate.
      Resident discussion panel, chief resident presentation, city tour, simulation center tour, meeting with fellowship directors.
      Table 2 shows the components of the interview day at US Internal Medicine residency programs. For the majority of programs, the interview day included a tour of the hospital and facilities (264, 97.8%), lunch (249, 92.2%), program overview/slideshow by the program director (228, 84.4%), and morning report (187, 69.3%). University-based programs were more likely than community-based programs to include a greeting by the Department of Medicine Chair (76, 83.5% vs 8, 21.6%, respectively; odds ratio [OR] 18.4, P <.0001) and dinner the evening before the interview day (112, 41.5% vs 7, 18.9%, respectively; OR 11.3, P <.0001). A greater percentage of community-based programs had candidates attend noon conference compared with university-based programs (27, 73.0% vs 29, 31.9%, respectively; OR 5.8, P <.0001). Less than 10% of all programs invited candidates to participate on patient care rounds with residents during the interview day.

       Estimates of Recruitment Costs

      Estimates of the costs of recruitment to US Internal Medicine residency programs are shown in Table 3. The estimated median (IQR) total cost of recruitment per residency program was $148,345 ($95,966-234,704). The median total cost per PGY1 matriculate was $9899 ($6494-13,271), and this estimate was similar across program types. It cost each residency program approximately $1042 ($733-1565) per interview conducted to recruit an intern class in the 2008-2009 academic year.
      Table 3Estimated Recruitment Costs of 270 US Internal Medicine Residency Programs
      Cost ComponentAll Programs (n = 270)University Based (n = 91)Community Based University Affiliated (n = 142)Community Based (n = 37)
      Median (IQR)Median (IQR)Median (IQR)Median (IQR)
      Program director effort$42,400 ($27,750-59,375)$50,000 ($37,500-72,521)$37,500 ($25,000-56,250)$37,500 ($25,000-50,000)
      Associate program director effort
      Includes total effort by all associate program directors at each program.
      $52,022 ($23,376-89,291)$86,703 ($57,820-140,253)$38,781 ($19,273-60,692)$28,910 ($18,700-63,602)
      Chief resident effort
      Includes total effort by all chief residents at each program.
      $12,206 ($4346-25,450)$25,425 ($12,226-42,383)$8198 ($1380-15,282)$5794 ($0-16,953)
      Administrative staff effort
      Includes effort by program administrators and secretaries.
      $29,650 ($19,582-41,919)$40,420 ($27,562-56,113)$26,257 ($17,505-35,134)$19,693 ($14,190-31,004)
      Department of medicine chair effort$369 ($0-854)$699 ($427-932)$0 ($0-777)$0 ($0-0)
      Food$4850 ($2400-10,000)$10,000 ($4850-24,000)$4225 ($2000-5000)$2000 ($900-4850)
      Supplies$600 ($300-1000)$1000 ($600-2000)$600 ($250-1000)$500 ($200-600)
      Hotel$0 ($0-700)$0 ($0-4500)$0 ($0-1000)$0 ($0-0)
      Total estimated recruitment cost per program$148,345 ($95,966-234,704)$239,821 ($162,815-317,526)$120,920 ($87,933-187,916)$105,256 ($81,880-140,980)
      Cost per PGY1 matriculate$9899 ($6494-13,271)$9639 ($6438-12,898)$9809 ($6347-13,337)$10,347 ($7382-13,293)
      Cost per NRMP matched position)
      n = 263 for this variable because 3 programs did not participate in the match and 4 programs did not fill any of their available positions in the match.
      $14,162 ($9741-22,605)$11,058 ($7792-14,708)$16,387 ($11,162-24,334)$20,939 ($13,647-28,096)
      IQR = interquartile range; NRMP = national resident matching program; PGY1 = post-graduate year 1.
      Includes total effort by all associate program directors at each program.
      Includes total effort by all chief residents at each program.
      Includes effort by program administrators and secretaries.
      § n = 263 for this variable because 3 programs did not participate in the match and 4 programs did not fill any of their available positions in the match.
      The largest components of recruitment costs were effort dedicated by program directors and associate program directors, which together accounted for approximately 64% of the estimated total cost (Table 3). Effort contributed by program administrators and secretaries was an estimated $29,650 ($19,582-41,919) and 20% of overall cost. Programs spent about $5000 on food throughout the recruitment season. Few programs covered hotel costs for applicants.
      The estimated median (IQR) total cost of recruitment per position filled through the NRMP match for all programs was $14,162 ($9741-22,605). Community-based programs spent significantly more than university-based programs on recruitment per position filled through the NRMP match ($20,939 vs $11,058, P <.0001); however, community programs filled a lower percentage of their total positions through the match in 2009 compared with university programs (54% vs 91%), resulting in a higher cost per position within the match.
      Table 4 shows bivariate and multivariable quantile regression models examining relationships between residency program characteristics and total costs of recruitment. In bivariate analysis, program characteristics associated with higher total costs of recruitment were:
      • University affiliation (β = +$119,244; P <.0001),
      • Government affiliation (β = +$91,467; P <.0001),
      • Location in the West (β = +$75,832; P = .01),
      • Having >3 unfilled positions (β = +$74,027; P = .0003),
      • A higher percentage of volunteer faculty (β = −$1069; P <.0001), and
      • Larger PGY1 class size (β = +$6787; P <.0001).
      In multivariate analysis, PGY1-approved positions (β = +$6073; P <.0001) and the number of positions filled outside the NRMP match (β = −$5446; P = .0001) were the only residency program variables independently associated with total cost of recruitment. These results may be interpreted as: For every one additional PGY1 categorical position, the total cost of recruitment increased by $6073, and for every one additional PGY1 categorical position taken outside the NRMP match, the total cost of recruitment decreased by $5446.
      Table 4Multivariate Quartile Regression Model of Relationships between Residency Program Characteristics and Total Costs of Recruitment (n = 270 Programs)
      n (%)Median (IQR)
      Median (IQR) is $148,345 ($95,966-234,704) for all 270 included programs.
      UnadjustedAdjusted
      Intercept for multivariate (adjusted) model is $158,042 (11,291).
      β (SE)P Valueβ (SE)P Value
      Census region
       West35 (13.0%)$215,881 ($122,032-317,526)+$75,832 (30,694).01+$22,282 (21,476).14
       South68 (25.2%)$144,031 ($95,424-222,730)+$7032 (20,816).76−$15,139 (14,260).45
       Midwest63 (23.2%)$160,243 ($96,514-247,841)+$20,193 (27,885).43+$3459 (12,226).83
       Northeast
      Reference group.
      104 (38.5%)$139,831 ($91,892-218,772)$140,050 (11,710)-
      Program description
       Community-based37 (13.7%)$105,256 ($81,880-140,980)−$15,321 (12,086).08+$14,717 (10,251).13
       University-based91 (33.7%)$239,821 ($162,815-317,526)+$119,244 (17,677)<.0001+$31,899 (20,540).05
       Community-based, university affiliated
      Reference group.
      142 (52.6%)$120,920 ($87,933-187,916)$120,577 (8522)-
      Accreditation cycle length, y
       <326 (9.6%)$126,666 ($79,272-239,054)−$33,482 (33,340).21−$3081 (24,184).63
       ≥3-<572 (26.7%)$117,690 ($88,053-214,281)−$50,981 (16,299).03−$20,724 (9891).08
       ≥5
      Reference group.
      172 (63.7%)$169,669 ($104,694-242,558)$169,441 (10,790)-
      Government affiliation
       Yes62 (23.0%)$227,776 ($140,294-317,526)+$91,467 (19,655)<.0001+$32,328 (27,317).49
       No208 (77.0%)$133,193 ($91,892-212,075)$135,960 (8749)-
      Presence of unfilled positions
       >392 (34.1%)$201,555 ($115,724-269,792)+$74,027 (17,228).0003+$5035 (13,543).99
       ≤3
      Reference group.
      178 (65.9%)$126,216 ($88,583-216,368)$126,890 (9400)-
      Mean (SD)Expected Median (SE)
      ABIM rolling pass rate 2006-2008, %92.6 (6.1)$149,957 (8446)+$1952 (1401).16+$420 (744).50
      Volunteer faculty, %40.7 (35.4)$163,162 (8092)−$1069 (208)<.0001−$190 (158).35
      Positions filled outside NRMP Match, n4.5 (5.5)$149,808 (9950)−$2786 (1136).04−$5446 (1069).0001
      PGY1 categorical positions, n18.6 (11.1)$165,978 (5076)+$6787 (652)<.0001+$6073 (981)<.0001
      ABIM = American Board of Internal Medicine; IQR = interquartile range; PGY1 = post-graduate year 1; SE = standard error.
      Intercept for multivariate (adjusted) model is $158,042 (11,291).
      Median (IQR) is $148,345 ($95,966-234,704) for all 270 included programs.
      Reference group.

      Discussion

      This study provides national data on characteristics and costs of recruitment to US Internal Medicine residency programs. These data can help programs optimize their recruitment strategies in this era of declining interest in Internal Medicine,
      • Schwartz M.D.
      • Durning S.
      • Linzer M.
      • Hauer K.E.
      Changes in medical students' views of internal medicine careers from 1990 to 2007.
      • Phillips Jr., R.L.
      • Dodoo M.S.
      • Petterson S.
      • et al.
      Specialty and geographic distribution of the physician workforce: what influences medical student & residency choices?.
      • Hauer K.E.
      • Durning S.J.
      • Kernan W.N.
      • et al.
      Factors associated with medical students' career choices regarding internal medicine.
      potential funding cuts to graduate medical education,
      • Iglehart J.K.
      The uncertain future of Medicare and graduate medical education.
      and the NRMP “all-in” requirement.

      National Resident Matching Program. NRMP policy change: all positions in the match. Available at: http://www.nrmp.org/allinnewrulesept11.pdf. Accessed November 20, 2012.

      To optimize recruitment success, residency programs attempt to design interview day experiences to highlight program characteristics and meet the expectations of applicants. Multiple studies have described factors that are important in an applicant's creation of a rank list.
      • Wang T.
      • Wong B.
      • Huang A.
      • et al.
      Factors affecting residency rank-listing: a Maxdiff survey of graduating Canadian medical students.
      • Crace P.P.
      • Nounou J.
      • Engel A.M.
      • Welling R.E.
      Attracting medical students to surgical residency programs.
      • Flynn T.C.
      • Gerrity M.S.
      • Berkowitz L.R.
      What do applicants look for when selecting internal medicine residency programs? A comparison of rating scale and open-ended responses.
      • Laskin D.M.
      • Lesny R.J.
      • Best A.M.
      The residents' viewpoint of the matching process, factors influencing their program selection, and satisfaction with the results.
      • Nuthalapaty F.S.
      • Jackson J.R.
      • Owen J.
      The influence of quality-of-life, academic, and workplace factors on residency program selection.
      • Pretorius E.S.
      • Hrung J.
      Factors that affect national resident matching program rankings of medical students applying for radiology residency.
      • Raymond M.J.
      • Sokol R.J.
      • Vontver L.A.
      • Ginsburg K.A.
      Candid candidate comments: the relationship between residency program selection factors and match list placements from ranked applicants.
      • Sanfilippo J.A.
      • Sharkey P.F.
      • Parvizi J.
      Criteria used by medical students to rank orthopedic surgery residency programs.
      • Khan K.
      • Levstik M.
      Ranking in Canadian gastroenterology residency match: what do residents and program directors want?.
      • Embi P.J.
      • Desai S.
      • Cooney T.G.
      Use and utility of Web-based residency program information: a survey of residency applicants.
      • Aagaard E.M.
      • Julian K.
      • Dedier J.
      • Soloman I.
      • Tillisch J.
      • Perez-Stable E.J.
      Factors affecting medical students' selection of an internal medicine residency program.
      • DeSantis M.
      • Marco C.A.
      Emergency medicine residency selection: factors influencing candidate decisions.
      Certain factors, for example, geographic location of a program, cannot be modified. However, among the modifiable factors are applicants' interactions with current residents and observation of the workplace environment. One way to expose applicants to residents and showcase the workplace environment without increasing interview day costs is to have applicants participate in patient care rounds with residents. At the time of this study, only 9.3% of programs were doing this. In this analysis, dinner with residents the evening before the interview day (conducted more frequently by university-based compared with community-based programs) is another opportunity for applicants to interact with residents that represented a small portion of total cost.
      The median total cost of recruitment per program participating in this survey was approximately $148,000; when these costs are extrapolated to all 362 US Internal Medicine programs (excluding military-based and unincorporated territory programs), recruitment costs exceed $50 million annually. As would be expected, program size is a major determinant of overall recruitment cost. Residency programs that are considering expanding their resident complement to meet challenges posed by duty hour restrictions,
      • Nasca T.J.
      • Day S.H.
      • Amis Jr., E.S.
      ACGME Duty Hour Task Force
      The new recommendations on duty hours from the ACGME task force.
      increasing resident workload
      • Arora V.M.
      • Georgitis E.
      • Siddique J.
      • et al.
      Association of workload of on-call medical interns with on-call sleep duration, shift duration, and participation in educational activities.
      • Ong M.
      • Bostrom A.
      • Vidyarthi A.
      • McCulloch C.
      • Auerbach A.
      House staff team workload and organization effects on patient outcomes in an academic general internal medicine inpatient service.
      and patient complexity, or to accommodate the growing number of graduates from new or expanding US medical schools

      Liaison Committee on Medical Education (LCME). Developing medical education programs. Available at: http://www.lcme.org/newschoolprocess.htm. Accessed November 20, 2012.

      can anticipate spending at least $6000 per additional incremental PGY1 position for recruitment, based on the results of the multivariate model in this study. Institutions planning to expand their training programs may find this information helpful in determining program budgets.
      In May 2012, the NRMP implemented a significant policy change referred to as the “all-in” rule, requiring residency programs to either place all of their positions within the NRMP match or withdraw their entire program from the match.

      National Resident Matching Program. NRMP policy change: all positions in the match. Available at: http://www.nrmp.org/allinnewrulesept11.pdf. Accessed November 20, 2012.

      In the 2013 recruitment year, all residency programs must decide whether they will go “all-in” or “all-out” of the match. Internal Medicine program directors have expressed concerns about the adverse impact of this policy change on recruitment costs; particularly for smaller community-based programs.
      • Adams M.
      • Morrison T.B.
      • Call S.
      • et al.
      Changes in the national residency matching program policy: are internal medicine program directors “all in”?.
      The results of this study validate these concerns, showing that programs accustomed to filling a portion of positions outside the match that now decide to go “all-in” can expect to spend approximately $5500 more on recruitment per position converted from outside to inside the match. It is important to note that this estimate is conservative because programs will likely need to interview a greater number of applicants to fill a position within the match than they did to fill a position via an “out of match offer.” Among the 270 programs in this analysis, 24.7% of all PGY1 positions were filled outside the match, a similar percentage as reported in other studies.
      • Wetz R.V.
      • Seelig C.B.
      • Khoueiry G.
      • Weiserbs K.F.
      Out-of-match residency offers: the possible extent and implications of prematching in graduate medical education.
      If all 362 US programs were to go “all-in” in 2013, this would add an estimated $8.9 million to recruitment costs nationwide, which does not take into account the costs of processing a greater number of applications that may occur as a result of this policy.
      • Adams M.
      • Morrison T.B.
      • Call S.
      • et al.
      Changes in the national residency matching program policy: are internal medicine program directors “all in”?.
      These data will help residency programs that have historically filled a percentage of their positions outside the match to approximate resources necessary for recruitment should they decide to go “all-in.”
      This study has several limitations. The cross-sectional design does not aim to address causality, but rather provides descriptive information and associations among variables pertaining to residency recruitment. The estimates of resources spent on recruitment were subjectively reported by Internal Medicine program directors and are subject to recall bias. Salary estimates for individuals other than program director are approximations using publicly available data and are intentionally conservative, thus true costs of recruitment are underestimated. As such, cost data are estimates intended to provide a gestalt description of resources required for Internal Medicine residency recruitment.
      In summary, this study is among the first to describe recruitment practices to US Internal Medicine residency programs and associated costs. Programs planning to expand their resident complement or fill a greater number of positions through the NRMP may anticipate increased recruitment costs. Data from this study should help Internal Medicine residency programs assess their recruitment practices and make informed adjustments in response to new policy changes.

      Acknowledgment

      We are grateful for the support of the Association of Program Directors of Internal Medicine (APDIM), the members of the Survey Committee, and the residency program directors who completed the APDIM survey. While we are reporting the results of the APDIM Survey Committee, we are not presuming to speak for the organization, and our article does not constitute an official policy statement of APDIM, the APDIM Council, or any other organization with which any of the authors may be affiliated. We are grateful to the Mayo Clinic Survey Research Center for assistance with the survey design and data collection.

      References

        • Schwartz M.D.
        • Durning S.
        • Linzer M.
        • Hauer K.E.
        Changes in medical students' views of internal medicine careers from 1990 to 2007.
        Arch Intern Med. 2011; 171: 744-749
        • Phillips Jr., R.L.
        • Dodoo M.S.
        • Petterson S.
        • et al.
        Specialty and geographic distribution of the physician workforce: what influences medical student & residency choices?.
        The Robert Graham Center, Washington, DCMarch 2009 (Available at:) (Accessed November 20, 2012)
        • Hauer K.E.
        • Durning S.J.
        • Kernan W.N.
        • et al.
        Factors associated with medical students' career choices regarding internal medicine.
        JAMA. 2008; 300: 1154-1164
        • Iglehart J.K.
        The uncertain future of Medicare and graduate medical education.
        N Engl J Med. 2011; 365: 1340-1345
      1. National Resident Matching Program. NRMP policy change: all positions in the match. Available at: http://www.nrmp.org/allinnewrulesept11.pdf. Accessed November 20, 2012.

        • Adams M.
        • Morrison T.B.
        • Call S.
        • et al.
        Changes in the national residency matching program policy: are internal medicine program directors “all in”?.
        J Grad Med Educ. 2012; 4: 148-153
        • Nasca T.J.
        • Day S.H.
        • Amis Jr., E.S.
        • ACGME Duty Hour Task Force
        The new recommendations on duty hours from the ACGME task force.
        N Engl J Med. 2010; 363: e3
        • Arora V.M.
        • Georgitis E.
        • Siddique J.
        • et al.
        Association of workload of on-call medical interns with on-call sleep duration, shift duration, and participation in educational activities.
        JAMA. 2008; 300: 1146-1153
        • Ong M.
        • Bostrom A.
        • Vidyarthi A.
        • McCulloch C.
        • Auerbach A.
        House staff team workload and organization effects on patient outcomes in an academic general internal medicine inpatient service.
        Arch Intern Med. 2007; 167: 47-52
      2. Liaison Committee on Medical Education (LCME). Developing medical education programs. Available at: http://www.lcme.org/newschoolprocess.htm. Accessed November 20, 2012.

      3. American Board of Internal Medicine. Resident and fellow workforce data. Available at: http://www.abim.org/about/examInfo/data-workforce.aspx. Accessed November 20, 2012.

      4. AMGA Medical Group Compensation and Financial Survey 2008. Salary by specialty and region. Available at: http://www.morganmdsearch.com/resources/AMGACompensationFinancial%20Survey2008.pdf. Accessed November 20, 2012.

      5. U.S. Census Bureau. Census regions and division of the United States. Available at: http://www.census.gov/geo/www/us_regdiv.pdf. Accessed November 20, 2012.

      6. American Medical Association. Graduate medical education: Fellowship and Residency Electronic Interactive Database Access system online (FRIEDA). Available at: https://freida.ama-assn.org/Freida/user/viewProgramSearch.do. Accessed November 20, 2012.

      7. Clayton CP. AIM-APM 2008 survey. Salary survey highlights from 2008. Available at: http://www.im.org/Publications/Insight/Archives/Documents/Vol6Issue2/AIMAPMSurvey.pdf. Accessed November 20, 2012.

      8. Associate Program Directors in Internal Medicine (APDIM) Program Administrators 2009 Survey. Regional survey reports. Available at: http://www.im.org/toolbox/surveys/APDIMSurveyData/Pages/default.aspx. Accessed November 20, 2012.

      9. Bureau of Labor Statistics. 2010 annual mean medical secretary salary data by state. Available at: http://www.bls.gov/oes/2010/may/oes436013.htm. Accessed November 20, 2012.

      10. American Board of Internal Medicine. Residency program pass rates 2006-2009. Available at: http://www.abim.org/pdf/pass-rates/residency-program-pass-rates.pdf. Accessed December 1, 2009.

      11. Accreditation Council for Graduate Medical Education. Accredited programs – internal medicine. Available at: http://www.acgme.org/adspublic/. Accessed December 1, 2009.

      12. National Residency Match Program (NRMP). Program results 2009 main residency match. Available at: http://www.nrmp.org/data/resultsanddata2009.pdf. Accessed November 20, 2012.

        • Wang T.
        • Wong B.
        • Huang A.
        • et al.
        Factors affecting residency rank-listing: a Maxdiff survey of graduating Canadian medical students.
        BMC Med Educ. 2011; 11: 61
        • Crace P.P.
        • Nounou J.
        • Engel A.M.
        • Welling R.E.
        Attracting medical students to surgical residency programs.
        Am Surg. 2006; 72: 485-490
        • Flynn T.C.
        • Gerrity M.S.
        • Berkowitz L.R.
        What do applicants look for when selecting internal medicine residency programs? A comparison of rating scale and open-ended responses.
        J Gen Intern Med. 1993; 8: 249-254
        • Laskin D.M.
        • Lesny R.J.
        • Best A.M.
        The residents' viewpoint of the matching process, factors influencing their program selection, and satisfaction with the results.
        J Oral Maxillofac Surg. 2003; 61: 228-233
        • Nuthalapaty F.S.
        • Jackson J.R.
        • Owen J.
        The influence of quality-of-life, academic, and workplace factors on residency program selection.
        Acad Med. 2004; 79: 417-425
        • Pretorius E.S.
        • Hrung J.
        Factors that affect national resident matching program rankings of medical students applying for radiology residency.
        Acad Radiol. 2002; 9: 75-81
        • Raymond M.J.
        • Sokol R.J.
        • Vontver L.A.
        • Ginsburg K.A.
        Candid candidate comments: the relationship between residency program selection factors and match list placements from ranked applicants.
        Am J Obstet Gynecol. 2005; 193: 1842-1847
        • Sanfilippo J.A.
        • Sharkey P.F.
        • Parvizi J.
        Criteria used by medical students to rank orthopedic surgery residency programs.
        Am J Orthop. 2006; 35: 512-514
        • Khan K.
        • Levstik M.
        Ranking in Canadian gastroenterology residency match: what do residents and program directors want?.
        Can J Gastroenterol. 2010; 24: 369-372
        • Embi P.J.
        • Desai S.
        • Cooney T.G.
        Use and utility of Web-based residency program information: a survey of residency applicants.
        J Med Internet Res. 2003; 5: e22
        • Aagaard E.M.
        • Julian K.
        • Dedier J.
        • Soloman I.
        • Tillisch J.
        • Perez-Stable E.J.
        Factors affecting medical students' selection of an internal medicine residency program.
        J Natl Med Assoc. 2005; 97: 1264-1270
        • DeSantis M.
        • Marco C.A.
        Emergency medicine residency selection: factors influencing candidate decisions.
        Acad Emerg Med. 2005; 12: 559-561
        • Wetz R.V.
        • Seelig C.B.
        • Khoueiry G.
        • Weiserbs K.F.
        Out-of-match residency offers: the possible extent and implications of prematching in graduate medical education.
        J Grad Med Ed. 2010; 2: 327-333