_pre-_and_post-program_participant_ratings_of_research_confidence_domains_acro.tif)
Background
Hospital medicine is expanding rapidly, with the number of hospitalists growing from several hundred in 1996 to over 50,000 in 2016. Despite the dramatic growth in the field, the number of hospitalist clinician investigators has increased much more slowly, and most hospitalists do not receive research training.1,2
For early-career academic hospitalists, lower research productivity can negatively impact their academic advancement and promotion. Hospitalists often participate in activities that could be disseminated (e.g., quality improvement [QI] projects, operational exercises, or educational initiatives) but lack the mentorship and skills to disseminate that work and receive appropriate credit. For example, a recent analysis of COVID-19-related articles published in the 4 top internal medicine journals demonstrated that hospitalists served as first, senior, or middle author in only 3-5% of publications.3 Lack of dissemination also limits hospitalists’ ability to receive funding for what often begins as volunteer work. Major barriers to research productivity in hospital medicine include not having protected time for research,1 lack of funding,4 and lack of mentorship.1,4 A recent report indicated that only 2% of hospitalist faculty members conducted research for >50% of the time, and only 14% of hospital medicine groups had a research training program for hospitalist investigators.4
While most academic hospitalists do not desire to be grant-funded researchers5 there is an important opportunity to build research capacity in hospital medicine so that academic hospitalists can be recognized for their diverse contributions to research. Inpatient clinical research would not be successful without the contributions of academic hospitalists who support research through their operational, QI, or educational leadership roles. Furthermore, dissemination of hospital medicine research is necessary so others can learn best practices for inpatient care.
In 2021, to support academic promotion and research growth of clinical track hospitalist faculty, we developed the Utah Quality Advancement Laboratory (UQuAL) Scholars program, a mentorship program focused on building research and dissemination skills. Here, we describe the UQuAL Scholars program and report its impact on research confidence and productivity for the first two scholar cohorts (n=8) between 2021-2023.
Methods
Program development
As part of program development, in May 2021 we surveyed our institution’s 58 hospital medicine clinicians (including physician hospitalists and advanced practice clinicians). The survey assessed experience with research/QI and sought to identify areas of interest and perceived barriers to research productivity (see supplement for survey).
Based on survey results, two hospitalists with varied experience in research developed the UQuAL Scholars program with a curriculum focused on guiding clinicians through the steps of refining a QI research question, planning for and conducting a research project, and disseminating the results (see Table 1 for curriculum). The program was able to obtain limited funding for the UQuAL program leaders (5% full-time equivalent, FTE), a statistician/data analyst (20% FTE), and a project manager to support the scholars (estimated total cost ~$57,000 annually). UQuAL Scholars were provided no protected time for research.
UQuAL Scholars Selection
In July of 2021 and 2022, interested hospitalists were invited to submit a short application for UQuAL Scholars, including: a) identifying any relevant content mentor(s), b) a description of their interest in the program, c) prior QI/research experience, including dissemination products, and d) research interest/question to be answered during the scholars program. Applicants were asked to commit (via signature) to 80% didactic attendance, to submit an abstract within 1 year, and to submit a manuscript within 2 years. To be eligible for participation, scholars had to be clinical-track faculty with at least 1 year of clinical experience and be interested in QI or clinical research. The UQuAL Scholars program leaders met to select scholars, prioritizing those with either interests similar to the leaders’ or an identified content mentor, and those with feasible research questions and project scope. Projects that aligned with existing institutional roles (e.g., QI or operational roles), which often came with additional resources, were preferentially selected based on the likelihood of completion. Four UQuAL Scholars were selected for each cohort.
UQuAL Scholars Program
Each UQuAL Scholar was assigned one of the two program leaders as a mentor. The didactic curriculum consisted of 14 sessions over 9 months (Sept-May; see Table 1). Each session was attended by 1-2 program leaders, had assigned pre-work (e.g., writing or reading assignment), and prioritized active participation (e.g., reviewing/revising writing assignments) during the session, with only very brief lectures. Writing assignments were specific to each scholar’s individual project. At 6-months post-program, scholars were required to provide an update on their progress at a divisional works-in-progress meeting.
Program outcome assessments
We evaluated the UQuAL Scholars program in 4 ways: First, we administered a modified version of the Clinical Research Appraisal Inventory (CRAI-19)11 via REDCap to evaluate pre- and post-program research confidence. The CRAI assesses 5 domains: study design/data analysis; collaboration/grant preparation; manuscript writing; responsible conduct of research; and dissemination of results, using a 10-point Likert scale (see the supplement for survey questions). To assess the scholar’s ability to plan and manage research, we added 3 questions: creating a realistic project timeline, keeping themselves and collaborators accountable, and identifying and addressing barriers to project completion. Second, after program completion, we surveyed scholars’ perceptions of program strengths, weaknesses, and usefulness, including a free-text section for open-ended comments. Third, we tracked scholars’ academic productivity (e.g., abstracts and peer-reviewed publications) for up to 2 years following program participation. Finally, we compared the promotion rate from assistant to associate professor among scholars and non-scholar assistant professors.
Analysis
Characteristics and survey responses of all hospitalists and the UQuAL Scholars were summarized using descriptive statistics. To assess pre- vs. post-program differences in research confidence domains, we used paired t-tests with Bonferroni correction for multiple comparisons (p-values < 0.05 were considered significant). Free-text comments were reviewed for common themes. UQuAL Scholars’ academic products were described, and rates of promotion to associate professor relative to all other hospital medicine assistant professors were compared using logistic regression, controlling for duration at the assistant professor rank. We followed EQUATOR reporting guidelines (STROBE checklist in Appendix). This project was reviewed by the University of Utah’s Institutional Review Board and received an exempt determination.
Results
Hospital medicine group pre-program survey
Demographics and academic/research experience
Pre-program survey response rates from the hospital medicine and UQuAL Scholars groups were 62% (36/58) and 100% (8/8), respectively. For the hospital medicine group, the majority of respondents had a rank of assistant professor (51%) and were White (91%), not Hispanic/Latino (94%), and female (51%). Most respondents had previously participated in a QI or research project (69% and 75%, respectively), and roughly half (54%) reported presenting a QI or research project at a national meeting and submitting to a peer-reviewed journal. 43% had published QI or research in a peer-reviewed journal. Current involvement in QI or research projects was lower (35% and 38%, respectively), with less than half reporting a leadership role in these projects. Demographics and academic/research experience of the scholars appeared similar to those of the larger hospital medicine group, with the exception of a higher proportion of females (75%) and current involvement in a QI project (63%). Just 2/8 (25%) of the scholars reported having previously led or assisted a research study. Characteristics of the hospital medicine and scholars’ groups are in Table 2.
Academic and research interests and perceived barriers
When asked about academic interests, the majority of both the hospital medicine and scholars groups reported being “very” or “extremely” interested in education (67% and 63%, respectively), with lower levels of interest in research or administration (Table 3). A larger proportion of scholars (88%) were “very” or “extremely” interested in advancing clinical practice (including QI) than in the larger hospital medicine group (44%). The top reported barriers to conducting QI or clinical research were limited time, limited/no funding, limited knowledge, and statistical support. The barriers of a lack of good ideas and a lack of interest were rated as the least problematic (Table 3).
Program outcomes
Ten hospitalists applied to UQuAL scholars over the initial 2 years (7 in the first year [2 accepted in year 2; 5 in the second year). Of the 8 proposed projects, 4 (50%) were QI, 1 (12.5%) operational, 1 (12.5) % outcomes/data analysis, and 2 (25%) qualitative research.
Research confidence
Median post-program ratings of overall research confidence, across both scholar cohorts, were significantly higher than pre-program reported confidence (p<0.01, Figure 1). The domains of design/analysis, collaboration, writing manuscripts, and responsible research conduct all improved significantly (p<0.01).
_pre-_and_post-program_participant_ratings_of_research_confidence_domains_acro.tif)
Program evaluation
Program evaluation results are presented in Table 4 and indicate agreement/strong agreement that the program was useful for career development, conducting and disseminating high-quality research, and developing organizational skills. The value of peer relationships with other scholars received the highest ratings (7/8 “strongly agree”). There was ambivalence (4/8 scholars neither agreeing or disagreeing) regarding the ability of the program to help with academic promotion.
When asked to identify the most helpful aspect of the program, scholars reported benefits that fell into 3 main categories:
-
Mentorship (e.g., “Having instruction and support from [mentors] was very helpful in conducting my own project, and I feel as though I could go to either of them with questions any time in the future.”)
-
Having a timeline for task completion (e.g., “UQuAL made research more accessible for me. By breaking things down into bite-sized pieces, the idea of research as a whole is MUCH less stressful.” and “I really liked having us do the assignments and keeping us accountable - PICOTS for our projects, introduction, methods, etc.”), and
-
Working closely with peers (e.g., “I got to know my UQuAL peers better and think they will be a good resource moving forward.” And [it was beneficial having] “other people learning the same thing at the same time.”).
In response to the question “was there anything else you wished the program provided,” two scholars responded with suggestions. The first requested information/discussion about setting a realistic timeline for a QI project while working in a full-time clinical position. The other scholar suggested it would have been helpful to have topics from the final lecture (reference software and time management) earlier in the program. This person also would have wanted more time between sessions to complete assignments.
All 8 UQuAL Scholars who completed the program disseminated their UQuAL projects at national meetings, and 2 scholars received awards. In the 24-month post-program period, one scholar successfully published their project as a first author. Reasons for scholars not publishing their project results included: results were not novel/publishable according to the scholar (n=1); barriers to project completion (lack of essential stakeholder buy-in, unable to recruit study participants (n=2); barriers to manuscript completion (incomplete analysis/interpretation of data, lack of protected time/competing time demands (n=4). Subsequent to the program, four scholars applied for or were included on grant applications, and two became co-investigators on externally funded grants. The scholars program appeared to have additional unplanned benefits. We noted a “trickle down” effect as most scholars (63% [5/8]) included additional learners (resident physicians) in their projects, and a “halo” effect with 2 scholars publishing work closely related to their original projects.
At the time of reporting, 50% (4/8) of scholars had been promoted to associate professor compared with 17% (8/47) of assistant professors who had not participated in the program. After controlling for duration (years) in the assistant professor rank, promotion to associate professor remained significantly higher among scholars (odds ratio [OR]=5.061; 95% CI 1.03-24.83).
Discussion
The UQuAL Scholars program—a mentorship program for clinical-track hospitalists—was associated with greater research confidence, successful dissemination, and higher academic promotion than non-scholar peers. The program was designed to address the need to improve research/QI knowledge and productivity among clinical-track hospitalists through mentored didactic and experiential learning. Our preprogram data indicated that while the majority of our hospital group had performed QI or research projects (69-75%), a smaller percentage had presented (54%) or published (43%) the work. Publication/dissemination of research/QI results was a major aim for the scholars program, and all eight scholars successfully disseminated their work at national meetings—though, to date, only one has published a first-author manuscript on their program project. For clinical faculty at our institution, presentations meet promotion criteria for “dissemination.” Post-program, four scholars became involved in grant-funded work suggesting their research scholarship will continue. Exploring ways to sustain project momentum post-program is a priority, and we are piloting expansion of UQuAL Scholars to a 2-year program with the second year focused on finishing the proposed manuscript.
Unlike other research mentorship models, UQuAL Scholars was not designed to create or support tenure-track research faculty. The vast majority of academic hospitalists do not report research as their primary interest. Even so, research training offers multiple potential benefits for clinical-track faculty. First, understanding how to proactively and systematically examine a research or clinical question prior to beginning a QI, educational, or operational project can improve the project’s impact on clinical practice. Second, for clinical track hospitalist faculty, dissemination—via poster, presentation, or publication—is critical for promotion to Associate Professor. Finally, demonstrable research knowledge and experience may make clinical track faculty better situated to become co-investigators, site principal investigators, or program leaders, which may offer protected time. For example, a unit leader who knows how to review the literature and ask and answer a scientifically answerable question is better poised to implement successful quality improvement interventions.
Post-program evaluations indicated that peer relationships developed during the program were highly valued and could serve as a resource for continued collaboration. Facilitating a peer-mentoring structure after the program could help sustain focus and momentum; however, maintaining a peer-mentoring network requires time and effort. In turn, experience with UQuAL Scholars could evolve into mentorship of more junior colleagues and other learners, thereby increasing the organization’s overall mentorship capacity. Given that hospital medicine has a small number of senior faculty, this capacity building is critical to supporting the continued academic success of the hospitalist program. Though not directly explored, the fact that most UQuAL Scholar projects included residents could positively impact our program’s ability to recruit talented academic hospitalists.
The post-program survey indicated some ambivalence (4/8 scholars neither agreeing or disagreeing) regarding the value of the program to help with academic promotion. This may be due to the short duration of the scholars program (9 months) relative to the time course of promotion (years). It is also true that academic hospitalists have multiple paths to promotion; promotion for clinical faculty is possible without strong research scholarship.
Limitations and Strengths
First, the small number of program participants and lack of a true comparison group limit our ability to broadly assess the impact of UQuAL Scholars. The scholars were motivated to participate in this program, which could mean they were more aggressively working toward promotion and research dissemination compared to others. It is likely that the scholars who were promoted were well on their way before they began the program. This self-selection of highly motivated scholars may be viewed as a strength because other clinical-track hospitalist faculty may pursue alternative paths to advancement (e.g., education) that better align with their interests. We suspect that targeting the scholars program to those most interested increases the likelihood of success and impact. Second, while the UQuAL Scholars program appeared to enhance research capabilities and address important barriers, there are other barriers, such as lack of funding and protected time that hinder productivity and may require institutional support to overcome. In both cohorts, mentees were highly motivated to attend sessions; however, enlisting additional institutional support for time to attend sessions and perform research/QI-related tasks may enhance engagement. We acknowledge that financial support is challenging to obtain. Finally, UQuAL Scholars was enabled by the leadership of two hospital medicine researchers. One of the program leaders is a tenure-track hospital medicine researcher, while the other is a clinical-track faculty member with experience conducting high-quality hospital medicine research. Not all hospital medicine programs have access to such leadership or the resources needed to protect their time for mentorship. Despite these limitations, we demonstrated that a hospitalist mentorship program for clinical track faculty is feasible, requires minimal resources, and improves junior clinical track faculty knowledge and productivity while contributing to the academic culture of a hospital medicine group. Specifically, this program increased research engagement and grant funding among clinical-track hospitalist faculty with limited research experience at the program’s outset. In conclusion, we found that a research mentoring program for clinical-track hospitalists is feasible, improves knowledge and research productivity, and may contribute to the promotion of junior faculty, while advancing the academic culture of a hospital medicine group.
Disclosures/Conflicts of Interest
No authors have disclosures or conflicts of interest.
Funding
This work was funded in part by the University of Utah Department of Internal Medicine. Dr. Vaughn was supported by a Career Development Award from the Agency for Healthcare Research and Quality [K08HS026530]. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research.
Corresponding Author
Andrea T. White, PhD
30 North Mario Capecchi Drive, 3rd floor South
Salt Lake City, UT 84112
Email: Andrea.white@hsc.utah.edu