Meta-analysis question : How effective are leadership training programs in improving leadership efficacy?

A broad literature search was undertaken in order to identify as many studies as possible where a form of training or intervention was conducted, and leader efficacy measured. We specifically searched for this Boolean string (and their translation in French): (“leader efficacy” OR “leadership efficacy” OR “leader self-efficacy”) AND (“training” OR “intervention”). We aimed to find studies published in English and French mainly, although work in other languages was also considered were feasible. We included any work that included our combined search terms, setting no exclusion criteria in order to be as inclusive as possible and avoid missing relevant work. We started our search on Google Scholar, to find all work that cited the article by Quigley (2013) and work that cited Hannah et al. (2008). We then looked at relevant meta-analyses (i.e., Avolio et al., 2009; Lacarenza et al., 2017) to include the studies that were included in their meta-analyses. Then, we searched the most prominent databases (i.e., PsychInfo, Proquest, ERIC, and Medline), using a full text search.

We also sought to include theses and dissertations by searching Proquest Dissertations and Theses Global, based on a title and abstract search. We also searched national databases of countries and regions with theses and dissertations in French or English (i.e., Australian Education Index, PASCAL, Theses Canada). We then searched conference proceedings of relevant scholarly conferences (i.e., AIPTLF, AOM, ASAC, SIOP, EAWOP, SMA, WAM, EAM, Midwestern AOM, British AOM, African AOM, Asia AOM, European AOM, Australian and NZ AOM, Indian AOM).

To research how disabled participants favor against non-disabled participants in selection measures, I utilized the following search terms across three databases - Business Source Complete, Academic Search Complete, and APA PsycINFO:

"Employment interviews" OR "job interviews" OR "hiring interviews" OR "general mental ability" OR "general mental ability test" OR "general mental ability tests" OR "general mental ability exam" OR "general mental ability exams" OR "general cognitive ability" OR "general cognitive ability test" OR "general cognitive ability tests" OR "general cognitive ability exam" OR "general cognitive ability exams"OR "situational judgment test" OR "situational judgment tests" OR "sjt" OR "hiring process" OR "hiring practice" OR "employee selection" OR "personnel selection" OR "resume" OR "personality assessment" OR "personality test" OR "personality tests" OR "personality exam" OR "personality exams" OR "employment assessment" OR "employment test" OR "employment tests" OR "employment exam" OR "employment exams" OR "assessment center" OR "personnel selection" OR "employee selection" OR "performance evaluation" OR "performance appraisal" OR "performance assessment" OR "performance expectation" OR "decision to hire" OR "adverse impact" OR "employment discrimination" OR "workplace discrimination" OR "temperament test" OR "temperament tests" OR "temperament exam" OR "temperament exams" OR "job knowledge" OR "job knowledge test" OR "job knowledge tests" OR "job knowledge exam" OR "job knowledge exams" OR "physical ability" OR "physical ability test" OR "physical ability tests" OR "physical ability exam" OR "physical ability exams" OR "spatial ability" OR "spatial ability test" OR "spatial ability tests" OR "spatial ability exam" OR "spatial ability exams" OR "psychomotor ability" OR "psychomotor ability test" OR "psychomotor ability tests" OR "psychomotor ability exam" OR "psychomotor ability exams" OR "biodata" OR "accomplishment record" OR "work sample"
AND
”disability" OR "disabled" or "disabilities" OR "disorder" OR "disease" OR "health condition" OR "neurodiversity" OR "neurodiverse" OR "neurodivergent" OR "neurodivergence" OR "physically impaired" OR "physical impairment"

From this search, with the default filter as “Select a Field - Optional,” I got 29,535 results - my library caps exporting into one file at 25,000. If I filtered by all text, I got 205,601. Filtering out by titles only resulted in 300 results. Filtering by subject terms resulted in 11,345 results. Filtering by abstract resulted in 6,395 results.

In this case, would filtering by subject terms be my best option? I know we discussed the cons of using it yesterday, but I’m surprised that filtering by the abstract has limited my search so much.

I conducted a systematic literature search to identify studies that examine how male allyship influences perceptions of support for female leaders, with a specific focus on differences in perceptions between male and female observers. I searched the following electronic databases: PsycINFO, Web of Science, ABI Inform, and Google Scholar. The search string included combinations of terms related to male allyship, perceptions, and female leadership. The Boolean string used was: "male allyship" OR "men as allies" AND "female leaders" OR "female leadership" AND "perception." The search was conducted on full-text articles and returned a total of 15,029 results. A record of the search process, including the number of studies retrieved and excluded at each stage, will be maintained and reported in the final version of the paper.

Our focus is on employee ethics training programs in organizations vs ethics education / ethical conduct of research

Informed by:

• Lacerenza, C. N., Reyes, D. L., Marlow, S. L., Joseph, D. L., & Salas, E. (2017). Leadership training design, delivery, and implementation: A meta-analysis. Journal of Applied Psychology, 102(12), 1686–1718.  https://doi.org/10.1037/apl0000241
• Tkachenko, O., & Hedayati, A. (2022). Employee ethics training: Literature review and integrative perspective. In Academy of Management Proceedings (Vol. 2022, No. 1, p. 17746).

Hypotheses

Hypothesis 1a: Ethics training programs have a positive effect on trainee reactions.

Hypothesis 1b: Ethics training programs have a positive effect on affective-, cognitive-, and skill-based learning outcomes (i.e., measured at the end of training).

Hypothesis 1c: Ethics training programs lead to the transfer of trained affective-, cognitive-, and skill-based concepts (i.e., measured on the job, after training).

Hypothesis 1d: Ethics training programs positively influence organizational outcomes.

Hypothesis 2: Ethics training programs delivered in multiple training sessions result in greater effects on reactions (H2a), learning (H2b), transfer (H2c), and results (H2d) compared to programs delivered in one training session.

Hypothesis 3: Ethics training programs implementing ‘active learning’ result in greater effects on reactions (H3a), learning (H3b), transfer (H3c), and results (H3d) compared to programs that rely only on lectures and/or presentations (i.e., demonstration and information).

(active learning = discussions, group activities, case studies, etc.).

Hypothesis 4: Face-to-face and hybrid ethics training programs increase positive trainee reactions (H4a), learning (H4b), transfer (H4c), and results (H4d) to a greater degree than virtual programs.

…

Search

We identified and selected relevant empirical studies in several ways. First, we conducted literature searches in the databases PsycINFO (xyz–June 2025), Business Source Premiere (xyz–June 2025), Web of Science (xyz–June 2025) and ProQuest Dissertations and Theses (xyz–June 2025).

We used the following keywords in our initial search: “ethics training,” “ethics programs,” “moral development,” “moral education,” “ethics education,” and “ethical development.” As our focus was on employee ethics training, we excluded abstracts with the following terms: “student” or “classroom.”

Second, we cross-checked reference lists from previous meta-analyses on pertinent topics (e.g., Steele et al., 2016; Watts et al., 2017). We also included unpublished studies (xyz) in order to reduce the potential for publication bias.

This initial search returned XYZ studies.

Each article was then reviewed and included if the following conditions were met:

(a) it reported an empirical evaluation of employee ethics training (moral development) program;

(b) the study used a repeated measures design, independent groups design, or an independent groups design with repeated measures;

(c) the participants were employees, including medical residents and pre-service teachers (e.g., not MBAs/undergraduate students);

(d) the study was published in English;

(e) it provided the sample size and sufficient information to calculate an effect size.

To wit, studies that focused on ethics education in academia (i.e., ethics courses and ethical conduct of research) were not considered.

Applying the above criteria, we received a final sample of XYZ independent studies (N=XYZ).

References:

Steele, L. M., Mulhearn, T. J., Medeiros, K. E., Watts, L. L., Connelly, S., & Mumford, M. D. (2016). How do we know what works? A review and critique of current practices in ethics training evaluation. Accountability in Research, 23(6), 319-350. https://doi.org/10.1080/08989621.2016.1186547

Watts, L. L., Medeiros, K. E., Mulhearn, T. J., Steele, L. M., Connelly, S., & Mumford, M. D. (2017). Are ethics training programs improving? A meta-analytic review of past and present ethics instruction in the sciences. Ethics & Behavior, 27(5), 351-384. https://doi.org/10.1080/10508422.2016.1182025

I will search for published articles in APA PsycINFO, Web of Science, Business Source Complete, ABI-Inform, and ProQuest Dissertation and Theses Global. I will search for unpublished articles in PsyArXiv, SocArXiv, and the Society for Industrial Organizational Psychology and Academy of Management conference programs. I will then review the reference sections of previous meta-analyses on similar topics (affirmative action reactions, DEI ideology, applicant attraction) (Chapman et al., 2005; Greig et al., 2023; Harrison et al., 2006; Leslie et al., 2014, 2020; Swider et al., 2015; Uggerslev et al., 2012).

Based on the meta-analyses of existing topics (i.e., Chapman et al., 2005; Greig et al., 2023; Harrison et al., 2006; Leslie et al., 2014, 2020; Swider et al., 2015; Uggerslev et al., 2012), the following search terms will be use: diversity, signal, DEI, race, gender, identity-safety, identity-threat, multicultural, colorblind, affirmative action , identity, meritocracy, justice, inclusion, exclusion, intergroup ideology, fairness, equal opportunity, corporate social responsibility, and personnel selection, recruitment, job applicant, job recruit, employee, job advertisement, applicant attraction, applicant attitude, applicant reaction, job choice, job acceptance, organizational attraction).

For this meta-analysis, I conducted a comprehensive search for empirical studies on destructive leadership types and employee identity that were available as of June 2025. Based on Schyns and Schilling’s (2013) review of destructive leadership types, and with the help of ChatGPT, I first collated a list of destructive leadership types and behaviors (both overt and covert) and identity-related keywords that are prevalently used in the identity literature. This resulted in the following Boolean search string: ("destructive leadership" OR "abusive leadership" OR "social undermining" OR "harassment" OR "petty tyranny" OR "coercive control" OR "abusive supervisory behavior*" OR "unsupportive managerial behavior*" OR "aversive leadership" OR "tyrannical leadership" OR "despotic leadership" OR "toxic leadership" OR "passive leadership" OR "exploitative leadership" OR "gaslighting" OR "bullying" OR "narcissistic leadership" OR "psychological aggression" OR "undermining leadership" OR "incivility" OR "supervisor mistreatment") AND ("employee identity" OR "professional identity" OR "work identity" OR "occupational identity" OR "role identity" OR "organizational identification" OR "self-concept" OR "identity threat" OR "identity work" OR "self-verification" OR "identity disruption" OR "authenticity at work" OR "sense of self"). Second, I search three scholarly databases (i.e., APA PsychInfo, ABI/INFORM Collection, Business Source Complete) for journal articles, dissertations and theses, conference papers and proceedings, working papers, books, and chapters in the full text of the manuscript. Lastly, I further searched the ProQuest Dissertation & Theses Global database for unpublished dissertations and theses, including the key terms from the Boolean string that only had “leadership” in them in the search. Combined, after removing duplicates, these efforts resulted in 9,896 articles for screening.

The Causal Effect of Leadership Interventions: A Meta-Analysis of Experimental and Quasi-Experimental Evidence on Individual and Team Performance

For this meta-analysis, we conducted a comprehensive search for experimental studies on leadership available by June 2025, encompassing publications in English, Spanish, and Portuguese. Our aim was to synthesize causal evidence about leadership by examining experimental manipulations of leader behaviors and styles across diverse contexts. The search process began with an examination of studies included in Avolio et al.'s (2009) foundational meta-analysis to establish continuity with prior research. We then expanded our search to include in-press and online-first articles from Financial Times 50-listed journals through systematic searches of PsycINFO, Web of Science, and ScienceDirect. To ensure thorough coverage, we additionally searched conference proceedings from the Academy of Management (AOM) and Society for Industrial and Organizational Psychology (SIOP), as well as unpublished dissertations through ProQuest Dissertations and Theses. Finally, we employed Google Scholar to identify potentially overlooked studies, with particular attention to international research that might not appear in conventional databases. This multi-phase approach combined traditional systematic review methods with proactive efforts to minimize publication bias and capture emerging work.

To build a comprehensive and representative dataset for our meta-analysis on the relationship between inclusive leadership and burnout, we conducted a broad literature search as of June 2025. Our aim was to identify empirical studies that reported a statistical association (e.g., correlations, effect sizes) between inclusive leadership and employee burnout.

We began by using Google Scholar to locate studies that cited foundational works on inclusive leadership (such as Carmeli et al., 2010, and Randel et al., 2018) and commonly used burnout measures (like the Maslach Burnout Inventory). We then reviewed recent literature reviews and meta-analyses that explored related constructs such as servant leadership, psychological safety, and employee well-being, in order to identify additional relevant studies and ensure broad coverage of the topic.

To expand the search, we manually reviewed the reference lists of key empirical papers and targeted top-tier journals in management, psychology, and organizational behavior. This included journals listed on the Financial Times 50 list (e.g., Academy of Management Journal, Journal of Applied Psychology) as well as those ranked 3 or above in categories like “HRM & Employment,” “Organizational Studies,” and “Work & Organizational Psychology” in the Chartered Association of Business Schools’ Academic Journal Guide. We also checked early access articles and online-first publications from relevant journals.

In addition to peer-reviewed sources, we searched databases such as PsycINFO, ProQuest Dissertations and Theses, Web of Science, and SSRN to capture unpublished studies, conference papers, dissertations, and technical reports. This step was taken to reduce publication bias and address the risk of underreporting non-significant or null findings.

Finally, we used HubMeta, a web-based tool for synthesizing evidence in the behavioral and organizational sciences. Within HubMeta, we searched for relevant constructs using their internal taxonomy and explored studies that measured inclusive leadership behaviors and burnout indicators, including emotional exhaustion, cynicism, and depersonalization. Boolean logic and structured search filters helped us refine and verify the relevance of studies for inclusion in our dataset.

You may want to cross-check reference lists from previous meta-analyses on pertinent topics.

On a different note: 'selection measures' sounds as if you're looking at tests and assessments (e.g., is job interview a selection measure?)

Hi Julia!

This sounds super interesting. Do you think you'll also search for dissertations and theses on the topic? I imagine there's been a boost in interest in these areas especially in the last five years, so there may not be as many published as unpublished manuscripts to consider.

Thanks for sharing!

We conducted a literature search for empirical work across three databases: PsycInfo, Web of Science, and ProQuestion Dissertations and Theses. We searched for terms that included combinations of phrases synonymous with “trait fit” (e.g., personality congruence), as well as terms that indicate a team-level personality construct (e.g., “team personality”). A complete list of Boolean operators and all terms is listed in Appendix A. Additionally, we conducted a hand-search of articles that had cited (as indicated by google scholar) two prior meta-analyses on this topic (Bell et al., 2007; Prewett et al., 2009).

___________

Question: I was playing with Boolean operators/search terms yesterday and I ended up with an extremely long string in order to avoid the issue of "personality" and "teams" returning 500k+ results. Is there a limit to the number of search terms that's appropriate? Is there any downside to an extremely long string and reporting that string exactly (e.g., in an Appendix as above).

___________

Edit: When I updated this w/a typo correction, it defaulted the name to Sam Stallion . . . sorry, Sam!

As a first step in our literature search, we did a full-text search in three main databases (i.e., PsycINFO, Web of Science, ABI/Inform), using the following search terms: "self-initiative" or "job crafting" or "work proactivity" or "proactive personality" or "personal initiative" or "proactive work behavior" or "proactive work behaviour” or "work proactivity" or "proactive behavior" or "proactive behaviour" or "proactivity" or "career proactivity" or "proactive career" or "taking charge" or "proactive feedback seeking" or “proactive career" or “proactive person-environment fit" or "proactive coping" or “voice” or “proactive information seeking” and “training” or “intervention” or “quasi-experiment”. Second, we searched the Proquest Dissertation and Theses database to attain a more balanced distribution of significant and nonsignificant findings, thereby limiting the effects of publication bias (Kepes et al., 2012). Finally, as a supplementary step, we did a search in Google scholar. We screened the search findings until 100 search results in a row were no longer relevant.

To identify relevant studies, a comprehensive literature search was conducted in June of 2025 across multiple scholarly databases (i.e., PsycINFO, ProQuest Dissertations and Theses, Web of Science, ScienceDirect). The search focused on empirical studies examining the effects of fringe job benefits (e.g., health insurance, paid leave, retirement plans, tuition reimbursement) on employee attitudes and behaviors relevant to recruitment and retention. Boolean search terms combined constructs related to fringe benefits (“fringe benefits” OR “employee benefits” OR “workplace benefits”), psychological mechanisms (“perceived organizational support” OR “POS” OR “person organization fit” OR “P O fit”), and outcome variables (“job pursuit intentions” OR “organizational attractiveness” OR “job acceptance” OR “affective commitment” OR “turnover” OR “employee retention”).

Our search was not limited to specific journals or publication formats, and we extensively searched dissertation and theses databases to minimize publication bias. However, our search did focus on manuscripts published in English and focused on United States samples due to the complexity of employment benefits in the country (i.e., at-will laws, absence of universal healthcare). Additional studies were identified through reference chaining of prior reviews and meta-analyses as well as conference proceedings and abstracts (i.e., AOM, SIOP, EAWOP, SMA). Next, we searched publicly available datasets (i.e., data.gov) for relevant statistics on fringe job benefits and outcomes. Lastly, we searched Google Scholar where articles were screened until we went through 100 results with no relevant studies. A PRISMA-style flow diagram was used to document the identification, screening, and inclusion process.

Hi Kathleen! 

My one suggestion for including your write-up would be to clarify this sentence: "We aimed to find studies published in English and French mainly, although work in other languages was also considered were feasible." I think it's the phrasing "work in other languages was also considered were feasible" that's throwing me off. What other languages did you consider? Did you look at studies that had English or French abstracts but a main body in another language, and ask for translation help? I think more clarity around that decision making would improve your methods section!