Mustafa Akben, a faculty member in the Martha and Spencer Love School of Business, created a machine learning model effective in evaluating job performance and leadership potential simply by analyzing emails.
Is it possible to use artificial intelligence to identify job candidates with leadership skills simply by reading their email responses to fictional situations in the workplace?
Such was the challenge for Assistant Professor Mustafa Akben, who developed an AI model over the winter months that was more accurate than two dozen other entries in an international contest hosted by the Society for Industrial and Organizational Psychology.
The SIOP 2023 Machine Learning Competition centered on natural language processing, a subfield of artificial intelligence focused on enabling computers to understand, interpret, and generate human language most recently in headlines because of its use in ChatGPT.
Here’s how the contest worked:
In a work assessment center, job candidates were asked to reply by email to fictional scenarios they received during a daylong simulation. The way they analyzed scenarios, and the tone and the content of their replies, were scored by a panel of human judges.
Teams in the machine learning competition were given a spreadsheet with the prompts and the job candidates’ replies. However, they only received a limited numbers of the scores. The task: determine how the human judges scored the second batch of replies by building a model that looked at the scores for the first batch.
Akben researches the way artificial intelligence impacts human productivity and creativity. The AI model he developed performed the best overall of predicting how judges had rated the second batch of candidates in the public board, leading to a virtual presentation he delivered in April to in-person and online attendees at the 2023 SIOP Annual Conference in Boston.
“I’m absolutely thrilled by this win, both on a personal and professional level,” Akben said. “It feels amazing to see that all the hard work, dedication, and persistence I’ve poured into my AI, machine learning and organizational behavior research is paying off. This achievement strengthens my dedication to excellence within the academic community at Elon. It’s a great example for our students, illustrating the value of lifelong learning and the pursuit of success in their areas of interest.
“Plus, it really showcases how interdisciplinary research can drive innovation and tackle some of the most intricate challenges we face today in modern workplaces.”
That doesn’t mean AI is on the verge of overtaking human assessments for job performance and leadership traits, Akben said. Though his model was the best at predicting, on average, how candidates were rated in the simulation, the resulting effect size was only moderate.
Akben said he would like to see the effect sizes grow higher and that part of the continued improvement in artificial intelligence will also be in the elimination of bias. Citing the parameters of the competition, he noted that machines might not account for English being someone’s second language. Machines may count syntax or grammatical errors against those whose other talents are downplayed by a focus on communication.
Akben joined Elon University’s Department of Management and Entrepreneurship in 2022 after earning his undergraduate degree at Marmara University, a Master of Science at the University of Pennsylvania, and his doctorate at Temple University.
His scholarly interests in proactivity, creativity, social network analysis, biohacking, and machine learning are reflected in many of the experiential learning assignments he’s already created for his principles of management courses at the university.
“As an educator,” Akben said, “I incorporate emerging AI into my classroom – for example, ChatGPT and other large language models – and my students often express their enthusiasm and fascination when exposed to these technologies. I am eager to collaborate on AI-related projects involving workplace or educational applications, creativity, and proactivity with Elon University students or faculty members.”