AI Tool GPT-4 Gives BCG Employees a Productivity Power-Up!
The impact of generative artificial intelligence on jobs and employment has been a topic of ongoing discussion since its emergence. It is widely believed that AI will render certain jobs obsolete while simultaneously creating new opportunities. However, little research has been conducted on the collaborative potential of AI and human workers. This changed with a recent study conducted at the Boston Consulting Group (BCG), which examined the cumulative effects of an AI-assisted work environment by providing some participants with a GPT-4 powered AI tool.
The study was conducted by Harvard Business School and a preprint version has been uploaded to the Social Science Research Network. The study, titled Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality, collected data from 758 BCG consultants. These participants were randomly assigned to one of three conditions—no AI access, GPT-4 AI access, or GPT-4 AI access with a quick design review.
BCG consultants with GPT-4 AI outperform those without it
After the groups were divided, all participants were given 18 realistic consulting tasks. The study claims that the tasks were designed so that some of them would be easy for the AI to handle, while others were beyond its current capabilities. The study found that consultants who used GPT-4 were significantly more productive than their counterparts.
The AI group completed an average of 12.2 percent more tasks, which was completed 25.1 percent faster than the control group. In addition, the AI group produced more than 40 percent higher quality tasks compared to the group without it.
“Consultants from different skill sets benefited significantly from the addition of artificial intelligence, as those below the average performance threshold increased by 43 percent and above 17 percent compared to their own scores,” the study states.
Interestingly, for a task selected outside of AI capabilities, consultants using AI produced 19 percent fewer correct solutions compared to those without AI, highlighting the downside of relying on AI tools.
The study also found two different patterns that people showed when using AI for given tasks. The study said, “One group of consultants acted as ‘centaurs’, like a mythical half-horse/half-human creature, sharing and delegating their solution creation activities to AI or themselves. Another group of consultants acted more like ‘cyborgs’, fully integrating their tasks with AI and constantly interacting with the technology .
