A recent study by Boston Consulting Group (BCG) reveals that generative AI, a type of artificial intelligence that can create novel content, excels at creative tasks but flops at business problem solving. The study, conducted with the help of researchers from Harvard, MIT, Wharton and Warwick, tested the use of OpenAI’s GPT-4, a powerful language model, on more than 750 BCG consultants worldwide.
GenAI Boosts Creative Performance by 40%
The study found that participants who used GPT-4 for a creative product innovation task outperformed the control group, who completed the task without using GPT-4, by 40%. The task involved coming up with ideas for new products and go-to-market plans based on a brief description of a customer segment and a problem to solve.
The researchers attribute this remarkable improvement to the ability of generative AI to generate creative, novel and useful ideas based on the vast amounts of data on which it has been trained. They also found that participants did best when they did not attempt to edit GPT-4’s output, suggesting that human intervention may hamper the quality of the generated content.
GenAI Misleads Business Problem Solving by 23%
However, when participants used GPT-4 for business problem solving, a task that required them to identify the root cause of a company’s challenges based on performance data and interviews with executives, they performed 23% worse than those who did not use GPT-4. The task was outside the tool’s current competence, as it involved weighing qualitative and quantitative data to answer a complex question.
The researchers found that many participants who used GPT-4 for this task accepted the tool’s erroneous output at face value, even when they were warned about the possibility of wrong answers. They suggest that this may be due to the persuasive abilities of GPT-4, which can generate convincing but inaccurate content.
GenAI Adoption Requires Change Management and Leadership
The study highlights the potential and pitfalls of generative AI for different types of tasks. It also emphasizes the need for change management and leadership to help people use the new technology in the right way, for the right tasks and to continually adjust and adapt in the face of GenAI’s ever-expanding frontier.
The researchers recommend that organizations adopt generative AI in a systematic way, starting with pilot projects and scaling up gradually. They also suggest that leaders should foster a culture of experimentation and learning, provide clear guidance and feedback, and monitor the impact and risks of generative AI.
Generative AI will be a powerful enabler of competitive advantage for companies that crack the code of adoption. But it will also pose new challenges and ethical dilemmas that require careful consideration and responsible use.
