19 We also want to be careful not to generalize our results beyond what we have presented. Our findings might be GPT-4 specific. Moreover, other idea domains than the one considered here are likely to lead to another ranking of the prompting strategies shown in Table 2. Nevertheless, we believe that CoT has a fundamental advantage. Yet, specific personas might or might not change if we used another LLM. Implications and Conclusions Our results have strong implications for anyone who wants to use LLMs for augmenting human idea generation. First, we confirm our hypothesis that generative AI currently produces less diverse ideas than a group of humans. By comparing the diversity of ideas generated by a whole array of different prompting strategies with the diversity of ideas generated by groups of humans, we find that humans still seem to have a slight advantage in coming up with diverse ideas compared to state of the art large language models and prompting. Second, and maybe not surprisingly, we show that prompting strategies do make a difference. Prompt engineering (i.e., prompting “the right way”) dramatically increases idea diversity. Specifically, we find that longer and more elaborate prompts work well. This is, as hypothesized, especially true of Chain-of-Thought (CoT) prompting. Finally, we find that the overlap that is obtained from different prompts is relatively low. This makes hybrid prompting, i.e., generating smaller pools of ideas with different prompting strategies and then combining these pools, an attractive strategy. After many years of research in the field of creativity and innovation with the objective of teaching and managing the innovation process (Terwiesch and Ulrich 2009), the new technology of LLM’s now enables aspiring innovators to use our enhanced understanding of the innovation process to automate it. When done so with human supervision, this should allow for more, better, (and if prompted correctly) also more diverse ideas.

Prompting Diverse Ideas: Increasing AI Idea Variance - Page 19 Prompting Diverse Ideas: Increasing AI Idea Variance Page 18 Page 20