3 outcome metrics we use the cosine similarity (Manning 2008) of the idea pool, the total number of unique ideas that can be identified by a prompt, and the speed at which the idea generation process gets exhausted and ideas start repeating themselves (see Kornish and Ulrich 2011). The domain of idea generation we consider is the search for a new product to be developed and launched. Specifically, we seek a new consumer product targeted to college students that can be sold for $50 or less. The main reason for this choice is that we have a pool of comparison from a Wharton MBA class and have used this idea domain in prior studies. Our main findings are as follows: ● We confirm the diversity achilles of AI generated brainstorming by showing that pools of ideas generated by GPT-4 with no special prompting are less diverse than ideas generated by groups of human subjects. Specifically, we find that (a) Cosine similarity for ideas generated by groups of humans is around 0.243 compared to 0.255 - 0.432 for GPT-4 generated ideas. ● Comparing an array of prompts that vary in wording and in problem solving strategy, we show that the diversity of AI generated ideas can be substantially improved using prompt engineering. For example, we show that instructing GPT-4 to think like Steve Jobs is effective in increasing the diversity of the resulting ideas (0.368 cosine similarity versus the baseline of 0.377) while prompting GPT-4 with recommended creativity tools published by the Harvard Business Review (cosine similarity of 0.387) less so. Overall, we compare 35 prompts in their ability to reduce cosine similarity, increase the number of unique ideas, and keep the ideation process from fatiguing. ● In the comparison of prompting strategies, we show that Chain-of-Thought (CoT) prompting leads to one of the highest diversity of the idea pools of all prompts we evaluated and was able to obtain a diversity nearly as high as groups of humans. CoT prompting breaks up the overall brainstorming task into a series of micro tasks and has been found to be highly effective in other LLM applications such as solving mathematical problems (see Wei et al 2022). We further show that CoT increases the number of unique ideas that can be generated in our domain from around 3700 for the base prompt to 4700. Theory and Hypotheses: The Importance of Idea Diversity The atomic unit of analysis in our study is an idea. We define an idea as a novel match between a solution and an unmet need. As mentioned above, our focus in this paper is on ideas for new

Prompting Diverse Ideas: Increasing AI Idea Variance - Page 3 Prompting Diverse Ideas: Increasing AI Idea Variance Page 2 Page 4