Reimagining Customer Service Journeys with LLMs: A Framework for Chatbot Design and Workflow Integration

Reimagining Customer Service Journeys with LLMs: A Framework for Chatbot Design and Workflow Integration Lennart Meincke and Christian Terwiesch1 January 23, 2025 Mack Institute for Innovation Management, The Wharton School, University of Pennsylvania Introduction Whether assisting travelers with inquiries about their upcoming flight or destination, providing mental health support, helping students with homework assignments, or aiding in financial planning, chatbots powered by generative artificial intelligence (GenAI) in the form of large language models (LLMs) can help deliver efficient, scalable, and personalized customer support. Building an effective chatbot involves navigating a series of design decisions that influence user experience, technical implementations, and various legal and ethical issues. This white paper outlines a general framework for the functionality of chatbots and their operational use. Our aim is to help executives navigate the complex decisions that need to be made when leveraging the power of LLMs to support their customers, be they travelers, patients, students, or investors. In the first part of this paper, we articulate five design choices that need to be made when building a chatbot. These choices provide the answers to five important managerial questions: 1. Focused vs. Broad Knowledge Base (“How much does the chatbot know?”) 2. Isolated Interactions vs. Long-time Relationships (“Does the chatbot remember users over multiple episodes?”) 3. Proactive vs. Responsive (“Does the user reach out to the chatbot or the chatbot to the user?”) 4. Static vs. Dynamic (“Does the chatbot learn about a user or a user population over time?”) 5. Quality Assurance (“How is the quality of the chatbot’s output assured?”) These five dimensions create a taxonomy of chatbots. In the second part of the paper, we discuss different modes of interactions between the chatbot and the human operators. Indeed, a successful use of chatbots is more than just a matter of designing and building a new technology. Rather, the chatbot must be integrated into the workflow of the organization. At the 1 Meincke, Terwiesch: The Wharton School, 500 Huntsman Hall, 3730 Walnut Street, Philadelphia, PA 19104, lennart@sas.upenn.edu, terwiesch@wharton.upenn.edu