Configuration 5 moves even more of the work to the chatbot. While in configuration 4 human operators still applied some “finishing touches” to the support request, in configuration 5 they only oversee the work of the chatbot. Such an oversight might involve a formal approval of a response back to the customer or handling an exception where the chatbot cannot help. This reduces the touch time dramatically, allowing one human operator to oversee multiple chatbots in parallel. Ideally, the chatbots are capable of explicitly calling for human intervention when they are uncertain about a particular customer request. This could be achieved by having a second chatbot monitoring the interaction with the customer and, in real time, alert the human operator to intervene or provide explicit tools to the first chatbot that make it aware of its capabilities. Configuration 5 has much larger efficiency potential than configuration 4. The challenge is to determine how many chatbots a single human operator can oversee at the same time. In our example, patients submit requests to the chatbot on the hospital’s website and the chatbots provide a response that, before being posted in the chat, needs to be approved by the human operator. Configuration 6: Chatbot response with offline auditing by human operator The highest level of autonomy is realized by providing the chatbot with the authority to handle a customer support request without a human in the loop. Returning to our example, patients approach the chatbots with questions and the chatbot provides the answers. This approach reduces the touch time to zero and thus has the highest efficiency potential. However, the fact that there is no human in the loop does not imply that there should be no oversight. In configuration 6, management audits the automatically executed chats periodically, potentially with the help of an LLM to refine strategies and improve the customer support experience. During these audits, management can get a sense (though with a delay) about the quality of the support provided and what changes might be necessary. Configuration 7: Hybrid configurations and implementation A seventh configuration to consider is a combination of any of the previously discussed configurations. An organization might roll out a GenAI initiative by taking 100% of its calls using configuration 1 (human operators only) and use the accumulating data to fine-tune an LLM. Then, as a second step, it might conduct a set of experiments that would confront chatbots with the situations encountered by the human operators and estimate a level of confidence with which the chatbots are handling requests correctly. It is also conceivable that an organization might deploy different configurations based on the nature of the support request. For high-risk or high-compliance tasks, configurations 1 or 2 might be the best option, as they leave most of the autonomy with the human operator. For complex decision-making tasks, configurations 3 or 4 can remove cognitive burden from the human operator and allow them to quickly

Reimagining Customer Service Journeys with LLMs: A Framework for Chatbot Design and Workflow Integration - Page 11 Reimagining Customer Service Journeys with LLMs: A Framework for Chatbot Design and Workflow Integration Page 10 Page 12