In our discussions with companies, we often notice that the performance of existing bots is often critically questioned. The common feedback is as follows:
This occurs almost regardless of whether the intelligent software package is provided by companies such as IBM Watson, Cognigy, RASA, moin.ai, OMQ, Zendesk or others. But what is really behind this?
The intelligence of enterprise chatbots is not always sufficient, as they are trained on specific questions and processes and therefore are not able to handle unpredictable or complex requests. In addition, insufficient data quality and quantity can contribute to chatbots not always being able to provide accurate answers.
Furthermore, maintaining and updating content is often costly. Thus, it is crucial to provide the chatbot with accurate and relevant information to provide appropriate responses to queries. This requires the collaboration of different departments within the company to ensure that all relevant information is available. It may also be necessary to regularly review and update the content to ensure it is up to date and accurate.
To tackle these challenges, companies need to continuously train the chatbot, and permanently review and update its database. From our experience, this is where the main problem lies: The chatbot was initially trained on simple questions and processes and therefore has difficulties to process complex queries. However, since the respective data is not available, the chatbot cannot provide precise answers to such questions.
Improving the intelligence of a chatbot can be achieved using an AI-based knowledge management solution. This solution allows the chatbot to access a wide range of information and knowledge sources from your organization to provide more accurate and appropriate answers to customer questions. One such AI-based search engine, also known as an enterprise search system, is IntraFind iFinder. The enterprise search engine helps to increase the accuracy and relevance of answers by selecting and providing relevant information from the knowledge base or other connected data sources. Via so-called connectors, the iFinder can easily access various systems in your company, such as Microsoft 365 and SharePoint, Atlassian Confluence and Jira, Wikis and others.
In addition, an AI knowledge management solution also enables more efficient management and maintenance of data. When data is added, updated or deleted in the connected sources, the AI knowledge management solution automatically keeps the enterprise search engine up to date.
Overall, a high-quality AI knowledge management solution can improve a chatbot's intelligence by providing it with a broad, up-to-date knowledge base while giving it the ability to effectively use that base to provide relevant answers to user questions.
Alternatively, you can integrate generative models such as ChatGPT from OpenAI or Luminous from the German manufacturer Aleph Alpha into your question-answer communication instead of a specially trained chatbot. The advantage of combining an AI-based search engine like iFinder with an integrated generative AI model is that it only uses data that is effectively available to the users – whether based on their permissions or on the model that has been trained. This significantly minimizes the risk of "hallucinating", which can occur with generative models.
Our approach to improving your chatbots includes a Natural Language Question Answering module (NLQA) in the form of a question-answering system. This powerful search extension is specifically designed for interactive question-answering chatbots. With this solution, your chatbot can access enterprise-wide information quickly and efficiently. This, in turn, allows you to optimize your chatbot dialogues and significantly increase user satisfaction.
The NLQA module ensures that predefined answers are extracted from your company data by the iFinder and seamlessly integrated into the chatbot dialogue. The iFinder is an AI-based knowledge management solution that extracts terms, texts, keywords, and corresponding questions from various data sources and delivers matching results. The application searches not only your chatbot database, but also other data sources such as Confluence, SharePoint, other content management systems and even voice or chat transcripts.
The maintenance effort is reduced considerably, since the iFinder already provides many concepts out of the box, which minimizes the effort to extract the "intents" (recognition or understanding mechanisms) or questions. In addition, it draws on existing knowledge from the available information. For example, the search can automatically add synonyms to the search query so that it searches not only for "pension" but also for "retirement pension", "old-age pension", "occupational pension", "statutory pension", "old-age insurance" and "old-age provision" and returns corresponding hits. The textual preview of the search results already provides the context to the answer customers are looking for. The ranking by relevance allows a precise setting to select the best answer from the available information.
We believe that the advantages of Natural Language Question Answering systems outweigh pure chatbot applications in many respects. Therefore, we focus on implementing the question-answering system. Compared to a modelled chatbot, a question-answering system offers clear advantages, including lower initial project investments, shortened project runtimes and reduced maintenance efforts.
We would like to emphasize that a chatbot is helpful when users do not know exactly what they are looking for. In addition, a chatbot can be useful in situations where the search function is not used, whether due to lack of awareness or lack of access to data. Our experience shows that a search usually provides an answer faster than a guided dialogue.
Our question-answer system also allows voice-controlled searches. The iFinder provides an optional Speech-to-Search function. Our solution can be deployed on-premises on organization-owned servers or as a cloud-based SaaS service.
In our view, chatbot applications achieve the greatest added value when they directly complete the information transfer process and are actively involved in the handling of processes. For example, when ordering a pizza or taking out an insurance policy. If the main purpose of the application is to find and pass on information, the concept of generating answers directly from the existing content is the optimal solution.
A generative language model like ChatGPT cannot replace a search engine or knowledge base because it does not have up-to-date and detailed expertise. The better option is a search engine that integrates a generative model. A search engine can provide text passages from documents that are likely to contain answers to a posed question, and the language model can convert this information into an understandable answer that can be used in a chatbot.
Therefore, the combination of an AI-based search engine with automated text extraction, such as the iFinder, and a generative language model similar to ChatGPT is the ideal solution for companies. In this combination, the generative AI takes over the human-like, automated dialogue, while the iFinder extracts the relevant content from the company's data sources.
We would be happy to discuss your individual use case in person. We are experts in the field of knowledge management and have been working with various large-scale language models for many years. We offer various implementation proposals with their respective advantages and disadvantages that provide you with an optimal decision-making basis for your project or use case.
