Configuration

InSpire AI Chat and Insight Services for RAG

Introduction

Mindbreeze InSpire AI Chat offers the possibility of a conversational interface based on generative AI. Thanks to the Insight Services for Retrieval Augmented Generation (RAG), a Large Language Model (LLM) using Mindbreeze InSpire is able to generate answers from the user's input based on the hits in Mindbreeze.

Preparation

Two prerequisites must be fulfilled:

• Activation of the corresponding features in Mindbreeze InSpire so that the configuration can be carried out.
• Operation of an LLM.

To activate the features, please contact Mindbreeze Support (support@mindbreeze.com) for details on activating the features via Mindbreeze InSpire Feature Flags.

For the operation of an LLM, please contact Mindbreeze Sales (sales@mindbreeze.com) to use an LLM. On-premise customers have the option of using GPU appliances and Mindbreeze SaaS customers can use a remote LLM. For the use of existing LLM infrastructures, the Huggingface TGI interface is currently supported and OAuth can be used as an authorisation option.

Activating the feature

Please contact Mindbreeze Support (support@mindbreeze.com) and, if available, submit your existing feature flag configuration "/etc/mindbreeze/enabled_features.json" to receive an updated version. Then update the new enabled_features.json that you received from Mindbreeze Support.

Then restart the InSpire container. To do this, open the "Setup" menu item in the Management Centre and then click on the "InSpire" submenu item. You can restart InSpire in the "Container Management" area.

Once the InSpire has been restarted and the feature has been activated correctly, you should see in the menu item "Insight Services" the new submenu item "RAG" in the Management Centre.

Configuration

Insight Services for Retrieval Augmented Generation

Open the menu item "Configuration" in the Mindbreeze Management Center. Create a new service in the "Indices" tab by clicking on "Add Service". Give the service a name under "Display Name" and select the option "Insight Services for RAG" in the "Service" setting.

Make sure that in the "Base Configuration" section the configured "Bind port" is not yet occupied. If necessary, activate the setting "Include prompt in app.telemetry". With this setting, the questions entered by the users and the prompts that are sent to the LLM are included in the app.telemetry entries. This is deactivated by default for security reasons.

„Log HTTP Requests in verbose Log Mode“ is enabled per default. If this setting and the setting „Full Logging“ (“Advanced Settings” must be activated in Client Services) are enabled, the logs will also log the details for the sent requests.

The setting "Generate with empty Results" is activated by default. If you disable this option, a response will only be generated in the chat if results are found in the index, otherwise an error message will be displayed, this can be useful to prevent responses being given without reference to the indexed data.

The setting "Path to Store Base" can be used to optionally configure the path of the service.

If “Path to Store Base” is defined, based on the path, the service configuration may not be included in the snapshots. If no “Path to Store Base” is defined only the service configurations (pipelines and LLMs) get packed in the snapshot but not the service data (datasets). If all data should be packed into a snapshot, the “Path to Store Base” should be set to a directory which content is fully included in a snapshot. For more information about migrating with a Snapshot, see Handbook - Backup & Restore - Creating a snapshot.

Attention: If the setting "Path to Store Base" is not set, it can occur that the service data for the pipeline and the LLM is not displayed after the application of a Snapshot. To solve this, see the chapter Service data for the pipeline and LLM vanished after applying a Snapshot.

Advanced Settings

Activate the "Advanced Settings" for the following settings.

Self-Signed SSL Certificate

In the "Security Configuration" section, activate the setting "Disable SSL Certificate Validation". This setting is deactivated by default.

Finally, save the changes by clicking on "Save".

Impersonation Configuration

In the section “Impersonation Configuration”, you can enter the endpoint mapping pattern, among other things. The endpoints are defined in the “Network” tab in the Configuration of the Management Center.

With this setting, the retriever retrieves the client certificate for the client service by comparing it with the pattern defined there. The following placeholder can be used for the pattern:

- client_service_id: ID of the client service

The endpoint mapping only takes effect if no "Trusted Peer Certificate" is selected and the HTTP header configured in the Impersonation Identifier Setting is also sent in the Generate request.

Finally, save the changes by clicking on "Save".

InSpire AI Chat

Switch to the "Client Services" tab and activate the advanced settings by ticking the box "Advanced Settings".

If available, use an existing client service. If no client service is available, add a client service by clicking on "Add Client Service". Give the client service a name with the setting "Display Name".

Go to the "Chat UI Settings" section and select the service you created in the "Indices" tab in the setting "Chat Service".

After the configuration, you can access the AI Chat using the path apps/chat, similar to the Insight App Designer. The full path looks like this: https://example.com/apps/chat/.

Personalised theme (optional configuration)

If a personalised theme is desired in AI Chat, this can be set as follows:

In the Management Center in the menu item "File Manager", the folder /data/apps/chat-theme must be created, if it has not yet been created, in which the following files are stored:

Activate the "Advanced Settings" in the "Client Services" tab and now create an "Additional Context" in your client service in the section "Web Applications Contexts Settings". Activate "Override Existing Path" there and enter /apps/chat/theme as the "URL Path" and /data/apps/chat-theme as the "File Path".

Save the configuration by clicking on "Save".

Activating the NLQA index

Activate "Natural Language Question Answering" (NLQA) in the desired indexes. For more information about the configuration, see Create NLQA index and Activate NLQA on existing index.

Setting up the Large Language Model (LLM)

On-premise customers have the option to use GPU appliances or Mindbreeze SaaS customers can use a remote LLM. Please contact sales@mindbreeze.com regarding options for running an LLM with Mindbreeze.

For the use of existing, external LLM models, the Huggingface Text Generation Interface (TGI) is currently supported. If necessary, OAuth can be used as an authorisation option.  If you want to use a self-hosted LLM with a different interface, please contact Mindbreeze Support (support@mindbreeze.com) to check compatibility.

To set up the LLM, click on the menu item "Insight Services" in the Management Center and open the submenu item "RAG". Select the "LLMs" area there.

Click on "Add" to configure a new LLM. The following values are supplied by the respective LLM:

Save your configuration by clicking on "Save". You can find more details on the administration of models and pipelines in Whitepaper - Administration of Insight Services for Retrieval Augmented Generation.

External LLM with authorisation

If you want to use an external LLM, please contact sales@mindbreeze.com. The sales team will discuss your specific situation with you and provide you with a tailored offer.

The mapping of LLM in the RAG Administration to the required credentials is done via Endpoint Mapping. To configure the authorisation, open the menu item "Configuration" in the Management Center and go to the tab "Network". Create a new credential of type "OAuth 2" with the information provided by Mindbreeze. See the following screenshot for an example.

Create a new endpoint with this credential. The "Location" of the endpoint is the URL of the LLM.

Creating a pipeline and using InSpire AI Chat

To use InSpire AI Chat, a pipeline with a model must be created. The steps required for this can be found in Whitepaper - Administration of Insight Services for Retrieval Augmented Generation.

Authorisation of Management Center users for RAG administration

To administer Insight Services for RAG, a user must have the following roles:

An administrator can assign these roles to a user or group in the Management Center under the menu item "Setup". To do this, click on the "Credentials" submenu item and assign the necessary roles to the user or group. For more information on assigning roles, see Configuration – Back-End Credentials.

Troubleshooting

Service Configuration data for the pipeline and LLM is not displayed after applying a Snapshot

After the application of a Snapshot, the problem can occur that already existing service configuration data for a pipeline and LLM is not displayed anymore. The reason is that there is no defined path in the setting “Path to Store Base” and therefore the pipeline and LLM data are not displayed in the RAG service.

To solve this, please restart your RAG service. After the restart, the service configuration data should be displayed again.