Home
Home
German Version
Support
Imprint
25.2 Release ►

Start Chat with Collection

Main Navigation

  • Preparation
  • Datasources
  • Configuration
  • Operations
  • User Manual
  • SDK
  • Release Notes
  • Security
  • Product Information
Home

Path

Sure, you can handle it. But should you?
Let our experts manage the tech maintenance while you focus on your business.
See Consulting Packages

Configuration
InSpire AI Chat and Insight Services for RAG

IntroductionPermanent link for this heading

Mindbreeze InSpire AI Chat offers the possibility of a conversational interface based on generative AI. Thanks to the Insight Services for Retrieval Augmented Generation (RAG), a Large Language Model (LLM) using Mindbreeze InSpire is able to generate answers from the user's input based on the hits in Mindbreeze.

PreparationPermanent link for this heading

Two prerequisites must be fulfilled:

  • Activation of the corresponding features in Mindbreeze InSpire so that the configuration can be carried out.
  • Operation of an LLM.

To activate the features, please contact Mindbreeze Support (support@mindbreeze.com) for details on activating the features via Mindbreeze InSpire Feature Flags.

For the operation of an LLM, please contact Mindbreeze Sales (sales@mindbreeze.com) to use an LLM. On-premise customers have the option of using GPU appliances and Mindbreeze SaaS customers can use a remote LLM. For the use of existing LLM infrastructures, the Huggingface TGI interface is currently supported and OAuth can be used as an authorisation option.

Activating the featurePermanent link for this heading

Please contact Mindbreeze Support (support@mindbreeze.com) and, if available, submit your existing feature flag configuration "/etc/mindbreeze/enabled_features.json" to receive an updated version. Then update the new enabled_features.json that you received from Mindbreeze Support.

Then restart the InSpire container. To do this, open the "Setup" menu item in the Management Centre and then click on the "InSpire" submenu item. You can restart InSpire in the "Container Management" area.

Once the InSpire has been restarted and the feature has been activated correctly, you should see in the menu item "Insight Services" the new submenu item "RAG" in the Management Centre.

ConfigurationPermanent link for this heading

Insight Services for Retrieval Augmented GenerationPermanent link for this heading

Open the menu item "Configuration" in the Mindbreeze Management Center. Create a new service in the "Indices" tab by clicking on "Add Service". Give the service a name under "Display Name" and select the option "Insight Services for RAG" in the "Service" setting.

Make sure that in the "Base Configuration" section the configured "Bind port" is not yet occupied. If necessary, activate the setting "Include prompt in app.telemetry". With this setting, the questions entered by the users and the prompts that are sent to the LLM are included in the app.telemetry entries. This is deactivated by default for security reasons.

„Log HTTP Requests in verbose Log Mode“ is disabled per default. If this setting and the setting „Full Logging“ are enabled, the logs will also log the details for the sent requests. The setting “Full Logging” can be found in the section “Support Mode”, which is visible when “Advanced Settings” are activated.

The setting "Generate with empty Results" is activated by default. If you disable this option, a response will only be generated in the chat if results are found in the index, otherwise an error message will be displayed, this can be useful to prevent responses being given without reference to the indexed data.

The setting "Path to Store Base" can be used to optionally configure the path of the service.

If “Path to Store Base” is defined, based on the path, the service configuration may not be included in the snapshots. If no “Path to Store Base” is defined only the service configurations (pipelines and LLMs) get packed in the snapshot but not the service data (datasets). If all data should be packed into a snapshot, the “Path to Store Base” should be set to a directory which content is fully included in a snapshot. For more information about migrating with a Snapshot, see Handbook - Backup & Restore - Creating a snapshot.

Attention: If the setting "Path to Store Base" is not set, it can occur that the service data for the pipeline and the LLM is not displayed after the application of a Snapshot. To solve this, see the chapter Service data for the pipeline and LLM vanished after applying a Snapshot.

Advanced SettingsPermanent link for this heading

Activate the "Advanced Settings" for the following settings.

Self-Signed SSL CertificatePermanent link for this heading

In the "Security Configuration" section, activate the setting "Disable SSL Certificate Validation". This setting is deactivated by default.

Finally, save the changes by clicking on "Save".

Impersonation ConfigurationPermanent link for this heading

In the section “Impersonation Configuration”, you can enter the endpoint mapping pattern, among other things. The endpoints are defined in the “Network” tab in the Configuration of the Management Center.

C:\Users\silke.lindenberger\AppData\Local\Microsoft\Windows\INetCache\Content.MSO\6E8B577A.tmp

With this setting, the retriever retrieves the client certificate for the client service by comparing it with the pattern defined there. The following placeholder can be used for the pattern:

- client_service_id: ID of the client service

The endpoint mapping only takes effect if no "Trusted Peer Certificate" is selected and the HTTP header configured in the Impersonation Identifier Setting is also sent in the Generate request.

C:\Users\silke.lindenberger\AppData\Local\Microsoft\Windows\INetCache\Content.MSO\25133D8C.tmp

Finally, save the changes by clicking on "Save".

InSpire AI ChatPermanent link for this heading

Switch to the "Client Services" tab and activate the advanced settings by ticking the box "Advanced Settings".

If available, use an existing client service. If no client service is available, add a client service by clicking on "Add Client Service". Give the client service a name with the setting "Display Name".

Go to the "Chat UI Settings" section and select the service you created in the "Indices" tab in the setting "Chat Service".

After the configuration, you can access the AI Chat using the path apps/chat, similar to the Insight App Designer. The full path looks like this: https://example.com/apps/chat/.

Personalised theme (optional configuration)Permanent link for this heading

If a personalised theme is desired in AI Chat, this can be set as follows:

In the Management Center in the menu item "File Manager", the folder /data/apps/chat-theme must be created, if it has not yet been created, in which the following files are stored:

logo.png (necessary)

The logo at the top left of the chat.

custom.css

The Custom Stylesheet.

custom.js

The Custom JavaScript.

favicon.png (necessary)

The icon on the left-hand side of the generated response.

The recommended size of the icon is 14 x 14 pixels.

favicon.svg

The favicon in the browser tab.

If no favicon.svg is available, the favicon.png is used here.

Activate the "Advanced Settings" in the "Client Services" tab and now create an "Additional Context" in your client service in the section "Web Applications Contexts Settings". Activate "Override Existing Path" there and enter /apps/chat/theme as the "URL Path" and /data/apps/chat-theme as the "File Path".

Save the configuration by clicking on "Save".

Activating the NLQA indexPermanent link for this heading

Activate "Natural Language Question Answering" (NLQA) in the desired indexes. For more information about the configuration, see Create NLQA index and Activate NLQA on existing index.

Setting up the Large Language Model (LLM)Permanent link for this heading

On-premise customers have the option to use GPU appliances or Mindbreeze SaaS customers can use a remote LLM. Please contact sales@mindbreeze.com regarding options for running an LLM with Mindbreeze.

For the use of existing, external LLM models, the Huggingface Text Generation Interface (TGI) is currently supported. If necessary, OAuth can be used as an authorisation option.  If you want to use a self-hosted LLM with a different interface, please contact Mindbreeze Support (support@mindbreeze.com) to check compatibility.

To set up the LLM, click on the menu item "Insight Services" in the Management Center and open the submenu item "RAG". Select the "LLMs" area there.

Click on "Add" to configure a new LLM. The following values are supplied by the respective LLM:

Setting

Description

Name

Name of the Large Language Model

URL

URL

User Message Token

User Message End Token

Assistant Message Token

Assistant Message End Token

Message End Token

To be filled in depending on the model.

Preprompt

A Pre-Prompt is used to apply specific roles, intents and limitations to each subsequent prompt of a Model.

Maximum amount of tokens

Limits the tokens of a prompt.

"0" does not limit the tokens. No limitation can decrease the speed of generation when the prompt is too long.

A value of 2000 is recommended.

Save your configuration by clicking on "Save". You can find more details on the administration of models and pipelines in Whitepaper - Administration of Insight Services for Retrieval Augmented Generation.

External LLM with authorisationPermanent link for this heading

If you want to use an external LLM, please contact sales@mindbreeze.com. The sales team will discuss your specific situation with you and provide you with a tailored offer.

The mapping of LLM in the RAG Administration to the required credentials is done via Endpoint Mapping. To configure the authorisation, open the menu item "Configuration" in the Management Center and go to the tab "Network". Create a new credential of type "OAuth 2" with the information provided by Mindbreeze. See the following screenshot for an example.

Create a new endpoint with this credential. The "Location" of the endpoint is the URL of the LLM.

Creating a pipeline and using InSpire AI ChatPermanent link for this heading

To use InSpire AI Chat, a pipeline with a model must be created. The steps required for this can be found in Whitepaper - Administration of Insight Services for Retrieval Augmented Generation.

Authorisation of Management Center users for RAG administration Permanent link for this heading

To administer Insight Services for RAG, a user must have the following roles:

An administrator can assign these roles to a user or group in the Management Center under the menu item "Setup". To do this, click on the "Credentials" submenu item and assign the necessary roles to the user or group. For more information on assigning roles, see Configuration – Back-End Credentials.

TroubleshootingPermanent link for this heading

Service Configuration data for the pipeline and LLM is not displayed after applying a SnapshotPermanent link for this heading

After the application of a Snapshot, the problem can occur that already existing service configuration data for a pipeline and LLM is not displayed anymore. The reason is that there is no defined path in the setting “Path to Store Base” and therefore the pipeline and LLM data are not displayed in the RAG service.

To solve this, please restart your RAG service. After the restart, the service configuration data should be displayed again.

Download PDF

  • Configuration - InSpire AI Chat and Insight Services for Retrieval Augmented Generation

Content

  • Introduction
  • Preparation
  • Activating the feature
  • Configuration
  • Authorisation of Management Center users for RAG administration
  • Troubleshooting

Download PDF

  • Configuration - InSpire AI Chat and Insight Services for Retrieval Augmented Generation