Home
Home
German Version
Support
Impressum
25.2 Release ►

Start Chat with Collection

    Main Navigation

    • Preparation
      • Connectors
      • Create an InSpire VM on Hyper-V
      • Initial Startup for G7 appliances
      • Setup InSpire G7 primary and Standby Appliances
    • Datasources
      • Configuration - Atlassian Confluence Connector
      • Configuration - Best Bets Connector
      • Configuration - Box Connector
      • Configuration - COYO Connector
      • Configuration - Data Integration Connector
      • Configuration - Documentum Connector
      • Configuration - Dropbox Connector
      • Configuration - Egnyte Connector
      • Configuration - GitHub Connector
      • Configuration - Google Drive Connector
      • Configuration - GSA Adapter Service
      • Configuration - HL7 Connector
      • Configuration - IBM Connections Connector
      • Configuration - IBM Lotus Connector
      • Configuration - Jira Connector
      • Configuration - JVM Launcher Service
      • Configuration - LDAP Connector
      • Configuration - Microsoft Azure Principal Resolution Service
      • Configuration - Microsoft Dynamics CRM Connector
      • Configuration - Microsoft Exchange Connector
      • Configuration - Microsoft File Connector (Legacy)
      • Configuration - Microsoft File Connector
      • Configuration - Microsoft Graph Connector
      • Configuration - Microsoft Loop Connector
      • Configuration - Microsoft Project Connector
      • Configuration - Microsoft SharePoint Connector
      • Configuration - Microsoft SharePoint Online Connector
      • Configuration - Microsoft Stream Connector
      • Configuration - Microsoft Teams Connector
      • Configuration - Salesforce Connector
      • Configuration - SCIM Principal Resolution Service
      • Configuration - SemanticWeb Connector
      • Configuration - ServiceNow Connector
      • Configuration - Web Connector
      • Configuration - Yammer Connector
      • Data Integration Guide with SQL Database by Example
      • Indexing user-specific properties (Documentum)
      • Installation & Configuration - Atlassian Confluence Sitemap Generator Add-On
      • Installation & Configuration - Caching Principal Resolution Service
      • Installation & Configuration - Mindbreeze InSpire Insight Apps in Microsoft SharePoint On-Prem
      • Mindbreeze InSpire Insight Apps in Microsoft SharePoint Online
      • Mindbreeze Web Parts for Microsoft SharePoint
      • User Defined Properties (SharePoint 2013 Connector)
      • Whitepaper - Mindbreeze InSpire Insight Apps in Salesforce
      • Whitepaper - Web Connector - Setting Up Advanced Javascript Usecases
    • Configuration
      • CAS_Authentication
      • Configuration - Alerts
      • Configuration - Alternative Search Suggestions and Automatic Search Expansion
      • Configuration - Back-End Credentials
      • Configuration - Chinese Tokenization Plugin (Jieba)
      • Configuration - CJK Tokenizer Plugin
      • Configuration - Collected Results
      • Configuration - CSV Metadata Mapping Item Transformation Service
      • Configuration - Entity Recognition
      • Configuration - Exporting Results
      • Configuration - External Query Service
      • Configuration - Filter Plugins
      • Configuration - GSA Late Binding Authentication
      • Configuration - Identity Conversion Service - Replacement Conversion
      • Configuration - InceptionImageFilter
      • Configuration - Index-Servlets
      • Configuration - InSpire AI Chat and Insight Services for Retrieval Augmented Generation
      • Configuration - Item Property Generator
      • Configuration - Japanese Language Tokenizer
      • Configuration - Kerberos Authentication
      • Configuration - Management Center Menu
      • Configuration - Metadata Enrichment
      • Configuration - Metadata Reference Builder Plugin
      • Configuration - Mindbreeze Proxy Environment (Remote Connector)
      • Configuration - Personalized Relevance
      • Configuration - Plugin Installation
      • Configuration - Principal Validation Plugin
      • Configuration - Profile
      • Configuration - Reporting Query Logs
      • Configuration - Reporting Query Performance Tests
      • Configuration - Request Header Session Authentication
      • Configuration - Shared Configuration (Windows)
      • Configuration - Vocabularies for Synonyms and Suggest
      • Configuration of Thumbnail Images
      • Cookie-Authentication
      • Documentation - Mindbreeze InSpire
      • I18n Item Transformation
      • Installation & Configuration - Outlook Add-In
      • Installation - GSA Base Configuration Package
      • JWT Authentication
      • Language detection - LanguageDetector Plugin
      • Mindbreeze Personalization
      • Mindbreeze Property Expression Language
      • Mindbreeze Query Expression Transformation
      • SAML-based Authentication
      • Trusted Peer Authentication for Mindbreeze InSpire
      • Using the InSpire Snapshot for Development in a CI_CD Scenario
      • Whitepaper - AI Chat
      • Whitepaper - Create a Google Compute Cloud Virtual Machine InSpire Appliance
      • Whitepaper - Create a Microsoft Azure Virtual Machine InSpire Appliance
      • Whitepaper - Create AWS 10M InSpire Appliance
      • Whitepaper - Create AWS 1M InSpire Appliance
      • Whitepaper - Create AWS 2M InSpire Appliance
      • Whitepaper - Create Oracle Cloud 10M InSpire Application
      • Whitepaper - Create Oracle Cloud 1M InSpire Application
      • Whitepaper - MMC_ Services
      • Whitepaper - Natural Language Question Answering (NLQA)
      • Whitepaper - SSO with Microsoft AAD or AD FS
      • Whitepaper - Text Classification Insight Services
    • Operations
      • Adjusting the InSpire Host OpenSSH Settings - Set LoginGraceTime to 0 (Mitigation for CVE-2024-6387)
      • app.telemetry Statistics Regarding Search Queries
      • CIS Level 2 Hardening - Setting SELinux to Enforcing mode
      • Configuration - app.telemetry dashboards for usage analysis
      • Configuration - Usage Analysis
      • Deletion of Hard Disks
      • Handbook - Backup & Restore
      • Handbook - Command Line Tools
      • Handbook - Distributed Operation (G7)
      • Handbook - Filemanager
      • Handbook - Indexing and Search Logs
      • Handbook - Updates and Downgrades
      • Index Operating Concepts
      • Inspire Diagnostics and Resource Monitoring
      • Provision of app.telemetry Information on G7 Appliances via SNMPv3
      • Restoring to As-Delivered Condition
      • Whitepaper - Administration of Insight Services for Retrieval Augmented Generation
    • User Manual
      • Browser Extension
      • Cheat Sheet
      • iOS App
      • Keyboard Operation
    • SDK
      • api.chat.v1beta.generate Interface Description
      • api.v2.alertstrigger Interface Description
      • api.v2.export Interface Description
      • api.v2.personalization Interface Description
      • api.v2.search Interface Description
      • api.v2.suggest Interface Description
      • api.v3.admin.SnapshotService Interface Description
      • Debugging (Eclipse)
      • Developing an API V2 search request response transformer
      • Developing Item Transformation and Post Filter Plugins with the Mindbreeze SDK
      • Development of a Query Expression Transformer
      • Development of Insight Apps
      • Embedding the Insight App Designer
      • Java API Interface Description
      • OpenAPI Interface Description
    • Release Notes
      • Release Notes 20.1 Release - Mindbreeze InSpire
      • Release Notes 20.2 Release - Mindbreeze InSpire
      • Release Notes 20.3 Release - Mindbreeze InSpire
      • Release Notes 20.4 Release - Mindbreeze InSpire
      • Release Notes 20.5 Release - Mindbreeze InSpire
      • Release Notes 21.1 Release - Mindbreeze InSpire
      • Release Notes 21.2 Release - Mindbreeze InSpire
      • Release Notes 21.3 Release - Mindbreeze InSpire
      • Release Notes 22.1 Release - Mindbreeze InSpire
      • Release Notes 22.2 Release - Mindbreeze InSpire
      • Release Notes 22.3 Release - Mindbreeze InSpire
      • Release Notes 23.1 Release - Mindbreeze InSpire
      • Release Notes 23.2 Release - Mindbreeze InSpire
      • Release Notes 23.3 Release - Mindbreeze InSpire
      • Release Notes 23.4 Release - Mindbreeze InSpire
      • Release Notes 23.5 Release - Mindbreeze InSpire
      • Release Notes 23.6 Release - Mindbreeze InSpire
      • Release Notes 23.7 Release - Mindbreeze InSpire
      • Release Notes 24.1 Release - Mindbreeze InSpire
      • Release Notes 24.2 Release - Mindbreeze InSpire
      • Release Notes 24.3 Release - Mindbreeze InSpire
      • Release Notes 24.4 Release - Mindbreeze InSpire
      • Release Notes 24.5 Release - Mindbreeze InSpire
      • Release Notes 24.6 Release - Mindbreeze InSpire
      • Release Notes 24.7 Release - Mindbreeze InSpire
      • Release Notes 24.8 Release - Mindbreeze InSpire
      • Release Notes 25.1 Release - Mindbreeze InSpire
      • Release Notes 25.2 Release - Mindbreeze InSpire
    • Security
      • Known Vulnerablities
    • Product Information
      • Product Information - Mindbreeze InSpire - Standby
      • Product Information - Mindbreeze InSpire
    Home

    Path

    Sure, you can handle it. But should you?
    Let our experts manage the tech maintenance while you focus on your business.
    See Consulting Packages

    Configuration
    InSpire AI Chat and Insight Services for RAG

    IntroductionPermanent link for this heading

    Mindbreeze InSpire AI Chat offers the possibility of a conversational interface based on generative AI. Thanks to the Insight Services for Retrieval Augmented Generation (RAG), a Large Language Model (LLM) using Mindbreeze InSpire is able to generate answers from the user's input based on the hits in Mindbreeze.

    PreparationPermanent link for this heading

    Two prerequisites must be fulfilled:

    • Activation of the corresponding features in Mindbreeze InSpire so that the configuration can be carried out.
    • Operation of an LLM.

    To activate the features, please contact Mindbreeze Support (support@mindbreeze.com) for details on activating the features via Mindbreeze InSpire Feature Flags.

    For the operation of an LLM, please contact Mindbreeze Sales (sales@mindbreeze.com) to use an LLM. On-premise customers have the option of using GPU appliances and Mindbreeze SaaS customers can use a remote LLM. For the use of existing LLM infrastructures, the Huggingface TGI interface is currently supported and OAuth can be used as an authorisation option.

    Activating the featurePermanent link for this heading

    Please contact Mindbreeze Support (support@mindbreeze.com) and, if available, submit your existing feature flag configuration "/etc/mindbreeze/enabled_features.json" to receive an updated version. Then update the new enabled_features.json that you received from Mindbreeze Support.

    Then restart the InSpire container. To do this, open the "Setup" menu item in the Management Centre and then click on the "InSpire" submenu item. You can restart InSpire in the "Container Management" area.

    Once the InSpire has been restarted and the feature has been activated correctly, you should see in the menu item "Insight Services" the new submenu item "RAG" in the Management Centre.

    ConfigurationPermanent link for this heading

    Insight Services for Retrieval Augmented GenerationPermanent link for this heading

    Open the menu item "Configuration" in the Mindbreeze Management Center. Create a new service in the "Indices" tab by clicking on "Add Service". Give the service a name under "Display Name" and select the option "Insight Services for RAG" in the "Service" setting.

    Make sure that in the "Base Configuration" section the configured "Bind port" is not yet occupied. If necessary, activate the setting "Include prompt in app.telemetry". With this setting, the questions entered by the users and the prompts that are sent to the LLM are included in the app.telemetry entries. This is deactivated by default for security reasons.

    „Log HTTP Requests in verbose Log Mode“ is disabled per default. If this setting and the setting „Full Logging“ are enabled, the logs will also log the details for the sent requests. The setting “Full Logging” can be found in the section “Support Mode”, which is visible when “Advanced Settings” are activated.

    The setting "Generate with empty Results" is activated by default. If you disable this option, a response will only be generated in the chat if results are found in the index, otherwise an error message will be displayed, this can be useful to prevent responses being given without reference to the indexed data.

    The setting "Path to Store Base" can be used to optionally configure the path of the service.

    If “Path to Store Base” is defined, based on the path, the service configuration may not be included in the snapshots. If no “Path to Store Base” is defined only the service configurations (pipelines and LLMs) get packed in the snapshot but not the service data (datasets). If all data should be packed into a snapshot, the “Path to Store Base” should be set to a directory which content is fully included in a snapshot. For more information about migrating with a Snapshot, see Handbook - Backup & Restore - Creating a snapshot.

    Attention: If the setting "Path to Store Base" is not set, it can occur that the service data for the pipeline and the LLM is not displayed after the application of a Snapshot. To solve this, see the chapter Service data for the pipeline and LLM vanished after applying a Snapshot.

    Advanced SettingsPermanent link for this heading

    Activate the "Advanced Settings" for the following settings.

    Self-Signed SSL CertificatePermanent link for this heading

    In the "Security Configuration" section, activate the setting "Disable SSL Certificate Validation". This setting is deactivated by default.

    Finally, save the changes by clicking on "Save".

    Impersonation ConfigurationPermanent link for this heading

    In the section “Impersonation Configuration”, you can enter the endpoint mapping pattern, among other things. The endpoints are defined in the “Network” tab in the Configuration of the Management Center.

    C:\Users\silke.lindenberger\AppData\Local\Microsoft\Windows\INetCache\Content.MSO\6E8B577A.tmp

    With this setting, the retriever retrieves the client certificate for the client service by comparing it with the pattern defined there. The following placeholder can be used for the pattern:

    - client_service_id: ID of the client service

    The endpoint mapping only takes effect if no "Trusted Peer Certificate" is selected and the HTTP header configured in the Impersonation Identifier Setting is also sent in the Generate request.

    C:\Users\silke.lindenberger\AppData\Local\Microsoft\Windows\INetCache\Content.MSO\25133D8C.tmp

    Finally, save the changes by clicking on "Save".

    InSpire AI ChatPermanent link for this heading

    Switch to the "Client Services" tab and activate the advanced settings by ticking the box "Advanced Settings".

    If available, use an existing client service. If no client service is available, add a client service by clicking on "Add Client Service". Give the client service a name with the setting "Display Name".

    Go to the "Chat UI Settings" section and select the service you created in the "Indices" tab in the setting "Chat Service".

    After the configuration, you can access the AI Chat using the path apps/chat, similar to the Insight App Designer. The full path looks like this: https://example.com/apps/chat/.

    Personalised theme (optional configuration)Permanent link for this heading

    If a personalised theme is desired in AI Chat, this can be set as follows:

    In the Management Center in the menu item "File Manager", the folder /data/apps/chat-theme must be created, if it has not yet been created, in which the following files are stored:

    logo.png (necessary)

    The logo at the top left of the chat.

    custom.css

    The Custom Stylesheet.

    custom.js

    The Custom JavaScript.

    favicon.png (necessary)

    The icon on the left-hand side of the generated response.

    The recommended size of the icon is 14 x 14 pixels.

    favicon.svg

    The favicon in the browser tab.

    If no favicon.svg is available, the favicon.png is used here.

    Activate the "Advanced Settings" in the "Client Services" tab and now create an "Additional Context" in your client service in the section "Web Applications Contexts Settings". Activate "Override Existing Path" there and enter /apps/chat/theme as the "URL Path" and /data/apps/chat-theme as the "File Path".

    Save the configuration by clicking on "Save".

    Activating the NLQA indexPermanent link for this heading

    Activate "Natural Language Question Answering" (NLQA) in the desired indexes. For more information about the configuration, see Create NLQA index and Activate NLQA on existing index.

    Setting up the Large Language Model (LLM)Permanent link for this heading

    On-premise customers have the option to use GPU appliances or Mindbreeze SaaS customers can use a remote LLM. Please contact sales@mindbreeze.com regarding options for running an LLM with Mindbreeze.

    For the use of existing, external LLM models, the Huggingface Text Generation Interface (TGI) is currently supported. If necessary, OAuth can be used as an authorisation option.  If you want to use a self-hosted LLM with a different interface, please contact Mindbreeze Support (support@mindbreeze.com) to check compatibility.

    To set up the LLM, click on the menu item "Insight Services" in the Management Center and open the submenu item "RAG". Select the "LLMs" area there.

    Click on "Add" to configure a new LLM. The following values are supplied by the respective LLM:

    Setting

    Description

    Name

    Name of the Large Language Model

    URL

    URL

    User Message Token

    User Message End Token

    Assistant Message Token

    Assistant Message End Token

    Message End Token

    To be filled in depending on the model.

    Preprompt

    A Pre-Prompt is used to apply specific roles, intents and limitations to each subsequent prompt of a Model.

    Maximum amount of tokens

    Limits the tokens of a prompt.

    "0" does not limit the tokens. No limitation can decrease the speed of generation when the prompt is too long.

    A value of 2000 is recommended.

    Save your configuration by clicking on "Save". You can find more details on the administration of models and pipelines in Whitepaper - Administration of Insight Services for Retrieval Augmented Generation.

    External LLM with authorisationPermanent link for this heading

    If you want to use an external LLM, please contact sales@mindbreeze.com. The sales team will discuss your specific situation with you and provide you with a tailored offer.

    The mapping of LLM in the RAG Administration to the required credentials is done via Endpoint Mapping. To configure the authorisation, open the menu item "Configuration" in the Management Center and go to the tab "Network". Create a new credential of type "OAuth 2" with the information provided by Mindbreeze. See the following screenshot for an example.

    Create a new endpoint with this credential. The "Location" of the endpoint is the URL of the LLM.

    Creating a pipeline and using InSpire AI ChatPermanent link for this heading

    To use InSpire AI Chat, a pipeline with a model must be created. The steps required for this can be found in Whitepaper - Administration of Insight Services for Retrieval Augmented Generation.

    Authorisation of Management Center users for RAG administration Permanent link for this heading

    To administer Insight Services for RAG, a user must have the following roles:

    An administrator can assign these roles to a user or group in the Management Center under the menu item "Setup". To do this, click on the "Credentials" submenu item and assign the necessary roles to the user or group. For more information on assigning roles, see Configuration – Back-End Credentials.

    TroubleshootingPermanent link for this heading

    Service Configuration data for the pipeline and LLM is not displayed after applying a SnapshotPermanent link for this heading

    After the application of a Snapshot, the problem can occur that already existing service configuration data for a pipeline and LLM is not displayed anymore. The reason is that there is no defined path in the setting “Path to Store Base” and therefore the pipeline and LLM data are not displayed in the RAG service.

    To solve this, please restart your RAG service. After the restart, the service configuration data should be displayed again.

    Download PDF

    • Configuration - InSpire AI Chat and Insight Services for Retrieval Augmented Generation

    Content

    • Introduction
    • Preparation
    • Activating the feature
    • Configuration
    • Authorisation of Management Center users for RAG administration
    • Troubleshooting

    Download PDF

    • Configuration - InSpire AI Chat and Insight Services for Retrieval Augmented Generation