Whitepaper

Administration of Insight Services for Retrieval Augmented Generation

Motivation and overview

Retrieval Augmented Generation (RAG) is a natural language processing technique that combines the strengths of query-based and generative artificial intelligence (AI) models. In a RAG-based AI system, a query model is used to find relevant information from existing data sources. Meanwhile the generative model takes the queried information, synthesises all the data and transforms it into a coherent, contextual response.

Configuration of a Large Language Model (LLM)

Initial Setup

You can find the instructions on how to set up an LLM in Configuration - InSpire AI Chat and Insight Services for Retrieval Augmented Generation.

Integration of a LLM

To configure a Large Language Model (LLM) for your pipelines, switch in the menu item “RAG” to the area "LLMs".

Click on "Add" and select the respective LLM to configure it. There are currently four LLMs available for Integration:

OpenAI

Azure OpenAI

InSpire LLM

Aleph Alpha

The following settings can be configured for the respective LLM.

OpenAI

When creating an OpenAI LLM a Dialog opens that refers to the Data Privacy Disclaimer. To continue you have to accept this Disclaimer.

Attention: When using the OpenAI API, chat inputs from the user and information indexed by your organization are transmitted to the respective endpoints via prompts. The handling of the transmitted information is governed by the data protection regulations of the respective AI provider. Mindbreeze is not responsible for the further data processing. The AI provider is neither a vicarious agent nor a subcontractor of Mindbreeze. We would like to point out that, according to current assessments, the lawful use of AI services is not guaranteed (precautionary note pursuant to GDPR Art. 28 para. 3 sentence 3). For further information and risks, please refer to the relevant privacy policies of the respective AI provider.

For more Information visit https://openai.com/enterprise-privacy .

By confirming the checkbox, you as the Data Controller instruct Mindbreeze to carry out this transmission nevertheless and acknowledge the note as outlined above.

Connection

With “Test Connection” you can check if the given values are valid and if a connection can be established.

For the areas “General”, “Prompt” and “Test”, please see the chapter General Parts of the LLM Settings.

Azure OpenAI

When creating an Azure OpenAI LLM a Dialog opens that refers to the Data Privacy Disclaimer. To continue you have to accept this Disclaimer.

Attention: When using the Azure OpenAI API, chat inputs from the user and information indexed by your organization are transmitted to the respective endpoints via prompts. The handling of the transmitted information is governed by the data protection regulations of the respective AI provider. Mindbreeze is not responsible for the further data processing. The AI provider is neither a vicarious agent nor a subcontractor of Mindbreeze. We would like to point out that, according to current assessments, the lawful use of AI services is not guaranteed (precautionary note pursuant to GDPR Art. 28 para. 3 sentence 3). For further information and risks, please refer to the relevant privacy policies of the respective AI provider.

For more Information visit https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy

By confirming the checkbox, you as the Data Controller instruct Mindbreeze to carry out this transmission nevertheless and acknowledge the note as outlined above.

Connection

Model

For the areas “General”, “Prompt” and “Test”, please see the chapter General Parts of the LLM Settings.

InSpire LLM

Connection

Model

For the areas “General”, “Prompt” and “Test”, please see the chapter General Parts of the LLM Settings.

Aleph Alpha

When creating an Aleph Alpha LLM a Dialog opens that refers to the Data Privacy Disclaimer. To continue you have to accept this Disclaimer.

Attention: When using the Aleph Alpha API, chat inputs from the user and information indexed by your organization are transmitted to the respective endpoints via prompts. The handling of the transmitted information is governed by the data protection regulations of the respective AI provider. Mindbreeze is not responsible for the further data processing. The AI provider is neither a vicarious agent nor a subcontractor of Mindbreeze. We would like to point out that, according to current assessments, the lawful use of AI services is not guaranteed (precautionary note pursuant to GDPR Art. 28 para. 3 sentence 3). For further information and risks, please refer to the relevant privacy policies of the respective AI provider.

For more Information visit https://aleph-alpha.com/data-privacy/

By confirming the checkbox, you as the Data Controller instruct Mindbreeze to carry out this transmission nevertheless and acknowledge the note as outlined above.

Connection

With “Test Connection” you can check if the given values are valid and if a connection can be established.

For the areas “General”, “Prompt” and “Test”, please see the chapter General Parts of the LLM Settings.

General Parts of the LLM Settings

General

Prompt

Prompt Examples

These examples are displayed in the Mindbreeze InSpire AI Chat as sample questions. Accordingly, frequently asked questions are suitable as prompt examples. By clicking on a prompt example, this question is automatically entered in the Mindbreeze InSpire AI Chat. A prompt example can be created by clicking on "Add" and filling in the following fields:

Click on "Save" to save the prompt example. Any number of prompt examples can be created. Once all prompt examples have been created, save the entire LLM to save the changes.

Test

On this page the configuration can be tested. If LLM parameters have been defined, these will be summarised here again. Keep in mind that the generated text is not based on retrieved documents.

After testing the LLM Settings click on “Save” to save the LLM.

Creation of a pipeline

To create a pipeline, switch to the section "Generative Pipelines". Click on "Add" to start creating a new pipeline.

The creation of a pipeline is divided into five sections:

1. General
2. Examples
3. Retrieval
4. Generation
5. Test

The individual sections are described in more detail in the following chapters.

General

In the section “General”, the following general settings can be set:

In the section "Prompt Examples" example questions can be added to a pipeline that are displayed in the Mindbreeze InSpire AI Chat. If no example questions are defined in the pipeline, the example questions are taken from the LLM. If no example questions are defined in the LLM either, no example questions are displayed in the Mindbreeze InSpire AI Chat. For more information on prompt examples, see the chapter Creation of Prompt Examples.

Once the necessary settings have been made, you can continue to the next section by clicking "Next" or by clicking on the desired section in the navigation bar on the left.

Examples

This area currently has no impact on the pipeline and can be skipped.

In the "Examples" section, you can create a new dataset or add an existing dataset. This area currently has no effect on the pipeline. The data sets in this section can be used in the future to evaluate the pipeline, so that the effects of a change can be analysed.

Creating a new dataset

If a new dataset is created here, it is automatically added to the pipeline. For more information, see the chapter Adding data.

Adding an existing dataset

If you have already created one (or more) datasets that you would like to use in this pipeline, you can add them by clicking on "Add an existing Dataset". In the context menu, you can select several datasets and use them in your pipeline by clicking "Add".

Retrieval

Only indices that have the "Semantic Sentence Similarity Search" feature enabled can provide answers for generation in the Mindbreeze InSpire AI Chat.

The retrieval part of the RAG can be configured in the "Retrieval" section. The following settings are available:

In the section "Constraints", the pipeline can be refined and customized to the respective requirements.

The section "Effective Data sources" provides an overview of the effective data sources of the selected search service.

Generation

In the section “Generation”, you can configure the prompt generation, select the LLM and the prompt templates that are filled with the search results and then sent to the configured LLM.

Select the LLM you have created in the setting "Model". By selecting the LLM, you will receive a summary made up of the following points:

Evaluation

Settings here are only necessary if a pipeline has been explicitly created, e.g. for data set refinement.

Test

In the "Test" section, you can test the pipeline settings and check whether the settings you have made fulfil the requirements.

Publication of the created pipeline

Pipeline status

A pipeline can have several versions and each of which has a status:

• Draft: This pipeline currently has a draft version. There can only be one draft version per pipeline. The draft version can be edited at any time.
• Released: A released version can no longer be changed retrospectively. If changes are subsequently made to a released version, a new edit version is automatically created if one does not yet exist.
• Published: Only one released version can be published per pipeline. It is also possible to publish an earlier released version. So, the last released version does not necessarily have to be published. Published versions can also be withdrawn by removing the publication.

Release of the pipeline version

After a pipeline has been created or edited, there is an draft version for it. To finalise the draft version, it must be released. To do this, select the pipeline you have created and click on "Release Version". Enter a version name and optionally a version description. Then click on "Release Version".

Publication of the pipeline version

To use a pipeline in Mindbreeze InSpire AI Chat, it is necessary to release a version. To do this, select a pipeline that already contains a released version and click on "Publish". Only released pipeline versions can be published.

In the dialogue box, select a pipeline version that you want to publish and check the display name and description. If a version of the pipeline has already been published, you will find information on the published version of the pipeline above the selection field.

Then click "Publish" to publish the selected version. After publishing, the version number of the published version should appear in the column "Published".

It is then possible to select and use the created pipeline in the Mindbreeze InSpire AI Chat.

Remove publication

Select a published pipeline and click on "Publish". You will find information about the published version in the dialogue box. Then click on "Unpublish".

The pipeline should no longer have a value for "Published" in the overview. The pipeline is now no longer available in the Mindbreeze InSpire AI Chat.

Producer Consumer Setup

If you have a Producer Consumer Setup, the RAG configuration can be synchronized to all Nodes by clicking “Sync to consumer”.

Creation of datasets

In this section, datasets can be created which are necessary for evaluating pipelines.

Creation of a new dataset

Adding data

Click on "Create new dataset" and give the dataset a name under "Name of dataset". Then add data to a dataset by clicking on "Add".

The following fields can be filled in:

Contexts

The expected answers can be entered for each question in the contexts.

Save the changes in the contexts by clicking on "Save" in the window below.

Placeholders

Placeholders can be defined for each question, where the value of the placeholder is to be inserted in the prompt. The comparison is carried out using the key.

Save the changes in the placeholders by clicking on "Save" at the bottom of the window.

Finally, click on "Save" to save the data set.

Uploading a file

In addition to the manual creation of data, a file can also be uploaded. To do this, click on "Upload file". Specify whether the data from the file should extend or overwrite the existing entries.

Please note that only one file can be uploaded. The following file types are compatible: JSON and CSV.

Once the file has been successfully loaded, click on "Add". Then save the dataset.

JSON Upload

The following values can be entered for each question:

Example of a JSON File:

{

  "data": [

    {

      "question": "How can I configure SAML based Authentication in Mindbreeze InSpire?",

      "answer": "Configuring SAML in Mindbreeze InSpire is a four-step Process.",

      "remark": "This is a Remark for this Question",

      "contexts": [

        {

          "text": "Configuring SAML in Mindbreeze is a four-step process: 1. Adding an SSL certificate that is used to generate the service provider metadata 2. Configuring the SAML authenticator 3. Configuring the parameter 4. Activating SAML for individual services",

          "source": "https://help.mindbreeze.com/en/index.php?topic=doc/SAML-based-Authentication/index.html"

        }

      ],

      "pipeline_key": "special-pipeline",

      "query_constraints": [

        "extension:pdf",

        "fqcategory:Web"

      ],

      "queries": [

        "What are Authentication Possibilities on {product}?",

        "What is {authentication}?"

      ],

      "promptPlaceholders": {

        "authentication": "SAML",

        "product": "Mindbreeze InSpire"

      }

    },

    {

      "question": "Another question"

    }

  ]

}

CSV Upload

For the CSV file, make sure that the separator is a semicolon and that the file was saved in UTF-8 format, as the areas may not be read correctly or there may be errors when reading.

The file must have a header.

Example of a CSV File:

question;answer;remark;queries;queryconstraints;context;source;context;source;pipelinename;pp_product_name;pp_authentication

How can I configure SAML based Authentication in Mindbreeze InSpire?; Configuring SAML in Mindbreeze InSpire is a four-step Process.;This is a Remark for this Question;"What are Authentication Possibilities on {product}?

What is {authentication}?";"extension:pdf

fqcategory:Web";Configuring SAML in Mindbreeze is a four-step process: 1. Adding an SSL certificate that is used to generate the service provider metadata 2. Configuring the SAML authenticator 3. Configuring the parameter 4. Activating SAML for individual services;"https://help.mindbreeze.com/en/index.php?topic=doc/SAML-based-Authentication/index.html";This is a second Context.;another_source.pdf; special_pipeline;Mindbreeze InSpire;SAML

Evaluation of a pipeline

To test the effectiveness of a pipeline, it is possible to evaluate pipelines in this respect.

Overview

The evaluations are listed in the overview and listed according to the start time.

Create a new Evaluation

With “New Evaluation” a new evaluation can be created.

Evaluation Overview

Summary of the Evaluation Result

Overview over the results from every single question

With “Details” you can check the evaluation details of the specific question.

Question Result

General

Feedback

Here you get the possibility to give feedback on the evaluation result for a question.

After you changed the feedback, you have to click on “Save” to save your feedback.

Generation

This section gives you Information about the answer generation.

In the table below, there is the expected answer from the dataset compared to the generated answer from the LLM.

Retrieval

This section gives you Information about the retrieval process.

In the table below, the expected contexts from the dataset are compared to the contexts you got in the retrieval process from the Client Service.