Installation and Configuration

Caching Principal Resolution Service

Introduction

This documentation contains basic information about the configuration and management of Caching Principal Resolution Services. You will find a general explanation of the terms and functions, an example configuration, a list of general settings, a description of how to carry out monitoring with app.telemetry, as well as troubleshooting and use case examples.

The information in this documentation applies to all Caching Principal Resolution Services. If specific information is required for a data source, you can find this in the corresponding documentation. A list of all supported data sources, including references to the respective documentation, can be found in the chapter Supported data sources.

Hint: The terms "Caching Principal Resolution Service" and "Principal Resolution Service" are used synonymously in this document and have therefore the same meaning.

What are principals?

A "principal" is a central concept in Mindbreeze that represents different types of entities. These entities can be users, groups, roles or data source-specific authorisation concepts. Principals are used for the uniform administration and identification of these entities in different authorisation systems.

Example:

• A user name like "David Porter" or an e-mail address like "david.porter@mindbreeze.com" can be regarded as a principal. The same user can have several aliases thus several representations.
• Groups and roles such as "Mindbreeze employee" or "Mindbreeze Invoicing" are also referred to as principals.
• Data source-specific authorisation concepts are also principals. For example, the list of all users with read authorisation for a Microsoft Sharepoint site is also saved as a principal.

Why is a Principal Resolution Service needed?

When a user logs in to a Mindbreeze Client Service, only the user's login information is known. To determine the associated aliases, roles and groups of the user, it is necessary to obtain this information from the respective data source.

With a Caching Principal Resolution Service, all users and their aliases, roles and groups are downloaded from a data source and stored to resolve them efficiently at the time of the search.

Functionality of the Caching Principal Resolution Service

The Caching Principal Resolution Service has the task of resolving all principals of a logged in user in a search query. This is crucial to determine which documents this user can access according to their authorisations.

Initialisation of the service

Before the Caching Principal Resolution Service is ready for use, it must retrieve all principals (users, roles, groups, etc.) from a data source and store them in a cache. These are stored efficiently in the Caching Principal Resolution Service, which means that the resolution can take place in real time, even if millions of principals are stored in the cache.

After initialisation, the cache is updated regularly. For information on how to configure the updating of the cache, see the chapter Cache Update Settings.

Procedure at the time of the search

The following process illustrates in a simplified form the essential steps that are performed at the time of the search:

1. A search is started and the principals (login information) of the searching user "David Porter" are sent to the Mindbreeze Index.
2. The principals of David Porter are forwarded to the Principal Resolution Service.
3. The Principal Resolution Service resolves the associated principals of "David Porter".
4. The ACLs (Access Control List) of the documents are compared with the resolved principals of David Porter.
5. Finally, David Porter only sees the documents in the search result to which he is authorised to have access.

Supported data sources

Each data source has at least one supported Principal Resolution Service. Below you will find a list of supported data sources and the link to the corresponding documentation:

Basic configuration

Preparation

Selecting the correct Caching Principal Resolution Service:

In the chapter Supported data sources you will find a detailed list of data sources with the respective Caching Principal Resolution Services and the reference to the corresponding documentation.

In some cases, several Principal Resolution Services can be selected for a single data source. If this is the case, please note the following points:

• Identify the identity sources or directory services used in your organisation for each data source in order to select the appropriate Principal Resolution Service.
  • For example, LDAP is often used for the company-wide single sign-on (SSO) registration. A CachingLdapPrincipalResolution Service should be used here, if available.
• If relevant principals for documents from a specific data source are distributed across different sources, a parent/child configuration of the services is necessary. This configuration links the services together. For more information on parent/child configurations, see the chapter Parent/child cache - resolution of principals from multiple data sources.
  • An example of this is Microsoft Sharepoint Online, where information of the principal can be found in the respective Sharepoint Online sites and on the Microsoft Azure platform.

Preparation of the data source:

Basically, it is necessary to create the required login information (credentials) in the data source. A user with read authorisation for all users, roles, groups etc. is usually required. The following must be noted:

• Carry out the specific preparation steps according to the documentation of the respective Principal Resolution Service.
• If a connector has already been configured for the data source, it may be possible to use the same credentials for the Principal Resolution Service.

Preparation of the network configuration:

For some data sources, it is necessary to edit the global network configuration in the Mindbreeze Appliance. The following must be noted:

• Carry out the specific preparation steps according to the documentation of the respective Principal Resolution Service.
• Gather information about the specific infrastructure of the organisation, including firewall and proxy settings, to ensure a configuration without problems.

Overview of the configuration steps

The basic configuration of a Principal Resolution Service involves the following steps in the Mindbreeze Management Center:

1. Creation of a suitable Principal Resolution Service for the selected data source.
2. Configuration of the login data (credentials) for the data source. Depending on the data source, the credentials can be configured in two ways:
   • Creating and assigning a Mindbreeze credential object with the login data.
   • Direct entry of the credentials in the configuration of the Principal Resolution Service.
3. Assignment of the created Principal Resolution Service to the used index.

In the following chapter, these steps are demonstrated using an example configuration. There, a Principal Resolution Service for Microsoft Dynamics 365 is configured.

Example configuration for Microsoft Dynamics 365

This section describes the configuration of a Microsoft Dynamics CRM Principal Resolution Service for an index with a Microsoft Dynamics 365 data source. The following steps must be carried out:

1. Carry out the preparation steps described in the Microsoft Dynamics CRM Principal Resolution documentation. Create a Microsoft Azure application with the necessary read authorizations.

1. Open the Management Centre and then the menu item "Configuration" in the page navigation. Create a service in the "Indices" tab. Give the service a name in the setting "Display Name". In the context of this example, the display name is "Microsoft Dynamics 365 Principal Resolution Service".

1. Select in the "Service" setting the option "Microsoft Dynamics 365 Principal Resolution Service"

1. Switch to the "Network" tab and create a new Mindbreeze credential for OAuth2 authentication. Use the Microsoft Azure application that you created in step 1. In this case, a credential of the type "OAuth2" is required.

1. Fill in the fields "Access Token URL", "Client ID" and "Client Secret" accordingly. Give the credential a suitable name like for example "Microsoft Dynamics 365 Oauth2 Credential".

1. Switch back to the "Index" tab to configure the Principal Resolution Services credentials in the section "Connection Settings".

1. First, enter the URL for "CRM URL" and "Organisational Service URL" of the Dynamics 365 instance. Select the credential "Microsoft Dynamics 365 OAuth2 Credential" created by you in step 4 as the "Credential". Also activate the setting "Is Microsoft Dynamics CRM Online".

1. Configure the SharePoint Online connector in the "Indices" tab of the desired index in the section "Data Sources". Select the created service in the setting "Caching Principal Resolution Service". In this example, "Microsoft Dynamics 365 Principal Resolution Service" must be selected.

1. Then save the changes by clicking on "Save".

Once the Principal Resolution Service has been configured and connected to the index, you can check whether the cache has been set up successfully and the principals are resolved as expected. You can find more information on this in the following chapter Monitoring.

Hint: As soon as the basic configuration has been completed, it is recommended that you also adjust the setting "Cache Update Interval" in the section "Cache Update Settings". This setting determines how often the principals of the Principal Resolution Services are updated. You can find more information on this in the chapters Cache Settings and Cache Update Settings.

Monitoring

After configuring a caching principal resolution service, it is important to ensure that the cache has been successfully established and that the service works as expected. The build-up of the cache is started by default as soon as the configuration has been saved.

The status and behaviour of the Caching Principal Resolution Service can be monitored via the app.telemetry Reporting Dashboard. It provides you with real-time information about the status of the build-up of the cache and the resolution of the principals.

In addition, a REST API is available that is hosted by each Caching Principal Resolution Service. This API enables the monitoring and management of the service, including updating the cache, checking the cache content and analyzing the resolved principals.

Monitoring and management with the REST API

This chapter explains how you can use the REST API to manage and monitor the caching principal resolution services. The API calls enable to:

• Trigger a cache update
• Query the current status of the build-up of the cache
• Analyzing the resolved principals
• Viewing the content of the cache
• Export of the cache content

Hint: Please note that a direct connection to the Mindbreeze appliance is required to use the REST API.

API calls for status information

API calls for resolving the principals

API calls for cache content

API calls for managing the cache

Monitoring with app.telemetry

You will find app.telemetry in the Management Centre in the page navigation under the menu item "Reporting". Click on "Reporting" and additional menu items will appear. Open the submenu item "Telemetry Details".

You will be directed to the area "Applications". In the section "Mindbreeze" click on "Principal Resolution Service".

Then, open "View Telemetry Data".

Finally, you are now in the section "Software telemetry". Here you can view and analyze various aspects of the created cache. For example, you can query the status of the cache, check if errors have occurred, view the duration of the build-up of the cache, check when the last cache update was performed and other aspects.

Hint: If certain information is not displayed, the configuration of the columns could be helpful. To do this, open "Columns" at the bottom left. There you can select which columns should be displayed. It is recommended to select the columns "Start Time", "Duration", "Running", "Service", "Agent" and "Errors".

List of general settings

This chapter describes all the settings that can be configured for each Caching Principal Resolution Service. If there are any differences, these are explained in the individual documentation for the respective Caching Principal Resolution Service.

Cache settings

In the section "Cache Settings", the properties of the cache can be customized, including the definition of the database path. Please note that this configuration is optional.

Cache Update Settings

In the section "Cache Update Settings", the structure of the cache and the frequency of the cache update can be configured.

Health Check Settings

Health checks are configured in the section "Health Check Settings". With a health check the availability of the Caching Principal Resolution Service is checked automatically. If connection issues are detected, the service will be restarted. A health check network request is sent to the service for this purpose. If no response is received from the service within the specified timeout, it will be recognised as an error. This process is repeated until the maximum number of errors has been exceeded or a response is received within the timeout. If the maximum number of errors is exceeded, the Caching Principal Resolution Service is automatically restarted.

Service Settings

The behaviour of the Principal Resolution Service as an independent service and the communication with other services is configured in the section "Service Settings".

Parent Cache Settings

Parent/child configurations can be created in the section "Parent Cache Settings". A parent cache is used to link two or more Principal Resolution Services with each other. This is necessary if the relevant principals for documents from a specific data source are distributed in different sources. A practical example of this can be found in the chapter Use cases.

Producer Consumer Services

Mindbreeze InSpire can be operated with dedicated producer and consumer nodes. The producer node is responsible for the preparation by building the index, crawling and collecting the groups. Once the producer node is finished with the preparation, the Principal Resolution Service data is copied and the consumer node is updated with it.

The consumer node is queried in the query service and updated by the producer node. If the producer node is updated, it automatically causes the consumer node to be updated.

For more information on producer and consumer nodes, see the documentation Handbook - Distributed Operation (G7) - Introduction.

The following explains how to configure the cache of the producer node and the consumer node.

Producer cache

Click in the Producer Node (= created service) in the "Indices" tab in the section "Consumer Services" on "Add Property". Then configure the following settings:

Consumer cache

If the setting "Readonly on Consumer" is configured on the producer node in a Mindbreeze InSpire environment with producer and consumer nodes, local cache updates on consumer nodes are deactivated. In the case of other environments, the setting "Readonly" must be selected manually in the “Cache Update Settings” in the consumer cache as displayed in the screenshot below. No further configuration is required. The consumer cache is automatically synchronised after each update in the producer cache.

Use cases

Parent/Child Cache - resolution of principals from multiple data sources

Parent and Child Principal Resolution Services are required if the relevant principals for documents from a specific data source are distributed in different sources. An example of this is Microsoft Sharepoint Online, where principal information can be found in the respective Sharepoint Online sites and on the Microsoft Azure platform.

The Parent/Child Configuration makes it possible to combine data from different Principal Resolution Services without duplicating the data or giving up the autonomy of the parent cache.

Procedure for the resolution of principals with Parent/Child Service:

The resolution of principals by the Parent/Child Service follows a defined process:

1. Forwarding to the Parent Cache by the Child Service:
   • The Child Service forwards all received principals to the Parent Cache before resolving the principal.
2. Resolution by the Parent Service:
   • The Parent Service resolves all principals known to it and sends them back to the Child Service.
3. Final resolution by the Child Service:
   • Finally, the Child Service resolves all the principals it has received, so that the complete resolution is done.

Selection of the Parent and Child Service:

The decision which service should be the Parent Service or the Child Service can be made based on the following criteria:

• Parent Service:
  • The Parent Service should have general principal information that is relevant for multiple data sources. This service can also be used for other Child Services and act independently as a Caching Principal Resolution Service.
  • Example: Microsoft Azure, LDAP.
• Child Service:
  • The Child Service should have specific principal information that is relevant for a specific data source.
  • Example: Sharepoint Online, OpenText Documentum.

General configuration

Configuration of the standalone Services:

1. Configure all services independently, whereby the services each have their own data sources and settings. You can find more information about the configuration of a Principal Resolution Service in the chapter Basic configuration.
2. After the configuration, ensure that the individual caches of the Principal Resolution Services have been set up and that the principals are resolved correctly. You can find more information on checking the cache and the resolution of the principals in the chapter Monitoring.

Defining a Principal Resolution Service as a Parent Cache:

To set the Parent/Child Configuration, the following steps must be carried out:

1. Go to the section "Parent Cache Settings" in the child service.

1. Activate the setting "Use Parent Principals Cache Service".
2. Enter the "Webservice Port" of the Parent Service in the setting "Parent Principals Cache Service Port".
3. Save the changes.

Configuration in the index settings:

Now the Child Principal Resolution Service must be assigned to the corresponding data source in your index.

Configuration example with Microsoft SharePoint Online

Microsoft SharePoint Online is an example of a data source that requires a Parent/Child Configuration. The principal information that is relevant for this data source is located on the respective SharePoint Online sites and also on the Microsoft Azure platform.

To ensure the correct resolution of the principals that are relevant for SharePoint Online, it is necessary to link all principals from both sources with a Parent/Child Configuration.

Hint: If the creation of credentials is also required, you will find the necessary information in the documentation Configuration - Microsoft SharePoint Online Connector - Configuration credentials and endpoints and Configuration - Microsoft Azure Principal Resolution Service - Configuring the Microsoft Azure Principal Resolution Service.

The creation of the Parent/Child Configuration is carried out as follows:

1. Creation of an index and crawler

Create a new index if no index exists yet. Select "Microsoft SharePoint Online" as the data source. In this example, the index name is "Microsoft SharePoint Online Index".

1. Creation of the Parent Service for Microsoft SharePoint Online

Create a new Caching Principal Resolution Service and in the setting "Service” click "Add Service" and select the option "SharepointOnlinePrincipalCache". Give the cache a name such as "Microsoft SharePoint Online Cache".

1. Configuration of the Webservice Port for the Microsoft SharePoint Online Principal Resolution Service

Enter a web service port in the "Service Settings" section of the Principal Resolution Services. The availability of this port is important, otherwise the service will not work. In this example, port "23901" is used.

1. Creation of the Principal Resolution Service for Microsoft Azure

Now create a second Caching Principal Resolution Service for Microsoft Azure. Select in "Service", the option "Microsoft Azure Principal Resolution Service". Give the service a name such as "Microsoft Azure Cache".

1. Configuration of the Webservice Port for the Microsoft Azure Principal Resolution Service

Specify a "Webservice Port" for the Microsoft Azure Caching Principal Resolution Service. In this example, the port "23902" is used.

1. Definition of the Parent Cache and the Child Cache for the resolution of the principals

The Microsoft Azure Principal Resolution Service is defined as the Parent Service and the Microsoft SharePoint Online Principal Resolution Service is defined as the Child Service. The reasons for this are:

• The Microsoft Azure Principal Resolution Service provides common principal information for multiple data sources. So this must be the Parent Service.
• The Microsoft SharePoint Online Principal Resolution Service provides specific principal information for SharePoint Online only. This must therefore be the Child Service.

1. Configuring the service as the Child Service

Open the "Microsoft SharePoint Online Cache" and enter the port of the "Microsoft Azure Cache" in the section "Parent Cache Settings in the setting "Parent Principals Cache Service Port". In this example, the port "23902" is required.

1. Linking the Microsoft SharePoint Online Principal Resolution Service with the index

Open the index and in the "Data Sources" section go to the setting "Caching Principal Resolution Service". Then select the corresponding service. In this example, "Microsoft SharePoint Online Cache" must be selected.

1. Finalising the configuration

Finally, save the configuration to finalise the creation of the Parent/Child configuration.

Using (LDAP/SAML) identity attributes to resolve principals

What is a Mindbreeze Identity Object?

When a user logs into a search application, the user information is saved in the form of a Mindbreeze Identity Object. When this user starts a search, this Identity Object is forwarded to the Principal Resolution Service to resolve the principals.

What are Identity Attributes?

Depending on the authentication system used, identity objects can contain optional attributes. For example, if LDAP is used for single sign-on authentication in your search application, all LDAP attributes supplied are automatically adopted as identity attributes. The same applies to the use of SAML authentication with the corresponding SAML attributes.

Default Behaviour when Resolving Principals

By default, only the user name (and all known aliases) of the logged-in user is used to resolve the principals in a Principal Resolution Service. The identity attributes are ignored. For example, if your data source uses email addresses to identify users, resolving the principals based on the user name alone will not provide any results.

Configuration for Identity Attributes for Resolving Principals

To solve this problem, the identity attribute, in this case the email address, must be marked as an alias for the Principal Resolution Service. This ensures that the value of this attribute is considered for the resolution of the principal. This can be done with the setting "Identity Alias Name Property" in the section "Service Settings".

Configuration example for Microsoft Azure/SAML authentication with Box as data source

In this example, we look at the data source Box. It uses email addresses to identify users.

The Mindbreeze Search Client is configured to use SAML-based Sign-On via Microsoft Azure (Entra) using the Microsoft Azure Principal Resolution Service.

Example SAML Claims Configuration in Microsoft Azure (Entra):

Since Box identifies users with e-mail addresses, using only the username in the resolution of the principals is insufficient. Without further configuration, our searching users can therefore not find any results.

Configuration

For the necessary configuration, the identity attribute matching the Microsoft Azure Claim Name "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress" must be marked as an alias.

This can be done in the "Service Settings" in the setting "Identity Alias Name Property" of the Principal Resolution Services (see screenshot). This attribute is provided by Microsoft Azure and is forwarded to the Mindbreeze Search Client as a SAML attribute during the SSO login.

Result

With this configuration, the "Microsoft Azure Principal Resolution Service" should now be able to successfully identify all users by their email addresses and resolve the principals properly. The users should therefore now be able to search for Box documents without any problems.

It is now possible to test whether the resolution of the principals works as expected. For more information on how to check the resolution of the principals, see the chapter Monitoring.

Exclusion of principals with extensive rights

In some cases, certain principals have very extensive rights in a system. This grants them access to a lot of data and resources. Whic can represent a potential security risk. To minimise this risk and ensure data security, such principals can be excluded from the Principal Resolution Service as a preventative measure.

Configuration example for the exclusion of principals according to a specific pattern

Assume that all principals whose names end with "_admin" are to be excluded. Such names often characterise users with very extensive rights who potentially have unrestricted access to data and resources.

Configuration

To achieve this, the setting "Exclude Principals Pattern" must be configured with a regex pattern. The regex pattern removes all principals with the ending "_admin" from the Principal Resolution Service.

The required regex pattern in this case is

.*_admin.*

After configuration, the changes must be saved in the Principal Resolution Service.

Result

This configuration excludes all principals that end with "_admin" and may have very far-reaching rights from the Principal Resolution Service. This minimises the security risk and ensures that users with such rights do not have unrestricted access to all files and data.

Troubleshooting

Troubleshooting differences between upper and lower case spelling of principals

Differences in the upper and lower case spelling of principals can mean that the resolution of users does not work as expected or that the resolved principals do not correspond to the upper and lower case spelling of the document ACLs.

Case 1: The resolved principals do not correspond to the document ACLs

Such a case could look as follows:

• Document ACLs: sales, marketing.
• Stored principals in the Principal Resolution Service: SALES, MARKETING.

Solution:

In the section "Service Settings", the setting "Lowercase Principals" must be activated for the Principal Resolution Service. This setting enables the conversion of all principals to lowercase, regardless of their original spelling.

Case 2: Principals of logged-in users are not resolved

Such a case could look like this:

• User credentials: "david porter".
• Stored principals in the Principal Resolution Service: "David Porter", "David.Porter", “david.porter@mindbreeze.com”.

Solution 1:

In the section "Service Settings", the setting "Lowercase Principals" must be activated for the Principal Resolution Service. This setting enables the conversion of all principals to lowercase, regardless of their original spelling.

Attention: Activating this setting only solves the problem if the login information is in lowercase.

Solution 2:

Alternatively, the setting "Case Insensitive Member Resolution" for the Principal Resolution Service can be activated in the section "Service Settings". This setting enables the correct resolution of principals, regardless of upper and lower case.