SharePoint Online Connector

Installation and Configuration

Installation

Before installing the SharePoint Online connector, make sure that the Mindbreeze server is installed and the SharePoint Online connector is included in the license. Use the Mindbreeze Management Center to install or update the connector.

Plugin installation via Mindbreeze Management Center

To install the plug-in, open the Mindbreeze Management Center. Select “Configuration” from the menu pane on the left-hand side. Then navigate to the “Plugins” tab. Under “Plugin Management,” select the appropriate zip file and upload it by clicking “Upload.” This automatically installs or updates the connector, as the case may be. In the process, the Mindbreeze services are restarted.

Configuring Mindbreeze

Select the “Advanced” installation method for configuration.

Configuring the index

To create a new index, navigate to the “Indices” tab and click the “Add new index” icon in the upper right corner.

Enter the path to the index and change the display name as necessary.

Configuring the data source

Add a new data source by clicking the “Add new custom source” icon at the top right. Select the category “Microsoft SharePoint Online” and configure the data source according to your needs.

In Sharepoint Online

In the "Sharepoint Online" area you can define your Microsoft SharePoint Online installation that is to be indexed. The following options are available:

In Azure Endpoints

Only enter the URL for the Azure ACS endpoint in the “Azure ACS endpoint” field if your SharePoint environment is hosted in a special environment (such as Germany).

The following environments require special URLs:

A complete list for Azure ACS endpoints can also be found at https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/extending-sharepoint-online-for-germany-china-usgovernment-environments.

In App-Only Authentication

Configure the options as follows:

App Registration in Sharepoint: Step 1

To generate a client ID and a client secret, enter the following URL in the browser:
<Server URL>/_layouts/15/appregnew.aspx
(e.g. https://mycompany.sharepoint.com/_layouts/15/appregnew.aspx)

Click the two buttons "Generate" (for "client Id" and for "client secret") and enter the other information as follows:

• "Title": user-definable
• "App Domain": "localhost"
• “Redirect URI”: https://localhost/

Then click “Create."

Then enter the client id and the client secret into the Mindbreeze InSpire configuration. Otherwise you will not be able to access the client secret later.

App Registration in Sharepoint: Step 2

Now enter the following URL in the browser:
<Server URL>/_layouts/15/appinv.aspx
(e.g. https://mycompany.sharepoint.com/_layouts/15/appinv.aspx)

Enter the client id in the “App Id” field and click “Lookup.” “Title,” “App Domain,” and “Redirect URL” will be filled in automatically. Then enter the following in the “Permission Request XML” field:

<AppPermissionRequests AllowAppOnlyPolicy="true">

    <AppPermissionRequest
        Scope="http://sharepoint/content/sitecollection/web"

        Right="FullControl"

    />

</AppPermissionRequests>

Note: "FullControl" is required so that Mindbreeze InSpire has access to the access rights of the SharePoint documents to be indexed in order to map the authorizations in Mindbreeze InSpire.

Then click “Create."

App Registration in Sharepoint: Step 3

Additional rights are required so that the ACL information on the users and groups required by the Principal Resolution Service can also be downloaded from SharePoint Online.

Now enter the following URL in the browser:

<Admin Site URL>/_layouts/15/appinv.aspx
(z.B. https://mycompany-admin.sharepoint.com/_layouts/15/appinv.aspx)

ATTENTION: Make sure that you are on the admin page. For example, if the URL is https://mycompany.sharepoint.com, then the admin page is usually https://mycompany-admin.sharepoint.com.

Enter the Client Id in the "App Id" field and activate the "Lookup" button. "Title", "App Domain" and "Redirect URL" will be filled in automatically. Then enter the following in the "Permission Request XML" field:

<AppPermissionRequests AllowAppOnlyPolicy="true">

    <AppPermissionRequest

        Scope="http://sharepoint/content/tenant"

        Right="FullControl" />

</AppPermissionRequests>

Then activate the "Create" button.

Bereich „Content Settings“

Configuring the principal resolution service

Select “Advanced Settings” to configure the following settings.

Enable the option “Enforce ACL Evaluation.”

Add a new service under “Services” by clicking on “Add new service.” Select “SharepointOnlinePrincipalCache” and assign a display name.

In Sharepoint Online

Enter the information about your Microsoft SharePoint Online installation under “Sharepoint Settings.” “Server URL” and “Site Relative URL” must match the settings in the “Data Source” area.

With the option "Enable Delta Update", you can set whether only the changes to the groups should be fetched from Sharepoint Online after the first cache creation, instead of fetching all groups each time. This is especially recommended for very large Sharepoint instances, as a regular cache update can otherwise take a long time. With the advanced option "Dump Change Responses", the changes that we receive from Sharepoint Online during the delta update can also be dumped into a file. This is very helpful for troubleshooting.

Under “Regex for your organization” you can enter a regular expression that defines whether or not a user belongs to your organization. The regular expression can refer to the e-mail address, the ObjectSID, or the ObjectGUID from LDAP.

The option "Parallel Request Count" can be used to define how many HTTP requests are sent simultaneously by the crawler. The higher the value, the faster the crawl run should be, but too high a value can also lead to a lot of "Too Many Requests" errors on the Sharepoint side. A value above 30 is not recommended.

In App-Only Authentication

This is only necessary if you have also configured app-only authentication for the data source.

In Graph API

If you have not configured “AD Connect” in the Azure Active Directory, select “AD Connect is NOT configured” and fill in the fields “Tenant Context ID,” “Application ID,” “Generated Key,” and “Protected Resource Hostname.” You can find the corresponding values in the Azure Portal.
You have to register a new app under the tab "Azure Active Directory" and then under the tab "App registrations":

In the newly generated app you will find the application id and under the tab "Certificates & Secrets" you can generate a key.

If AD Connect is set up in your Azure Active Directory, do not enable the “AD Connect is NOT configured” option.

The following table lists the protected resource hostnames for different cloud environments:  

A complete list of protected resource hostnames can also be found at https://developer.microsoft.com/en-us/graph/docs/concepts/deployments

In LDAP Settings

An LDAP cache is required to remove users from the active directory. The following link describes how to set up a caching principal resolution service: https://help.mindbreeze.com/de/index.php?topic=doc/Installation--Konfiguration---Caching-Principal-Resolution-Service/index.htm

The following values should be entered in the LDAP cache under “User Alias Name LDAP Attributes” or “User Alias Name LDAP Attributes”:

mail

cn

objectGUID

objectSID

Enter the information about the LDAP cache under “LDAP Settings.” Enable the option “Use LDAP Principal Cache Service” and enter the corresponding port of your LDAP principal cache.

Under “Cache Settings,” configure where you want the database for the cache to be located and set the desired interval for the updates.

Under “Service Settings,” enter a free port to be used for the principal cache and enable the “Lowercase Principals” option so that the SharePoint groups can be resolved correctly.

In Azure Endpoints

Only enter the URLs for Azure AD Endpoint and Azure ACS Endpoint in the “Azure AD Endpoint” and “Azure ACS endpoint” fields if your SharePoint environment is hosted in a special environment (such as Germany).

The following environments require special URLs for Azure AD Endpoint:

The following environments require special URLs for Azure ACS Endpoint:

A complete list for Azure ACS endpoints can also be found at https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/extending-sharepoint-online-for-germany-china-usgovernment-environments.

REST API for testing

1. To test the caching principal resolution service, you can use the Principal Resolution Service REST API.

Configuring credentials and endpoints

If you are using app-only authentication, this section is NOT applicable to you. Otherwise, proceed as follows:

Navigate to the “Network” tab and add a new credential for Microsoft SharePoint Online under “Credentials” by clicking “Add Credential.”

Enter the credentials for the user you want to use for indexing and assign a name for the credential. Select a user with adequate permissions to read all relevant pages and authorizations.

Then add a new endpoint for the credential you just created by clicking on “Add Endpoint” under “Endpoints.” Enter the server URL of your Microsoft SharePoint Online installation as the location and select the credential you just created.

Hints for crawling OneDrive

Configuration

With the help of the Sharepoint Online Connector, OneDrive pages can also be crawled.
For this you have to consider some points for the configuration:

• In the Server URL -my has to be appended to the tenant name, e.g. https://mycompany-my.sharepoint.com
• All OneDrive pages must be explicitly specified in the Site Relative URL. Site Discovery is not supported, e.g. /personal/my_user_mycompany_onmicrosoft_com

Appendix

Used API Endpoints