Installation and Configuration
Microsoft Loop Connector

Introduction

The Microsoft Loop Connector can be used to index Microsoft Loop pages with their metadata and content. The Loop Connector works differently compared to other connectors. To index Loop, a Loop Sitemap Generator is created, which then provides a sitemap. This sitemap is then crawled by a Web Connector configured specifically for Loop.

For more information, please see the chapter Limitations.

Configuration of Microsoft Loop

Providing a Microsoft Loop User

• Create a new user in Microsoft Loop that can be used in the Mindbreeze Microsoft Loop Sitemap Generator to create a Microsoft Loop sitemap. The user must have access to all the content that is to be indexed.

Configuration of Mindbreeze

In the Mindbreeze Management Center, open the ‘Configuration’ section to configure the Microsoft Loop Sitemap Generator and the Microsoft Loop Principal Resolution Service.

Configuration of the Microsoft Loop Sitemap Generator

Add a new service in the tab “Indices” with “+Add Service”. Then select “Microsoft Loop Sitemap Generator” for the setting “Service” in the new service.

Now configure the Microsoft Loop Sitemap Generator with the settings in the section “Connection Settings”.

Connection Settings

Configuration of the Microsoft Loop Principal Resolution Service

Add a new service in the tab “Indices” with “+Add Service”. Then select “Microsoft Loop Principal Resolution Service” for the setting “Service” in the new service.

Now configure the Microsoft Loop Principal Resolution Service with the settings in the section “Connection Settings”.

Hint: For more information about the creation, basic configuration of a cache for a Principal Resolution Service and other configuration options, see Installation & Configuration - Caching Principal Resolution Service.

Connection Settings

Configuration of the Index

Add a new index in the tab “Indices” with “+Add Index”. Select the desired “Index Node” and “Client Service” and select “Web” as “Data Source”. Then confirm your entries with “Apply”.

Setup of the Web Connector

To set up the Web Connector, you can copy the Config XML from the Web Connector Import/Export XML section and import it using the Import/Export button:

In addition, the following changes are also necessary:

1. Link the crawler to the Microsoft Loop Principal Resolution Service. To do this, select the added Microsoft Loop Principal Resolution Service in the setting “Caching Principal Resolution Service”.

1. Furthermore, the port specified for the setting “Crawling Root” must match the configured “Bind Port” of the Microsoft Loop Sitemap Generator.

1. Finally, the credential of the Loop User must be added to the configured “Credential Scripts” for the two scripts “MS Login Username” and “MS Login Password”. Activate “Advanced Settings” to add or edit “Credential Scripts”.

Appendix

Limitations

The following limitations should be noted:

Only workspaces to which the specified Loop User has access, can be indexed.

The Loop Sitemap Generator can only process up to a maximum of 1000 workspaces.

A workspace may contain a maximum of 1000 users.

Web Connector Import/Export XML

The following XML can be used for setting up the web connector, among other things. For more information, see the chapter Setup of the Web Connector.

<settings>

    <id>plugin:com.mindbreeze.datasource.Crawler/Web</id>

    <attributes>

        <attribute name="category" value="Web"/>

        <attribute name="categoryinstance" value="Microsoft Loop"/>

        <attribute name="datasource" value="Web"/>

        <attribute name="processtype" value="command"/>

        <attribute name="interval" value="6"/>

        <attribute name="intervalmult" value="3600"/>

        <attribute name="launchedservice" value="true"/>

    </attributes>

    <properties>

        <property name="com.mindbreeze.datasource.enable_javascript" value="true"/>

        <property name="com.mindbreeze.datasource.include_network_resources_hostname">

            <![CDATA[login.microsoftonline.com

aadcdn.msauth.net

aadcdn.msftauth.net

login.live.com

.*data.microsoft.com

graph.microsoft.com

substrate.office.com

ecs.office.com

odc.officeapps.live.com

clients.config.office.net

.*sharepoint.com

config.edge.skype.com

.*cdn.office.net

.*hubblecontent.osi.office.net

loop.cloud.microsoft]]>

        </property>

        <property name="com.mindbreeze.datasource.credential_scripts" value="composite">

            <property name="com.mindbreeze.datasource.credential_scripts.script_name" value="MS Login Username"/>

            <property name="com.mindbreeze.datasource.credential_scripts.script_allowed_hosts" value="login.microsoftonline.com"/>

            <property name="com.mindbreeze.datasource.credential_scripts.script_selector_type" value="XPATH"/>

            <property name="com.mindbreeze.datasource.credential_scripts.script_trigger_selector" value="//*[@type=\&quot;email\&quot;]"/>

            <property name="com.mindbreeze.datasource.credential_scripts.script" value="// 24.7&#xA;event = new Event('change')&#xA;usernameField = document.evaluate(&quot;//*[@type=\&quot;email\&quot;]&quot;, document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;&#xA;submitButton = document.evaluate(&quot;//*[@type=\&quot;submit\&quot;]&quot;, document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;&#xA;usernameField.value = mesCredential.username;&#xA;usernameField.dispatchEvent(event);&#xA;submitButton.click(); "/>

            <property name="com.mindbreeze.datasource.credential_scripts.scrip_credential" value="377777061136392"/></property>

        <property name="com.mindbreeze.datasource.maxhopsfromcrawlingroot" value="1"/>

        <property name="com.mindbreeze.datasource.crawlingroot" value="http://localhost:23950/sitemap.xml"/>

        <property name="com.mindbreeze.datasource.robotshonoringpolicytype" value="IGNORE"/>

        <property name="com.mindbreeze.datasource.presence_selectors" value="composite">

            <property name="com.mindbreeze.datasource.presence_selectors.content_presence_selector_url_patterns" value="https://loop.cloud.microsoft/.*"/>

            <property name="com.mindbreeze.datasource.presence_selectors.content_selector_type" value="XPATH"/>

            <property name="com.mindbreeze.datasource.presence_selectors.content_presence_selector" value="//meta[@name=\&quot;isready\&quot;]"/></property>

        <property name="com.mindbreeze.datasource.exclude_javascript_url_pattern">

            <![CDATA[.*robots.txt]]>

        </property>

        <property name="com.mindbreeze.datasource.enable_verbose_logging" value="false"/>

        <property name="com.mindbreeze.datasource.allowed_resource_types">

            <![CDATA[DOCUMENT

          STYLESHEET

          IMAGE

          MEDIA

          FONT

          SCRIPT

          XHR

          FETCH

          PING

          CSPVIOLATIONREPORT

          OTHER]]>

        </property>

        <property name="com.mindbreeze.datasource.skip_head_request" value="true"/>

        <property name="com.mindbreeze.datasource.parallelqueuecount" value=""/>

        <property name="com.mindbreeze.datasource.crawlerthreadcount" value="5"/>

        <property name="com.mindbreeze.datasource.scripts" value="composite">

            <property name="com.mindbreeze.datasource.scripts.script_name" value="Reload Loop"/>

            <property name="com.mindbreeze.datasource.scripts.script_url_patterns" value="https://loop.cloud.microsoft/.*"/>

            <property name="com.mindbreeze.datasource.scripts.script_selector_type" value="XPATH"/>

            <property name="com.mindbreeze.datasource.scripts.script_trigger_selector" value="//*[@id=\&quot;loopApp-menu2\&quot;]"/>

            <property name="com.mindbreeze.datasource.scripts.script" value="// 24.7&#xA;location.reload();"/></property>

        <property name="com.mindbreeze.datasource.scripts" value="composite">

            <property name="com.mindbreeze.datasource.scripts.script_name" value="Press KMSI"/>

            <property name="com.mindbreeze.datasource.scripts.script_url_patterns" value="https://login.microsoftonline.com/common/login"/>

            <property name="com.mindbreeze.datasource.scripts.script_selector_type" value="XPATH"/>

            <property name="com.mindbreeze.datasource.scripts.script_trigger_selector" value="//*[@type=\&quot;submit\&quot;]"/>

            <property name="com.mindbreeze.datasource.scripts.script" value="// 24.7&#xA;document.evaluate(&quot;//*[@type=\&quot;submit\&quot;]&quot;, document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue.click();"/></property>

        <property name="com.mindbreeze.datasource.credential_scripts" value="composite">

            <property name="com.mindbreeze.datasource.credential_scripts.script_name" value="MS Login Password"/>

            <property name="com.mindbreeze.datasource.credential_scripts.script_allowed_hosts" value="login.microsoftonline.com"/>

            <property name="com.mindbreeze.datasource.credential_scripts.script_selector_type" value="XPATH"/>

            <property name="com.mindbreeze.datasource.credential_scripts.script_trigger_selector" value="//*[@id=\&quot;idA_PWD_ForgotPassword\&quot;]"/>

            <property name="com.mindbreeze.datasource.credential_scripts.script" value="// 24.7&#xA;passwordField =  document.evaluate(&quot;//*[@type=\&quot;password\&quot;]&quot;, document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;&#xA;submitButton =  document.evaluate(&quot;//*[@type=\&quot;submit\&quot;]&quot;, document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;&#xA;passwordField.value = mesCredential.password;&#xA;passwordField.dispatchEvent(event);&#xA;submitButton.click();"/>

            <property name="com.mindbreeze.datasource.credential_scripts.scrip_credential" value="377777061136392"/></property>

        <property name="com.mindbreeze.datasource.page_load_timeout_seconds" value="20"/>

        <property name="com.mindbreeze.datasource.network_timeout" value="20"/>

        <property name="com.mindbreeze.datasource.isdeltarun" value="complete"/>

        <property name="com.mindbreeze.datasource.on_new_document_script">

            <![CDATA[// 24.7

window.open = function(...args) {

console.log("Popup blocked: window.open was called, but no action was taken.");

};

window.print = function () {

     window.onbeforeprint();

     const meta = document.createElement('meta');

     meta.name = "isready";

     meta.content = "true";

     document.head.appendChild(meta);

};]]>

        </property>

        <property name="com.mindbreeze.datasource.match_network_resources_hostnames_as_regex" value="true"/>

        <property name="com.mindbreeze.datasource.content_presence_selector" value=""/>

    </properties>

</settings>