Microsoft Exchange Connector

Installation and Configuration

Installation

Before installing the Microsoft Exchange Connector ensure that the Mindbreeze Server is already installed and this connector is also included in the Mindbreeze license.

Needed Rights for Crawling User

• Mailbox permissions:
  • Access rights: FullAccess

Granting the needed rights for the Crawling User

Execute the following power shell commands on the Microsoft Exchange server:

Per Mailbox:

Get-Mailbox | Add-MailboxPermission -user <domain>\<user>
-accessrights FullAccess

Or per mailbox database:

Get-MailboxDatabase -identity “Mailbox 1” |Add-ADPermission -user <domain>\<user> –accessrights GenericALL

The crawling user can read the mailbox of other users but he cannot send emails on behalf of these users. The crawling user should have read permission on user objects in AD in order to read the msexchmailboxsecuritydescriptor attribute containing mailbox permissions during crawling ACLs.

Overwriting default Throttling Policies

For better crawling performance its recommended to overwrite the default Throttling Policies (EWSFindCountLimit = 1000 , EWSMaxConcurrency =10) according to current infrastructure. Therefor a new Throttling Policy should be defined for service user.

New-ThrottlingPolicy -Name serviceUserPolicy -EWSFindCountLimit 1000

-EWSMaxConcurrency 25

Set-Mailbox <domain>\<serviceuser> -ThrottlingPolicy serviceUserPolicy

Configuration of Mindbreeze

Click on the “Indices” tab and then on the “Add new index” symbol to create a new index.

Enter the index path, e.g. “/data/indices/exchange/2”. Change the Display Name of the Index Service and the related Filter Service if necessary.

Add a new data source with the symbol “Add new custom source” at the bottom right.

Configuration of Data Source

Caching Principal Resolution Service

A CachingLdapPrincipalResoution can be selected as the caching principal resolution service. It is then used to terminate a user's AD group membership when searching.

For details on configuring the caching principal resolution service, see Caching Principal Resolution Service.

Microsoft Exchange Server Connection

This information is important for the configuration of the connection with the Microsoft Exchange Server:

• Microsoft Exchange Server: Fully qualified domain name of the Microsoft Exchange  Server (e.g. exchange2010.mycompany.com).
• Domain: Fully qualified domain name of user. (Optional)
• Privileged Logon Account: User name of the privileged user. This is only needed when the user authentication fails. Detailed information on the Crawler’s integrated authentication is found under ‟2. CONFIGURING THE INTEGRATED AUTHENTICATION OF THE ”. (Opional)
• Password: The password of the privileged user. (Only for BASIC authentication)

Context Actions

Hits of emails, attachments and calendars can be downloaded via the context menu.

To open a hit through Outlook Web Access, the “Outlook Web Access URL” must be configured and the option “Use Outlook Web Access” must be checked.  

If the “Use Outlook Web Access Open as Default Action” option is selected, this action will be performed as a standard context action for every search result, i.e. the link of the hit opens Outlook Web Access.

Source Scope Constraints (Advanced Options)

• “Mailbox Selection Pattern”: It is possible to restrict the data to be crawled. For example, it is possible to crawl only documents from a specific mailbox. The name of the mailbox to be indexed must then correspond to a regular expression specified in the "Mailbox Selection Pattern" field.
• For example, to restrict your selection to the mailbox of David Porter (david.porter@mycompany.com), enter "david.porter@mycompany.com" as the selection pattern. However, you can use any regular expression (regex) as a pattern. The syntax of the regular expression follows the POSIX convention.  
• “Folders items traversal method”: The following methods for detecting changes in the mailboxes can be selected.
  • “All Folders Items”: Every item in each directory is compared with the corresponding item in the index.
  • “Changed Folders Items”: Each item in a directory containing at least one modified item is compared with the corresponding item in the index.  
  • "All Folders Items On Startup Only": corresponds to the "All Folders Items" method for the first crawler run after starting the service and "Changed Folders Items" for the subsequent crawler runs.
• “EWS Search Method”: "Search Filter" and "Query String" are the two methods that allow for restricting the objects found in Exchange Server. It is advisable to use the "Search Filter" method unless you only want to restrict certain objects using a defined AQS. If the "Query String" method is selected, the AQS string must be entered in the "Query String (AQS)" field. The use of query strings is not supported for the public folder.
• With the "Custom Search Filter Query" field additional filter conditions can be entered via XML. These only apply if "Search Filter" has been selected as "EWS Search Method". If, for example, all emails whose subject begins with "Automatic reply:" or "Accepted:" are to be excluded, this can be achieved with the following configuration:

<Not xmlns="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ns2="http://schemas.microsoft.com/exchange/services/2006/messages">

            <Or>

              <Contains ContainmentMode="Prefixed" ContainmentComparison="Exact">

                <FieldURI FieldURI="item:Subject"/>

                <Constant Value="Automatische Antwort:"/>

              </Contains>

              <Contains ContainmentMode="Prefixed" ContainmentComparison="Exact">

                <FieldURI FieldURI="item:Subject"/>

                <Constant Value="Accepted:"/>

              </Contains>

            </Or>

</Not>

Details on the available XML-Operations can be found in the official Microsoft documentation.

Restriction by time interval (advanced view)

To index only the objects within a given time interval, select the corresponding time unit from "Past Time Unit" options and enter the number of these time units in the "Number of Past Time Units" field.  This time interval will shift after every delta run. During this process, new objects are indexed and older ones are deleted from the index.

Search settings

To restrict the user search to the relevant primary user mailbox and public folders, select "Restrict to Primary Mailbox and Public Folder". This does not return hits from shared mailboxes and the shared folders of other users.

For Microsoft Exchange 2003 and 2007, it is necessary to re-index the Microsoft Exchange data source (if indices have already been created with a version prior to 2015 Spring Release). Delta indexing is not sufficient.

Resource parameters (advanced view)

In this section, you can define settings that influence resource utilization on the Microsoft Exchange Server as well as on the Mindbreeze enterprise search server.  The following parameters can be set:

• “Crawler Queue Size”: Maximum number of entries in the indexing queue.
• “EWS Dispatcher Thread Count”: Number of threads connecting to the Microsoft Exchange Server simultaneously.
• “Mindbreeze Dispatcher Thread Count”: Number of threads that send data to the index simultaneously.

Dump requests/responses (advanced view)

The “dump requests/responses” field allows advanced debugging and logs queries and responses to the configured dump directory. With the preconfigured "On Error" option, log files are automatically created whenever an error occurs. With "Never" no dumps are generated and with "Always" they are generated for each request.

The “Dump Directory” field specifies a directory in which the dumps are stored. If no dump directory is configured, the dump feature is not available.

Note: Do not activate "Always" permanently in production mode.

Configuring the integrated authentication of the Microsoft Exchange Crawler

Windows:

If the installation is made on a Microsoft Windows Server, the Kerberos authentication of the current Mindbreeze Service user can also be used for the Microsoft Exchange Crawler. In this case the Service user must be authorized to access the Microsoft Exchange Web Services.

Linux:

For installations under Linux, you can use the managementcenter to generate the keytab.

• Upload the keytab:

• Configure the keytab and the contained principal (in the authentication tab):

IMPORTANT: The keytab must contain the key of the abovementioned user. The keytab for the Client Service cannot be used here.

Problem-solving options

Generally speaking, problems with the indexing of Exchange data sources can be found first and foremost in the corresponding log files in the Mindbreeze log folder.

In the Mindbreeze base log folder there is a corresponding subfolder for the configured Exchange crawler which may, for example, be named as follows:

C:\logs\current\log-mescrawler_launchedservice-Microsoft_Exchange

This contains a date subfolder for each crawl run with two log files:

• log-mescrawler_launchedservice.log: Log file with all relevant log information and possible errors

Invalid LDAP credentials

If no credentials are configured in the Network tab under LDAP Settings , the crawler user credentials are used to connect to the LDAP server. The following message is logged in the log file if the credentials are wrong:

com.mindbreeze.enterprisesearch.ldapclient.LDAPQuery INTERNALWARNING: Unable to connect to ldap server <x> Invalid Credentials (49)

Unauthorized crawling user

Problem behavior:

The crawler does not receive documents from Exchange and does not create a file list in the log file mindbreeze-dispatcher.csv.

Many error messages of the following type can be found in the log file log-mescrawler_launchedservice.log:

com.mindbreeze.enterprisesearch.connectors.exchange.ewsclient.callables.SOAPCallable WARNING: Call find folders of mailbox <x> - SOAP call: ERROR ErrorItemNotFound The specified object was not found in the store.

Problem description and solution:

Due to authorization problems, it is not possible for the service user to query folders from Exchange. See the section "Required authorizations for the crawling user" above.

In addition, you can send the following POST query to EWS via curl to check the authorization:

curl -X POST   https://mail.mycompany.com/ews/exchange.asmx   -v -H 'cache-control: no-cache' --ntlm -negotiate -u cr'<?xml version="1.0" encoding="utf-8"?>-type: text/xml'   -d '

<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

               xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages"

               xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"

               xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">

  <soap:Header>

    <t:RequestServerVersion Version="Exchange2010" />

  </soap:Header>

  <soap:Body>

   <m:FindFolder Traversal="Deep">

      <m:FolderShape>

        <t:BaseShape>IdOnly</t:BaseShape>

        <t:AdditionalProperties>

          <t:FieldURI FieldURI="folder:DisplayName" />

        </t:AdditionalProperties>

      </m:FolderShape>

      <m:IndexedPageFolderView MaxEntriesReturned="100"

                               Offset="0"

                               BasePoint="Beginning" />

      <m:ParentFolderIds>

        <t:DistinguishedFolderId Id="msgfolderroot">

               <t:Mailbox>

               <t:EmailAddress>user@mycompany.com</t:EmailAddress>

               </t:Mailbox>

        </t:DistinguishedFolderId>

      </m:ParentFolderIds>

    </m:FindFolder>

  </soap:Body>

</soap:Envelope>'