Microsoft Exchange Connector

Installation and Configuration

Copyright ©

Mindbreeze GmbH, A-4020 Linz, 2018.

All rights reserved. All hardware and software names used are registered trade names and/or registered trademarks of the respective manufacturers.

These documents are highly confidential. No rights to our software or our professional services, or results of our professional services, or other protected rights can be based on the handing over and presentation of these documents.

Distribution, publication or duplication is not permitted.

The term ‘user‘ is used in a gender-neutral sense throughout the document.

InstallationPermanent link for this heading

Before installing the Microsoft Exchange Connector ensure that the Mindbreeze Server is already installed and this connector is also included in the Mindbreeze license.

Extending Fabasoft Mindbreeze Enterprise for use with the Microsoft Exchange ConnectorPermanent link for this heading

The Microsoft Exchange Connector is available as a ZIP file. This file must be registered with the Fabasoft Mindbreeze Enterprise Server via mesextension.exe as follows:

mesextension --interface=plugin --type=archive --file=MicrosoftExchangeConnector<version>.zip install

PLEASE NOTE: The Connector can be updated by calling the same mesextention. Fabasoft Mindbreeze Enterprise will automatically carry out the required update.

Needed Rights for Crawling UserPermanent link for this heading

  • Mailbox permissions:
    • Access rights: FullAccess

Granting the needed rights for the Crawling UserPermanent link for this heading

Execute the following power shell commands on the Microsoft Exchange server:

Per Mailbox:

Get-Mailbox | Add-MailboxPermission -user <domain>\<user>
-accessrights FullAccess

Or per mailbox database:

Get-MailboxDatabase -identity “Mailbox 1” |Add-ADPermission -user <domain>\<user> –accessrights GenericALL

The crawling user can read the mailbox of other users but he cannot send emails on behalf of these users. The crawling user should have read permission on user objects in AD in order to read the msexchmailboxsecuritydescriptor attribute containing mailbox permissions during crawling ACLs.

Overwriting default Throttling PoliciesPermanent link for this heading

For better crawling performance its recommended to overwrite the default Throttling Policies (EWSFindCountLimit = 1000 , EWSMaxConcurrency =10) according to current infrastructure. Therefor a new Throttling Policy should be defined for service user.

New-ThrottlingPolicy -Name serviceUserPolicy -EWSFindCountLimit 1000

-EWSMaxConcurrency 25

Set-Mailbox <domain>\<serviceuser> -ThrottlingPolicy serviceUserPolicy


Configuration of MindbreezePermanent link for this heading

Select the “Advanced” installation method:

Click on the “Indices” tab and then on the “Add new index” symbol to create a new index.

Enter the index path, e.g. “/data/indices/exchange/2. Change the Display Name of the Index Service and the related Filter Service if necessary.

Add a new data source with the symbol “Add new custom source” at the bottom right.

Configuration of Data SourcePermanent link for this heading

Caching Principal Resolution ServicePermanent link for this heading

A CachingLdapPrincipalResoution can be selected as the caching principal resolution service. It is then used to terminate a user's AD group membership when searching.

For details on configuring the caching principal resolution service, see Caching Principal Resolution Service.

Microsoft Exchange Server ConnectionPermanent link for this heading

This information is important for the configuration of the connection with the Microsoft Exchange Server:

  • Microsoft Exchange Server: Fully qualified domain name of the Microsoft Exchange  Server (e.g. exchange2010.mycompany.com).
  • Domain: Fully qualified domain name of user. (Optional)
  • Privileged Logon Account: User name of the privileged user. This is only needed when the user authentication fails. Detailed information on the Crawler’s integrated authentication is found under ‟2. CONFIGURING THE INTEGRATED AUTHENTICATION OF THE ”. (Opional)
  • Password: The password of the privileged user. (Only for BASIC authentication)

Context ActionsPermanent link for this heading

Hits of emails, attachments and calendars can be downloaded via the context menu.

To open a hit through Outlook Web Access, the “Outlook Web Access URL” must be configured and the option “Use Outlook Web Access” must be checked.  

If the “Use Outlook Web Access Open as Default Action” option is selected, this action will be performed as a standard context action for every search result, i.e. the link of the hit opens Outlook Web Access.

Source Scope Constraints (Advanced Options)Permanent link for this heading

  • “Mailbox Selection Pattern”: It is possible to restrict the data to be crawled. For example, it is possible to crawl only documents from a specific mailbox. The name of the mailbox to be indexed must then correspond to a regular expression specified in the "Mailbox Selection Pattern" field.
  • For example, to restrict your selection to the mailbox of David Porter (david.porter@mycompany.com), enter "david.porter@mycompany.com" as the selection pattern. However, you can use any regular expression (regex) as a pattern. The syntax of the regular expression follows the POSIX convention.  
  • “Folders items traversal method”: The following methods for detecting changes in the mailboxes can be selected.
    • “All Folders Items”: Every item in each directory is compared with the corresponding item in the index.
    • “Changed Folders Items”: Each item in a directory containing at least one modified item is compared with the corresponding item in the index.  
    • "All Folders Items On Startup Only": corresponds to the "All Folders Items" method for the first crawler run after starting the service and "Changed Folders Items" for the subsequent crawler runs.
  • “EWS Search Method”: "Search Filter" and "Query String" are the two methods that allow for restricting the objects found in Exchange Server. It is advisable to use the "Search Filter" method unless you only want to restrict certain objects using a defined AQS. If the "Query String" method is selected, the AQS string must be entered in the "Query String (AQS)" field. The use of query strings is not supported for the public folder.

Restriction by time interval (advanced view)Permanent link for this heading

To index only the objects within a given time interval, select the corresponding time unit from "Past Time Unit" options and enter the number of these time units in the "Number of Past Time Units" field.  This time interval will shift after every delta run. During this process, new objects are indexed and older ones are deleted from the index.

Search settingsPermanent link for this heading

To restrict the user search to the relevant primary user mailbox and public folders, select "Restrict to Primary Mailbox and Public Folder". This does not return hits from shared mailboxes and the shared folders of other users.

For Microsoft Exchange 2003 and 2007, it is necessary to re-index the Microsoft Exchange data source (if indices have already been created with a version prior to 2015 Spring Release). Delta indexing is not sufficient.

Resource parameters (advanced view)Permanent link for this heading

In this section, you can define settings that influence resource utilization on the Microsoft Exchange Server as well as on the Mindbreeze enterprise search server.  The following parameters can be set:

  • “Crawler Queue Size”: Maximum number of entries in the indexing queue.
  • “EWS Dispatcher Thread Count”: Number of threads connecting to the Microsoft Exchange Server simultaneously.
  • “Mindbreeze Dispatcher Thread Count”: Number of threads that send data to the index simultaneously.

Dump requests/responses (advanced view)Permanent link for this heading

The “dump requests/responses” field allows advanced debugging and logs queries and responses to the configured dump directory. With the preconfigured "On Error" option, log files are automatically created whenever an error occurs. With "Never" no dumps are generated and with "Always" they are generated for each request.

The “Dump Directory” field specifies a directory in which the dumps are stored. If no dump directory is configured, the dump feature is not available.

Note: Do not activate "Always" permanently in production mode.

Configuring the integrated authentication of the Microsoft Exchange CrawlerPermanent link for this heading

Windows:

If the installation is made on a Microsoft Windows Server, the Kerberos authentication of the current Mindbreeze Service user can also be used for the Microsoft Exchange Crawler. In this case the Service user must be authorized to access the Microsoft Exchange Web Services.

Linux:

For installations under Linux, you can use the managementcenter to generate the keytab.

  • Upload the keytab:

  • Configure the keytab and the contained principal (in the authentication tab):

IMPORTANT: The keytab must contain the key of the abovementioned user. The keytab for the Client Service cannot be used here.

Problem-solving optionsPermanent link for this heading

Generally speaking, problems with the indexing of Exchange data sources can be found first and foremost in the corresponding log files in the Mindbreeze log folder.

In the Mindbreeze base log folder there is a corresponding subfolder for the configured Exchange crawler which may, for example, be named as follows:

C:\logs\current\log-mescrawler_launchedservice-Microsoft_Exchange

This contains a date subfolder for each crawl run with two log files:

  • log-mescrawler_launchedservice.log: Log file with all relevant log information and possible errors

Invalid LDAP credentialsPermanent link for this heading

If no credentials are configured in the Network tab under LDAP Settings , the crawler user credentials are used to connect to the LDAP server. The following message is logged in the log file if the credentials are wrong:

com.mindbreeze.enterprisesearch.ldapclient.LDAPQuery INTERNALWARNING: Unable to connect to ldap server <x> Invalid Credentials (49)

Unauthorized crawling userPermanent link for this heading

Problem behavior:

The crawler does not receive documents from Exchange and does not create a file list in the log file mindbreeze-dispatcher.csv.

Many error messages of the following type can be found in the log file log-mescrawler_launchedservice.log:

com.mindbreeze.enterprisesearch.connectors.exchange.ewsclient.callables.SOAPCallable WARNING: Call find folders of mailbox <x> - SOAP call: ERROR ErrorItemNotFound The specified object was not found in the store.

Problem description and solution:

Due to authorization problems, it is not possible for the service user to query folders from Exchange. See the section "Required authorizations for the crawling user" above.

In addition, you can send the following POST query to EWS via curl to check the authorization:

curl -X POST   https://mail.mycompany.com/ews/exchange.asmx   -v -H 'cache-control: no-cache' --ntlm -negotiate -u cr'<?xml version="1.0" encoding="utf-8"?>-type: text/xml'   -d '

<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

               xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages"

               xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"

               xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">

  <soap:Header>

    <t:RequestServerVersion Version="Exchange2010" />

  </soap:Header>

  <soap:Body>

   <m:FindFolder Traversal="Deep">

      <m:FolderShape>

        <t:BaseShape>IdOnly</t:BaseShape>

        <t:AdditionalProperties>

          <t:FieldURI FieldURI="folder:DisplayName" />

        </t:AdditionalProperties>

      </m:FolderShape>

      <m:IndexedPageFolderView MaxEntriesReturned="100"

                               Offset="0"

                               BasePoint="Beginning" />

      <m:ParentFolderIds>

        <t:DistinguishedFolderId Id="msgfolderroot">

               <t:Mailbox>

               <t:EmailAddress>user@mycompany.com</t:EmailAddress>

               </t:Mailbox>

        </t:DistinguishedFolderId>

      </m:ParentFolderIds>

    </m:FindFolder>

  </soap:Body>

</soap:Envelope>'

Uninstalling the Microsoft Exchange ConnectorPermanent link for this heading

To uninstall the Microsoft Exchange Connector, first delete all Microsoft Exchange Crawlers and then carry out the following command:

mesextension --interface=plugin --type=archive --file=MicrosoftExchangeConnector<version>.zip uninstall