Home
Home
German Version
Support
Impressum
20.5 Release ►

    Main Navigation

    • Preparation
      • Connectors
      • Initial Startup for G6 appliances (before January 2018)
      • Initial Startup for G7 appliances
      • Setup InSpire G7 primary and Standby Appliances
    • Datasources
      • Configuration - Atlassian Confluence Connector
      • Configuration - Best Bets Connector
      • Configuration - Data Integration Connector
      • Configuration - Documentum Connector
      • Configuration - Dropbox Connector
      • Configuration - Egnyte Connector
      • Configuration - GitHub Connector
      • Configuration - Google Drive Connector
      • Configuration - GSA Adapter Service
      • Configuration - HL7 Connector
      • Configuration - IBM Connections Connector
      • Configuration - IBM Lotus Connector
      • Configuration - Jira Connector
      • Configuration - JiveSoftware Jive Connector
      • Configuration - JVM Launcher Service
      • Configuration - LDAP Connector
      • Configuration - Microsoft Dynamics CRM Connector
      • Configuration - Microsoft Exchange Connector
      • Configuration - Microsoft File Connector (Legacy)
      • Configuration - Microsoft File Connector
      • Configuration - Microsoft SharePoint Connector
      • Configuration - Salesforce Connector
      • Configuration - SAP KMC Connector
      • Configuration - SemanticWeb Connector
      • Configuration - ServiceNow Connector
      • Configuration - SharePoint Online Connector
      • Configuration - Web Connector
      • Configuration - Yammer Connector
      • Data Integration Guide with SQL Database by Example
      • Indexing user-specific properties (Documentum)
      • Installation & Configuration - Atlassian Confluence Sitemap Generator Add-On
      • Installation & Configuration - Caching Principal Resolution Service
      • Installation & Configuration - Jive Sitemap Generator
      • Mindbreeze InSpire Insight Apps in Microsoft SharePoint Online
      • Mindbreeze InSpire Search Apps in Microsoft SharePoint 2010
      • Mindbreeze InSpire Search Apps in Microsoft SharePoint 2013
      • Mindbreeze Web Parts for Microsoft SharePoint
      • User Defined Properties (SharePoint 2013 Connector)
    • Configuration
      • CAS_Authentication
      • Cognito JWT Authentification
      • Configuration - Alternative Search Suggestions and Automatic Search Expansion
      • Configuration - Back-End Credentials
      • Configuration - Chinese Tokenization Plugin (Jieba)
      • Configuration - CJK Tokenizer Plugin
      • Configuration - Collected Results
      • Configuration - CSV Metadata Mapping Item Transformation Service
      • Configuration - Entity Recognition
      • Configuration - Exporting Results
      • Configuration - GSA Late Binding Authentication
      • Configuration - Index-Servlets
      • Configuration - Item Property Generator
      • Configuration - Japanese Language Tokenizer
      • Configuration - Kerberos Authentication
      • Configuration - Management Center Menu
      • Configuration - Metadata Enrichment
      • Configuration - Metadata Reference Builder Plugin
      • Configuration - Notifications
      • Configuration - Personalized Relevance
      • Configuration - Plugin Installation
      • Configuration - Principal Validation Plugin
      • Configuration - Profile
      • Configuration - Reporting Query Logs
      • Configuration - Reporting Query Performance Tests
      • Configuration - Request Header Session Authentication
      • Configuration - Vocabularies for Synonyms and Suggest
      • Configuration of Thumbnail Images
      • Cookie-Authentication
      • Documentation - Mindbreeze InSpire
      • Google Search Appliance Migration to Mindbreeze InSpire
      • I18n Item Transformation
      • Installation & Configuration - Outlook Add-In
      • Installation - GSA Base Configuration Package
      • Language detection - LanguageDetector Plugin
      • Mindbreeze Personalization
      • Mindbreeze Prediction Service Text Classification
      • Mindbreeze Property Expression Language
      • Mindbreeze Query Expression Transformation
      • Non-Inverted Metadata Item Transformer
      • SAML-based Authentication
      • Trusted Peer Authentication for Mindbreeze InSpire
      • Using the InSpire Snapshot for Development in a CI_CD Scenario
    • Operations
      • app.telemetry Statistics Regarding Search Queries
      • Configuration Usage Analysis
      • Deletion of Hard Disks
      • Handbook - Backup & Restore
      • Handbook - Command Line Tools
      • Handbook - Distributed Operation (G7)
      • Handbook - Filemanager
      • Handbook - Synchronized Operation (G6)
      • Index Operating Concepts
      • Indexing and Search Logs
      • Inspire Diagnostics and Resource Monitoring
      • InSpire Support Documentation
      • Mindbreeze InSpire SFX Update
      • Provision of app.telemetry Information on G7 Appliances via SNMPv3
      • Restoring to As-Delivered Condition
    • User Manual
      • Cheat Sheet
      • iOS App
      • Keyboard Operation
    • SDK
      • api.v2.alertstrigger Interface Description
      • api.v2.export Interface Description
      • api.v2.personalization Interface Description
      • api.v2.search Interface Description
      • api.v2.suggest Interface Description
      • api.v3.admin.SnapshotService Interface Description
      • Debugging (Eclipse)
      • Developing an API V2 search request response transformer
      • Developing Item Transformation and Post Filter Plugins with the Mindbreeze SDK
      • Development of Insight Apps
      • Java API Interface Description
    • Release Notes
      • Release Notes 20.1 Release - Mindbreeze InSpire
      • Release Notes 20.2 Release - Mindbreeze InSpire
      • Release Notes 20.3 Release - Mindbreeze InSpire
      • Release Notes 20.4 Release - Mindbreeze InSpire
      • Release Notes 20.5 Release - Mindbreeze InSpire
      • Release Notes 2018 Spring - Mindbreeze InSpire
      • Release Notes 2018 Winter - Mindbreeze InSpire
      • Release Notes 2019 Fall - Mindbreeze InSpire
      • Release Notes 2019 Winter - Mindbreeze InSpire
    • Security
      • Known Vulnerablities
    • Product Information
      • Product Information - Mindbreeze InSpire - Standby
      • Product Information - Mindbreeze InSpire
    Home

    Path

    Indexing and Search Logs

    Copyright ©

    Mindbreeze GmbH, A-4020 Linz, 2020.

    All rights reserved. All hardware and software names are brand names and/or trademarks of their respective manufacturers.

    These documents are strictly confidential. The submission and presentation of these documents does not confer any rights to our software, our services and service outcomes or other protected rights.

    The dissemination, publication or reproduction hereof is prohibited.

    For ease of readability, gender differentiation has been waived. Corresponding terms and definitions apply within the meaning and intent of the equal treatment principle for both sexes.

    .

    Log filesPermanent link for this heading

    The directory for the Mindbreeze enterprise search services log files can be configured in "Advanced Settings".

    It is entered by default under Linux “/var/opt/mindbreeze/log“ and under Windows “c:/Users/<service user>/App Data/Local/Temp“. In the following chapters, this directory is referred to as "Log Directory".

    Client service log filesPermanent link for this heading

    The client service log files are located in the directory "Log Directory/log-mesclientservice-<client-port>", where a subdirectory with the corresponding time stamp is created for each restart.

    Query service log filesPermanent link for this heading

    • The query service log files are located under "Log Directory/log-mesindex<index path>" with subdirectories for each restart. Relevant query service log files are:
    • log-mesindex.log
    • performance-findservlet.csv  

    The following columns of the "performance-findservlet.csv" file have an essential role in the analysis of search queries:

    • Starting Timestamp: Start time
    • Query: Contains the search term
    • Authorized Hit Count: The number of authorized documents
    • Rejected Hit Count: The number of unauthorized documents

    The authorization and contextualization as part of the query service will be performed in separate processes. The log files are located under "Log Directory/log-messdkjavasandbox-bootstrapped-<query-port>", "Log Directory/log-messdkcsandbox-<query-port>" or in the file "Log Directory/log-messdknetsandbox-<query-port>.txt", depending on the implementation (Java, C or .NET).

    Crawler log filesPermanent link for this heading

    The crawler logs are located in the "Log Directory/log-mescrawler-launchedservice-<Category>_<Source Name>" directory.

    Microsoft Exchange 2010Permanent link for this heading

    log-mescrawler_launchedservice_Microsoft_Exchange_<Source Name>.log includes all information such as: start time, mailboxes found (AD), crawler status fetch, time when a mailbox search is completed, and end time with the collected statistics on the total number of documents found, indexed, and deleted.

    Additionally, CSV log files are generated. The names of the CSV log files correspond to the associated "crawl run" or "scope". The following files have a significant role in the analysis of delta indexing:

    per-folder-finditems-<crawlrun>_<scope>.csv: Contains the information of all found folders of the mailboxes that were created or modified within the time frame (scope). The main columns are:

    • Timestamp: Start time
    • Mailbox: E-mail address
    • Path: Path of the folder <mailbox>/ordner/unterordner/…
    • PerFolderFoundItemsSize: Number of found items that are newly created or modified.

    mindbreeze-dispatcher-<crawlrun>_<scope>.csv: All documents or e-mails that have been sent to the filter service. The main columns are:

    • Timestamp: Start time
    • Key:
    • Mailbox: E-mail address
    • Folder: The folder in which the e-mail is located

    FilesystemPermanent link for this heading

    "Log Directory/log-mesfscrawler_<indexpath>.log" contains all information that is generated by the filesystem crawler.

    Filter service log filesPermanent link for this heading

    The filter service log files are located in the directory "Log Directory/log-mesfilterservice-<port>." The statistic-filter-rpc.csv file provides information, for example about the filter plugin used, file extensions and possible errors in filtering. The main columns are:

    • Timestamp: Start time
    • Extension: file extension
    • Used Filter ID: Filter plugin
    • Internal Exception: possible errors

    The actual filtering happens in separate processes. Depending on the implementation of the filter plugins (Java, C or .NET), the "Log Directory/log-messdkjavasandbox2-Filter@<port>", "Log Directory/log-messdkcsandbox2-Filter@<port>" directories or the "Log Directory/log-messdknetsandbox-<port>.txt" file will be generated.

    Index service log filesPermanent link for this heading

    The index service log files are located in the "Log Directory/log-mesindex<index path>“ directory. All information concerning the state of the indexing (inverting, merging and announcements) is located in the file log-mesindex.log.

    performance-servlet-mes3_indexeddocuments.csv provides information about the number of documents that have been obtained by the index service.

    performance-servlet-mes3_deletedocuments.csv provides information about the number of documents that have been deleted from the index.

    performance-servlet-mes3_updatedocuments.csv provides information about the number of documents that have been updated in the index.

    System log files (G7)Permanent link for this heading

    Mindbreeze InSpire appliances collect logs from sources including the following:

    System logs of the host system

    Logs for the docker container

    Logs of the reverse proxy and the Keycloak server

    Logs of Mindbreeze processes

    These are automatically forwarded to the syslog server in the InSpire container and to app.telemetry.

    Distribution to log filesPermanent link for this heading

    /var/log/messagescontains messages from Mindbreeze services as well as start messages from the host and the containers

    /var/log/reverse-proxycontains messages from the reverse proxy server

    /var/log/croncontains messages from the cron jobs

    /var/log/securecontains audit and authentication messages (such as login, logout)

    Filtered log pools in app.telemetryPermanent link for this heading

    In app.telemetry, the log messages are visible in the log pool: “Syslog Events.”
    Additionally, there are the following “filtered log pools”:

    Syslog Audit Events: is equivalent to /var/log/secure

    Syslog Reverse Proxy Events: is equivalent to /var/log/reverse-proxy

    Use Case: retrieve audit logsPermanent link for this heading

    The audit logs (/var/log/secure) can be retrieved using the file manager:

    Forwarding to an external syslog serverPermanent link for this heading

    It is also possible to forward the above logs to a syslog server.
    To do this, the server must be entered in the file “/config/rsyslog/remote.conf” as follows:

    *.*@<hostname oder ip>:<port> (uses UDP)

    or

    *.*@@<hostname oder ip>:<port> (uses TCP)

    This forwards all log messages. To configure the forwarding more precisely, please use the documentation for rsyslog.

    Encrypted transport of syslog messages is also possible if it is supported by the target server. For this you have to configure the correct certificates. E.g:

    $DefaultNetstreamDriverCAFile /config/rsyslog/ca-cert.cer

    $DefaultNetstreamDriverCertFile /config/rsyslog/client-cert.cer

    $DefaultNetstreamDriverKeyFile /config/rsyslog/cliet-cert.key

    $DefaultNetStreamDriver gtls

    $ActionSendStreamDriverMode 1

    $ActionSendStreamDriverAuthMode anon

    For configuration details please also refer to rsyslog.

    Thread dumpsPermanent link for this heading

    Thread dumps can be created to obtain information about the current state of a process.

    The process ID (PID) is required for all types of processes. It is located in the log directory of the process (xxxxx.pid) and xxxxx corresponds to the PID.

    Thread dumps of Java processesPermanent link for this heading

    To create a thread dump of Java processes, the “QUIT” signal must be sent to the respective process.

    kill-QUIT <PID>

    The thread dump is then located in the log file of the process.

    Thread dumps of native processesPermanent link for this heading

    To create a thread dump of a native process (e.g. mesindex) you can use the auxiliary script “thread-dump-native.sh.”

    /opt/mindbreeze/scripts/thread-dump-native.sh <PID>

    The thread dump is output directly, but can be redirected to a file.

    Download PDF

    • Indexing and Search Logs

    Content

    • Log files
    • System log files (G7)
    • Thread dumps

    Download PDF

    • Indexing and Search Logs