Release Notes for Mindbreeze InSpire

Version 23.1

Innovations and new features

Graph search through semantic linking of unstructured content

Mindbreeze InSpire links content via a knowledge graph representation, which can be used for graph search as well as for many other features. Examples include 360-degree views with charts, as well as the ability to use calculated metadata for display or as a filter.

The Mindbreeze InSpire 23.1 release now offers additional options to create relationships between results. Relationships between content can now be created using any metadata value. The added value lies in the possibility to be able to generate links from unstructured content, for example by recognizing entities in the content. The relationships can be established bidirectionally: On the one hand, entities can be assigned to concrete persons, while on the other hand, persons can be related to the entities in the texts.

The figure below shows an example of a concrete use case. When searching for a qualification, in this case "Pipeline Automation", Mindbreeze InSpire extracts both the content that has been entered for the topic and the corresponding internal experts. By linking the people in the company with the author information, the corresponding experts are quickly found. Skills or expertise related to "pipeline automation" result from the context of the metadata of the created documents and therefore do not have to be extracted manually.

Example of a customized Insight App

Link to the documentation

Stop characters with exact search

With this new Mindbreeze InSpire release, you have the ability to see more accurate search results for search queries with stop and special characters. For search terms such as "axis", Mindbreeze InSpire finds terms such as "X-axis" by simply removing punctuation characters, such as the hyphen (-), period (.) or comma (,). However, for a search query with a stop character, such as ".Net Framework", it is beneficial to include the stop character in the search.

With a new configurable functionality, Mindbreeze can detect if stop characters are part of the search term and uses these stop characters to accurately find such occurrences. This is especially relevant when stop signs indicate a difference in meaning. For technical terms, for example, it can make a difference whether something is separated by a hyphen or a comma.

In programming languages such as C, C# or C++, stop characters also have a difference in meaning. In the figure below, you can see such a use case where C# is searched for and only results with the relevant programming language are found.

Example of a customized Insight App

Link to the documentation

New and Advanced Connectors

Web Connector: Extension of the JavaScript Crawler for Interactive Web Content

JavaScript crawling functionality has been extended to index content from web pages with login masks or cookie popups, for example. This makes it possible to script user input for interactive web pages and delay the indexing of web pages until the desired content is loaded. For this purpose, user-defined JavaScript snippets can be stored in the crawler settings, which are applied during the crawling process.

This should allow web content to be indexed, as long as it can be displayed in a web browser. New parameters are now available in the crawler configuration to restrict the use of the scripts only to certain content.

Link to the documentation

Microsoft Graph Connector: Supports Extended Metadata, Documents Can Be Skipped based on metadata

The Microsoft Graph connector now supports the evaluation of on-premises extension attributes from Microsoft Graph. This enables the creation of new Mindbreeze Insight apps that take into account additional custom metadata for search. This feature is enabled by default for existing and new connector installations as of this release and can be configured in the crawler settings.

Furthermore, it is now possible to skip indexing documents during the crawl process if certain document metadata match a configured value. This allows only required documents to be indexed, which can lead to significant performance gains.

Link to the documentation

Yammer Connector: Document References from Sharepoint Online Can Now Be Indexed

Attachments from Yammer stored in Microsoft SharePoint Online can now be made searchable using the Yammer Connector. The attachments are added as a metadata reference to the corresponding Yammer documents. This requires a configured Microsoft SharePoint Online connector which exists on the same index to reference the documents. This feature is necessary to be able to search for attachments from Yammer communications when Yammer is connected to Office 365.

Example of a customized Insight App

Link to the documentation

Microsoft File Connector: Improved Compatibility with Fileshare Protocols

The automatic detection of fileshare protocols improves the compatibility of the Microsoft File connector with your existing infrastructure. Additionally, there are more options in the crawler settings to manually configure the SMB dialect.

Link to the documentation

Technical Advancements

Insight App Designer: New Display Options for Search Windows

The Insight App Designer now supports adding search windows that search over specific metadata. This allows any user metadata to be searched over or further constrained via a filter. In the example below, this enables compact and extensive input of filter parameters to narrow down search results.

Example of a customized Insight App

Link to the documentation

Web Connector: Performance Optimization

The Web Connector now supports intelligent crawl behavior for crawling websites without a sitemap. The crawler evaluates the last modified date of each downloaded document and compares it with the already indexed version. The document is indexed only if a change is detected. This option is enabled by default and enables delta-like crawl behavior even without a sitemap.

Link to the documentation

Security relevant changes

23.1.0.410

• Patched: Lodash CVEs (CVE-2021-23337, CVE-2020-28500, CVE-2020-8203, CVE-2019-1010266, CVE-2019-10744, CVE-2018-16487, CVE-2018-3721)
• Patched: momentJS and DataTableJS vulnerabilities (CVE-2016-4055, CVE-2020-28458)
• Updated: Python certify (CVE-2022-23491)

Additional changes

23.1.0.410

• Updated: Ini4j
• Updated: org.json
• Updated: CoreOS
• Updated: commons-io
• Atlassian Confluence Connector: Added support for Confluence 7.13 and 7.19
• Graph Connector: Added the ability to filter users based on metadata values as well as the indexing of the onPremisesExtensionAttributes
• InceptionImageFilter: Split classified Labels at ","
• Microsoft Azure Principal Resolution Service: Made API Base URL configurable to support Gov Clouds
• Microsoft File Connector: The normalization of ACLs can be disabled to improve crawling performance, if the user principals can be provided in SID format by the principal resolution cache.
• Microsoft File Connector: The SMB2 dialects and the data encryption were made configurable.
• Microsoft SharePoint Online Connector: implemented a new setting that allows for ignoring errors during the Site Discovery.
• Personalization: support for middle- and rightclick
• SharePoint Connector: The recommended settings are set per default for new connectors.
• ServiceNow Principal Cache: added feature to allow disabling resolution of User Criteria for certain content types
• Web Connector: JavaScript crawler enhancements for crawling of interactive web content
• Web Connector: Implemented the Setting "Skip Up-to-date Documents", which can speed up re-crawls of normal websites
• Renamed: SharePoint Online Connector to Microsoft SharePoint Online
• Yammer Connector: can now reference attachments saved in Microsoft SharePoint Online that were indexed by the Microsoft SharePoint Online Connector.
• Added: aria-selected="true" and role="option" in suggest autocomplete
• Added: new data-hide-filter-options to FilteredFacets component
• Added: Support for referencing between documents via string references and inverse string references
• Added: a new css class for matching and non-matching Values in "Recent Searches"
• Added: Highlighting the suggested search list.
• Added: Consideration of stop characters during exact search
• Changed: Plugin Packaging of InceptionImageFilter
• Fix for: (Web Connector) Invalid Document Remover now utilizes the, in the settings configured, User-Agent
• Fix for: The Web Connector now reports the correct amount of indexed items after crawling a large amount of documents
• Fix for: Certain searches produce a error in the JS Console
• Fix for: Not showing last event in app.telemetry when detach/attach is used
• Fix for: Typo, invertion is replaced with inversion
• Fix for: Default app.telemetry columns are missing
• Fix for: gdb has incorrect offsets in core dumps
• Fix for: app.telemetry log definition may not be updated if app.telemetry startup is slow
• Fix for: Principal resolution producer/consumer synchronization problem caused by an inspire node's role change or by creating and importing configuration snapshots.
• Fix for: Sources and tabs can not be saved on the server
• Fix for: Corrupt Snapshots may appear to be imported twice
• Fix for: In some cases the required available space on disk is not checked
• Fix for: Direct Upgrade from very old versions to the new Installer do not work
• Fix for: Custom app.telemetry checks for tasks are broken
• Fix for: Possible index crash when document boostings are used for non-existing dynamic aggregated metadata
• Fix for: Alias terms lead to different results than when searching via element
• Fix for: Duplicate documents, showing deleted documents with query 'ALL AND mes:docid:[A TO B] '
• Fix for: Memory Usage issue in ServiceNow Principal Resolution Service.
• Fix for: SharePoint Online document changes could be lost while the index was unavailable or set to readonly
• Fix for: Potential ConcurrentModificationException in SharePoint Online Principal Resolution Service
• Fix for: Export stops if a query service is not available
• Fix for: java.util.logging log-level is not set correctly, what lead to performance issues
• Fix for Correct Inverter positions for mes:uniformdocid
• Fix for: Relevancy is not optimal when using Aliases
• Fix for: Check consistency with inverted index fails when inverted index is not loaded
• Fix for: Query-Log-Dashboards config changes had no effect
• Fix for: app.telemetry event with ID: ResultEnrichment:getThumbnail has no description
• Improved: usability of principals resolution cache producer/consumer configuration.
• Improved: UI for chart tooltips in QueryLogs Dashboard
• Improved: tools for modifying binary resources that are downloaded through the client service.