Vocabularies for Synonyms and Suggest

Introduction

Vocabularies define preferred and alternative terms for search terms. These can be defined in the Mindbreeze Management Center under "Search Experience" > "Vocabulary".

Behind this is a CSV file in which the vocabularies are defined.

An Example:
SearchTerm;PrefLabel;AltLabel
advanced;help;cheat sheet
easter egg;hidden feature;inside joke

When the VocabularySynonymTransformer plugin is installed, entries are treated as synonyms. The search for „advanced“ is extended to „help“ and „cheat sheet“, whereas the prefLabel is always given more weight than the altLabel. Synonyms are also used reversely; a search for „cheat sheet“ also searches for „advanced“ and „help“. Here is to be noted, that there are no quotes allowed in the synonym-files.

Configuration

Configurations can be performed under the „Indices“ tab in section „Query Transformation Services“ as well as under the „Client Services“ tab in section „Suggest Settings“.

The path to the CSV file can be declared in section „Suggest Settings (Concept CSV).

As an alternative, you can specify in “CSV File Pattern” a pattern to select the CSV-file dynamically. Therefore, “CSV File Path” is interpreted as directory, and all containing CSV-files are loaded at startup. If a search sends a SuggestRequest, you can specify additional options that are being sent. These options, together with the “CSV File Pattern”, select the concrete CSV-File used for this SuggestRequest. E.g. If the “CSV File Pattern” suggests_{{lang}}_{{source}}.csv is set and a SuggestRequest contains the options lang:en, source:default, the concrete CSV-File suggests_en_default.csv will be selected for this request.

The setting has to be enabled:

Next, a new Query Transformation Service must be added (Indices tab).
(Choose dropdown value and klick „Add“)

By clicking the plus sign the lately added service is expanded and the file path to the vocabulary file is displayed.

The plugin can also remove special words like "stop words" or other unwanted words or phrases from the search. To do this, a file with stop phrases must be specified in the "Stop Phrases File Path" option. The file must be a text file with one or more words in each line.

You can find an example for a vocabulary CSV file in .

As an alternative to the CSV-File format, you can use the Google-Search-Appliance (GSA) file format. Choose “Vocabulary File Format“: Google GSA and provide for “Vocabulary File Path“ the vocabulary file paths. You can provide multiple files.

For GSA-Files, you can also provide a base directory and a regular expression. All matching files, that are located directly in the base directory, will be loaded.

If a regular expression is used, new files are not detected automatically. However, if an old GSA-File is modified, the whole List of GSA-Files are reloaded (including possible new Files)

Special treatment of acronyms

Acronyms are abbreviations such as "IT" for "Information Technology". It makes sense to include acronyms as synonyms to improve the search. An acronym consists exclusively of capital letters. In order to improve the quality of the results, acronyms are searched  exactly and case-sensitive. This means, a search for e.g. "IT" will find documents like "Information Technology", but not documents like "When it rains" (because "it" is written in lower case letters).

This applies to both the CSV and GSA file format.

Handling terms with numerical suffix

In special cases, when searching for e.g. event123, you may want to use the first part (event) as the basis for synonym processing. For example, you want to find documents with appointment123 by defining a synonym from event to appointment. To make this possible the option "Compound Split Numeric Suffixes" must be activated. Note: enabling this feature can cause unexpected side effects and should only be used with caution in exceptional cases.

Removing stop words in exact searches

By default, stop words (defined in "Stop Phrases File Path") are removed during exact searches (Quoted Terms). However, if this should not be performed, it can be disabled with the setting "Remove Stop Phrases from Quoted Terms".

In the following situation, disabling this setting can be beneficial. For example, when using the stopword of, an exact search for "Game of Life" will only find the document with Game Life, but not the expected document with Game of Life, because of has been removed and now an exact search for "Game Life" is effectively performed.

Differences between the Plugins Synonym Transformer and the Vocabulary Synonym Transformer

Both plugins transform queries to include synonyms. However there are a few differences:

Synonym Transformer Plugin:

• Uses a single CSV-File
• CSV-Syntax: 2 or more Columns
• All synonyms have the same weight
• Only single terms without whitespace are supported

Vocabulary Synonym Transformer Plugin:

• Can be used with multiple CSV or GSA syntax files
• CSV-Syntax: 3 columns
• Depending on the column the synonyms have less weight
• Support for multi-word-terms

Autocomplete

Allows you to use autocomplete (autosuggestion) in an input field.

Example:

<input data-template="suggest"        

       data-placeholder="search"

       class="mb-query"

       name="query"

       type="search"

       data-aria-label="search"

       autofocus

data-source-id-pattern="database"

data-grouped="true"

data-show-onfocus="true">

Settings

• property: The metadata to be used for the completion. Default: title
• property-constraint: A search term to limit the possible values. Default: empty
• count: The number of suggestions. Default: 4
• source: DOCUMENT_PROPERTY supplies suggestions from metadata, QUERY_TERM from previous searches and CONCEPT from a concept index. For QUERY_TERM and CONCEPT additional settings in the Mindbreeze configuration are necessary. Default value: DOCUMENT_PROPERTY
• service: URL for a  Suggest service. Default value: empty – the same as search
• disabled: Suggestions are disabled. Default: empty
• source-id-pattern:  recent_query provieds suggestions from recent searches
• grouped: Groups the results (true/false) – Default: true
• show-onfocus: Show suggestions when Inputelement is focused (true/false) – Default: false

Settings for suggested search terms can be performed as follows:

Similar Terms

Enable Suggesting of Similar Queries: If checked, suggests are fetched from the App.Telemetry database

ID: An ID might be assigned (e.g. for Styling purpose)

Source Name: The source name might be for example “Popular Searches”

Database JDBC URL: URL to the App.Telemetry database

Max Number of Database Connections: Maximum number of database connections. Default value: 10, minimum value: 2.

Table Name: Table name from the App.Telemetry configuration

Table Column for Query: Selection for count of the results (from the App.Telemetry configuration)

Table Column for Score: Selection for column names of the results (from the App.Telemetry configuration)

Table Column for View ID: Selection for column names of the query (from the App.Telemetry configuration)

Additional WHERE Clause: An aditonal “WHERE” clause can be created and added

Suggest If User Query Is Empty: if “true”, suggestions are displayed, also when the user has not searched yet.

Concept CSV

Enable Suggesting from CSV File: If checked, suggests are fetched from an CSV file

ID: An ID might be assigned (e.g. for Styling purpose)

Source Name: The source name might be for example “Words and Terms”

CSV File Path: The path to the CSV file