Sure, you can handle it. But should you?
Let our experts manage the tech maintenance while you focus on your business.
Let our experts manage the tech maintenance while you focus on your business.
Configuration
Personalized Relevance
Introduction
This plugin can be used to influence the Mindbreeze relevance model (also called document boosting). This includes the search behavior of all users. The basic functionality of this plugin is that documents that have been clicked more often (open/preview action) will be displayed above for future (same) search queries.
The following factors are also considered:
- How many different users have performed an action on the document
- In how many different sessions was an action carried out on the document?
- How often was the same search query executed (relative to all executed search queries)?
Please note that the search behavior of all users is included in the relevance calculation and not that separate relevance calculations take place for each user.
Installation
The plugin is already included in Mindbreeze InSpire and therefore does not need to be installed additionally.
Preconditions
For the plugin to function, personalization must be activated in the Client Service. This logs all user searches and actions in the app.telemetry. This option is active by default. Check this by opening the tab "Client Services" in the menu "Configuration" in the Mindbreeze Management Center, activate the "Advanced Settings" and see under "Personalization Settings" that the option "Enable Personalization" is active.
The records in app.telemetry are stored for a maximum of 6 months by default. Also note that if "Enable Personalization" was not active, no data was recorded and therefore the PersonalizedRelevanceTransformer cannot have any effect on the relevance model.
To use the PersonalisedRelevanceTransformer, the "referer" must also be sent in the search query for every search query. Otherwise, the transformer has no effect on the relevance model.
Configuration
To configure the plugin, open the tab "Indices" in the menu "Configuration" of the Mindbreeze Management Center. Add a new service in the "Services" section by clicking on the "+ Add Service" button. Select the plugin "PersonalizedRelevanceTransformer" from the list "Service".
Depending on whether the QueryExprTransformationService is to be available for all indexes or only for selected indexes, you must proceed as follows:
- For all indices: In the "Query Transformation Services" area, you can select the configured service (<Display Name>@PersonalizedRelevanceTransformer).
- For selected indices: You can select the configured service from the selected index in the "Query Transformation Services" area.
In both cases, click on the "+ Add" button to add the service.
Then make sure that the PersonalizedRelevanceTransformer Service is at the top of the list. To change the order of the service, use the arrow buttons on the right (
).
In the Services area you can now configure the service (activate the "Advanced Settings" to see all configuration options):
"Base Configuration"
Option | Description |
Bind Port | The port on which the service is to run. |
Boosting Data Cache Refresh Interval Seconds | The service uses an internal cache to process search queries efficiently. Depending on how up-to-date the data must be for the relevance calculation, you can specify the interval for renewing the cache in seconds. If "0" is specified, the cache is not built. Instead, the Refresh Servlet under /api/boostings/refresh can be used to start the cache refresh manually. |
Boosting Type (Advanced Setting) | So that the plugin also works with older Mindbreeze InSpire versions, the option "QueryExpr" can be selected. For performance reasons, however, you should always select "FQCategory and Key" if possible. |
Report Heap Dump on out of Memory (Advanced Setting) | If active and an OutOfMemoryError occurs, a heap dump is stored as a .hprof file in the current log directory. |
Default Servlet Page Size (Advanced Setting) | The default page size of the Servelts |
Maximum Number of Boosting Exports (Advanced Setting) | Boostings are automatically exported at each refresh. The last 10 exports are kept by default. How many exports are kept can be configured with this option. |
Application Definitions | To separate the Boostings from different Search Clients, "Application Definitions" can be specified:
A search query uses the boostings of the first application that corresponds to the URL of the search query. By default there is also an "Application Definition" with the name "default", which matches all URLs (.*) - but this is not visible in the configuration. |
"Boost settings"
Option | Description |
Relative Minimum Queries To Boost | The relative number of identical search queries that have been submitted at least previously to boost documents with a particular query. For example, if "mindbreeze" was searched 1000 times and there were 500000 queries in the past, the Relative Queries To Boost for the query "mindbreeze" is 1000/500000 = 0.002 |
Boosting Base | The base of the boosting value (corresponds to the minimum value) |
Boosting Factor | The factor how strong the boosting effect is |
Add Empty Boostings (Advanced Settings) | Also adds boostings that are below the configured "Relative Minimum Queries To Boost" (with boosting factor 1). This option has no effect on the relevance model, but these boostings are displayed in the servlet /api/boostings. |
Action Frequency Weight Exponent | Comma values (with . as decimal separator) between 0 and 1 are valid (0 = is not included in the calculation, 1 = is fully included in the calculation). |
User Frequency Weight Exponent | Effect on boosting how many different users have performed an action on the document. Decimal values between 0 and 1 are valid. |
Session Frequency Weight Exponent | Impact on the boosting of how many different sessions an action was performed on the document. Decimal values between 0 and 1 are valid. |
Query Frequency Weight Exponent | Effect on boosting of how often the same search query (with the same query string) was executed (relative to all executed search queries). Decimal values between 0 and 1 are valid. |
Excluded Users Patterns | Patterns (Java RegEx) for users who should not influence the boosting by their search behavior. Multiple patterns can be specified, separated by line breaks. Can be used, for example, to ensure that the searches of automated or manual tests have no effect on boosting. The patterns are separated with line breaks. This option also affects vote boostings. |
Vote Boost Settings
Boost Votes | If enabled, also up- and down-voted results are considered in the relevance model. The relevance model is influenced independently of the search term. |
Vote Dampening Up To | The number of votes needed until the maximum boosting factor is reached (maximum is 0 for down-votes, 2 for up-votes) |
Max Vote Boosting Factor Deviation | e.g. 2 will lead to boostings between 1-2=-1 (for down-votes) and 1+2=3 (for up-votes)) |
"DB Connection Settings"
Option | Description |
JDBC URL | The JDBC URL to the app.telemetry database |
Custom Credential | The login information to the app.telemetry database. This field can be left blank ("None"). However, if you have changed the default "JDBC URL" (jdbc:postgresql://localhost:5432/telemetrydb), you must configure the credentials explicitly. In the tab "Network" under "Credentials" add new credentials with the type "Username/Password". The default credentials can be found in Initial Startup for G7 Appliances - Reporting. If you are using a G6 appliance, you will find the default credentials in Initial Startup for G6 appliances (before January 2018) - Reporting. Please also note that in this case you have to adjust the option "Table Name" in the section "DB Schema" (Advanced Setting). |
Maximum Pooled Connections | Maximum number of database connections held in the connection pool. |
"DB Schema (Advanced Setting)"
These options do not normally need to be adjusted.
Option | Description |
Table Name | Default: clientservicequerylogmesclientservicequerylog If you have a G6 appliance in use, you must adjust this value. In the Mindbreeze Management Center, first go to Telemetry Details Configuration Log Pools Client Service Query Log and copy the "Database Table Prefix". In the Personalized Relevance Transformer configuration, set the Table Name option to “<Database Table Prefx>mesclientservicequerylog” (without quotation marks), e.g. “client_service_query_logmesclientservicequerylog”. |
Query Id Column | Default: log:query_id |
User Query Column | Default: query:user_query |
Search User Column | Default: log:username |
Action Column | Default: queryAction:action |
Key Column | Default: queryResultAction:key |
Weight Column | Default: queryResultAction:weight |
FQCategory Column | Default: queryResultAction:fqcategory |
Session ID Column | Default: log:session_id |
Referer Column | Default: log:referer |
Position Column | Default: queryResultAction:position |
Total Count Column | Only relevant for Reporting Query Logs |
Start Time Column | Only relevant for Reporting Query Logs |
Duration Column | Only relevant for Reporting Query Logs |
Operation Column | Only relevant for Reporting Query Logs |
Scrollposition in Percentage Column | Only relevant for Reporting Query Logs |
Duration in ms Column | Only relevant for Reporting Query Logs |
Appendix
Analysis Servlets
The following servlets are available. These can be retrieved via https://<appliance-hostname>:8443/index/<transformer-bind-port>/<path> (e.g. https://myappliance:8443/index/8989/api/boostings?application=default).
GET (HTML servlets via browser)
Path | Description |
/api or | HTML page that provides links to the other servlets. |
/api/boostings | Returns an HTML table containing all effective cached boostings (sorted by query, application, boosting). The following HTTP query parameters can be specified:
|
/api/boostings/refresh | Renews the boosting cache. The cache is automatically renewed periodically, depending on what is configured in the "Boosting Data Cache Refresh Interval Seconds" option. The following HTTP query parameters can be specified to overwrite config options for the cache refresh:
|
/api/queries | Returns an HTML table showing which queries were submitted how often (sorted by number, descending). The following HTTP query parameters can be specified:
|
/api/actions | Returns an HTML table that specifies how many user actions were performed for each document (a maximum of one action per document is counted per query ID). In addition, statistics on the position of the documents in the search results and on the boostings are provided. A link to /api/boosting will be provided to get the detailed boostings and queries. The following HTTP query parameters can be specified:
|
/api/referers | Returns a HTML table containing all referers (URLs from which a search was performed), to which application definition they belong and how often a search was performed. The following HTTP query parameters can be specified:
|
/api/voteboostings | Returns an HTML table containing all effective cached vote boostings (sorted by boosting, fqcategory, application). The following HTTP query parameters can be specified:
fqcategory: restriction to the metadata fqcategory of documents |
POST
/api/boostings/export | Boostings can be exported as a binary file and then re-imported using the Import servlet. This servlet can be used for backups or producer-consumer setups. Exactly one of the following HTTP query parameters must be specified:
|
/api/boostings/import | The boosting cache stored with the export servlet can be reimported using this servlet. Each time you import or refresh, an additional export is automatically stored in /data/currentservices/launchedservice-<service-name>/data/export/<uuid>, which can be re-imported using the "uuid" URL parameter. Exactly one of the following HTTP query parameters must be specified:
|
api/boostings/clear | Boosting and servlet caches can be deleted. GET /api/boostings/refresh can be used to rebuild these caches. |
Enabling and Disabling Personalization in the Search Client
On your Search Client you can deactivate the PersonalizedRelevanceTransformer for your searches under "Settings" -> "Search" with the option "Disable personalized search relevance". Note that this setting is not saved and will be lost if the page is reloaded.
Relevance Calculation in the Search Client
To check how strongly the boosting of personalized relevance has an effect, the URL query parameter "relevance-info=true" can be used in the Search Client. The boosting of the plugin flows into the "Document Boost (%)" column.