Mindbreeze GmbH, A-4020 Linz, 2018.
All rights reserved. All hardware and software names are trade names and/or trademarks of their respective owners.
These documents are confidential. The delivery and presentation of these documents alone does not justify any rights whatsoever to our software, our services and service performance results or other protected rights. The disclosure, publication or reproduction is not permitted.
For reasons of easier legibility, gender differentiation has been dispensed with. In terms of equal treatment, appropriate terms apply to both sexes.
Mindbreeze InSpire® can be operated using determined producer and consumer nodes.
One or more servers serve as Producer. Initial indexing and delta indexing are carried out according to the respective valid configuration on these nodes.
In addition, these servers operate all Mindbreeze indices as well as one Mindbreeze Filter Service each. On these servers, the indices are produced (indexed) and they also perform the delta indexing. The Producer nodes are thus pure producers of indices.
The thus generated or renewed indices are automatically distributed by copying to the consumer nodes.
The following are running on the consumer node (also spread out over several producers):
These Consumer servers are responsible for answering search queries and providing client services. In order to ensure an efficient distribution of newly created or updated indices, the use of an elastic index is necessary. This feature enables automatic sizing of indices and allows only the actual changes in the index to be transferred from the producer to the consumer.
An elastic index automatically scales with the amount of indexed objects. Therefore, an index is only limited by the hardware on which the index runs.
If one uses an elastic index, only those files of the index in which the data resides will be adjusted during delta Indexing. This makes it possible to copy only those files that have changed in the distribution of updated indices. The transfer amount and the time needed to update the data between producer and consumer are reduced to a minimum.
Each configuration is generated on the Producer server(s) for all involved nodes. The generated configuration is then distributed to all other servers (Consumer). This concept also ensures the failsafe configuration of Mindbreeze InSpire. For this, load distributors are needed / required.
First, determine the configured service user under /etc/mindbreeze/runtime.conf (normally mes).
Then, make sure that it is possible to copy without interaction via SCP to the Consumer from the Producer.
An SSH PubKey without passphrase must be present on the Producer, and this must be entered in the $HOME/.ssh/authorized_keys of the Consumer (ssh-keygen ssh-copy-id).
Also the known hosts connections are to be made, for example, by manually creating the connection from the Producer via ssh:
su mes (execute on Consumer and Producer)
ssh –vt mes@Producer (execute on Consumer)
ssh –vt mes@Consumer (execute on Producer)
Create RSA Key:
Must be done for Consumer and Producer.
ssh-keygen -t rsa:
Generating public/private rsa key pair.
Enter file in which to save the key (/home/demo/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/demo/.ssh/id_rsa.
Your public key has been saved in /home/demo/.ssh/id_rsa.pub.
The key fingerprint is:
The key's randomart image is:
+--[ RSA 2048]----+
| .oo. |
| . o.E |
| + . o |
| . = = . |
| = S = . |
| o + = + |
| . o + o . |
| . o |
The public key is filed under /home/mes/.ssh/id_rsa.pub. The private key (identification) can be found under /home/mes/.ssh/id_rsa.
su mes (executed on Consumer and Producer)
ssh-copy-id mes@consumervm (executed on Producer)
ssh-copy-id mes@producervm (executed on Consumer)
Create a directory on the consumer node that has enough space for an index, and to which the producer node can copy the indexing deltas.
The default on Linux is /data/incoming, the service user should be the owner and the group:
Create a Producer index in the Manager UI, whereby the following settings are to be applied (under "Advanced Settings"):
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 8443 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 8444 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 23000 –s <producervm/consumervm> -j ACCEPT
-A INPUT -p tcp -m tcp –dport <INDEXPORT> –s <producervm/consumervm> -j ACCEPT
-A INPUT -p tcp -m tcp --dport 5432 –s <producervm/consumervm> -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
The consumer nodes that are checked will be automatically configured as query service, and coupled to the Producer as synchronization goals.
If a synchronization was initiated manually or via script, the index produced will be distributed to all consumer query services for which the "Sync" has been selected.
Make sure in "Query engines" under "Client Services" that the above-configured consumer "Query Services" are checked.
Restart the services on the consumernode:
If there is no index-folder after full start of the services (for instance /data/indices/producer) follow the steps below:
Once the crawling is complete, synchronization via mescontrol can be done manually
mescontrol >:> syncdelta
If everything worked correctly, no output is given.
Filterable properties and regex-matchable properties can be defiend by:
This requires a reinvertion of these properties. The procuder does this automatically after changing the configuration. On the consumer this leads to the following error message:
Failed to start IndexQueryService: Readonly index: Failed to convert index (needsCompleteReinvertion=0,needsDocInfoReinvertion=1). Please convert in readwrite mode
To prevent this please follow these instructions instead of saving and automatically applying the configuration: