Mindbreeze GmbH, A-4020 Linz, 2020.
All rights reserved. All hardware and software names are trade names and/or trademarks of their respective owners.
These documents are confidential. The delivery and presentation of these documents alone does not justify any rights whatsoever to our software, our services and service performance results or other protected rights. The disclosure, publication or reproduction is not permitted.
For reasons of easier legibility, gender differentiation has been dispensed with. In terms of equal treatment, appropriate terms apply to both sexes.
Mindbreeze InSpire® can be operated using determined producer and consumer nodes.
One or more servers serve as Producer. Initial indexing and delta indexing are carried out according to the respective valid configuration on these nodes.
In addition, these servers operate all Mindbreeze indices as well as one Mindbreeze Filter Service each. On these servers, the indices are produced (indexed) and they also perform the delta indexing. The Producer nodes are thus pure producers of indices.
The thus generated or renewed indices are automatically distributed by copying to the consumer nodes.
The following are running on the consumer node (also spread out over several producers):
These Consumer servers are responsible for answering search queries and providing client services. In order to ensure an efficient distribution of newly created or updated indices, the use of an elastic index is necessary. This feature enables automatic sizing of indices and allows only the actual changes in the index to be transferred from the producer to the consumer.
An elastic index automatically scales with the amount of indexed objects. Therefore, an index is only limited by the hardware on which the index runs.
If one uses an elastic index, only those files of the index in which the data resides will be adjusted during delta Indexing. This makes it possible to copy only those files that have changed in the distribution of updated indices. The transfer amount and the time needed to update the data between producer and consumer are reduced to a minimum.
Each configuration is generated on the Producer server(s) for all involved nodes. The generated configuration is then distributed to all other servers (Consumer). This concept also ensures the failsafe configuration of Mindbreeze InSpire. For this, load distributors are needed / required.
First, determine the configured service user under /etc/mindbreeze/runtime.conf (normally mes).
Then, make sure that it is possible to copy without interaction via SCP to the Consumer from the Producer.
An SSH PubKey without passphrase must be present on the Producer, and this must be entered in the $HOME/.ssh/authorized_keys of the Consumer (ssh-keygen ssh-copy-id).
Also the known hosts connections are to be made, for example, by manually creating the connection from the Producer via ssh:
su mes (execute on Consumer and Producer)
ssh –vt mes@Producer (execute on Consumer)
ssh –vt mes@Consumer (execute on Producer)
Create RSA Key:
Must be done for Consumer and Producer.
ssh-keygen -t rsa:
Generating public/private rsa key pair.
Enter file in which to save the key (/home/demo/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/demo/.ssh/id_rsa.
Your public key has been saved in /home/demo/.ssh/id_rsa.pub.
The key fingerprint is:
The key's randomart image is:
+--[ RSA 2048]----+
| .oo. |
| . o.E |
| + . o |
| . = = . |
| = S = . |
| o + = + |
| . o + o . |
| . o |
The public key is filed under /home/mes/.ssh/id_rsa.pub. The private key (identification) can be found under /home/mes/.ssh/id_rsa.
su mes (executed on Consumer and Producer)
ssh-copy-id mes@consumervm (executed on Producer)
ssh-copy-id mes@producervm (executed on Consumer)
Create a directory on the consumer node that has enough space for an index, and to which the producer node can copy the indexing deltas.
The default on Linux is /data/incoming, the service user should be the owner and the group:
Create a Producer index in the Manager UI, whereby the following settings are to be applied (under "Advanced Settings"):
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 8443 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 8444 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 23000 –s <producervm/consumervm> -j ACCEPT
-A INPUT -p tcp -m tcp –dport <INDEXPORT> –s <producervm/consumervm> -j ACCEPT
-A INPUT -p tcp -m tcp --dport 5432 –s <producervm/consumervm> -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
The consumer nodes that are checked will be automatically configured as query service, and coupled to the Producer as synchronization goals.
If a synchronization was initiated manually or via script, the index produced will be distributed to all consumer query services for which the "Sync" has been selected.
Make sure in "Query engines" under "Client Services" that the above-configured consumer "Query Services" are checked.
Until now only the configuration of the producer has changed. For the consumer to apply the same configuration it has to be synced. Run the following command on the producer:
It is advised to set up a Cron-job on the producer to do this in an regular interval.
Additional data is synchronized as well e.g: Kerberos configuration an all files from /data/resources.
Once config sync and the crawling is complete, synchronization via mescontrol can be done manually.
The prefered method is Sync Delta:
mescontrol >:> syncdelta
Check the index logs to verify that the synchronization worked (e.g: /data/logs/log-mesindex_<myindex>/current/log-mesindex.log).
If filterable or regexmatchable properties are changed using
Then it is neccesary that these properties are rebuild. The producer does this automatically after the config change. In this case Sync Delta would not work anymore, please use Sync Full instead:
mescontrol >:> syncfull
The index also synchronizes the index configuration to the consumer. This ensures that the configuration matches the index.
If the index has to be re-synchronized by schema changes, we require the multi-index layout for the consumer. If the synchronization is complete, the index remains in the consumer until the index is completely synchronized.