Mindbreeze Property Expression Language
Copyright ©
Mindbreeze GmbH, A-4020 Linz, 2023.
All rights reserved. All hardware and software names are brand names and/or trademarks of their respective manufacturers.
These documents are strictly confidential. The submission and presentation of these documents does not confer any rights to our software, our services and service outcomes or other protected rights.
The dissemination, publication or reproduction hereof is prohibited.
For ease of readability, gender differentiation has been waived. Corresponding terms and definitions apply within the meaning and intent of the equal treatment principle for both sexes.
Introduction
The property expression language can be used to extract metadata, or to create filter values that can then be limited in the search.
Language elements
Literals
Constant literal values such as numbers or strings can be specified.
Example: | |
String | "text" |
Integer | 3 |
Boolean value | true |
Identifiers
An identifier always refers to the current context.
This context can be a document, but also an item.
If a reference is tracked, this refers to the referenced document.
Example: | |
Regular metadata of a document | title |
Reference metadata of a document | reference |
Property within an item | itemProperty |
Note that the following character class is valid without escaping: [A-Za-z0-9_:\-]
The following characters in an identifier do not have to be escaped:
- Capital letters (A-Z)
- Lower case letters (a-z)
- The following special characters: "_", ":", "-"
All other characters must be escaped with a backslash (\) so that the character string is recognized as an identifier. For example, property.value must be escaped as follows: property\.value
Lists
A list of several expressions, that are all emitted.
Example: | |
Expression | What is emitted … |
list("text1", "text2", "text3") | "text1", "text2", "text3" |
Removing Duplicates
The unique operator removes duplicates from its parameters while maintaining the order. It can be applied to all expressions that have a string representation.
Examples: | |
Expression | What is emitted … |
unique("a", "b", "c", "b") | "a", "b", "c" |
unique("a", "a", "a") | "a" |
Paths
If metadata of the referenced document should be accessed, the path operator “/” can be used.
Example: | ||
Variant 1 | reference/title | If the current document has a reference property reference, the title of the referenced document is emitted |
Variant 2 | reference1/reference2/title | If the current document has a reference property reference1, and the referenced document in turn has a reference property reference2 that refers to another document, that title is emitted |
Inverse References
The rev operator can be used to follow inverse references. To do this, you need to configure the reference property as Inverted Reference Metatdata Keys in the Reference Settings of the index. Also see Configuration Mindbreeze InSpire – “Indices” Tab (section: Reference Settings).
Example: | |
rev(parent)/mes:key | All mes:key values of the inverted reference parent are emitted. |
String References
In order to follow string references, the lookup operator can be used. To do this, you need to configure the reference property as Inverted Hash Reference Metadata Keys and Hash Reference Target Metadata Keys in the Reference Settings of the index, also see Configuration Mindbreeze InSpire – “Indices” Tab (section: Reference Settings).
Example: | |
lookup(author_email)/name | If the current document has an author_email property, the name of the referenced person is emitted. |
lookup(creator_email,last_edited_email)/name | If the current document has a creator_email or last_edited_email property, the names of the referenced persons is emitted. Note: 1 or more parameters must be specified for lookup. |
The following limitations apply when using the lookup operator:
- The lookup operator only supports string metadata. Built-in metadata is not supported. A list of all built-in metadata can be found in Index Operating Concepts - Built-In Metadata Keys .
- Regarding references, the lookup operator cannot support cross-index references. Therefore, the references must be within an index.
It is also recommended to limit the number of different values of Hash Reference Target Metadata Keys to avoid ambiguities and to ensure reasonable performance. Such a limitation can be implemented using "Precomputed Synthesized Metadata". This is demonstrated below using an example of a use case:
Example of a Use Case: Synthesise the name of the author of a document. | |
In the following example, documents and people are in the index. The documents already have an author_email metadata with the author's email address. The goal of this example is to synthesise a metadata author_fullname for all documents by creating references to the persons (authors). Document 1 with the following metadata: key: "doc1", datasource/category: "DataIntegration", author_email: "max.mustermann@mindbreeze.com" Person 1 with the following metadata: key: "person1", datasource/category: "Microsoft Graph", fullname: "Max Mustermann", email: "max.mustermann@mindbreeze.com" The result should be a metadata author_fullname: "Max Mustermann" on document 1. This requires the following configuration steps in the index configuration: | |
Precomputed Synthesized Metadata | |
Name | target_email |
Property Expression | regex_match(datasource\/category, "Microsoft Graph") && email |
Reference Settings | |
Inverted Hash Reference Metadata Keys | author_email |
Hash Reference Target Metadata Keys | target_email |
Synthesized Metadata | |
Name | author_fullname |
Property Expression | lookup(author_email)/fullname |
Result | |
Document 1 with the following metadata: key: "doc1", datasource/category: "DataIntegration", author_email: "max.mustermann@mindbreeze.com", author_fullname: "Max Mustermann" Person 1 with the following metadata: key: "person1", datasource/category: "Microsoft Graph", fullname: "Max Mustermann", email: "max.mustermann@mindbreeze.com", target_email: "max.mustermann@mindbreeze.com" | |
For more information, see Configuration - Metadata Enrichment - Precomputed Synthesized Metadata .
Inverse String References
In order to follow inverse string references, the rev_lookup operator can be used in combination with the path operator “/”. To do this, you need to configure the reference property as Inverted Hash Reference Metadata Keys and Hash Reference Target Metadata Keys in the Reference Settings of the index, also see Configuration Mindbreeze InSpire – “Indices” Tab (section: Reference Settings).
Example: | |
rev_lookup(email,author_email)/title | All title values of the inverted string reference email are emitted. Example use case: Documents reference persons (document metadata author_email references a person with metadata email). The property expression rev_lookup(email,author_email)/title returns the titles of all documents written by the referenced person. |
rev_lookup(list(email,email_alias), | All title values of the inverted string references email and email_alias are emitted. Example use case: Documents reference people (document metadata creator_email and last_edited_email reference persons with metadata email and email_alias). The Property Expression (see left column) returns the titles of all documents created or last edited by the referenced person. Note: 2 or more parameters must be specified for rev_lookup. The first parameter of rev_lookup is the target metadate the reference points to. Multiple properties can be specified here using list. All other parameters represent the properties from which the references originate. |
Transitive Closure
The dfs operator can be used to emit all items starting from an expression that are reachable via an expression in depth first search (DFS) traversal. dfs takes two or three arguments:
Examples: | |
dfs(.,parent)/mes:key | All mes:key values starting at the current context of all items reachable via the parent expression. |
dfs(.,rev(parent))/mes:key | All mes:key values starting at the current context of all items reachable via the rev(parent) expression. |
dfs(.,parent,"terminal")/mes:key | mes:key value of the last (terminal) item that is reachable via the parent expression from the current context. If the graph has a tree form, this expression yields the mes:key property of the root of the tree. |
dfs(.,rev(parent),"terminal")/mes:key | mes:key value of the last (terminal) item that is reachable via the rev(parent) expression from the current context. If the graph has a tree form, this expression emits all the mes:key properties of leaves reachable from the current context. |
Filters
The filter operator can be used to define additional constraints which must apply (normally for items) for selected expressions.
Example: | ||
Variant 1 | title[../mes:numericdate < 1444381468000] | The title of the current document will only be emitted if the date is prior to 2015-10-09T09:04:28.000Z (as milliseconds since 1970) |
Comparison operators
The following comparison operators may be used:
either in combination with logic operations
Examples in combination with logical operators: | ||
Expression | What is emitted … | |
Equality | 3 = 3 && "true" || "false" | "true" |
Inequality | 3 != 3 && "true" || "false" | "false" |
Less than | 3 < 3 && "true" || "false" | "false" |
Less than or equal to | 3 <= 3 && "true" || "false" | "true" |
Greater than | 3 > 3 && "true" || "false" | "false" |
Greater than or equal to | 3 >= 3 && "true" || "false" | "true" |
Examples in combination with filter(s): | ||
Expression | What is emitted … | |
Equality | title[3 = 3] | Title of the current document |
Inequality | title[3 != 3] | Nothing |
Less than | title[3 < 3] | Nothing |
Less than or equal to | title[3 <= 3] | Title of the current document |
Greater | title[3 > 3] | Nothing |
Greater than or equal to | title[3 >= 3] | Title of the current document |
Logical Operators
AND operator &&
In the AND operation, only the last item in the &&-chain is emitted, if and only when all previous elements have emitted something and their value is not interpreted as false.
Example: | ||
Expression | What is emitted … | |
Literal 1 | "text" && "text2" && "text3" | "text3" |
Boolean expressions | 0 && "text2" | nothing |
1=1 && "true" || "false" | "true" | |
1!=1 && "true" || "false" | "false" |
OR operator ||
In the OR operator, the first element in the ||-chain that emits something is emitted. In combined expressions containing both && and ||, && takes precedence over the ||.
Example: | ||
Expression | What is emitted … | |
Literal 1 | "text1" || "text2" || "text3" | "text1" |
Boolean expressions | 0 || "text2" | "text2" |
0=0 && "true" || "text2" | "true" |
Arithmetic operators
The following arithmetic operators can be used:
Expressions | What is emitted … | |
Addition | 3 + 2 3.2 + 5 | 5 8.2 |
Subtraction | 3 – 2 3.2 – 5 | 1 -2.2 |
Multiplication | 3 * 2 3.2 * 5 | 6 16.0 |
Division | 3 div 2 3.2 div 5 | 1 0.64 |
Modulo | mod(3, 2) | 1 |
Operator precedence is supported for compound expressions, so multiplication and division have higher precedence than addition and subtraction. The precedence of partial expressions can be changed by using parentheses:
Expressions | What is emitted … | |
Precedence | 3 + 1 * 2 – 1 (3 + 1) * 2 – 1 (3 + 1) * (2 – 1) | 4 7 4 |
Parent context
The Parent-operator .. can be used to access the parent context.
This operator is particularly useful in combination with the filter operator, see Inverse References.
Example: | ||
Variant 1 | reference/title/.. | Corresponds to reference |
Variant 2 | reference/title/../.. | Corresponds to the evaluated document itself |
Name
The function name can be used to deduce the name of the metadata item or the item property from a value context.
Example: | ||
Variant 1 | name(regex_match(title, "^T.*$")) | Title emitted, if the title of the current document matches the regular expression |
Strings
Regex match
The function regex_match can be used to restrict to those strings which correspond to a regular expression.
Example: | ||
Variant 1 | regex_match(title, "^T.*$") | Titles that begin with "T" are emitted (implicitly case sensitive) |
Variant 2 | regex_match(title, "^T.*$", "i") | Titles that begin with "T" are emitted (explicitly case insensitive) |
Conversions to upper/lower case
With the functions to_lower and to_upper, expressions can be converted to uppercase or lowercase.
Example: | ||
to_lower("UPPER") | "upper" is emitted | |
to_upper("lower") | "LOWER" is emitted |
String contexts (concat)
The concat function concatenates multiple values from metadata.
Example: | |
concat("Hauptplatz", " ", "123") | "Hauptplatz 123" is emitted |
Character replacement
With the replace function, characters (character strings) can be replaced; when doing this, regular expressions can be used.
Example: | ||
Removing letters | replace("12ab34cd56", "[a-z]", "") | "123456" is emitted |
replace("0123456789","[0-9]{9}$","") | "0" is emitted |
Type conversions
Conversion to an Integer Value
With the function int, a string or a date value can be converted to an integer.
Example: | ||
Expression | What is emitted: | |
Variant 1 | int("3") | Number 3 |
Variant 2 | int(dateproperty) | Date value of the property "dateproperty" as milliseconds since 1970 |
Variant 3 | int(now()) | Current time as milliseconds since 1970 |
Conversion to a Floating Point Value
The float function can be used to convert a string or date value to a floating point number.
Example: | ||
Expression | What is emitted: | |
Variant 1 | float("3.124") | Number 3.124 |
Variant 2 | float(10) | 10 as floating point number |
Variant 3 | float(now()) | Current time as milliseconds since 1970 |
Splitting Strings
With the function split(<string-expr>, <pattern>) a string can be converted to a list of values.
Example: | ||
Expression | What is emitted: | |
split("a,b and c", " *(and|,) *") | List: a,b,c |
With the function split_path(<string-expr>, <pattern>) a string can be converted to a hierarchical list of values. This hierarchical list can be displayed as a hierarchical facet.
Expression | What is emitted: |
split_path(“/path/to/documents”, “/”) | Hierarchical List: path, to, documents |
Parsing JSON expressions
The function json_parse(<json>) can be used to parse a JSON expression.
Beispiel:
Ausdruck | What is emitted: |
json_parse("\"a\"") | a |
json_parse("[\"a\", \"b\"]") | List: a,b |
json_parse("{\"int\": 1, \"str\": \"a\"}") | the parsed JSON expression is represented hierarchically as an item |
Parsing date values
With the function parse_date, strings can be interpreted as a date value.
Parameters are:
String value to be parsed
Date formatting parameter, see http://userguide.icu-project.org/formatparse/datetime#TOC-Date-Time-Format-Syntax
(Optional) Locale, see http://userguide.icu-project.org/locale#TOC-Examples
Example: | ||
Expression | What is emitted: | |
Variant 1 | parse_date("1970-01-01 00:00:00 UTC", "yyyy-MM-dd HH:mm:ss zzz") | Corresponding date value |
Variant 2 | parse_date("October 2015", "MMMM y", "en") | Corresponding date value |
Now (current time)
The function now can be used to refer to the current time. The value is expressed in milliseconds since January 1, 1970, which is important in arithmetic expressions.
Example: | ||
Expression | What is emitted: | |
Variant 1 | now() | Current time |
Variant 2 | now() - 3600000 | Time one hour ago |
Variant 3 | mes:date[../mes:date < now()] | Date values of documents that are in the past |
Variant 4 | -(now(), dateproperty) > 3600000 && "true" || "false" | "true" |