Sure, you can handle it. But should you?
Let our experts manage the tech maintenance while you focus on your business.
Let our experts manage the tech maintenance while you focus on your business.
Mindbreeze Property Expression Language
Introduction
The property expression language can be used to extract metadata, or to create filter values that can then be limited in the search.
Language elements
These elements are available during the inversion as well as during the search.
Literals
Constant literal values such as numbers or strings can be specified.
Example:
String | "text" |
Integer | 3 |
Boolean value | true |
Identifiers
An identifier always refers to the current context. This context can be a document, but also an item.
If a reference is tracked, this refers to the referenced document.
Example:
Regular metadata of a document | title |
Reference metadata of a document | reference |
Property within an item | itemProperty |
Note that the following character class is valid without escaping: [A-Za-z0-9_:\-]
The following characters in an identifier do not have to be escaped:
- Capital letters (A-Z)
- Lower case letters (a-z)
- The following special characters: "_", ":", "-"
All other characters must be escaped with a backslash (\) so that the character string is recognized as an identifier. For example, property.value must be escaped as follows: property\.value
Lists
A list of several expressions, that are all emitted.
Example:
Expression | What is emitted… |
list("text1", "text2", "text3") | "text1", "text2", "text3" |
Removing Duplicates
The unique operator removes duplicates from its parameters while maintaining the order. It can be applied to all expressions that have a string representation.
Example:
Expression | What is emitted… |
unique("a", "b", "c", "b") | "a", "b", "c" |
unique("a", "a", "a") | "a" |
Filters
The filter operator can be used to define additional constraints which must apply (normally for items) for selected expressions (e.g. comparison).
Example:
Variant 1 | title[../mes:numericdate < 1444381468000] | The title of the current document will only be emitted if the date is prior to 2015-10-09T09:04:28.000Z (as milliseconds since 1970). |
Comparison operators
The following comparison operators may be used:
Examples in combination with logical operators:
Expression | What is emitted… | |
Equality | 3 = 3 && "true" || "false" | "true" |
Inequality | 3 != 3 && "true" || "false" | "false" |
Less than | 3 < 3 && "true" || "false" | "false" |
Less than or equal to | 3 <= 3 && "true" || "false" | "true" |
Greater than | 3 > 3 && "true" || "false" | "false" |
Greater than or equal to | 3 >= 3 && "true" || "false" | "true" |
Examples in combination with filter(s):
Expression | What is emitted… | |
Equality | title[3 = 3] | Title of the current document |
Inequality | title[3 != 3] | Nothing |
Less than | title[3 < 3] | Nothing |
Less than or equal to | title[3 <= 3] | Title of the current document |
Greater | title[3 > 3] | Nothing |
Greater than or equal to | title[3 >= 3] | Title of the current document |
Logical Operators
AND operator &&
In the AND operation, only the last item in the &&-chain is emitted, if and only when all previous elements have emitted something and their value is not interpreted as false.
Example:
Expression | What is emitted… | |
Literal 1 | "text" && "text2" && "text3" | "text3" |
Boolean expressions | 0 && "text2" | nothing |
1=1 && "true" || "false" | "true" | |
1!=1 && "true" || "false" | "false" |
OR operator ||
In the OR operator, the first element in the ||-chain that emits something is emitted. In combined expressions containing both && and ||, && takes precedence over the ||.
Example:
Expression | What is emitted… | |
Literal 1 | "text1" || "text2" || "text3" | "text1" |
Boolean expressions | 0 || "text2" | "text2" |
0=0 && "true" || "text2" | "true" |
Arithmetic operators
The following arithmetic operators can be used:
Expression | What is emitted… | |
Addition | 3 + 2 3.2 + 5 | 5 8.2 |
Subtraction | 3 – 2 3.2 – 5 | 1 -2.2 |
Multiplication | 3 * 2 3.2 * 5 | 6 16.0 |
Division | 3 div 2 3.2 div 5 | 1 0.64 |
Modulo | mod(3, 2) | 1 |
Operator precedence is supported for compound expressions, so multiplication and division have higher precedence than addition and subtraction. The precedence of partial expressions can be changed by using parentheses:
Expression | What is emitted… | |
Precedence | 3 + 1 * 2 – 1 (3 + 1) * 2 – 1 (3 + 1) * (2 – 1) | 4 7 4 |
Strings
Regex match
The function regex_match can be used to restrict to those strings which correspond to a regular expression.
Parameter | Description | Example |
Parameter 1 – Property (Mandatory) | Name of the Identifier to be checked. | title |
Parameter 2 – Regex Expression (Mandatory) | The Regular Expression to be matched against the value of the Identifier. | ^T.*$ |
Parameter 3 – Options (Optional) | The available options of the regex pattern are for example:
| i |
Example:
Expression | What is emitted… | |
Variant 1 | regex_match(title, "^T.*$") | Titles that begin with "T" are emitted (implicitly case sensitive). |
Variant 2 | regex_match(title, "^T.*$", "i") | Titles that begin with "T" are emitted (explicitly case insensitive). |
Conversions to upper/lower case
With the functions to_lower and to_upper, expressions can be converted to uppercase or lowercase.
Example:
to_lower("UPPER") | "upper" is emitted. | |
to_upper("lower") | "LOWER" is emitted. |
String contexts (concat)
The concat function concatenates multiple parameters of Literals into one.
Example:
concat("Hauptplatz", " ", "123") | "Hauptplatz 123" is emitted. |
Character replacement
With the replace function, characters (character strings) can be replaced; when doing this, regular expressions can be used.
Parameter | Description | Example |
Parameter 1 – Input String (Mandatory) | The string of characters to be manipulated. | 12ab34cd56 |
Parameter 2 – Search String (Mandatory) | The string of characters to search for and that will be replaced. Here a regular expression is used. | [a-z] |
Parameter 3 – Replacement String (Mandatory) | The string of characters to replace the Search String. | 1 |
Example:
Expression | What is emitted… | |
Removing letters | replace("12ab34cd56", "[a-z]", "") | "123456" is emitted. |
replace("0123456789","[0-9]{9}$","") | "0" is emitted. |
Type conversions
Conversion to an Integer Value
With the function int, a string or a date value can be converted to an integer.
Example:
Expression | What is emitted… | |
Variant 1 | int("3") | Number 3 |
Variant 2 | int(dateproperty) | Date value of the property "dateproperty" as milliseconds since 1970 |
Variant 3 | int(now()) | Current time as milliseconds since 1970 |
Conversion to a Floating Point Value
The float function can be used to convert a string or date value to a floating point number.
Example:
Expression | What is emitted… | |
Variant 1 | float("3.124") | Number 3.124 |
Variant 2 | float(10) | 10 as floating point number |
Variant 3 | float(now()) | Current time as milliseconds since 1970 |
Splitting Strings
With the function split a string can be converted to a list of values.
Parameter | Description | Example |
Parameter 1 – Input String (Mandatory) | The string of characters to be manipulated. |
|
Parameter 2 – Search String (Mandatory) | The String of characters to split at. This uses a Regular Expression. |
|
Example:
Expression | What is emitted… |
split("a,b and c", " *(and|,) *") | List: a,b,c |
With the function split_path(<string-expr>, <pattern>) a string can be converted to a hierarchical list of values. This hierarchical list can be displayed as a hierarchical facet.
ExpressionExpression | What is emitted… |
split_path(“/path/to/documents”, “/”) | Hierarchical List: path, to, documents |
Parsing JSON expressions
The json_parse(<json>) function can be used to analyse and display a JSON expression in a structured manner.
Example:
Expression | What is emitted… |
json_parse("\"a\"") | a |
json_parse("[\"a\", \"b\"]") | List: a,b |
json_parse("{\"int\": 1, \"str\": \"a\"}") | The parsed JSON expression is represented hierarchically as an item. |
Parsing date values
With the function parse_date, strings can be interpreted as a date value.
Parameter | Description | Example |
Parameter 1 – Date value as a String (Mandatory) | The date value to be parsed as a string. Note: The date value as a string must match the defined pattern in parameter 2, otherwise the date value will not be parsed. |
|
Parameter 2 - Date pattern parameter (Mandatory) | Defines the pattern or structure of the date. For a complete table of all date field symbols, see Formatting Dates and Times | ICU Documentation. Note: The date pattern parameter must match the pattern of parameter 1, otherwise the date value will not be parsed. |
|
Parameter 3 – Locale (Optional) | Defines the locality or language. This parameter is useful if the string in parameter 1 could be interpreted differently depending on the language. Example:
For parameter examples, see Locales and Resources | ICU Documentation. | en |
Example:
Expression | What is emitted… | |
Variant 1 | parse_date("1970-01-01 00:00:00 UTC", "yyyy-MM-dd HH:mm:ss zzz") | Corresponding date value |
Variant 2 | parse_date("October 2015", "MMMM y", "en") | Corresponding date value |
Formatting date values
The function format_date can be used to convert date values into string.
Parameter | Description | Example |
Parameter 1 – Date value (Mandatory) | Value as a date string or as an integer (in milliseconds since January 1, 1970). | 1000 |
Parameter 2 – Date format string (Mandatory) | See https://unicode-org.github.io/icu/userguide/format_parse/datetime/ - datetime-format-syntax. |
|
Parameter 3 – Time zone (Optional) | See https://unicode-org.github.io/icu/userguide/format_parse/datetime/ - time-zone-display-names and https://www.unicode.org/reports/tr35/ - Time_Zone_Identifiers. |
|
Parameter 4 – locale (Optional) | See https://unicode-org.github.io/icu/userguide/locale/examples.html . | en |
Example:
Expression | What is emitted… | |
Variant 1 | format_date(1000, "yyyy-MM-dd HH:mm:ss zzz", "UTC") | The date 1000ms after Jan 1, 1970, in the UTC time zone: "1970-01-01 00:00:01 UTC" |
Variant 2 | format_date(store:modificationdate, "dd MMMM y HH:mm zzz", "Europe/Vienna", "en") | The date at which the item was last modified in the index, in the Vienna time zone and in English: e.g. "13 June 2024 09:03 GMT+2" |
Variant 3 | format_date(parse_date("13.06.24","dd.MM.yy"), "EEEE dd MMMM y", "en") | Changing the format of a date given as a string: "Thursday 13 June 2024" |
Variant 4 | int(format_date(store:modificationdate, "y")) | Extracting values, for example the year of the last modification date, as an integer: e.g. 2024 |
Now (current time)
The function now can be used to refer to the current time. The value is expressed in milliseconds since January 1, 1970, which is important in arithmetic expressions.
Example:
Expression | What is emitted… | |
Variant 1 | now() | Current time |
Variant 2 | now() - 3600000 | Time one hour ago |
Variant 3 | mes:date[../mes:date < now()] | Date values of documents that are in the past |
Variant 4 | -(now(), dateproperty) > 3600000 && "true" || "false" | "true" |
Annotation
The function annotation can be used to extract values of annotations. The corresponding annotations must be aggregatable.
Example:
Expression | What is emitted… |
annotation(author, "mes:key") | The value of the annotation mes:key for the property with name author is emitted. In the property example below, the following string value is emitted: "https://myorganization.com/people/id/1447" Property: property { name: "author" value { kind: STRING string_value: "John Doe" } annotation { name: "mes:key" value { kind: STRING string_value: "https://myorganization.com/people/id/1447" } flags: 24 |
Search specific language elements
These Language elements are only available for searches. Unlike generell elements, these language elements are not usable during the inversion.
Paths
If metadata of the referenced document should be accessed, the path operator “/” can be used.
Example:
Variant 1 | reference/title | If the current document has the reference property reference, the title of the referenced document is emitted. |
Variant 2 | reference1/reference2/title | If the current document has the reference property reference1, and the referenced document in turn has the reference property reference2 that refers to another document, that title is emitted. |
Inverse References
The rev operator can be used to follow inverse references. To do this, you need to configure the reference property as Inverted Reference Metatdata Keys in the Reference Settings of the index. Also see the section “Reference Settings” in Configuration Mindbreeze InSpire – “Indices” Tab.
Example:
rev(parent)/mes:key | All mes:key values of the inverted reference parent are emitted. |
String References
In order to follow string references, the lookup operator in combination with the path operator “/” can be used. To do this, you need to configure the reference property as Inverted Hash Reference Metadata Keys and Hash Reference Target Metadata Keys in the Reference Settings of the index. Also see the section “Reference Settings” in Configuration Mindbreeze InSpire – “Indices” Tab.
Example:
lookup(author_email)/name | If the current document has an author_email property, the name of the referenced person is emitted. |
lookup(creator_email,last_edited_email)/name | If the current document has a creator_email or last_edited_email property, the names of the referenced persons are emitted. Note: One or more parameters must be specified for the lookup operator. |
The following limitations apply when using the lookup operator:
- The lookup operator only supports string metadata. Built-in metadata is not supported. A list of all built-in metadata can be found in Index Operating Concepts - Built-In Metadata Keys .
- Regarding references, the lookup operator cannot support cross-index references. Therefore, the references must be within an index.
It is also recommended to limit the number of different values of Hash Reference Target Metadata Keys to avoid ambiguities and to ensure reasonable performance. Such a limitation can be implemented using "Precomputed Synthesized Metadata". This is demonstrated below with a use case:
Use Case: Synthesise the name of the author of a document. | |
In the following example, documents and people are in the index. The documents already have an author_email metadata with the author's email address. The goal is to synthesise the metadata author_fullname for all documents by creating references to the persons (authors). Document 1 with the following metadata: key: "doc1", datasource/category: "DataIntegration", author_email: "max.mustermann@mindbreeze.com" Person 1 with the following metadata: key: "person1", datasource/category: "Microsoft Graph", fullname: "Max Mustermann", email: "max.mustermann@mindbreeze.com" The result should be the metadata author_fullname: "Max Mustermann" on document 1. This requires the following configuration steps in the index configuration: | |
Precomputed Synthesized Metadata | |
Name | target_email |
Property Expression | regex_match(datasource\/category, "Microsoft Graph") && email |
Reference Settings | |
Inverted Hash Reference Metadata Keys | author_email |
Hash Reference Target Metadata Keys | target_email |
Synthesized Metadata | |
Name | author_fullname |
Property Expression | lookup(author_email)/fullname |
Result | |
Document 1 with the following metadata: key: "doc1", datasource/category: "DataIntegration", author_email: "max.mustermann@mindbreeze.com", author_fullname: "Max Mustermann" Person 1 with the following metadata: key: "person1", datasource/category: "Microsoft Graph", fullname: "Max Mustermann", email: "max.mustermann@mindbreeze.com", target_email: "max.mustermann@mindbreeze.com" | |
For more information, see Configuration - Metadata Enrichment - Precomputed Synthesized Metadata .
Inverse String References
In order to follow inverse string references, the rev_lookup operator can be used in combination with the path operator “/”. To do this, you need to configure the reference property as Inverted Hash Reference Metadata Keys and Hash Reference Target Metadata Keys in the Reference Settings of the index. Also see the section “Reference Settings” in Configuration Mindbreeze InSpire – “Indices” Tab.
Example:
rev_lookup(email,author_email)/title | All title values of the inverted string reference email are emitted. Example of a use case: Documents reference persons (document metadata author_email references a person with metadata email). The property expression (see left column) returns the titles of all documents written by the referenced person. |
rev_lookup(list(email,email_alias), | All title values of the inverted string references email and email_alias are emitted. Example of a use case: Documents reference people (document metadata creator_email and last_edited_email reference persons with metadata email and email_alias). The Property Expression (see left column) returns the titles of all documents created or last edited by the referenced person. Note: Two or more parameters must be specified for the rev_lookup operator. The first parameter of rev_lookup is the target metadate the reference points to. Multiple properties can be specified here using list. All other parameters represent the properties from which the references originate. |
Transitive Closure
The dfs operator can be used to emit all items starting from an expression that are reachable via an expression in Depth First Search (DFS) traversal. Two or three arguments must be specified for the dfs operator:
Example:
dfs(.,parent)/mes:key | All mes:key values starting at the current context of all items reachable via the parent expression. |
dfs(.,rev(parent))/mes:key | All mes:key values starting at the current context of all items reachable via the rev(parent) expression. |
dfs(.,parent,"terminal")/mes:key | mes:key value of the last (terminal) item that is reachable via the parent expression from the current context. If the graph has a tree form, this expression yields the mes:key property of the root of the tree. |
dfs(.,rev(parent),"terminal")/mes:key | mes:key value of the last (terminal) item that is reachable via the rev(parent) expression from the current context. If the graph has a tree form, this expression emits all the mes:key properties of leaves reachable from the current context. |
Parent context
The Parent-operator .. can be used to access the parent context.
This operator is particularly useful in combination with the filter operator, see Inverse References.
Example:
Variant 1 | reference/title/.. | Corresponds to reference . |
Variant 2 | reference/title/../.. | Corresponds to the evaluated document itself. |
Name
The function name can be used to deduce the name of the metadata item or the item property from a value context.
Example:
Variant 1 | name(regex_match(title, "^T.*$")) | Title emitted, if the title of the current document matches the regular expression. |