Atlassian Jira Connector

Installation and Configuration

Installation

Before you install the Atlassian Jira connector, you must ensure that the Mindbreeze server is installed and that the Atlassian Jira connector is included in the license. Use the Mindbreeze Management Center to install or update the connector.

Configuring the Index and Crawler

Navigate to the tab "Indices" and click on "+ Add Index" in the upper right corner to create a new index.

Add a new data source by clicking on "Add Data Source" in the upper right corner. Select the category "Atlassian Jira".

Configure the following required settings:

To create an API token, go to https://id.atlassian.com/manage-profile/security/api-tokens and log in with the Jira user that will be used for crawling. Then click on "Create API token" to create, name, and then copy the token.

The following settings are optional:

If your Jira instance contains a large amount of data, it is recommended to enable delta crawling:

Then save the configuration and restart.

Configuration of the Principal Resolution Service

In the new or existing service, select the Atlassian Jira Caching Principal Resolution Service option in the Service setting. For more information about additional configuration options and how to create a cache and how to do the basic configuration of a cache for a Principal Resolution Service, see Installation & Configuration - Caching Principal Resolution Service.

Configure the following required settings:

The following settings are optional:

Metadata indexed by default

By default, the following Jira fields are downloaded per issue and converted to metadata:

If additional Jira fields should be indexed, downloaded or eventually converted to metadata, follow the instructions in Configuration - Jira Connector.

Indexing of custom metadata

Overview

JIRA allows its users to create custom fields, display them in the UI and assign values to these fields, either when creating or editing issues.

In some cases, you may want the values of these fields as metadata on your indexed issues in Mindbreeze InSpire. This is possible via the following steps:

1. Find out the unique identifier of the custom fields in JIRA (see the chapter Locating the Field ID of custom JIRA fields).
2. Create a YAML file for custom metadata mapping
   1. In most cases here you only need to specify the unique identifier (Field ID) in the section “indexedFields”. For more information on the format of the metadata mapping and the section “indexedFields”, see the chapter Custom Metadata Mapping.
   2. The result of this step should be a YAML file like customMetadataMapping.yaml .
3. Upload the custom mapping to Mindbreeze InSpire.
   1. This can be done in the Mindbreeze InSpire Management Center in the menu item "File Manager". For example at /data/resources/mapping (this folder does not exist per default):
   2. 

1. Configure the JIRA Crawler
   1. Create a JIRA crawler and adapt the configuration to your needs. For more information about the configuration of the index and crawler, see the chapter Configuring the Index and crawler.
   2. Additionally, in the configuration set the advanced setting "Custom Metadata Mapping Path" to, for example, /data/resources/mapping/customMetadataMapping.yaml.

Attention: Specifying a custom metadata mapping does NOT override the default mapping. Rather, it extends the default mapping.

Locating the Field ID of custom JIRA fields

There are two approaches to locate the Field ID of a custom JIRA field. Either via the JIRA GUI or via the Mindbreeze crawler logs.

JIRA GUI

First, open the Jira administration page and click the menu item “Issues”.

On the left side, click the menu item “Custom Fields”.

Afterwards, go to the desired field and click on "Configure".

You will find the Field ID in the URL of the website.

The complete Field ID for this field is “customfield_10601“.

Mindbreeze Crawler Logs

The Field ID can also be located in the Mindbreeze crawler log directory. There the file jira-fields.csv can be found, which gives an overview over all JIRA fields.

The following information can be found there:

• Field ID
• Display name (in JIRA)
• Schema (according to JIRA)
• Configured schema (according to the default and the custom metadata mapping)
• Single-value or multi-value field
• Is the field currently configured to be indexed (metadata is going to be extracted) or not?

Custom Metadata Mapping

Overview

The metadata mapping is defined in the YAML format and consists of 2 main sections – the schemas and the indexedFields.

The metadata mapping describes which fields should be indexed ("indexedFields" section) and how this metadata should be created ("schemas" section).

Since all of the JIRA field types (JIRA schemas) are already predefined in the default metadata mapping and the needed schema for a JIRA field can be automatically detected, in most use cases, it is sufficient to only specify the indexed field, as shown in the chapter Automatic Schema Detection. This has to be done in the indexedFields section. See the chapter Complete Example for a finished mapping.yaml.

IndexedFields

An indexed field specifies a JIRA field from which metadata should be generated and which schema should be used.

An indexed field always consists of the FIeld ID and, optionally, of additional attributes.

Attention: In most cases, the specification of optional attributes is not necessary. The important attributes (schema, metadataBaseName and multiValueField) are automatically retrieved from JIRA.

Optional Attributes

Examples

Automatic Schema Detection

Objective: In your JIRA setup, you have a custom multi-user picker with the Field ID “customfield_10401” and the display name “Involved People”. Metadata should be extracted from this field.

The following field mapping would do the trick:

customfield_10401: {}

With this specification, the crawler would, via the JIRA fields endpoint, automatically find out that the configured field is a multi-value field with the schema "user" and the metadata base name "involved_people". In this case the "user" schema does not have to be specified manually, as almost all of the JIRA default schemas are already defined in the default metadata mapping.

If the metadata base name "involved_people" is not desired, a different name can be specified for it via the following configuration:

customfield_10401:

  metadataBaseName: 'my_custom_base_name'

Attention: The schema does not need to be specified.

Multi Value Field

Objective: You want to use a different schema for the “Involved People” field, than the default user schema. To achieve this, the schema must be specified manually. If a schema is specified manually, the attribute 'multiValueField' must be specified manually as well:

customfield_10401:

  schema: 'my-custom-user-schema'

  multiValueField: True

Ignore Field

Objective: Mindbreeze specifies the field “reporter” per default as indexed field. You want to avoid this, since this metadata is not needed.

To achieve this, the field must be set to ignored in the custom configuration:

reporter:

  ignoreField: True

Schemas

Schemas describe how different JIRA fields (from the API in JSON format) should be handled, what properties (JSON keys) there are, and how to create a metadata from them. A schema consists of a property definition, which can have various attributes. Those attributes are described in the following chapter.

Property Definition

Examples

To make the examples easier to understand, let's look at the following custom JIRA field first. Suppose you have created a custom field called “Code Reviewer” that is a “SingleUserPicker”.

When the issue JSON is fetched by Mindbreeze, the field with its value (representing a single user) looks like this. (note that the name of the ID of this custom field is created by JIRA. Iin this example, the ID is "customfield_10413"):

"customfield_10413": {

  "self": "http://jira.url/rest/api/2/user?username=john.doe",

  "name": "john.doe",

  "key": "john.doe",

  "emailAddress": "john.doe@email.com",

  "avatarUrls": {

    "48x48": "http://jira.url/secure/useravatar?avatarId=10122",

    "24x24": "http://jira.url /secure/useravatar?size=small&avatarId=10122",

    "16x16": "http://jira.url/secure/useravatar?size=xsmall&avatarId=10122",

    "32x32": "http://jira.url/secure/useravatar?size=medium&avatarId=10122"

  },

  "displayName": "John Doe",

  "active": true,

  "timeZone": "Europe/Vienna"

}

Example of a simple User Schema

Objective: Generate metadata for the values “displayName”, “emailAddress” and “key”. Furthermore, the names of the metadata should be slightly altered.

To do this, the schema must look like this (the default schema when a field of the type “user” is encountered):

my-user-schema:

  properties:

    - property: 'displayName'

      name: 'name'

    - property: 'emailAddress'

      name: 'email'

    - property: 'key'

First the schema name is defined, in this case the name is “my-user-schema“. The value of a schema is already a property definition. So all of the attributes, defined in the table, can be used here.

Since the value of customfield_10413 is on a (JSON) object, the sub-properties of this object need to be defined.

In the example shown, the desired properties are:

displayName (the name is overwritten to name)

emailAddress (the name will be overwritten with email)

key

Since none of these properties have other attributes specified (for example “prefixName“ or „parsingStrategy“) the resulting metadata will look like this:

Example of an Extended User Schema

Objective: Generate metadata for the values “displayName”, “emailAddress” and “key”. Furthermore, the names of the metadata should be slightly altered. Additionally, create a metadatum for avatar URL in the size 48 x 48 Pixel and make it non-searchable. So it cannot be found when for example the number “10122” is being searched.

The following schema accomplishes this:

my-user-schema:

  properties:

    - property: 'displayName'

      name: 'name'

    - property: 'emailAddress'

      name: 'email'

    - property: 'key'

    - property: 'avatarUrls'

      properties:

        - property: '48x48'

          searchable: False

Since the avatarUrls in the JSON also has an object as value, the sub-properties of that object need to be defined via the properties attribute.

The resulting metadata will look like this:

If the name should be code_reviewer_avatar_url rather than code_reviewer_avatarUrls_48x48, the property can be specified like this:

    - property: 'avatarUrls'

      name: ''# when creating the hierarchical name, empty names are skipped

      properties:

        - property: '48x48'

          name: 'avatar_url'

          searchable: False

Another way to achieve this is to use the following method:

    - property: 'avatarUrls'

      properties:

        - property: '48x48'

          name: 'code_reviewer_avatar_url'

          prefixName: False # defines that this name should not be prefixed

          searchable: False

Attention: This method is not recommended, since this schema only makes sense in combination with the code reviewer field and not with for example “SingleUserPickers”. Additionally, it is easier to create name conflicts with this method.

Complete Example

Objectives:

Add avatar URL for all Mindbreeze default indexed user metadata.

Add custom schema which does not convert a date field to a calendar, adds the value as directly as string and use it for a custom date picker.

Add the JIRA affected versions as new metadatum.

Ignore the metadatum (field) “reporter”.

Add avatar URL for users

To accomplish that, the schema “mindbreeze-user” must be overridden. This schema is the default schema for the fields “reporter”, “assignee” and “creator”.

First a copy of the default schema should be created, to not lose any already existing metadata. Then add the property "avatarUrls" with its sub-property "48x48" to the schema to get the URL for the largest avatar URL available. If desired, the names can be adjusted as well.

The schema should look like this:

mindbreeze-user:

  properties:

    - property: 'displayName'

      name: 'name'

    - property: 'key'

    - property: 'emailAddress'

      name: 'email'

    - property: 'avatarUrls'

      name: ''

      properties:

        - property: '48x48'

          name: '_avatar_url'

New schema for human readable date

The default schema for fields with the type "date" looks like this:

date:

    parsingStrategy: TO_CALENDAR

The goal here is to avoid the parsing step. This can easily be done by not specifying a parsing strategy. The new schema should look like this:

string_date: {}

The schema can now be used for date fields (e.g., custom date picker fields):

customfield_10401:         # example custom date picker field

    schema: 'string_date'

Ignore the field "reporter"

Like shown in a previous example, this can be accomplished with a simple field specification:

reporter:

    ignoreField: True

Add field "Affects Version/s”

The Field ID for this field is "versions", since the default schema, defined by JIRA, as well as the display name are sufficient. Just specify the field "versions" in the indexedFields:

versions: {}

Since the JIRA names (display names as shown in the JIRA UI) are converted to snake case when used as metadata names, the final metadata would be prefixed with "affects_versions" (special characters like in this case “/” are removed as well).

Final Mapping

When everything is brought together, the final custom metadata mapping should look like this:

schemaVersion: 1

schemas:

  string_date: {}

  mindbreeze-user:

    properties:

      - property: 'displayName'

        name: 'name'

      - property: 'key'

      - property: 'emailAddress'

        name: 'email'

      - property: 'avatarUrls'

        name: ''

        properties:

          - property: '16x16'

            name: '_avatar_url'

indexedFields:

  versions: {}

  customfield_10401:

    schema: 'string_date'

  reporter:

    ignoreField: True

View effects of a custom mapping configuration

The crawler logs show an overview over the effects of the custom mapping configuration at the start:

Found custom metadata mapping at: '/data/resources/mapping/metadataMapping.yaml'. This will have the following effects:

        - Schemas:

                - Newly added:

                        - string_date

                - Overridden:

                        - mindbreeze-user

        - Indexed Fields:

                - Newly added:

                        - customfield_10401

                        - versions

                - Ignored (removed):

                        - reporter

Appendix

Restrictions

• Jira permission schemes with "Group custom field value" in the "Browse projects" permission are not supported.
• Jira issue-level security is not supported. Jira issues with security level set cannot be found in the search if ACLs are enabled.
• For the Principal Resolution Service to resolve Project Roles or Groups for a user, the user’s email address needs to be public. You can configure this here. To do this, set the column "Who can see this?" to "Anyone" for the user's email address.

Troubleshooting

“ERROR: CrawlRun was unsuccessful” with URL Path “/search” and “status code 400”

Most likely an "Issue Constraint (JQL)" is configured, which causes the error

1. Check the configured "Issue Constraint (JQL)" using the "Advanced Search" function in Jira directly. In any case, the JQL must work within Jira.
2. Check that there are no forbidden keywords in the JQL, such as "ORDER BY". Remove these keywords.
3. The crawler does not use the entered JQL directly, but transforms it before performing Jira API requests. The entered JQL is enclosed in brackets and further constraints and sorting are performed. The transformed JQL is then passed as a URL parameter in the Jira API requests. The effective URL is then displayed in the error message.
   E.g.: https://jira.mycompany.com/ rest/api/2/search?jql=%28proect+%3D+%22Big+Data%22%29+AND+updated+%3E%3D+%272020-06-17+17%3A10%27+ORDER+BY+updated+ASC&startAt=100&maxResults=100&fields=key,updated,description...

This corresponds decoded e.g. to the effective JQL: (proect = \"Big Data\") AND updated >= '2020-06-17 17:10' ORDER BY updated ASC

If you open the effective URL from the error message in a browser, you will get a detailed error message about what exactly went wrong with the effective JQL. E.g.: "Field 'proect' does not exist or you do not have permission to view it."

Crawl run is not completed or certain issues were not indexed

If the time zone preference of the user used for crawling and the Jira server are different, various errors can occur. For example, it is possible that the crawl run is never completed or certain issues are skipped.

The former case can be detected by checking the log of the crawler. If the same URLs are called over and over again, the reason may be the user's time zone.

Therefore, you should always make sure that the user's time zone matches the server's time zone. This can be set in the user's profile.

A User is finding too little documents and calling ‘checkprinicpals’ on the Principal Resolution Service does not return Project Roles or Groups

As described in Section 4.1 Restrictions, the Principal Resolution Service can only resolve the Project Roles and Groups of users, if their Email Address is public. You can configure this here. To do this, set the column "Who can see this?" to "Anyone" for the user's email address.