Elasticsearch Mapping Management + Templates

Mapping Management + Templates

As described earlier, a mapping defines the shape of a field key. Once a mapping has been created, it cannot be modified except by adding fields. To change it, you need to recreate the index.

There are four ways to create or add mappings.

Create mappings automatically
Create mappings manually
Create mappings with templates
Add fields to an existing mapping

Create mappings automatically

Elasticsearch automatically creates mappings when creating documents in an index, as shown below.

Create a document in the mapping_test index

PUT /mapping_test/_doc/1
{
  "date":"2021/12/01 09:00:00+0900",
  "Tweet":"This is a mapping test."
}

Check the mapping

GET /mapping_test

Mapping contents

GET /mapping_test
{
  "mapping_test" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "Tweet" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "date" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    },

// ... omitted ...

From the result above, you can see that the following mappings were defined. The difference between the text and keyword types is as follows.

The date and Tweet keys are text type.
The date.keyword and text.keyword keys are keyword type.

The date key in the mapping above is registered as text, but there are cases where it should be registered as the date type. In that case, you need to create the mapping manually.

Create mappings manually

Manual mapping creation is done as follows.

Create a manual mapping

PUT /mapping_test2
{
  "mappings": {
    "properties": {
      "date":    { 
        "type": "date",
        "format": "yyyy/MM/dd HH:mm:ssZ"
      },  
      "tweet":  {
        "type": "text",
        "fields": {
          "keyword":{
            "type":"keyword"
          }
        }
      }
    }
  }
}

Result of creating the mapping manually

GET /mapping_test2

Manually created mapping contents

GET /mapping_test2
{
  "mapping_test2" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "date" : {
          "type" : "date",
          "format" : "yyyy/MM/dd HH:mm:ssZ"
        },
        "tweet" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword"
            }
          }
        }
      }
    },

... omitted ...

If it was created correctly, you can see that the date key is mapped as the date type.

When the format differs from the one specified in the mapping

If you create a document with a format different from the one specified in the mapping, an error occurs.

When running the following example, the date field format is "yyyy/MM/dd HH:mm:ssZ", but the data is inserted with the different format "yyyy/MM/dd HH:mm:Z" because seconds are not specified.

Create a document with a format different from the mapping

PUT /mapping_test2/_doc/1
{
  "date":"2021/12/01 09:00+0900",
  "tweet":"This is a mapping test."
}

Error message

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "failed to parse field [date] of type [date] in document with id '1'. Preview of field's value: '2021/12/01 09:00+0900'"
      }
    ],

... omitted ...

As expected, the message says that the date field cannot be parsed.

When the format matches the mapping

Now create a document with the correct format.

Create a document according to the mapping format

PUT /mapping_test2/_doc/2
{
  "date":"2020/11/01 09:00:00+0900",
  "tweet":"This is a mapping test."
}

When the document is created with the correct format, you can see that it succeeds.

Create mappings with templates

Creating the same mapping for each index is cumbersome. Templates solve this problem.

When an index with the specified name is created, a template creates the index using the template mapping. Use the template API to create a template.

Use the _template API.

PUT /_template/test_template
{
  "index_patterns": "test*",
  "mappings": {
    "properties": {
      "date":    { 
        "type": "date",
        "format": "yyyy/MM/dd HH:mm:ssZ"
      },  
      "tweet":  {
        "type": "text",
        "fields": {
          "keyword":{
            "type":"keyword"
          }
        }
      }
    }
  }
}

In the template above, because "index_patterns": "test*" is specified, the mapping is created for index names that have "test" as a prefix.

Now check how the template works.

Create the test index

PUT /test

Check the mapping of the test index

GET /test

Check the mapping result of the test index

GET /test
{
  "test" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "date" : {
          "type" : "date",
          "format" : "yyyy/MM/dd HH:mm:ssZ" // date field registered with the format specified in the template
        },
        "tweet" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword"
            }
          }
        }
      }
    },

... omitted ...

You can see that the date field was registered with the format specified in the template.

Add fields to an existing mapping

Use the mapping API to add a new field to an existing mapping.

Here, add the additional_field field to the mapping of the mapping_test2 index.

Contents of mapping_test2

GET /mapping_test2
{
  "mapping_test2" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "date" : {
          "type" : "date",
          "format" : "yyyy/MM/dd HH:mm:ssZ"
        },
        "tweet" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword"
            }
          }
        }
      }
    },
... omitted ...

Add a format to the mapping

PUT /mapping_test2/_mapping
{
  "properties":{
    "additional_field":{
      "type":"text"
    }
  }
}

Check that the additional_field format was added

GET /mapping_test2

Result confirming that the additional_field format was added

{
  "mapping_test2" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "additional_field" : { // additional_field was added
          "type" : "text"
        },
        "date" : {
          "type" : "date",
          "format" : "yyyy/MM/dd HH:mm:ssZ"
        },
        "tweet" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword"
            }
          }
        }
      }
    },
... omitted ...

You can confirm that the additional_field field was added.