Elasticsearch How To Create Index

Elasticsearch is one part of the popular ELK stack used for log analytics and search. Applications and systems are constantly logging data that can be very useful for troubleshooting and tracking problems. Using the ELK stack, you have the best tools to perform these tasks quickly and very easily.

In this quick tutorial, we will look at Elasticsearch, specifically how to create indices in the Elasticsearch engine. Although you do not need any comprehensive knowledge about ELK stack to follow this tutorial, having a basic understanding of the following topics might be advantageous:

Using the terminal, specifically, cURL
Basic knowledge of APIs and JSON
Making HTTP Request

NOTE: This tutorial also assumes that you have Elasticsearch installed and running on your system.

What Are Elasticsearch Indices?

Without oversimplifying or overcomplicating things, an Elasticsearch index is a collection of related JSON documents.

As mentioned in a previous post, Elasticsearch indices are JSON objects—considered the base unit of storage in Elasticsearch. These related JSON documents are stored in a single unit that makes up an index. Think of Elasticsearch documents as tables in a relational database.

Let's relate an Elasticsearch index as a database in the SQL world.

MySQL => Databases => Tables => Columns/Rows
Elasticsearch => Indices => Types => JSON Documents with Properties

How to Create an Elasticsearch Index

Elasticsearch uses a powerful and intuitive REST API to expose its services. This functionality allows you to use HTTP requests to perform operations on the Elasticsearch cluster. Therefore, we will use the create index API to create a new index.

For this guide, we will use cURL to send the requests and preserve integrity and usability for all users. However, if you encounter errors with cURL, consider using Kibana Console.

The syntax for creating a new index in Elasticsearch cluster is:

To create an index, all you have to do is pass the index name without other parameters, which creates an index using default settings.

You can also specify various features of the index, such as in the index body:

The settings for the index
Index aliases
Mappings for index fields

The index name is a required parameter; otherwise, you will get an error for the URIL (/)

curl -X PUT "localhost:9200"
{"error":"Incorrect HTTP method for uri [/] and method [PUT], allowed: [DELETE, HEAD, GET]","status":405}

To create a new index with the name single_index, we pass the request:

For cURL, use the command:

curl -X PUT "localhost:9200/single_index?pretty"

This command should result in HTTP Status 200 OK and a message with acknowledged: true as:

{
"acknowledged": true,
"shards_acknowledged" : true,
"index" : "single_index"
}

The request above creates an index single_index with default settings as we did not specify any configurations.

Index Naming Rules

When creating names for Elasticsearch indices, you must adhere to the following naming standards:

The index name must be in lower case only.
The index names cannot start with a dash (-), an underscore (_), or an addition sign (+)
The names cannot be . or ..
Index names cannot include special characters such as: \, /, *, ?, ", <, >, |, ` ` (space character), ,, #
The length of index names must be less than 255 bytes. Multi-byte characters will count in the total length of the index name. For example, if a single character is 8 bytes in length, the total remaining length of the name is 255 – 8
In the latest version of Elasticsearch, names that start with a . are reserved for hidden indices and internal indices used by Elasticsearch plugins.

How to Create an Index Body

When using the PUT request to create an index, you can pass various arguments that define the settings for the index you want to have created. Values you can specify in the body include:

Aliases: Specifies alias names for the index you want to have created; this parameter is optional.
Settings: This defines the configuration options for the index you want to have created. If you fail to specify any parameters, the index gets created using default configurations.
Mappings: This defines the mapping for fields in the index. The specifications you can include in mappings include:
- The field name
- The data type
- The mapping parameter

For an example of creating an index with body configurations, consider the request below:

PUT /single_index_with_body
{
"settings": {
"number_of_shards": 2,
"number_of_replicas": 2
},
"mappings": {
"properties": {
"field1": { "type": "object" }
}
}
}

For a cURL equivalent request:

curl -XPUT "http://localhost:9200/single_index_with_body" -H 'Content-Type: application/json' -d'{ "settings": { "number_of_shards": 2, "number_of_replicas": 2 }, "mappings": { "properties": { "field1": { "type": "object" } } }}'

The above request creates a new index with the name single_index_with_body with 2 numbers of shards and 2 replicas. It also creates a mapping with a field of name field1 and type as a JSON object.

Once you send the request, you will get a response with the status of the request as:

{
"acknowledged": true,
"shards_acknowledged" : true,
"index" : "single_index_with_body"
}

"Acknowledged" shows whether the index was successfully created in the cluster, while "shards_acknowledged" shows whether the required number of shard copies were started for every shard in the specified index before time out.

How to View Elasticsearch Index

To view the information about the index you created, use a similar request to that of creating an index, but use the HTTP method instead of PUT as:

GET /single_index_with_body

For cURL,

curl -XGET "http://localhost:9200/single_index_with_body"

This command will give you detailed information about the requested index as:

{
"single_index_with_body" : {
"aliases" : { },
"mappings" : {
"properties" : {
"field1" : {
"type" : "object"
}
}
},
"settings" : {
"index" : {
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
},
"number_of_shards" : "2",
"provided_name" : "single_index_with_body",
"creation_date" : "1611045687208",
"number_of_replicas" : "2",
"uuid" : "3TRkO7xmQcSUOOGtb6pXVA",
"version" : {
"created" : "7100299"
}
}
}
}
}

Conclusion

This guide discussed how to work with Elasticsearch to create index API to create new indices. We also discussed how to create suitable names for the indices and configuration settings.

By using this guide, you can now create and view indices using the Elasticsearch API.

About the author

My name is John and am a fellow geek like you. I am passionate about all things computers from Hardware, Operating systems to Programming. My dream is to share my knowledge with the world and help out fellow geeks. Follow my content by subscribing to LinuxHint mailing list