Elasticsearch demo

In part two of this series, I’ll cover the creation of your first elastic index, CRUD operation and simple search with paging and sorting. But before that I’ll give you some more background information about Elastic.

Elastic is Document-oriented Storage

According to Elasticsearch: The Definitive Guide, "Elastic is document-oriented, meaning that it stores entire objects or 'documents'. It not only stores them, but also indexes the contents of each document in order to make them searchable. In Elastic, you index, search, sort and filter documents—not rows of columnar data. Elastic uses JavaScript Object Notation, or JSON, as the serialization format for documents.

Creating Your First Elastic Index

According to the same guide, "The act of storing data in Elastic is called indexing … In Elasticsearch, a document belongs to a type, and those types live inside an index.… An Elasticsearch cluster can contain multiple indices (databases), which in turn contain multiple types (tables). These types hold multiple documents (rows), and each document has multiple fields (columns)."

You have a working installation of Elastic server and have successfully connected to it from your .Net program. Now, we will create your first Elastic index. Start by creating a simple type for this blog post itself, which we will store in Elastic. It will contain all the details of the article. Each document in our index will be of this type. This type will live on our newly created index. We will call this index “my_first_index”. This index will live on our Elastic node. We will define blog post type in separate “BlogPost.cs” file. Create it, and then enter the following code:

[ElasticType(IdProperty = "Id", Name = "blog_post")]

public class BlogPost

{

   [ElasticProperty(Name = "_id", Index = FieldIndexOption.NotAnalyzed, Type = FieldType.String)]

   public Guid? Id { get; set; }

   [ElasticProperty(Name = "title", Index = FieldIndexOption.Analyzed, Type = FieldType.String)]

   public string Title { get; set; }

   [ElasticProperty(Name = "body", Index = FieldIndexOption.Analyzed, Type = FieldType.String)]

   public string Body { get; set; }

   public override string ToString()

   {

       return string.Format("Id: '{0}', Title: '{1}', Body: '{2}'", Id, Title, Body);

   }

}

Please note that Elastic either automatically evolves a schema for you, or a schema is to be described manually by adding the appropriate mappings. I recommend describing a schema manually. As I have mentioned earlier we are using NEST library for communication with Elastic. NEST allows put mappings into Elastic in two ways either by using “Attribute based mapping” or “Code based mapping”. I am using “Attribute based mapping” in this example. There are the following two mapping attributes available in NEST:

ElasticType

ElasticProperty

ElasticType defines Elastic index type. Name and IdProperty option could be specified. Name – provides type name. IdProperty – specifies which property should be used as identity key.

ElasticProperty specifies how model properties should be indexed. For each property, we could assign the following four options: name, type, index option and analyzer. The “Name” option specifies what that property is called in Elastic index. The “Type” option specifies how your property is stored (Elastic internal type are the following: string, integer/long, float/double, boolean and null). The “Index” option could have the following values: Analyzed - value of property will be analyzed and processed before indexing it; NotAnalyzed - value of property will not be analyzed and will be indexed as it is; No - value of property will not be indexed.

Now we will use type and mapping information from attributes to create Index. Enter the following code into “Program.cs” file:

var res = elastic.CreateIndex(ci => ci

   .Index("my_first_index")

   .AddMapping(m => m.MapFromAttributes()));

Console.WriteLine(res.RequestInformation.Success);

Now save build and run program. You should see the following output in the console: "True." Congratulations, you just have created your first Elastic index!

CRUD Operations

We have created an index; the place there our data will be stored. Now, let’s put some documents into it. Enter the following code:

var blogPost = new BlogPost

{

   Id = Guid.NewGuid(),

   Title = "First blog post",

   Body = "This is very long blog post!"

};

var firstId = blogPost.Id;

var res = elastic.Index(blogPost, p => p

   .Index("my_first_index")

   .Id(blogPost.Id.ToString())

   .Refresh());

   Console.WriteLine(res.RequestInformation.Success);

Now, save your build and run the program. You should see the following output in the console: "True." When you save the document into the Elastic index, you should provide data, index name and ID. Use the “refresh” option if you need newly created data to be instantly visible in THE index. Otherwise, it will take some time before it is indexed.

The following example shows how you to retrieve document from index by its ID:

var resGet = elastic.Get(g => g

   .Id(firstId.ToString())

   .Index("my_first_index"));

Console.WriteLine(resGet.RequestInformation.Success);

Console.WriteLine(resGet.Source);

Now save the build and run the program. You should see the following output in the console:

True

Id: '0b6871fa-9c84-4b8a-8c7b-206c6b049362', Title: 'First blog post', Body: 'This is very long blog post!'

Please note that the value of the "ID" property in your output will be different because we used “Guid.NewGuid()” to generate it.

The following example shows how document could be deleted from index by ID:

var resDel = elastic.Delete(d => d

   .Id(firstId.ToString())

   .Index("my_first_index"));

Console.WriteLine(resDel.RequestInformation.Success);

Now save the build and run the program. You should see the following output in console: “True.”

By now, you have probably noticed that we are specifying the index name in each NEST method call. The good news is that we could avoid the specifying of an index name by providing a default index and then setting up a connection. Let’s update our code in order to provide a default index, as follows.

var local = new Uri("http://localhost:9200");

var settings = new ConnectionSettings(local, "my_first_index");

var elastic = new ElasticClient(settings);

var resGet = elastic.Get(g => g

   .Id(firstId.ToString());

That wraps up the CROD operations section. Now, we move on to searching.

Simple Search with Paging and Sorting

We’ll start by populating our index with new documents. The following code snippet will create ten blog posts. They are almost identical, except one number in the title and body. This number represents the index in order of document creation.

for (var i = 0; i < 10; i++)

{

   var blogPost = new BlogPost

   {

       Id = Guid.NewGuid(),

       Title = string.Format("title {0:000}", i),

       Body = string.Format("This is {0:000} very long blog post!", i)

   };

   elastic.Index(blogPost, p => p

       .Id(blogPost.Id.ToString())

       .Refresh());

}

Now that the index has some documents in it, let’s run a search on them. We will start from the simplest of all queries, “MatchAll”, which lists all documents. Our query consists of the following parts: “from”, “size”, “query” and “sort”. “From” tells Elastic to return matching documents starting from a specified index. “Size” tells how many of the matching documents Elastic should return (e.g. page size). “Query” tells Elastic which search operation to use (e.g. “MatchAll”). “Sort” tells Elastic in which order matched documents should be returned (e.g. sort results by “Title” property in ascending order). In this simple case, “query” consists only of the “MatchAll” operator:

var res = elastic.Search(s => s

   .From(0)

   .Size(5)

   .Query(q => q.MatchAll())

   .Sort(o => o.OnField(p => p.Title).Ascending()));

Console.WriteLine(res.RequestInformation.Success);

Console.WriteLine(res.Hits.Count());

foreach (var hit in res.Hits)

{

   Console.WriteLine(hit.Source);

}

Now save the build and run the program. You should see the following output in the console:

_True

5

Id: '9c01b315-95d2-470e-a448-1103c8e0fca6', Title: 'title 000', Body: 'This is 000 very long blog post!'

Id: 'cd280b3f-b1e6-40e1-a4cc-ecf0a88fae95', Title: 'title 001', Body: 'This is 001 very long blog post!'

Id: 'ae8f8b12-8a29-4999-a7dd-4c35ccd55c29', Title: 'title 002', Body: 'This is 002 very long blog post!'

Id: '6a366272-889e-4ce3-9aab-fb9dd2c7c432', Title: 'title 003', Body: 'This is 003 very long blog post!'

Id: '1aae4718-5718-4d37-8a21-1d4a85f229b8', Title: 'title 004', Body: 'This is 004 very long blog post!'

If you change the value of “from” to five you will get second page of results:

True

5

Id: 'f1868a97-19d9-4714-a6b8-59c1caa28d54', Title: 'title 005', Body: 'This is 005 very long blog post!'

Id: '5d7020c3-9845-4345-a21b-cde18506593d', Title: 'title 006', Body: 'This is 006 very long blog post!'

Id: '47eed745-a394-48e9-8f21-d04f52004b55', Title: 'title 007', Body: 'This is 007 very long blog post!'

Id: 'c30b8bb1-2385-4a1d-a522-3016886fd59d', Title: 'title 008', Body: 'This is 008 very long blog post!'

Id: '266ea063-1797-4856-8786-2a0258a8e16a', Title: 'title 009', Body: 'This is 009 very long blog post!'

_

You saw simple query (“match all”) in action we will now look into other query types.

That's it for part two of this series. In part three, we'll cover other types of queries, such as Match, Bool and Nested Types. Stay tuned!