Share via

Searching data in Elasticsearch using C#

  • Article

Introduction

In this article, we will see how to use Elasticsearch in our application to fetch data from Elasticsearch and show that data to the client application. The example is made of C# use under WinForm. This article is especially focusing on newcomers and anyone new wants to learn or thinking of using ES in their .NET program. This sample illustrates a way to let user search data from Elasticsearch from their app.

What is Elasticsearch?

Elasticsearch is a RESTful, NoSQL, distributed full-text database or search engine. Which means that this database is document based instead of using tables or schema, we use documents… lots and lots of documents. 

Features include:

  • Distributed and Highly Available Search Engine.
    • Each index is fully sharded with a configurable number of shards.
    • Each shard can have one or more replicas.
    • Read / Search operations performed on either one of the replica shards.
  • Multi-Tenant with Multi Types.
    • Support for more than one index.
    • Support for more than one type per index.
    • Index level configuration (number of shards, index storage, …).
  • Various set of APIs
    • HTTP RESTful API
    • Native Java API.
    • All APIs perform automatic node operation rerouting.
  • Document-oriented
    • No need for upfront schema definition.
    • The schema can be defined per type for customization of the indexing process.
  • Reliable, Asynchronous Write-Behind for long-term persistence.
  • (Near) Real-Time Search.
  • Built on top of Lucene
    • Each shard is a fully functional Lucene index
    • All the power of Lucene easily exposed through simple configuration / plugins.
  • Per operation consistency
    • Single document level operations are atomic, consistent, isolated and durable.
  • Open Source under Apache 2 License.

Building The Sample

Before we use Elasticsearch we need to configure it on our machine. Please follow the below steps:

Installing Elasticsearch

Installing Elastic Search on a Windows machine

  • Pre-requisites:
    • Java Runtime Environment (JRE)http://www.oracle.com/technetwork/java/javase/downloads/index.html
    • After installing JRE, do yourself a favor and set the JAVA___HOME environment variable
      • My Computer -> Properties -> Advanced tab -> Environment variables button
      • Under System Variables, click Add New
      • Name: JAVA___HOME
      • Path: c:\progra~1\Java\jre7
      • Assumes Java JRE is installed at c:\program files\Java\jre[X]
  • You can download the Elasticsearch runtime from the Elasticsearch website and follow the standard installation instructions (it's really quite simple)
  • OR, if you'd like the Elasticsearch engine to run as a Windows service, you can download an installer from here: http://ruilopes.com/elasticsearch-setup/
    • Basically, the installer just unzips the Elasticsearch package, then creates a Windows service wrapping the engine
    • Install to [DRIVE]:\Elasticsearch (avoid installing to the Program Files directory to help avoid UAC issues)
    • An Elasticsearch service will be created, but you may need to start the service manually (Administrative Tools -> Services) and set it to startup automatically
  • Confirm Elasticsearch is running browsing on http://localhost:9200/_cluster/health
    • If you receive a JSON response with a property named "status" that has a value of "green" or "yellow", all systems go.
    • By default, Elasticsearch listens on port 9200.

Preparing Document

Illustrative purposes, first we create a document called "character" in the index name "Disney".

An index is like a ‘database’ in a relational database. It has a mapping which defines multiple types. The easiest and most familiar layout clones what you would expect from a relational database. You can (very roughly) think of an index like a database.

  • SQL Server => Databases => Tables => Columns/Rows
  • Elasticsearch => Indices => Types => Documents with Properties

An Elasticsearch cluster can contain multiple Indices (databases), which in turn contain multiple Types (tables). These types hold multiple Documents (rows), and each document has Properties (columns).

Inserting data so if you want you could add more by using bulk insert method. Also, note that all the document in Elasticsearch is stored in JSON format. So you need to have a good grasp on JSON. We are using cURL commands to insert document values in Elasticsearch.

curl -XPUT "http://localhost:9200/disney/character/1" -d'
{
    "name": "Abby Mallard",
    "original_voice_actor": "Joan Cusack",
    "animated_debut": "Chicken Little"
}'
 
curl -XPUT "http://localhost:9200/disney/character/2" -d'
{
    "name": "Abigail Gabble",
    "original_voice_actor": "Monica Evans",
    "animated_debut": "The Aristocats"
}'
 
curl -XPUT "http://localhost:9200/disney/character/3" -d'
{
    "name": "Babyface Beagle",
    "original_voice_actor": "Terry McGovern",
    "animated_debut": "DuckTales"
}'

As we can see from the above data our document has three fields: "name", "original_voice_actor", "animated_debut" these are like the column name in the relational database. So, we would retrieve based on these three fields.

Developing the user end

Setting the app to connect to Elasticsearch

Connection to Elasticsearch is very simple. We can use the below code but first, we need to download the .NET client for Elasticsearch which is a NEST.

```

PM> Install-Package NEST

```

Then we use following code to connect to Elasticsearch. More info here.

var node = new  Uri("http://localhost:9200");
var settings = new  ConnectionSettings(node);
var client = new  ElasticClient(settings);

Then we need to define the response model of the field's datatype so that our program recognize which field's data and data type we are getting after making a search query. Remember response model is important, without it data won't get retrieve.

class disney
{
   public string  name { get; set; }
   public string  original_voice_actor { get; set; }
   public string  animated_debut { get; set; }
}

A Simple Query

Elasticsearch uses Query DSL based on JSON to define queries (more). So the LINQ query is used to retrieve document value is based on that.

var response = elasticClient.Search<disney>(s => s
  .Index("disney")
  .Type("character")
  .Query(q => q.QueryString(qs => qs.Query(tbxName.Text + "*"))));

You might have noticed that the Disney class that we created is set as the parameter in the Search<> API to hold the data that we get from running this query. The result we get from this query is also interesting. Because every result we get from this query is known as Hits in Elasticsearch a property of Search API. So, in order to show the result, we need to loop through the Hits one of the properties of "response" that we declared above. So, it will be like this:

foreach (var hit in response.Hits)
{
   hit.Source.name.ToString()
   hit.Source.original_voice_actor.ToString()
   hit.Source.animated_debut.ToString()
}

That's it you just only need the above step to make a WinForm app to use Elasticsearch. So the whole code will look like this:

User Interface

Backend Code

Getter Setter define class (disney.cs):

namespace ESsearchTest
{
    class disney
    {
        public string  name { get; set; }
        public string  original_voice_actor { get; set; }
        public string  animated_debut { get; set; }
    }
}

Main Code:

using Nest;
using System;
using System.Windows.Forms;
 
namespace ESsearchTest
{
    public partial  class Form1 : Form
    {
        ConnectionSettings connectionSettings;
        ElasticClient elasticClient;
 
        public Form1()
        {
            InitializeComponent();
        }
 
        private void  Form1_Load(object  sender, EventArgs e)
        {
            //Establishing connection with ES
            connectionSettings = new  ConnectionSettings(new Uri("http://localhost:9200/")); //local PC            
            elasticClient = new  ElasticClient(connectionSettings);
        }
 
        private void  tbxName_KeyPress(object sender, KeyPressEventArgs e)
        {
            if (e.KeyChar == '\b')
            {
                rtxSearchResult.Clear();
            }
        }
 
        //Retrieve information based on search textbox value
        private void  btnSearch_Click(object sender, EventArgs e)
        {
            //Search query to retrieve info
            var response = elasticClient.Search<disney>(s => s
                .Index("disney")
                .Type("character")
                .Query(q => q.QueryString(qs => qs.Query(tbxName.Text + "*"))));
 
            if (rtxSearchResult.Text != " ")
            {
                rtxSearchResult.Clear();
 
                foreach (var hit in response.Hits)
                {
                    rtxSearchResult.AppendText("Name: " + hit.Source.name.ToString()
                    + Environment.NewLine
                    + "Original_voice_actor: "  + hit.Source.original_voice_actor.ToString()
                    + Environment.NewLine
                    + "Animated_debut: "  + hit.Source.animated_debut.ToString()
                    + Environment.NewLine
                    + Environment.NewLine);
                }
            }
        }
    }
}

Now we completed implementing search and display part. Let’s start run our application and search some data and check the results.

You can download the Source Code from this link Download Source Code