A year ago I stumbled across something new which sounded promising. Meanwhile we are using it in several departments. I’m talking about Elastic.Just in case you haven’t heard about it Elastic is a company with offices world-wide which provides a stack of applications called Elastic stack formally known as ELK. ELK stands for Elasticsearch Logstash and Kibana:
Elasticsearch
Elasticsearch is a distributed, open source search and analytics engine, designed for horizontal scalability, reliability, and easy management. It combines the speed of search with the power of analytics via a sophisticated, developer-friendly query language covering structured, unstructured, and time-series data. [Elastic]
Logstash
Logstash is a flexible, open source data collection, enrichment, and transportation pipeline. With connectors to common infrastructure for easy integration, Logstash is designed to efficiently process a growing list of log, event, and unstructured data sources for distribution into a variety of outputs, including Elasticsearch. [Elastic]
Kibana
Kibana is an open source data visualization platform that allows you to interact with your data through stunning, powerful graphics. From histograms to geomaps, Kibana brings your data to life with visuals that can be combined into custom dashboards that help you share insights from your data far and wide. [Elastic]
They have stacked up even more their portfolio with things like Beats:
Beats
Beats are the future of open source data shippers for Elasticsearch. From network packets to log files, infrastructure data, and beyond, Beats is the platform for building a variety of lightweight shippers to help you leverage any type of data you like. [Elastic]
To bring it into one sentence Elastic is a stack where you can feed any kind of log files into a search engine and create visual reports from it. Which makes it real a all-in-one tools for a search engine, logging, monitoring, alerting, business intelligence & analytics. No wonder that several other companies using Elastic to create new products and services.
If you wanna quickly try out yourself you should download a version of Elasticsearch, Kibana and Topbeat. Just simply unzip them all into separate directories and then start:
elasticsearch-2.3.3\bin\elasticsearch.bat
To see a status of your Elasticsearch check on http://localhost:9200/
It should return something like
{
"name": "Marvel Man",
"cluster_name": "elasticsearch",
"version": {
"number": "2.3.3",
"build_hash": "218bdf10790eef486ff2c41a3df5cfa32dadcfde",
"build_timestamp": "2016-05-17T15:40:04Z",
"build_snapshot": false,
"lucene_version": "5.5.0"
},
"tagline": "You Know, for Search"
}
You can start now feeding data via a Beats:
topbeat-1.2.3\topbeat.exe
Now you can start up Kibana:
kibana-4.5.1\bin\kibana.bat
Kibana can be found under http://localhost:5601/app/kibana. If you then go to “Settings / Indices” you can create an index pattern “topbeat-*”. Under “Discover” and the right index (pattern) selected you’ll see the data send to Elasticsearch.
Under “Visualize” you can now play around and create your own charts …