9. Developing Elasticsearch Plugins
by Saurabh Chhajed, Marek Rogoziński, Rafał Kuć, Bharvi Dixit
Elasticsearch: A Complete Guide
- Elasticsearch: A Complete Guide
- Table of Contents
- Elasticsearch: A Complete Guide
- Elasticsearch: A Complete Guide
- Credits
- Preface
- 1. Module 1
- 1. Getting Started with Elasticsearch
- Introducing Elasticsearch
- Installing and configuring Elasticsearch
- Basic operations with Elasticsearch
- Summary
- 2. Understanding Document Analysis and Creating Mappings
- 3. Putting Elasticsearch into Action
- CRUD operations using elasticsearch-py
- CRUD operations using Java
- Creating a search database
- Elasticsearch Query-DSL
- Understanding Query-DSL parameters
- Search requests using Python
- Search requests using Java
- Sorting your data
- Document routing
- Summary
- 4. Aggregations for Analytics
- 5. Data Looks Better on Maps: Master Geo-Spatiality
- 6. Document Relationships in NoSQL World
- 7. Different Methods of Search and Bulk Operations
- 8. Controlling Relevancy
- 9. Cluster Scaling in Production Deployments
- Node types in Elasticsearch
- Introducing Zen-Discovery
- Multicasting discovery
- Unicasting discovery
- Configuring unicasting discovery
- Minimum number of master nodes: preventing split-brain
- 10. Backups and Security
- II. Module 2
- 1. Introduction to Elasticsearch
- 2. Power User Query DSL
- Default Apache Lucene scoring explained
- Query rewrite explained
- Query templates
- Handling filters and why it matters
- Choosing the right query for the job
- Query categorization
- The use cases
- Summary
- 3. Not Only Full Text Search
- Query rescoring
- Controlling multimatching
- Significant terms aggregation
- Documents grouping
- Relations between documents
- Scripting changes between Elasticsearch versions
- Summary
- 4. Improving the User Search Experience
- Correcting user spelling mistakes
- Improving the query relevance
- Summary
- 5. The Index Distribution Architecture
- 6. Low-level Index Control
- Altering Apache Lucene scoring
- Choosing the right directory implementation – the store module
- NRT, flush, refresh, and transaction log
- Segment merging under control
- When it is too much for I/O – throttling explained
- Understanding Elasticsearch caching
- The filter cache
- The field data cache
- The shard query cache
- Using circuit breakers
- Clearing the caches
- Index, indices, and all caches clearing
- Summary
- 7. Elasticsearch Administration
- Discovery and recovery modules
- The human-friendly status API – using the Cat API
- Backing up
- Federated search
- Summary
- 8. Improving Performance
- Using doc values to optimize your queries
- Knowing about garbage collector
- Benchmarking queries
- Very hot threads
- Scaling Elasticsearch
- Vertical scaling
- Horizontal scaling
- Using Elasticsearch for high load scenarios
- Summary
- 9. Developing Elasticsearch Plugins
- Creating the Apache Maven project structure
- Understanding the basics
- Creating custom REST action
- Creating the custom analysis plugin
- Implementation details
- Implementing TokenFilter
- Implementing the TokenFilter factory
- Implementing the class custom analyzer
- Implementing the analyzer provider
- Implementing the analysis binder
- Implementing the analyzer indices component
- Implementing the analyzer module
- Implementing the analyzer plugin
- Informing Elasticsearch about our custom analyzer
- Testing our custom analysis plugin
- Implementation details
- Summary
- III. Module 3
- 1. Introduction to ELK Stack
- The need for log analysis
- Challenges in log analysis
- The ELK Stack
- ELK data pipeline
- ELK Stack installation
- Installing Elasticsearch
- Running Elasticsearch
- Elasticsearch configuration
- Elasticsearch plugins
- Installing Logstash
- Running Logstash
- Logstash with file input
- Logstash with Elasticsearch output
- Configuring Logstash
- Installing Logstash forwarder
- Logstash plugins
- Installing Kibana
- Configuring Kibana
- Running Kibana
- Kibana interface
- Summary
- 2. Building Your First Data Pipeline with ELK
- 3. Collect, Parse and Transform Data with Logstash
- Configuring Logstash
- Logstash plugins
- Summary
- 4. Creating Custom Logstash Plugins
- 5. Why Do We Need Elasticsearch in ELK?
- 6. Finding Insights with Kibana
- 7. Kibana – Visualization and Dashboard
- 8. Putting It All Together
- 9. ELK Stack in Production
- 10. Expanding Horizons with ELK
- A. Bibliography
- 1. Introduction to ELK Stack
- Index
- Configuring unicasting discovery
- 1. Getting Started with Elasticsearch
In the previous chapter, we were focused on the performance and scaling of our Elasticsearch clusters. We looked at how doc values can help us improve query performance and lower the memory for queries, which deals with field data cache at the cost of slightly slower indexing. We looked at how garbage collector works and what to look at when changing its configuration. We've benchmarked our queries, and we've seen what Hot Threads API gives us. Finally, we discussed how to scale Elasticsearch. By the end of this chapter, you will have learned:
- How to set up the Apache Maven project for Elasticsearch plugins' development
- How to develop a custom REST action plugin
- How to develop a custom analysis plugin extending Elasticsearch analysis capabilities
Before we start with showing you how to develop a custom Elasticsearch plugin, we would like to discuss a way to package it so that it can be installed by Elasticsearch using the plugin command. In order to do that, we will use Apache Maven (http://maven.apache.org/), which is designed to simplify software projects' management. It aims to make your build process easier, provide a unifying build system, manage dependencies, and so on.
Also remember that the book you are holding in your hands is not about Maven but Elasticsearch, and we will keep Maven-related information to the required minimum.
Note
Installing Apache Maven is a straightforward task; we assume that you already have it installed. However, if you have problems with it, please consult http://maven.apache.org/ for more information.
-
No Comment