Elasticsearch: Difference between revisions

From DWIKI
mNo edit summary
 
(14 intermediate revisions by the same user not shown)
Line 1: Line 1:
=Links=
=Links=
*[https://opster.com/guides/elasticsearch/glossary/elasticsearch-threshold/ Elasticsearch DiskThreshold]
=Documentation=
==Configuration==
*http://jprante.github.io/2012/11/28/Elasticsearch-Java-Virtual-Machine-settings-explained.html
*http://jprante.github.io/2012/11/28/Elasticsearch-Java-Virtual-Machine-settings-explained.html
==Heap==
*https://www.elastic.co/blog/a-heap-of-trouble
*https://www.elastic.co/blog/a-heap-of-trouble
==Garbage collection==
*[https://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/geninfo/diagnos/garbage_collect.html Garbage collect]
*[https://sematext.com/blog/java-garbage-collection/#how-does-java-garbage-collection-work Garbage collection]
*[https://opster.com/guides/elasticsearch/capacity-planning/elasticsearch-heap-size-usage/ Elasticsearch Heap Size Usage and JVM Garbage Collection]
Ideally:
*Young GC is processed quickly (within 50 ms).
*Young GC is not frequently executed (about 10 seconds).
*Old GC is processed quickly (within 1 second).
*Old GC is not frequently executed (once per 10 minutes or more).
=HOWTO=
==Adjust log rotation==
Is done in '''/etc/elasticsearch/log4j.properties'''
===Number of logfiles to keep===
appender.rolling.strategy.max
==Start elasticsearch in safe mode==
su -s /bin/bash elasticsearch -c "usr/share/elasticsearch/bin/elasticsearch -E node.ingest=false -E node.master=false -E node.data=false"
=Monitoring elasticsearch=
=Monitoring elasticsearch=
== Things to monitor ==
=== heap size ===
*node jvm.mem.heap_used_percent
=== garbage collection ===
Something like
node.jvm.uptime_in_millis/node.jvm.gc.collectors.young.collection_count
*[https://www.datadoghq.com/blog/monitor-elasticsearch-performance-metrics/ How to monitor Elasticsearch performance]
*[https://www.datadoghq.com/blog/monitor-elasticsearch-performance-metrics/ How to monitor Elasticsearch performance]
*[https://sematext.com/blog/top-10-elasticsearch-metrics-to-watch/ Top 10 Elasticsearch Metrics to Monitor]
*[https://sematext.com/blog/top-10-elasticsearch-metrics-to-watch/ Top 10 Elasticsearch Metrics to Monitor]
*[https://www.datadoghq.com/blog/monitor-elasticsearch-performance-metrics/#key-elasticsearch-performance-metrics-to-monitor Performance metrics to monitor]
*[https://www.datadoghq.com/blog/monitor-elasticsearch-performance-metrics/#key-elasticsearch-performance-metrics-to-monitor Performance metrics to monitor]
*[https://opster.com/guides/elasticsearch/capacity-planning/elasticsearch-heap-size-usage/ Heap Size Usage]
*[https://opster.com/guides/elasticsearch/capacity-planning/elasticsearch-heap-size-usage/ Heap Size Usage]
==Query the web interface==
*'cluster': 'http://localhost:9200/_cluster/stats',
*    'nodes'  : 'http://localhost:9200/_nodes/stats',
*    'indices': 'http://localhost:9200/_stats',
*    'health' : 'http://localhost:9200/_cluster/health'


=Installing Elasticsearch=
=Installing Elasticsearch=
Line 19: Line 62:
==Percolation==
==Percolation==
https://spinscale.de/posts/2021-09-15-understanding-elasticsearch-percolate-query.html
https://spinscale.de/posts/2021-09-15-understanding-elasticsearch-percolate-query.html
==now throttling indexing for shard: segment writing can't keep up==
*[https://opster.com/analysis/elasticsearch-now-throttling-indexing-for-shard-segment-writing-can-t-keep-up/ one doc about this]


=FAQ=
=FAQ=
Line 24: Line 71:
*[http://chrissimpson.co.uk/elasticsearch-yellow-cluster-status-explained.html Yellow cluster status explained]
*[http://chrissimpson.co.uk/elasticsearch-yellow-cluster-status-explained.html Yellow cluster status explained]
and look for [https://www.datadoghq.com/blog/elasticsearch-unassigned-shards/ unassigned shards]
and look for [https://www.datadoghq.com/blog/elasticsearch-unassigned-shards/ unassigned shards]
==Update ingest-attachment==
/usr/share/elasticsearch/bin/elasticsearch-plugin remove ingest-attachment
/usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-attachment
==Get version of active elasticsearch==
wget -q -O - http://localhost:9200/_cluster/stats
or
wget -q -O - http://localhost:9200/_cluster/stats | sed 's/^.*versions\":\[\"\(.*\)\"\].*$/\1\n/g'
==Error messages==
===org.elasticsearch.ElasticsearchException: Failed to bind service===
?

Latest revision as of 11:07, 23 October 2024

Links

Documentation

Configuration

Heap

Garbage collection

Ideally:

  • Young GC is processed quickly (within 50 ms).
  • Young GC is not frequently executed (about 10 seconds).
  • Old GC is processed quickly (within 1 second).
  • Old GC is not frequently executed (once per 10 minutes or more).

HOWTO

Adjust log rotation

Is done in /etc/elasticsearch/log4j.properties

Number of logfiles to keep

appender.rolling.strategy.max


Start elasticsearch in safe mode

su -s /bin/bash elasticsearch -c "usr/share/elasticsearch/bin/elasticsearch -E node.ingest=false -E node.master=false -E node.data=false"

Monitoring elasticsearch

Things to monitor

heap size

  • node jvm.mem.heap_used_percent

garbage collection

Something like 

node.jvm.uptime_in_millis/node.jvm.gc.collectors.young.collection_count

Query the web interface

Installing Elasticsearch

On Debian

deb https://artifacts.elastic.co/packages/oss-6.x/apt stable main

packages is called elasticsearch-oss now?

Terminology

shards

"blocks" of index data distributed over the nodes

Percolation

https://spinscale.de/posts/2021-09-15-understanding-elasticsearch-percolate-query.html


now throttling indexing for shard: segment writing can't keep up

FAQ

Status is yellow

and look for unassigned shards

Update ingest-attachment

/usr/share/elasticsearch/bin/elasticsearch-plugin remove ingest-attachment
/usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-attachment


Get version of active elasticsearch

wget -q -O - http://localhost:9200/_cluster/stats

or 

wget -q -O - http://localhost:9200/_cluster/stats | sed 's/^.*versions\":\[\"\(.*\)\"\].*$/\1\n/g'


Error messages

org.elasticsearch.ElasticsearchException: Failed to bind service

?

Retrieved from "https://wiki.dhits.nl/index.php?title=Elasticsearch&oldid=9210"