These steps are for upgrading ELK 5.5.1 to 5.6.9 on ubuntu 16.04 and then installing the X-Pack addon. Repeat for each node in the cluster one by one.
Upgrading ELK 5.5.1 to 5.6.9
Disable shard allocation
# curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d' { "transient": { "cluster.routing.allocation.enable": "none" } } ' {"acknowledged":true,"persistent":{},"transient":{"cluster":{"routing":{"allocation":{"enable":"all"}}}}}
Stop non-essential indexing and perform a synced flush (Optional)
# curl -X POST "localhost:9200/_flush/synced"
Stop and upgrade a single node
# service elasticsearch stop # ps -ef | grep [e]lastic # apt-cache madison elasticsearch elasticsearch | 5.6.9 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.6.8 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.6.7 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.6.6 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.6.5 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.6.4 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.6.3 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.6.2 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.6.1 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.6.0 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.5.3 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.5.2 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.5.1 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.5.0 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.4.3 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.4.2 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.4.1 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.4.0 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.3.3 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.3.2 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.3.1 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.3.0 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.2.2 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.2.1 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.2.0 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.1.2 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.1.1 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.0.2 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.0.1 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 5.0.0 | https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 Packages elasticsearch | 1.7.3+dfsg-3 | http://azure.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages elasticsearch | 1.7.3+dfsg-3 | http://azure.archive.ubuntu.com/ubuntu xenial/universe Sources # apt-get upgrade elasticsearch=5.6.9 # dpkg -l | grep elasticsearch ii elasticsearch 5.6.9 all Elasticsearch is a distributed RESTful search engine built for the cloud. Reference documentation can be found at https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html and the 'Elasticsearch: The Definitive Guide' book can be found at https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html ii elasticsearch-curator 5.5.3 amd64 Have indices in Elasticsearch? This is the tool for you! Like a museum curator manages the exhibits and collections on display, Elasticsearch Curator helps you curate, or manage your indices. # dpkg -l | grep logstash ii logstash 1:5.6.9-1 all An extensible logging pipeline # dpkg -l | grep kibana ii kibana 5.6.9 amd64 Explore and visualize your Elasticsearch data # service elasticsearch start # ps -ef | grep [e]lastic elastic+ 47382 1 99 12:39 ? 00:00:32 /usr/bin/java -Xms8g -Xmx8g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet -Edefault.path.logs=/var/log/elasticsearch -Edefault.path.data=/var/lib/elasticsearch -Edefault.path.conf=/etc/elasticsearch
wait a bit….
.
.
.
Start the upgraded node
# curl -X GET "localhost:9200/_cat/nodes" 123.456.78.90 2 98 15 0.66 1.32 1.97 mdi - mygroovyelk-1 123.456.78.91 92 44 1.65 1.90 2.27 mdi * mygroovyelk-2
Re-enable shard allocation
# curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d' { "transient": { "cluster.routing.allocation.enable": "all" } } ' {"acknowledged":true,"persistent":{},"transient":{"cluster":{"routing":{"allocation":{"enable":"all"}}}}}
Wait for the node to recover
# curl -X GET "localhost:9200/_cat/health" 1527079301 12:41:41 milogging1 yellow 2 2 257 247 0 2 235 1 - 52.0% # curl -X GET "localhost:9200/_cat/health" 1527079312 12:41:52 milogging1 yellow 2 2 266 247 0 2 226 0 - 53.8% # curl -X GET "localhost:9200/_cat/health" 1527079346 12:42:26 milogging1 yellow 2 2 287 247 0 2 205 0 - 58.1% # curl -X GET "localhost:9200/_cat/health" 1527079845 12:50:45 milogging1 yellow 2 2 488 247 0 2 4 0 - 98.8% # curl -X GET "localhost:9200/_cat/health" 1527079994 12:53:14 milogging1 yellow 2 2 490 247 0 2 2 0 - 99.2% # curl -X GET "localhost:9200/_cat/health" 1527080383 12:59:43 milogging1 green 2 2 494 247 0 0 0 0 - 100.0% # curl -X GET "localhost:9200/_cat/health?v" epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent 1527080380 12:59:40 milogging1 green 2 2 494 247 0 0 0 0 - 100.0%
Restart kibana and logstash
# service kibana restart # service logstash restart
Repeat for the next node in the cluster. Once all nodes have been upgraded, you can install x-pack on them.
=========================================================================================================================================================
Once again, repeat for each node in the cluster one by one.
Install X-Pack 5.6.9 for Elasticsearch
# curl "localhost:9200/_nodes/settings?pretty=true" | grep home "home" : "/usr/share/elasticsearch" # cd /usr/share/elasticsearch/ # bin/elasticsearch-plugin install x-pack --batch -> Downloading x-pack from elastic [=================================================] 100% @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: plugin requires additional permissions @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ * java.io.FilePermission \.pipe* read,write * java.lang.RuntimePermission accessClassInPackage.com.sun.activation.registries * java.lang.RuntimePermission getClassLoader * java.lang.RuntimePermission setContextClassLoader * java.lang.RuntimePermission setFactory * java.net.SocketPermission * connect,accept,resolve * java.security.SecurityPermission createPolicy.JavaPolicy * java.security.SecurityPermission getPolicy * java.security.SecurityPermission putProviderProperty.BC * java.security.SecurityPermission setPolicy * java.util.PropertyPermission * read,write * javax.net.ssl.SSLPermission setHostnameVerifier See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html for descriptions of what these permissions allow and the associated risks. @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: plugin forks a native controller @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ This plugin launches a native controller that is not subject to the Java security manager nor to system call filters. -> Installed x-pack # service elasticsearch restart
You should now get a 401 response when using the elastic api
# curl -X GET "localhost:9200/_cat/health?v" {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/_cat/health?v]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}}],"type":"security_exception","reason":"missing authentication token for REST request [/_cat/health?v]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}},"status":401}
Install X-Pack 5.6.9 for Kibana
# cd /usr/share/kibana/ # sudo -u kibana bin/kibana-plugin install x-pack Attempting to transfer from x-pack Attempting to transfer from https://artifacts.elastic.co/downloads/kibana-plugins/x-pack/x-pack-5.6.9.zip Transferring 119595626 bytes.................... Transfer complete Retrieving metadata from plugin archive Extracting plugin archive Extraction complete Optimizing and caching browser bundles...Install X-Pack 5.6.9 for Logstash
# cd /usr/share/logstash/ # bin/logstash-plugin install x-pack Downloading file: https://artifacts.elastic.co/downloads/logstash-plugins/x-pack/x-pack-5.6.9.zip Downloading [=============================================================] 100% Installing file: /tmp/studtmp-9676d592993622aeb4bb6942519f37e19c147f9e9f745ee32dffd155c161/x-pack-5.6.9.zip Install successfulReset the logstash_system user’s password so that you can set it in the config
curl -XPUT -u elastic 'localhost:9200/_xpack/security/user/logstash_system/_password?pretty' -H 'Content-Type: application/json' -d' { "password": "mygroovypassword" } ' Enter host password for user 'elastic': { }By default the elastic user’s password is changeme, but you should change this, just like changing the logstash_user’s password to something else.
Add the elasticsearch basic auth to the logstash config, with the logstash_system user here…
# vim /etc/logstash/logstash.yml xpack.monitoring.enabled: true xpack.monitoring.elasticsearch.url: http://localhost:9200 xpack.monitoring.elasticsearch.username: logstash_system xpack.monitoring.elasticsearch.password: mygroovypassword…and the elastic user here
# vim /etc/logstash/conf.d/99_output_elasticsearch.conf output { if "beats" in [tags] { elasticsearch { hosts => ["http://localhost:9200"] user => "elastic" password => "mygroovypassword" index => "ls-%{[fields][program]}-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}" } } if "healthcheck" in [tags] { elasticsearch { hosts => ["http://localhost:9200"] user => "elastic" password => "mygroovypassword" index => "ls-healtcheck-%{+YYYY.MM.dd}" } } }Restart Logstash
# service logstash restartGo to kibana in your browser and you’ll be shown a login page (if you didn’t have auth enabled before), and some new menus granted by X-Pack. Happy elking.