Search Guard Installation for ELK Stack

Search Guard Installation and Concepts

Search Guard is an Open Source Elasticsearch plugin that offers encryption, authentication, and authorization and can be used to secure your Elasticsearch cluster by working with different industry standard authentication techniques such as Active Directory, LDAP, Kerberos, JSON web tokens and many more, and includes fine grained role-based access control to clusters, indices, documents and fields. It enables multi tenancy in Kibana, and stays compliant with GDPR, HIPAA, PCI, SOX and ISO by using audit logging.

With Community Edition Search Guard offers all basic security features for free. The Community Edition of Search Guard can be used for all projects, including commercial projects, at absolutely no cost. The Community Edition includes:

  •     Full data in transit encryption
  •     Node-to-node encryption
  •     Index level access control
  •     Document type based access control
  •     User-, role- and permission management
  •     HTTP basic authentication
  •     User Impersonation
  •     Proxy support

If you just want to use the free Community Edition, install Search Guard Enterprise and then disable all commercial features.

Basic workflow of Searchguard mechanism

  • A user wants to access an Elasticsearch cluster, for example by issuing a simple query.
  • Search Guard retrieves the user’s credentials from the request (they can be extracted from HTTP Basic Authentication headers, from a JSON web token or from a Kerberos ticket)
  • Search Guard authenticates the credentials against the configured authentication backend(s).
  • Search Guard authorizes the user by retrieving a list of the user’s roles from the configured authorization backend
  • Search Guard maps the user and backend roles to Search Guard roles.
  • Search Guard determines the permissions associated with the Search Guard role and decides whether the action the user wants to perform is allowed or not iewhich role,  can perform which action, against which cluster or index.
  • If Document- and Field-Level-Security is required you can also apply more fine grained permissions based on documents and individual fields.

Search Guard ships with pre-defined groups of permissions like READ, WRITE, SEARCH etc called action groups. An action group is an alias for a set of permissions. Action groups can be nested. For example, the following snippet shows two action groups, where the SUGGEST action group is referenced by the SEARCH action group:


  - "indices:data/read/search*"

  - "indices:data/read/msearch*"



  - "indices:data/read/suggest*"

Action groups can be used in the role configuration instead of or in combination with fine-grained permissions like indices:data/read/search*.

A first time installation of Search Guard on a cluster always requires a full cluster restart. TLS encryption is mandatory on the transport layer of Elasticsearch, and thus all nodes must have Search Guard installed in order to be able to talk to each other.

All configuration settings for Search Guard, such as users, roles and permissions, are stored as documents in a special Search Guard index. This index is secured so that only an admin user with a special SSL certificate may write or read this index. You can define one or more of these certificates, called admin certificates, in elasticsearch.yml.

The Search Guard configuration consists of the following files. These are shipped with Search Guard as templates.

sg_config.yml - configure authenticators and authorisation backends.
sg_roles.yml - define roles and the associated permissions.
sg_roles_mapping.yml - map backend roles, hosts and users to roles.
sg_internal_users.yml - stores users, roles and hashed passwords (hash with in the internal user database.
sg_action_groups.yml - define named permission groups.

Configuration settings are applied by pushing the content of one or more configuration files to the Search Guard secured cluster by using the sgadmin tool. For details, refer to the chapter sgadmin.

The installation Steps

  • Stop Elasticsearch
  • Install the Search Guard plugin
  • Generate or obtain TLS certificates
  • Add at least the TLS configuration to elasticsearch.yml
  • Restart Elasticsearch and check that the nodes come up
  • Configure authentication/authorization, users, roles and permissions by uploading the Search Guard configuration with sgadmin

Search Guard demo installer is an installation script that ships with Search Guard with certificates that you can use to run a PoC. This example uses the internal database for authentication.

Download Search Guard Elasticsearch plugin

[root@myhost bin]# wget
--2018-06-04 08:34:37--
Resolving (
Connecting to (||:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 19553206 (19M) [application/zip]
Saving to: ‘’

100%[===========================================================================================================================================================>] 19,553,206 226KB/s in 87s

2018-06-04 08:36:06 (219 KB/s) - ‘’ saved [19553206/19553206]

Install the plugin

[root@myhost bin]# ./elasticsearch-plugin install file:///usr/share/elasticsearch/bin/
-> Downloading file:///usr/share/elasticsearch/bin/
[=================================================] 100%  
@ WARNING: plugin requires additional permissions @
* /proc/sys/net/core/somaxconn read
* java.lang.RuntimePermission
* java.lang.RuntimePermission accessClassInPackage.sun.misc
* java.lang.RuntimePermission
* java.lang.RuntimePermission
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission accessUserInformation
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission shutdownHooks
* java.lang.reflect.ReflectPermission suppressAccessChecks
* getNetworkInformation
* getProxySelector
* * connect,accept,resolve
* getProperty.ssl.KeyManagerFactory.algorithm
* insertProvider.BC
* putProviderProperty.BC
* setProperty.ocsp.enable
* java.util.PropertyPermission * read,write
* doAs
* modifyPrivateCredentials
* * accept
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed search-guard-6
[root@myhost bin]#

Installing the demo configuration script

[root@myhost tools]# chmod a+x
[root@myhost tools]# ./
Search Guard 6 Demo Installer
** Warning: Do not use on production or public reachable systems **
Install demo certificates? [y/N] N
[root@bnghqsiem tools]# ./
Search Guard 6 Demo Installer
** Warning: Do not use on production or public reachable systems **
Install demo certificates? [y/N] y
Initialize Search Guard? [y/N] y
Cluster mode requires maybe additional setup of:
- Virtual memory (vm.max_map_count)

Enable cluster mode? [y/N] N
Basedir: /usr/share/elasticsearch
This script maybe require your root password for 'sudo' privileges
Elasticsearch install type: rpm/deb on CentOS Linux release 7.2.1511 (Core)
Elasticsearch config dir: /etc/elasticsearch
Elasticsearch config file: /etc/elasticsearch/elasticsearch.yml
Elasticsearch bin dir: /usr/share/elasticsearch/bin
Elasticsearch plugins dir: /usr/share/elasticsearch/plugins
Elasticsearch lib dir: /usr/share/elasticsearch/lib
Detected Elasticsearch Version: launchers-6.2.4
Detected Search Guard Version: 6.2.4-compliance-rc1

### Success
### Execute this script now on all your nodes and then start all nodes
### Search Guard will be automatically initialized.
### If you like to change the runtime configuration
### change the files in ../sgconfig and execute:
sudo /usr/share/elasticsearch/plugins/search-guard-6/tools/ -cd /usr/share/elasticsearch/plugins/search-guard-6/sgconfig -icl -key /etc/elasticsearch/kirk-key.pem -cert /etc/elasticsearch/kirk.pem -cacert /etc/elasticsearch/root-ca.pem -nhnv
### or run ./
### To use the Search Guard Configuration GUI see
### To access your Search Guard secured cluster open https://<hostname>:<HTTP port> and log in with admin/admin.
### (Ignore the SSL certificate warning because we installed self-signed demo certificates)
[root@myhost tools]# cd /etc/elasticsearch/
[root@myhost elasticsearch]# ls

Run the script for installing the demo configuration files

[root@myhost tools]# ls
hash.bat sgadmin.bat
[root@bnghqsiem tools]# ./
WARNING: JAVA_HOME not set, will use /bin/java
Search Guard Admin v6
Will connect to localhost:9300 ... done

*** This is a RC version and NOT suitable for production. ***
Elasticsearch Version: 6.2.4
Search Guard Version: 6.2.4-compliance-rc1
Connected as CN=kirk,OU=client,O=client,L=test,C=de
Contacting elasticsearch cluster 'elasticsearch' and wait for YELLOW clusterstate ...
Clustername: MY-SIEM
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
searchguard index already exists, so we do not need to create one.
Populate config from /usr/share/elasticsearch/plugins/search-guard-6/sgconfig/
Will update 'sg/config' with /usr/share/elasticsearch/plugins/search-guard-6/sgconfig/sg_config.yml
SUCC: Configuration for 'config' created or updated
Will update 'sg/roles' with /usr/share/elasticsearch/plugins/search-guard-6/sgconfig/sg_roles.yml
SUCC: Configuration for 'roles' created or updated
Will update 'sg/rolesmapping' with /usr/share/elasticsearch/plugins/search-guard-6/sgconfig/sg_roles_mapping.yml
SUCC: Configuration for 'rolesmapping' created or updated
Will update 'sg/internalusers' with /usr/share/elasticsearch/plugins/search-guard-6/sgconfig/sg_internal_users.yml
SUCC: Configuration for 'internalusers' created or updated
Will update 'sg/actiongroups' with /usr/share/elasticsearch/plugins/search-guard
SUCC: Configuration for 'actiongroups' created or updated
Done with success
[root@myhost tools]#

Download and install the Search Guard Kibana plugin

[root@myhost bin]# ./kibana-plugin install file:///usr/share/kibana/bin/
Attempting to transfer from file:///usr/share/kibana/bin/
Transferring 2958310 bytes....................
Transfer complete
Retrieving metadata from plugin archive
Extracting plugin archive
Extraction complete
Optimizing and caching browser bundles...
Plugin installation complete
[root@myhost bin]#

elasticsearch.yml settings

######## Start Search Guard Demo Configuration ########
# WARNING: revise all the lines below before you go into production
searchguard.ssl.transport.pemcert_filepath: esnode.pem
searchguard.ssl.transport.pemkey_filepath: esnode-key.pem
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: esnode.pem
searchguard.ssl.http.pemkey_filepath: esnode-key.pem
searchguard.ssl.http.pemtrustedcas_filepath: root-ca.pem
searchguard.allow_unsafe_democertificates: true
searchguard.allow_default_init_sgindex: true
- CN=kirk,OU=client,O=client,L=test, C=de

searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: ["sg_all_access"]
cluster.routing.allocation.disk.threshold_enabled: false
discovery.zen.minimum_master_nodes: 1
node.max_local_storage_nodes: 3
searchguard.enterprise_modules_enabled: false
######## End Search Guard Demo Configuration ########

kibana.yml settings

# Use HTTPS instead of HTTP
elasticsearch.url: "https://localhost:9200"
# # Configure the Kibana internal server user
elasticsearch.username: "kibanaserver"
elasticsearch.password: "kibanaserver"
# # Disable SSL verification because we use self-signed demo certificates
elasticsearch.ssl.verificationMode: none
# # Whitelist the Search Guard Multi Tenancy Header
elasticsearch.requestHeadersWhitelist: [ "Authorization", "sgtenant" ]

Internal user database installed by the demo installer script. All the passwords are hashed. You can use to hash passwords.

[root@myhost tools]# vim /usr/share/elasticsearch/plugins/search-guard-6/sgconfig/sg_internal_users.yml
# This is the internal user database
# The hash value is a bcrypt hash and can be generated with plugin/tools/

#password is: admin
readonly: true
hash: $2a$12$VcCDgh2NDk07JGN0rjGbM.Ad41qVR/YFJcgHp0UGns5JDymv..TOG
- admin
#no dots allowed in attribute names
attribute1: value1
attribute2: value2
attribute3: value3

#password is: logstash
hash: $2a$12$u1ShR4l4uBS3Uv59Pa2y5.1uQuZBrZtmNfqB3iM/.jL0XoV9sghS2
- logstash

#password is: kibanaserver
readonly: true
hash: $2a$12$4AcgAt3xwOWadA5s5blL6ev39OXDNhmOesEoo33eZtrq2N0YrU3H.

#password is: kibanaro
hash: $2a$12$JJSXNfTowz7Uu5ttXfeYpeYE0arACvcwlPBStB1F.MI7f0U9Z4DGC
- kibanauser
- readall

#password is: readall
hash: $2a$12$ae4ycwzwvLtZxwZ82RmiEunBbIPiAmGZduBAjKN0TXdwQFtCwARz2
#password is: readall
- readall

#password is: snapshotrestore
hash: $2y$12$DpwmetHKwgYnorbgdvORCenv4NAK8cPUg8AI6pxLCuWf/ALc0.v7W
- snapshotrestore

Roles configuratin

[root@myhost tools]# vim /usr/share/elasticsearch/plugins/search-guard-6/sgconfig/sg_roles.yml

# For logstash and beats
- indices:admin/template/get
- indices:admin/template/put

Logstash output configuration

Make sure that the url is modified with https and credentials are givem

output {
if [indexType]== "WIN-LOGS"
elasticsearch {
user => logstash
password => logstash
ssl => true
ssl_certificate_verification => false
hosts => "https://localhost:9200"
manage_template => false
#index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => doc
else if [indexType]== "SYS-LOGS"
elasticsearch {
user => logstash
password => logstash
ssl => true
ssl_certificate_verification => false
hosts => "https://localhost:9200"
manage_template => false
index => "syslog-%{+YYYY.MM.dd}"
document_type => doc

Seach Guard Community License shows “No license required because enterprise modules not enabled.” message

{"_nodes":{"total":1,"successful":1,"failed":0},"cluster_name":"MY-SIEM","sg_license":{"msgs":["No license required because enterprise modules not enabled."],"license_required":false},"modules":{"INTERNAL_USERS_AUTHENTICATION_BACKEND":

Kibana Login page