How to setup multi-site Search Head cluster

To deploy a multisite cluster, you configure the set of nodes for each site:

  • A single master resides on one of the sites and controls the entire multisite cluster.
  • A set of peer nodes resides on each site.
  • A search head resides on each site that searches cluster data.
  • If you want all searches to be local, you must install a search head on each site.
  •  

    Site 1                          New York City

     

    Splunk Instance Role IP Listing Port Replication Port
    Deployment Server 10.20.9.5

     

     

    CLUSTER-MASTER

    Also act  License Master

    10.20.9.11
    Site1-idx1 10.20.9.15 9997 9887
    Site1-idx2 10.20.9.16 9997 9887
    Site1-sh1 10.20.9.31 9887
    Site1-sh2 10.20.9.32 9887
     

    Site 2                   Las Vegas

     

    Splunk Instance Role IP Listing Port Replication Port
    Site2-idx1 10.10.9.15 9997 9887
    Site2-idx2 10.10.9.16 9997 9887
    Site2-sh1 10.10.9.31 9887
    Site2-sh2 10.10.9.32 9887

 

Search Head    SITE 1 New York City       Site1-sh1  (10.20.9.31)

/opt/splunk/bin/splunk edit licenser-localslave –master_uri http://10.20.9.11:8089

/opt/splunk/bin/splunk restart

This is instance license status

/opt/splunk/bin/splunk  licenser-localsalve

/opt/splunk/bin/splunk edit cluster-confg –master_uri http://10.20.9.5:8089 –mode searchhead –site site1 –replication_port 9887 –secret My_First_Splunk_Multi-site_Cluster

/opt/splunk/bin/splunk restart

 

Search Head  SITE 1 New York City       Site1-sh2  (10.20.9.32) 

/opt/splunk/bin/splunk edit licenser-localslave –master_uri http://10.20.9.11:8089

/opt/splunk/bin/splunk restart

This is instance license status

/opt/splunk/bin/splunk  licenser-localsalve

/opt/splunk/bin/splunk edit cluster-confg –master_uri http://10.20.9.11:8089 –mode searchhead –site site1 –replication_port 9887 –secret My_First_Splunk_Multi-site_Cluster

/opt/splunk/bin/splunk restart

 

Search Head  SITE 2 Las Vegas    Site2-sh1  (10.10.9.31)

/opt/splunk/bin/splunk edit licenser-localslave –master_uri http://10.20.9.11:8089

/opt/splunk/bin/splunk restart

This is instance license status

/opt/splunk/bin/splunk  licenser-localsalve

/opt/splunk/bin/splunk edit cluster-confg –master_uri http://10.20.9.11:8089 –mode searchhead –site site2 –replication_port 9887 –secret My_First_Splunk_Multi-site_Cluster

/opt/splunk/bin/splunk restart

 

Search Head    Peer SITE 2 Las Vegas   Site2-idx2  (10.10.9.32)

/opt/splunk/bin/splunk edit licenser-localslave –master_uri http://10.20.9.11:8089

/opt/splunk/bin/splunk restart

This is instance license status

/opt/splunk/bin/splunk  licenser-localsalve

/opt/splunk/bin/splunk edit cluster-confg –master_uri http://10.20.9.11:8089 –mode searchhead –site site2 –replication_port 9887 –secret My_First_Splunk_Multi-site_Cluster

/opt/splunk/bin/splunk restart

 

Site1-sh1

Bootstrap site1-sh1 to be the initial captain with the splunk bootstrap shcluster-captain

/opt/splunk/bin/splunk bootstrap shcluster-captain –servers_list http://10.20.9.31:8089,  http://10.20.9.32:8089, http://10.10.9.31:8089, http://10.10.9.32:8089

/opt/splunk/bin/splunk show shcluster-status

/opt/splunk/bin/splunk rolling-restart shcluster-members

Wait couple of minute

/opt/splunk/bin/splunk show shcluster-status

/opt/splunk/bin/splunk edit shcluster-config –shcluster_label cluster1

From captain

index=_internal sourcetype=splunkd  component=SHPRaftConsensus | reverse

 

Installed Splunk Enterprise on all the above host in table using the below commands.

wget -O splunk-7.0.1-2b5b15c4ee89-Linux-x86_64.tgz ‘https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.0.1&product=splunk&filename=splunk-7.0.1-2b5b15c4ee89-Linux-x86_64.tgz&wget=true’

sudo tar zxf splunk-7.0.1-2b5b15c4ee89-Linux-x86_64.tgz -C /opt

sudo chown -R splunker:splunk /opt/splunk

sudo ls -l /opt/splunk

sudo su – splunker

sudo /opt/splunk/bin/splunk start –accept-license –no-prompt -answer

sudo /opt/splunk/bin/splunk enable boot-start -user splunker

sudo /opt/splunk/bin/splunk status

/opt/splunk/bin/splunk show web-port -auth admin:changeme

/opt/splunk/bin/splunk show splunkd-port -auth admin:changeme

/opt/splunk/bin/splunk show appserver-ports -auth admin:changeme

/opt/splunk/bin/splunk show kvstore-port -auth admin:changeme

/opt/splunk/bin/splunk show servername -auth admin:changeme

/opt/splunk/bin/splunk show default-hostname -auth admin:changeme

netstat -tuplen | grep splunkd

netstat –an | grep 8000

 

 

 

 

Please follow and like us: