Architecting Splunk Deployment

Architecting Splunk Deployment

• Scale Splunk Enterprise functionality to handle the data needs for enterprises of any size and complexity.
• Access diverse or dispersed data sources.
• Achieve high availability and ensure disaster recovery with data replication and multisite deployment.

Splunk Enterprise performs three key functions as it processes data:

1. It ingests data from files, the network, or other sources.
2. It parses and indexes the data.
3. It runs searches on the indexed data.

Types of processing components

There are three main types of processing components:

• Forwarders
• Indexers
• Search heads
• Data input.Data enters the system through forwarders, which consume external data, perform a small amount of preprocessing on it, and then forward the data to the indexers.
• Two or three indexers receive, index, and store incoming data from the forwarders. The indexers also search that data, in response to requests from the search head. The indexers reside on dedicated machines.
• Search management.A single search head performs the search management function. It handles search requests from users and distributes the requests across the set of indexers, which perform the actual searches on their local data. The search head then consolidates the results from all the indexers and serves them to the users.

An Indexer cluster is a group of Splunk Enterprise indexers that are configured to replicate each other’s’ data, so that the system keeps multiple copies of all data.

• They include a capability to coordinate configuration updates easily across all indexers in the cluster.
• They include a built-in distributed search capability.
• They feature indexer discovery, which enables the set of forwarders to automatically load-balance across all indexers in the cluster.

The data pipeline has these segments:

• Input
• Paring
• Indexing
• Search

The correspondence between the three typical processing tiers and the four data pipeline segments is this:

• The data input tier handles the input segment.
• The indexing tier handles the parsing and indexing segments.
• The search management tier handles the search segment.

Types of management components

A deployment usually includes one or more of these management components:

• The license master  handles Splunk Enterprise licensing.
• The monitoring console performs centralized monitoring of the entire deployment.
• The deployment server  updates configurations and distributes apps to processing components, primarily forwarders.
• The “cluster master”, coordinates the activities of an indexer cluster. It also handles updates for indexer clusters.
• The search head cluster deployer handles updates for search head cluster.

These are some of the main types of deployments, based on size:

• A single instance that combines indexing and search management functions.
• Small enterprise ->   One search head with two or three indexers.
• Medium enterprise -> A small search head cluster, with several indexers.
• Large enterprise -> A large search head cluster, with large numbers of indexers.