Deploying Splunk on AWS
When deploying a non-clustered environment, either single-server or distributed, we recommend utilizing EBS volumes and EBS-optimized instance types.
- An EBS volume is persistent, even in the case of instance termination or crash. Each of these options come with their own feature set and price points. Exact storage selection will depend on a customer’s desired requirements and price sensitivity.
- When using Splunk premium solutions, such as Splunk Enterprise Security (ES) or Splunk® IT Service Intelligence (ITSI), we recommend indexer instance types with a larger memory footprint. The following assumptions also assume EBS gp2 volumes attached.
- Splunk provides an add-on that will automate ingest data from several AWS services, including Config and Config Rules, CloudTrail, CloudWatch, Inspector, CloudFront and more. Installing and configuring the add-on will allow data collection from any of the services a user selects directly to the user’s Splunk Enterprise deployment.
The assumptions below assume EBS gp2 volumes attached to the c4 instance types. The d2 instance types come with instance storage, so EBS is not required. In all situations, deploying on dedicated hosts to avoid potentially noisy neighbor situations is recommended.
| Indexers | ||
| Instance Type | Daily Indexing Volumes (GB) | |
| (c4 or d2).2xlarge | < 100 | |
| (c4 or d2).4xlarge | 100 – 200 | |
| (c4 or d2).8xlarge | 200 – 300 + | |
| Search Heads | ||
| Instance Type | Concurrent Users | Performance |
| (r4.4xlarge | Up to 8 | Good |
| (r4.8xlarge) | Up to 16 | Better |
| Deployment Server, License or Cluster Master | ||
| (c3.2xlarge ) | Good | |
| (c3.4xlarge) | Better | |
|
Small-Scale Deployment |
Medium-Scale Deployment |
Large-Scale Deployment
|
| Indexing up to 100 GB/day
Six concurrent searches
|
Indexing up to 500 GB/day
Search and load up to 8 to 16 users |
Indexing up to 1TB/day
With concurrent search load for 16 users
|
| 1 – c4.4xlarge with EBS-backed storage | • 3 – c4.8xlarge with EBS-backed storage (Indexers)
|
• 5 – c4.8xlarge with EBS-backed storage (Indexers) |
| 1 – c4.8xlarge with EBS-backed storage (Search Head) | • 1 – c4.8xlarge with EBS-backed storage (Search Head)
|
|
| • 1 – c3.xlarge (License Master) | • 1 – c3.xlarge (License Master)
|
BEST PRACTICE
Custom AMI creation
- Create your own AMI using Linux based or Splunk provided
- Leverage current configuration tooling with AMI (don’t have to use deployment server, but can be very helpful)
Search Head Clustering
- Deploy to the same AWS Region
- Replication and searches across Regions can be a challenge
- Use a Virtual Private Cloud (VPC)
