Skip to content

Latest commit

 

History

History
223 lines (174 loc) · 7.07 KB

README.md

File metadata and controls

223 lines (174 loc) · 7.07 KB

Splunk Platform Automator

License

Looking for a lightweight, VM-based Splunk deployment solution optimized for Apple Silicon Macs? This is the first framework that lets you easily build and test Splunk environments on modern ARM-based MacBooks, with native performance and minimal resource overhead.

This repository modernizes the original Splunk Platform Automator for the Apple Silicon era, replacing Vagrant with Terraform and leveraging OrbStack for efficient virtualization. Whether you need to test Splunk upgrades, experiment with indexer clustering, or verify configuration changes, you can now do it all natively on your M1/M2 Mac.

Quick Start

  1. Install prerequisites:
# On macOS
brew install ansible terraform [email protected]

# Install required Python packages
python -m pip install jmespath lxml  # Required for json_query and license checks
  1. Setup and deploy:
task setup-venv              # Create virtual environment
. ./.venv/bin/activate      # Activate environment
task setup:deps             # Install dependencies

task example:list                        # View available examples
task example:use -- idx_3shc_uf_orbstack # Use example config
task deploy                             # Deploy infrastructure and Splunk

For more complex deployments, check the examples directory.

Why This Fork?

This fork modernizes the infrastructure layer with Terraform, bringing several key benefits:

  1. Modern Infrastructure Management

    • Infrastructure as Code (IaC) best practices
    • Better state management and drift detection
    • Native support for multiple cloud providers (AWS planned)
    • Flexible virtualization support

  2. Streamlined Architecture

    graph LR
    %% Styling
    classDef task fill:#99f,stroke:#333,stroke-width:4px
    classDef config fill:#f9f,stroke:#333
    classDef ansible fill:#fbf,stroke:#333
    classDef infra fill:#bfb,stroke:#333
    
    %% Nodes
    Task(Task Workflow)
    Examples(Examples)
    Config(splunk_config.yml)
    Inventory(Ansible Inventory)
    Terraform(Terraform)
    Infra(Infrastructure)
    Ansible(Ansible Config)
    
    %% Main Pipeline
    Examples --> Config --> Inventory --> Terraform --> Infra --> Ansible
    
    %% Task Control
    Task --> Examples & Config & Inventory & Terraform & Ansible
    
    %% Apply styles
    class Task task
    class Examples,Config config
    class Inventory,Ansible ansible
    class Terraform,Infra infra
    Loading

    The architecture provides:

    • Single source of truth in splunk_config.yml
    • Clear separation between infrastructure and configuration
    • Task-based automation for all operations
    • Integrated validation and testing

Deployment Options

This framework supports two deployment approaches:

1. Terraform-based Deployment (Recommended)

The recommended approach using Terraform for infrastructure provisioning:

  1. Infrastructure Creation: Terraform automatically provisions and manages your infrastructure
  2. Inventory Management: Generates inventory/hosts file during provisioning
  3. Configuration: Manages host entries based on your splunk_config.yml

Benefits:

  • Automated infrastructure management
  • Consistent environment creation
  • Infrastructure state tracking
  • Easy scaling and updates

2. Manual VM Deployment

For scenarios where you prefer to manage VMs manually:

  1. Configuration Setup:

    # Copy and modify an example configuration
cp examples/idx_sh_uf_orbstack.yml config/splunk_config.yml

  2. VM Creation: Create virtual machines matching your configuration

    # Example using OrbStack (a lightweight virtualization solution for ARM-based macOS)
orb create almalinux:9 idx1
orb create almalinux:9 sh1
orb create almalinux:9 uf1

  3. Deployment:

    task orb:inventory > inventory/hosts  # Generate inventory
task ansible:deploy                   # Deploy Splunk

Example inventory/hosts:

# Generated by orb:inventory - Host configurations
idx1 ip_addr=198.19.249.75 public_dns_name=idx1.orb.local
sh1 ip_addr=198.19.249.134 public_dns_name=sh1.orb.local
uf1 ip_addr=198.19.249.177 public_dns_name=uf1.orb.local

Command Reference

Infrastructure Management

# Initialize and Plan
task tf:init         # Initialize Terraform working directory
task tf:validate     # Validate Terraform configurations
task tf:plan         # Plan infrastructure changes

# Apply and Destroy
task tf:apply        # Apply infrastructure changes
task tf:destroy      # Destroy infrastructure (interactive)
task tf:destroy-auto # Destroy infrastructure (non-interactive)

Configuration Management

task ansible:validate:config    # Validate configuration
task ansible:generate:inventory # Generate Ansible inventory
task ansible:deploy            # Deploy Splunk configuration

Example Management

task example:list     # List available examples
task example:use      # Use an example configuration
task example:restore  # Restore previous configuration

End-to-end Workflows

task deploy          # Deploy complete infrastructure and configure Splunk
task destroy:all     # Destroy all infrastructure and clean up

Support

Note: This framework is not officially supported by Splunk. It is being developed on best effort basis.

Project Status

Current Sprint: Improving Manual Provisioning

  • Simplified host management by consolidating all host information in inventory/hosts file
  • Removed redundant host mapping functionality
  • Streamlined manual provisioning process for better user experience

Previous Sprints

Sprint 3 (October 2024)

  • [c] Terraform module for OrbStack Linux machines
  • [c] Development and production environment setup
  • [c] Basic Ansible inventory generation
  • [c] Host file management system

Sprint 2 (September-October 2024)

  • [c] Task-based infrastructure management
  • [c] Example configurations for OrbStack
  • [c] Basic configuration validation
  • [c] Infrastructure state verification

Sprint 1 (September 2024)

  • [c] Project initialization
  • [c] Basic project structure
  • [c] Initial documentation
  • [c] Basic OrbStack integration

Planned Features

Next Sprint Priority

  • [p] AWS/vSphere provider implementation
    • AWS EC2 instance management
    • vSphere VM provisioning
    • Provider-specific networking
    • Security group configuration
    • Environment-specific provider configurations
  • [p] Cross-provider configuration templates
  • [p] Enhanced environment management
    • Cross-environment resource sharing
    • Environment isolation controls
    • Resource tagging by environment

Future Roadmap

  • [d] Enhanced logging and monitoring
  • [d] Automated testing pipeline
  • [d] Container support
  • [d] Additional cloud provider support (Azure, GCP)

Legend:

  • [c] Completed
  • [p] Planned (Current/Next Sprint)
  • [d] Deferred (Future Roadmap)

License

Apache License 2.0