• +91-8107108740
  • F-5, F-6 4th Floor Dana Pani Restaurant, Central Spine, Vidhyadhar Nagar Jaipur.
CASE STUDY · ELASTICSEARCH CCR

Cross Cluster ELK Replication Setup
for Reliable Data Synchronization

The client needed assistance in setting up Cross-Cluster Replication (CCR) between two Elasticsearch clusters running in a Windows environment to ensure reliable data synchronization and high availability.

Elasticsearch CCR Leader Cluster Follower Cluster Windows Environment Replication Setup
🖥️ 8Bit – CCR Architecture
● SOURCE
🗄️
Leader Cluster
Primary Data Source
cluster.nameleader-cluster
node.nameleader-node
ports9200 · 9300
CCR REPLICATION
⚡ Real-time Sync
Proxy Mode
Port 9300
● REPLICA
🗄️
Follower Cluster
Synchronized Replica
cluster.namefollower-cluster
node.namefollower-node
ports9200 · 9300
01 · DISCOVERY

Project Request and Initial Discovery

Before implementation, we conducted a comprehensive review of the client's environment to ensure compatibility and readiness for Cross-Cluster Replication.

Environment Assessment Checklist
Cluster version compatibility (Elasticsearch 7.16+)
Single-node cluster setup on both machines
Network configuration and connectivity verification
CCR license requirements and availability
No-security setup for simplified configuration
Identification of indices to be replicated
02 · PLANNING

Structured CCR Implementation Plan

A systematic approach to ensure reliable Cross-Cluster Replication setup across both environments.

🔍1
Discover
Assess environment & version compatibility
📋2
Plan
Define replication strategy & index scope
⚙️3
Configure
Set YAML settings on both cluster nodes
📁4
Replicate
Enable CCR follow on Follower cluster
5
Verify
Test data sync across both clusters
📈6
Monitor
Track health, lag, and error logs
03 · PRE-CONFIGURATION

Pre-Configuration Preparations

Essential prerequisites and initial setup steps to prepare both clusters for CCR implementation.

⚙️ Prerequisites
1
Install Elasticsearch version 7.16 or higher on both Leader and Follower machines
2
Ensure both clusters can communicate over the network (verify firewall settings)
3
Set up Java environment if not already configured
4
Verify compatible Elasticsearch configurations and CCR license availability
5
Take snapshots on the Leader cluster for data backup
6
Import the snapshot repository on the Follower cluster
04 · IMPLEMENTATION

Setup Leader and Follower Clusters

Configure both Elasticsearch clusters with proper settings to enable Cross-Cluster Replication.

L
Leader Cluster Configuration
elasticsearch.yml (Leader)
YAML
cluster.name: leader-cluster
node.name: leader-node
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: []
cluster.initial_master_nodes: ["leader-node"]
F
Follower Cluster Configuration
elasticsearch.yml (Follower)
YAML
cluster.name: follower-cluster
node.name: follower-node
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: []
cluster.initial_master_nodes: ["follower-node"]
⚠️
Restart Required
After modifying the elasticsearch.yml files on both machines, restart the Elasticsearch services to apply the configuration changes.
PROXY MODE CONFIGURATION

Configure Remote Cluster Connection (Proxy Mode)

Setup Remote Cluster on Follower
JSON
PUT /_cluster/settings
{
  "persistent": {
    "cluster.remote.leader-cluster.mode": "proxy",
    "cluster.remote.leader-cluster.proxy_address": "LEADER_CLUSTER_IP:9300",
    "cluster.remote.leader-cluster.skip_unavailable": true
  }
}
TRANSPORT LAYER – PROXY MODE CONNECTION
🗄️
Follower
follower-cluster
:9300
🛡️ PROXY
skip_unavailable: true
🗄️
Leader
leader-cluster
LEADER_IP:9300
05 · REPLICATION

Leader Index Creation and Follower Replication Setup

Create the source index on the Leader cluster and configure the Follower to replicate it.

1
Create Index on Leader Cluster
Create Leader Index
JSON
PUT /leader-index
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0
  }
}
2
Insert Sample Data into Leader Index
Add Document to Leader
JSON
POST /leader-index/_doc
{
  "field1": "value1",
  "field2": "value2"
}
3
Configure CCR on Follower Cluster
Setup CCR Following
JSON
PUT /follower-index/_ccr/follow
{
  "remote_cluster": "leader-cluster",
  "leader_index": "leader-index"
}
INDEX REPLICATION FLOW
📄
leader-index
Write Operations
LEADER
⚡ Write Event
CCR FOLLOW Auto-sync
transport · 9300
🗄️
follower-index
Auto-Replicated
FOLLOWER
06 · VERIFICATION

Replication Verification and Data Testing

Validate that Cross-Cluster Replication is working correctly by testing data synchronization.

1
Insert New Data on Leader
Add Test Document
JSON
POST /leader-index/_doc
                {
                 "field1": "new_value",
                  "field2": "additional_data"
                }
			
2
Verify Data on Follower
Search Follower Index
JSON
GET /follower-index/_search
VERIFICATION FLOW
🗄️
New Data Inserted
Leader Index
🔄
Replication Triggered
CCR Process
Follower Updated
Data Synchronized
07 · MONITORING

Post-Setup Monitoring

Implement continuous monitoring to ensure CCR remains healthy and performs optimally.

Cluster Health Checks
Monitor the overall health status of both clusters regularly.
HTTP
📋 Copy
GET /_cluster/health
Log Monitoring
Monitor Elasticsearch logs on Windows for CCR-related warnings and errors.
elasticsearch.log
Performance Metrics
Track CCR statistics and replication lag metrics.
HTTP
📋 Copy
GET /_ccr/stats
MONITORING STACK
📈
Cluster Health
Overall cluster status across both nodes
GET /_cluster/health
📄
Log Analysis
CCR warnings, errors, and connectivity events
elasticsearch.log
📊
CCR Metrics
Replication lag and throughput statistics
GET /_ccr/stats
08 · OUTCOME

Implementation Outcome

The client received a comprehensive, step-by-step approach to correctly set up Cross-Cluster Replication in their Windows environment, ensuring reliable data synchronization between both Elasticsearch clusters.

🛡️
High Availability
Redundant data storage across clusters ensures continuous operation even during failures.
Real-Time Syncing
Automated replication ensures follower clusters stay synchronized with minimal latency.
⚙️
Ease of Management
Simple configuration and monitoring make CCR straightforward to maintain and troubleshoot.
The client now has a detailed, production-ready guide to implement and maintain Cross-Cluster Replication for dependable data synchronization between both Elasticsearch clusters.