OpenSearch Installation & Cluster Architecture: A Production Deployment Guide

Configuring the Core Architecture

Once the binaries are present, the configuration files located in `/etc/opensearch/` must be explicitly tuned. Never use the default configurations in a production environment.

opensearch.yml Deep-Dive

The primary configuration file is `opensearch.yml`. Below is an annotated configuration for a data node within a cluster named `catalogue-cluster`. I have specifically included performance-tuning settings that I use in high-throughput environments.

cluster.name: catalogue-cluster
node.name: data-node-01

# Role Definition: Dictates what tasks this node can perform
node.roles: [ data, ingest ]

# Memory Lock: Crucial for preventing the JVM heap from being swapped to disk
bootstrap.memory_lock: true

# Network Binding
network.host: 10.0.1.15
http.port: 9200
transport.port: 9300

# Network Performance Tuning
# network.tcp.no_delay: true disables Nagle's algorithm, reducing latency for small requests
network.tcp.no_delay: true

# Discovery and Cluster Formation
discovery.seed_hosts: ["10.0.1.10", "10.0.1.11", "10.0.1.12"]
cluster.initial_cluster_manager_nodes: ["master-node-01", "master-node-02", "master-node-03"]

# Cluster Routing & Recovery Settings
# Limits concurrent recoveries to prevent I/O starvation during node restarts
cluster.routing.allocation.node_concurrent_recoveries: 2
# Throttles recovery speed to protect ongoing query performance
indices.recovery.max_bytes_per_sec: 50mb

# Security Plugin Initialization
plugins.security.ssl.transport.pemcert_filepath: certs/node-cert.pem
plugins.security.ssl.transport.pemkey_filepath: certs/node-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: certs/root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: certs/http-cert.pem
plugins.security.ssl.http.pemkey_filepath: certs/http-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: certs/root-ca.pem
plugins.security.allow_unsafe_democertificates: false
plugins.security.allow_default_init_securityindex: true

Setting `bootstrap.memory_lock: true` is vital. This tells the JVM to lock its heap in memory, preventing the OS from swapping it out, which would cause severe garbage collection pauses. Furthermore, configuring `cluster.routing.allocation.node_concurrent_recoveries` prevents a cascading failure when a node reboots by limiting how many shards it attempts to rebuild simultaneously.

JVM Options and Garbage Collection

Modify `/etc/opensearch/jvm.options` to set your heap size and garbage collector. For a server with 64GB of RAM, set the heap to 31GB.

-Xms31g
-Xmx31g

# Use G1GC (Garbage First Garbage Collector)
-XX:+UseG1GC
-XX:G1ReservePercent=25
-XX:InitiatingHeapOccupancyPercent=30

Security Plugin Configuration

Unlike Elasticsearch which historically paywalled advanced RBAC, OpenSearch includes enterprise-grade security by default. However, configuring it requires strict adherence to certificate generation and role mapping.

TLS Certificate Generation

You cannot run a secure cluster without TLS for both the HTTP (client) and Transport (node-to-node) layers. Here are the exact openssl commands to generate a Root CA, an admin certificate, and a node certificate. I run these commands in a secure vault before provisioning the cluster.

# Generate Root CA
openssl genrsa -out root-ca-key.pem 2048
openssl req -new -x509 -sha256 -key root-ca-key.pem -subj "/C=GB/ST=London/L=London/O=MODRACX/OU=Engineering/CN=root.modracx.internal" -out root-ca.pem -days 730

# Generate Admin Certificate
openssl genrsa -out admin-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in admin-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out admin-key.pem
openssl req -new -key admin-key.pem -subj "/C=GB/ST=London/L=London/O=MODRACX/OU=Engineering/CN=admin" -out admin.csr
openssl x509 -req -in admin.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out admin.pem -days 730

# Generate Node Certificate
openssl genrsa -out node-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in node-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out node-key.pem
openssl req -new -key node-key.pem -subj "/C=GB/ST=London/L=London/O=MODRACX/OU=Engineering/CN=data-node-01" -out node.csr
openssl x509 -req -in node.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out node.pem -days 730

Defining Users and Roles

The `internal_users.yml` file stores your local user database. You must generate bcrypt hashes for passwords using the bundled hash tool. Navigate to `/usr/share/opensearch/plugins/opensearch-security/tools/` and run `./hash.sh -p 'YourSecurePassword123!'`. You will receive a hash like `$2y$12$....`.

Next, define a role in `roles.yml`. Here is a concrete example of a read-only reporting role that only grants access to index data without allowing structural changes:

reporting_reader:
  cluster_permissions:
    - cluster_composite_ops_ro
  index_permissions:
    - index_patterns:
        - "sales_data_*"
        - "catalogue_*"
      allowed_actions:
        - read
        - search

Finally, bind the user to the role in `roles_mapping.yml`:

reporting_reader:
  reserved: false
  users:
    - "bi_reporting_user"

Index Mapping Deep-Dive and the "Mapping Explosion" Trap

OpenSearch is frequently sold as schema-less, meaning you can send it a JSON document and it will automatically generate mappings for the fields. This is technically true, but in production, dynamic mapping is highly dangerous and directly leads to structural failures.

The Mechanics of Mapping Explosion

When you send a new field to OpenSearch, it infers the data type. If a field contains a string, it will create two mappings by default: one `text` field for full-text search, and one `keyword` field for exact matching and aggregations.

If an application starts sending arbitrary JSON keys—perhaps generating dynamic keys based on user IDs or timestamps—OpenSearch will create new mapping definitions for every unique key. The cluster state, which is distributed to every node in the cluster by the master, must store these mappings. As the mappings grow into the tens of thousands, updating the cluster state becomes extremely slow, eventually freezing the entire cluster—a phenomenon known as a mapping explosion.

Explicit Mapping Enforcement

To prevent this, you must explicitly define your mappings and set `dynamic: strict` or `dynamic: false` on your indices. `strict` will throw an error if an unknown field is ingested, while `false` will store the field in `_source` for retrieval but will not index it for searching.

Here is a full product index mapping JSON demonstrating text and keyword multi-fields, nested objects for variants, integer types for stock levels, and a date type for created_at timestamps. In an archive index, I often disable `_source` entirely to save disk space, though I have left it enabled here for typical eCommerce use.

PUT /ecommerce_products
{
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "sku": {
        "type": "keyword"
      },
      "title": {
        "type": "text",
        "analyzer": "english",
        "fields": {
          "raw": {
            "type": "keyword"
          }
        }
      },
      "price": {
        "type": "scaled_float",
        "scaling_factor": 100
      },
      "stock_quantity": {
        "type": "integer"
      },
      "created_at": {
        "type": "date",
        "format": "strict_date_optional_time||epoch_millis"
      },
      "variants": {
        "type": "nested",
        "properties": {
          "color": { "type": "keyword" },
          "size": { "type": "keyword" },
          "sku_variant": { "type": "keyword" }
        }
      }
    }
  },
  "settings": {
    "index": {
      "number_of_shards": 3,
      "number_of_replicas": 1
    }
  }
}

By enforcing strict mappings, we ensure that a rogue microservice pushing malformed JSON will simply receive an ingestion error, rather than poisoning our cluster state. For complex eCommerce schemas, see our guide on Performance Optimization which discusses data structuring for search engines.

Index Lifecycle Management via ISM Policy

If you are storing logging or analytical data, you should automate index lifecycle phases using ISM (Index State Management) policies. Manual index deletion via cron jobs is a common anti-pattern that leads to disks filling up when cron silently fails.

Here is a complete ISM policy JSON that implements a Hot-Warm-Cold architecture. It keeps indices in a hot phase until they reach 50GB or 30 days of age, at which point it rolls them over. It then moves them to a warm phase where replicas are dropped to 0 and segments are force-merged. Finally, it transitions to a cold phase and eventually deletes the index after 365 days.

PUT _plugins/_ism/policies/logs_lifecycle
{
  "policy": {
    "description": "Hot-Warm-Cold-Delete lifecycle for log indices",
    "default_state": "hot",
    "states": [
      {
        "name": "hot",
        "actions": [
          {
            "rollover": {
              "min_size": "50gb",
              "min_index_age": "30d"
            }
          }
        ],
        "transitions": [
          {
            "state_name": "warm",
            "conditions": {
              "min_index_age": "30d"
            }
          }
        ]
      },
      {
        "name": "warm",
        "actions": [
          {
            "replica_count": {
              "number_of_replicas": 0
            }
          },
          {
            "force_merge": {
              "max_num_segments": 1
            }
          }
        ],
        "transitions": [
          {
            "state_name": "cold",
            "conditions": {
              "min_index_age": "90d"
            }
          }
        ]
      },
      {
        "name": "cold",
        "actions": [
          {
            "read_only": {}
          }
        ],
        "transitions": [
          {
            "state_name": "delete",
            "conditions": {
              "min_index_age": "365d"
            }
          }
        ]
      },
      {
        "name": "delete",
        "actions": [
          {
            "delete": {}
          }
        ],
        "transitions": []
      }
    ]
  }
}

Query Performance and Profiling

When queries begin returning slowly, guessing at the cause is futile. OpenSearch provides a Profile API that acts similarly to an SQL `EXPLAIN ANALYZE`. By appending `"profile": true` to your search request body, OpenSearch will return a detailed timing breakdown for every shard and every phase of the query execution.

Common Slow Query Causes

In my tuning engagements, three patterns account for 90% of performance issues:

1. Leading Wildcards: Searching for `*phone` forces Lucene to scan the entire dictionary because it cannot use the reverse index prefix tree. Never allow user input to generate leading wildcard queries.
2. Nested Query Depth: Heavy usage of the `nested` datatype requires significant CPU overhead to join child documents back to their parent at query time. Limit nested fields where possible.
3. Script Scoring: Using Painless scripts to recalculate relevance scores dynamically for every document retrieved will bottleneck your data nodes. Pre-calculate values at ingestion time whenever possible.

To identify slow queries proactively, enable the slow query log. This setting logs any query exceeding a specific threshold to your OpenSearch log directory:

PUT /_settings
{
  "index.search.slowlog.threshold.query.warn": "2s",
  "index.search.slowlog.threshold.query.info": "1s",
  "index.search.slowlog.threshold.fetch.warn": "1s"
}

Monitoring and Observability

Visibility into your cluster state is non-negotiable. OpenSearch Dashboards is the native visual interface for cluster management. Below is a docker-compose snippet that sets up Dashboards connected to an existing OpenSearch node.

version: '3'
services:
  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:2.11.0
    container_name: os-dashboards
    ports:
      - 5601:5601
    environment:
      OPENSEARCH_HOSTS: '["https://10.0.1.15:9200"]'
    # Mount custom dashboards config here

For automated health checks, a simple Python script querying the Cluster Health API can trigger alerts if the cluster state changes from green to yellow or red.

import requests
import sys

response = requests.get('https://10.0.1.15:9200/_cluster/health', auth=('admin', 'your_password'), verify=False)
data = response.json()

if data['status'] != 'green':
    print(f"CRITICAL: Cluster state is {data['status']}")
    # Trigger PagerDuty or Slack alert here
    sys.exit(1)
print("Cluster health is OK.")
sys.exit(0)

Additionally, you can leverage the native OpenSearch Notifications plugin to configure webhooks directly to Slack or Microsoft Teams whenever specific thresholds (e.g., CPU > 90%) are breached.

Snapshot and Backup Architecture

High availability is not a substitute for backups. Replicas protect against node failure; snapshots protect against human error (like accidental index deletion). You should configure daily automated snapshots to Amazon S3 or a compatible object store.

First, register the S3 snapshot repository via the REST API. You will need the repository-s3 plugin installed on all nodes.

PUT _snapshot/s3_backup
{
  "type": "s3",
  "settings": {
    "bucket": "opensearch-cluster-backups",
    "base_path": "production-cluster",
    "region": "eu-west-1",
    "compress": true
  }
}

Once the repository is registered, you can create a snapshot policy using the ISM plugin to automate daily backups and prune snapshots older than 30 days. This removes the need for brittle shell scripts managing backups via cron.

Cluster Upgrade Path (Rolling Upgrades)

Upgrading a live production cluster must be performed as a rolling upgrade to avoid downtime. A rolling upgrade involves updating one node at a time while the cluster remains active. Here is the strict sequence I follow during maintenance windows:

1. Disable Shard Allocation: Stop the master node from trying to rebalance shards while you reboot a data node. Run: PUT _cluster/settings { "persistent": { "cluster.routing.allocation.enable": "primaries" } }
2. Stop the Node: Execute systemctl stop opensearch on a single target node.
3. Upgrade Binaries: Run the package manager update (e.g., apt-get upgrade opensearch).
4. Restart and Wait: Start the node, tail the logs, and wait for it to join the cluster.
5. Re-enable Allocation: Execute PUT _cluster/settings { "persistent": { "cluster.routing.allocation.enable": "all" } } and wait for the cluster state to return to green.
6. Repeat: Proceed to the next node in the cluster. Always upgrade master nodes last.

When NOT to Use OpenSearch (The Honest Trade-Off)

OpenSearch is a heavyweight, JVM-based distributed system. It requires significant operational overhead, hardware provisioning, and architectural planning. You should not use OpenSearch in the following scenarios:

• Small Datasets (<10GB): If your entire catalog or dataset is under 10GB, a dedicated OpenSearch cluster is vast overkill. The JVM overhead and node management will cost more time and money than it is worth. In these cases, utilize PostgreSQL's native Full-Text Search (FTS) or specialized lightweight tools like Meilisearch or Typesense.
• Zero Dedicated Ops Capacity: If your team does not have a DevOps engineer or someone familiar with Elasticsearch/Lucene mechanics, running a self-managed cluster is dangerous. Clusters require monitoring for split-brain scenarios, unassigned shards, and JVM garbage collection spikes.
• Highly Relational Data: OpenSearch is a document store. If your search queries require deep, multi-level joins across different entities in real-time, OpenSearch is the wrong tool. Data must be denormalised before ingestion.

Production Readiness Checklist

Before launching traffic at your cluster, verify the following:

1. vm.max_map_count is set to at least 262144.
2. Swap is permanently disabled on all nodes.
3. File descriptors limit is set to 65536.
4. Default passwords (`admin:admin`) are changed in `internal_users.yml`.
5. TLS encryption is enabled on the transport layer (Node-to-Node communication).
6. Heap size is strictly 50% of RAM, capped at 31GB.
7. Shard allocation awareness is enabled if deploying across multiple availability zones.

10. OpenSearch Performance Tuning: Thread Pools, Circuit Breakers and Merge Policy

When dealing with a cluster under immense production load, relying on default thread management and memory circuit breakers is a direct path to unpredictable latency spikes and sudden node crashes. In this section, we will tune the specific engine parameters that govern how OpenSearch processes requests and merges Lucene segments on disk.

Managing Thread Pools for High Throughput

OpenSearch uses distinct thread pools for different types of operations, preventing an influx of writes from entirely starving read requests. By default, the search thread pool is sized based on the number of available processors. However, for high-concurrency read-heavy workloads, I prefer tuning this explicitly to balance queueing versus context switching.

You should configure the search thread pool size to the number of allocated CPUs plus one (thread_pool.search.size), with a queue size of 1000. This prevents excessive context switching overhead while providing a deep enough buffer to handle temporary query spikes. The write thread pool, which handles document indexing and bulk requests, should similarly be set to the number of CPUs plus one (thread_pool.write.size), but with a much tighter queue size of 200. If write requests queue up beyond 200, it is better that the node rejects the request (a 429 Too Many Requests response) rather than consuming heap memory with a massive backlog. This backpressure signals the client to retry with exponential backoff.

You can apply these settings dynamically via the cluster update API:

PUT /_cluster/settings
{
  "persistent": {
    "thread_pool.search.size": 17,
    "thread_pool.search.queue_size": 1000,
    "thread_pool.write.size": 17,
    "thread_pool.write.queue_size": 200
  }
}

(Assuming a 16-core machine)

Hardening Circuit Breakers

OpenSearch implements circuit breakers to prevent operations from causing an OutOfMemoryError. When a request attempts to allocate memory that would push the heap usage over a configured limit, the circuit breaker trips and aborts the operation.

The default parent circuit breaker limit (indices.breaker.total.limit) is 70% of the JVM heap. In tightly controlled environments where queries are profiled and predictable, I raise this to 95% to maximise memory utilisation, alongside tuning specific child breakers.

The fielddata circuit breaker (indices.breaker.fielddata.limit) controls how much heap is used for loading field data (typically for sorting or aggregating on text fields). I set this to 40%. If you exceed this, it indicates an architectural flaw—you should be using keyword fields (which use doc values on disk) rather than text fields for aggregations.

The request circuit breaker (indices.breaker.request.limit) limits the memory a single request can consume during its execution phase. I configure this to 60%. If a single query attempts to consume more than 60% of a 31GB heap (roughly 18GB of RAM), it is a poorly constructed query that must be killed before it impacts other tenants.

PUT /_cluster/settings
{
  "persistent": {
    "indices.breaker.total.limit": "95%",
    "indices.breaker.fielddata.limit": "40%",
    "indices.breaker.request.limit": "60%"
  }
}

Lucene Segment Merge Policy

Every time you index documents, OpenSearch writes them to new Lucene segments. Over time, having thousands of small segments destroys search performance. OpenSearch runs background threads to merge these segments.

The indices.merge.scheduler.max_thread_count setting determines how many threads are allowed to merge segments simultaneously. For modern NVMe SSDs, the default is half the number of CPUs. However, if you are forced to run on spinning mechanical disks, you must set this to 1. Spinning disks cannot handle concurrent sequential writes effectively; allowing multiple merge threads will cause severe I/O thrashing and cripple your read performance.

Furthermore, we must tune the Lucene tieredMergePolicy. By default, OpenSearch might allow segments to grow infinitely or merge too aggressively. I enforce a strict maxMergedSegmentMB of 5120 (5GB) and set segmentsPerTier to 10. This means that once a segment reaches 5GB, it will no longer be merged with other segments. This provides highly predictable I/O patterns. A 5GB segment is large enough to be extremely efficient for querying, but small enough that if it needs to be relocated to another node, it won't saturate the network interface for hours.

PUT /_settings
{
  "index.merge.policy.max_merged_segment": "5120mb",
  "index.merge.policy.segments_per_tier": 10
}

Disk I/O and Linux Scheduler Tuning

OpenSearch's background segment merging is incredibly I/O intensive. By default, many Linux distributions use the mq-deadline or bfq I/O scheduler. These schedulers are designed to provide fairness across multiple processes and prioritise interactive tasks, which is completely counterproductive for a dedicated database node.

For OpenSearch data nodes equipped with NVMe SSDs, you must change the I/O scheduler to none (or noop on older kernels). This allows the NVMe controller's internal hardware queues to handle request ordering natively, eliminating the kernel's CPU overhead for reordering I/O requests. In a production cluster running on AWS i3en instances, switching the scheduler to none reduced our disk wait times (iowait) from an average of 12% to under 2% during heavy ingestion phases.

You can verify your current scheduler with:

cat /sys/block/nvme0n1/queue/scheduler

And set it to none permanently by configuring /etc/udev/rules.d/60-scheduler.rules:

ACTION=="add|change", KERNEL=="nvme[0-9]*", ATTR{queue/scheduler}="none"

This seemingly minor operating system tweak often yields a greater performance uplift during write-heavy workloads than weeks of application-level query tuning.

11. Multi-Tenancy and Index Isolation Patterns

When building SaaS platforms or handling multiple distinct clients within the same OpenSearch cluster, architecting your multi-tenancy strategy is a critical early decision. The wrong pattern will either expose cross-client data or bring the cluster down under the weight of excessive metadata.

Strategy A: Index-per-Tenant (Hard Isolation)

The simplest and most secure approach is the index-per-tenant pattern. In this model, every client or tenant is allocated their own distinct index (e.g., client_a_catalogue, client_b_catalogue).

The primary advantage is hard isolation. There is mathematically zero risk of cross-tenant data leakage because queries are physically routed to different indices. It also allows for highly customised mappings per tenant, and you can easily delete a tenant's data simply by dropping their index (an O(1) operation).

However, this pattern incurs severe overhead at scale. Every index requires its own shards, and every shard consumes JVM heap for its metadata, regardless of whether it holds 10 documents or 10 million documents. In a typical cluster architecture, one cluster can hold approximately 200 active indices comfortably before master node overhead becomes measurable. Once you push past 500 indices, the cluster state updates become heavily constrained, and JVM memory is consumed entirely by idle shards. If you have thousands of clients, the index-per-tenant pattern will definitively fail.

Strategy B: Shared Index with Filtered Aliases

To scale to thousands or tens of thousands of tenants, you must use a shared index pattern. All documents for all tenants reside in a single massive index (e.g., global_catalogue). To segregate the data, every single document must contain a strictly enforced tenant_id field.

While you could simply rely on the application layer to append a term filter for the tenant_id on every query, relying on application logic for data security is inherently risky. A single missed filter in the codebase results in a catastrophic data breach.

Instead, OpenSearch provides a powerful mechanism known as filtered aliases. You create an alias for each tenant that transparently injects the tenant_id filter into every request routed through it. The application then queries the alias (e.g., client_a_alias) rather than the raw index.

Here is the exact API call to create a filtered alias using a terms filter:

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "global_catalogue",
        "alias": "tenant_1049_catalogue",
        "filter": {
          "term": {
            "tenant_id": "1049"
          }
        }
      }
    },
    {
      "add": {
        "index": "global_catalogue",
        "alias": "tenant_2081_catalogue",
        "filter": {
          "term": {
            "tenant_id": "2081"
          }
        }
      }
    }
  ]
}

When the application queries tenant_1049_catalogue, OpenSearch automatically rewrites the query at the coordinating node level to include the term: { "tenant_id": "1049" } filter.

This shared index pattern scales effortlessly to thousands of tenants, as creating an alias adds virtually zero overhead compared to creating a full index. The trade-off, however, is that all tenants must share exactly the same mapping schema. If Tenant A needs a custom field, it must be added to the global index, potentially causing mapping sparsity. Furthermore, deleting a tenant requires executing a Delete By Query operation (_delete_by_query), which is highly resource-intensive and leaves behind tombstone markers until the segments are merged.

Handling Tenant-Specific Routing

In the shared index model, a critical optimization is utilizing OpenSearch's custom routing. By default, when you index a document, OpenSearch hashes the document's _id to determine which primary shard will hold the data. When a query is executed without routing, it must be broadcast to all shards (the scatter-gather phase), and the results are merged.

For a multi-tenant shared index, broadcasting every query to all shards is massively inefficient. Instead, you should use the tenant_id as the routing key.

PUT /global_catalogue/_doc/prod_991?routing=1049
{
  "tenant_id": "1049",
  "sku": "X-100",
  "title": "Industrial Widget"
}

By explicitly setting the routing parameter, OpenSearch hashes the tenant_id instead of the document ID. This guarantees that all documents for Tenant 1049 are stored on the exact same single shard. When querying, you pass the same routing key:

GET /global_catalogue/_search?routing=1049
{
  "query": {
    "term": { "tenant_id": "1049" }
  }
}

The coordinating node now knows precisely which shard holds all of Tenant 1049's data. It sends the request to that single shard and skips the scatter-gather phase entirely. In a cluster with 30 primary shards, this reduces the query overhead by 96%, allowing a dramatically higher throughput on the exact same hardware footprint. This single routing parameter is often the difference between a cluster supporting 5,000 requests per second versus collapsing at 200 requests per second.

Frequently Asked Questions

How does OpenSearch differ from Elasticsearch?

OpenSearch is an Apache 2.0 licensed fork of Elasticsearch 7.10.2 created by Amazon after Elastic changed their licensing. It maintains compatibility with older Elasticsearch clients but has evolved its own security, machine learning, and index management plugins natively. It ensures that critical enterprise features remain free and open-source.

What is the recommended JVM heap size for OpenSearch?

Set the JVM heap (-Xms and -Xmx) to exactly 50% of the total system RAM, up to a strict maximum of 31GB. This leaves the remaining memory for the operating system's filesystem cache, which Lucene relies on for fast search operations. Exceeding this boundary will actively harm performance.

Why did my OpenSearch cluster state turn RED?

A RED cluster state means at least one primary shard is unassigned and data is missing. This usually occurs when a data node drops out of the cluster, or you run out of disk space triggering a high watermark block. It requires immediate intervention to restore the missing node or expand storage capacity.

How do I automate index backups?

Automate backups by registering an S3 repository via the Snapshot REST API, then bind it to an Index State Management (ISM) policy. The ISM plugin handles regular snapshots without relying on brittle cron jobs. This guarantees your snapshots run on schedule directly from the cluster manager.