Skip to main content

Hot/warm data tiering in Aiven for OpenSearch® Limited availability

Hot/warm data tiering lets you store recent, frequently queried data on fast nodes and older data on cheaper nodes—without splitting your cluster or changing how you search.

Hot/warm data tiering is in limited availability for Aiven for OpenSearch® version 2.19 and later. It is available on custom plans only.

tip

Contact Aiven to request a custom plan with hot/warm data tiering enabled.

How hot/warm data tiering works​

A tiered cluster has two groups of data nodes. Each group carries a node.attr.temp attribute:

  • Hot nodes use faster disks and are sized for active writes and frequent queries.
  • Warm nodes use larger, lower-cost disks and are sized for data that is queried less often.

OpenSearch shard allocation filtering places index shards on the correct tier using the index.routing.allocation.require.temp index setting. Set this to hot to pin new indices to hot nodes. When an index ages, change the setting to warm to move shards to warm nodes.

Index State Management (ISM) automates these transitions. An ISM policy rolls over an index when it reaches a size or age threshold, migrates it to warm storage after a retention period, and optionally deletes it.

Dynamic Disk Sizing​

Dynamic Disk Sizing adds capacity to both tiers at the same time. You cannot expand a single tier on its own. The added space is distributed proportionally to each tier's base volume size.

For example, if hot nodes have 100 GiB of total volume and warm nodes have 200 GiB, adding 90 GiB allocates 30 GiB to the hot tier (100 / 300 × 90) and 60 GiB to the warm tier (200 / 300 × 90). This behavior might change in the future.

Supported OpenSearch versions​

Hot/warm data tiering requires Aiven for OpenSearch® 2.19 or later. Version 3.3 and later is also supported.

Prerequisites​

  • A custom plan with hot and warm node groups. Contact Aiven to get one configured for your account.
  • Aiven for OpenSearch® 2.19 or later.

Related pages