Erasure coding topology properties
An erasure coding topology has these properties:
- A name and, optionally, a description.
- The number of HCP systems in the erasure coding topology. A topology can have from three through six systems.
- For an erasure coding topology with four or more systems, the type of the underlying replication topology: ring or fully connected. The topology type for three systems is always fully connected.
- A distribution method — chunk or full copy.
- A minimum size for erasure coding objects and object parts — 4 KB, 16 KB, 32 KB, 64 KB, 128 KB, 256 KB, 512 KB, or 1 MB. HCP erasure-codes only objects and object parts with a size greater than or equal to the minimum size you select. HCP uses whole-object protection for objects and parts below the minimum size, regardless of whether the containing namespace is using erasure-coded protection.NoteThe service plan associated with a namespace can include a metadata-only storage tier on all the systems in an erasure coding topology. With this configuration, objects and parts in the namespace that are smaller than the minimum size for erasure coding may have data on only one of those systems. As a result, those objects and parts are not protected against a single system failure.
- An erasure coding delay. This is the amount of time the Geo-distributed Erasure Coding service waits before reducing a full copy of the data for an object or object part to a chunk. With full-copy distribution, this is also the amount of time the service waits on systems where the object or part is metadata-only before requesting a chunk for the object.
- Optionally, a restore period. If an erasure coding topology has a restore period, after an object or object part that's subject to erasure coding is read from a system in the topology, a full copy of the object or part data is kept on that system at least until the restore period expires.
When setting the minimum size, erasure coding delay, and restore period, consider the tradeoffs with respect to storage efficiency and read performance.
Erasure coding delay
The erasure coding delay is counted from the time the ingest system receives the object or part data. You specify the delay as a number of days.
Even if the erasure coding delay has expired for an object, the Geo-distributed Erasure Coding service does not reduce a full copy of the data for the object or part to a chunk if doing so would leave the object or part insufficiently protected.
Restore period
You specify the restore period for an erasure coding topology as a number of days. If the specified number is zero, the topology does not have a restore period.
When an object or part that's subject to erasure coding is read from a given system, where that system is in an erasure coding topology with a restore period:
- If the system currently has a full copy of the object or part data, the restore period applies to that full copy.
- If the system currently has a chunk for the object or part on primary running storage, on an S Series Node, or on spindown storage and the chunk has never been tiered to extended storage, a full copy of the object or part data is restored on the current tier, and the chunk is deleted. The restore period applies to the restored full copy.
- While the system has only a full copy of the object or part data, the object or part is not counted as an erasure-coded object on that system.
- If the system currently has a chunk for the object or part and the chunk is or ever was on an extended storage tier, a full copy of the object or part data is restored on the highest storage tier that could have had a full copy of the data, and the chunk is deleted. The restore period applies to the restored full copy.NoteFor objects and parts that are subject to erasure coding, the storage tiers that can have a full copy of the object or part data are primary running storage, HCP S Series Node, and primary spindown storage. HCP never tiers a full copy of the object or part data to extended storage. Instead, when the Geo-distributed Erasure Coding service on a system reduces a full copy of object or part data to a chunk, if the object or part is due to be tiered, that service puts the chunk on the applicable storage tier.
The restore period for an object or part starts when the object or part is read. If the object or part is read again during the restore period, the restore period restarts from the time of that read operation.
The restore period for an object or part can restart only once on any given day. After the first restart on a day, subsequent reads on the same day do not cause the restore period to restart.
The restore period for an object or part can start before the erasure coding delay has expired. If the erasure coding delay then expires before the restore period expires, the full copy of the object or part data is not reduced to a chunk until the restore period expires. If the restore period expires before the erasure coding delay expires, the full copy of the object or part data is not reduced to a chunk until the erasure coding delay expires.