Distribution methods for erasure-coded protection

HCP supports two distribution methods for erasure-coded protection: chunk distribution and full-copy distribution.

Chunk distribution

With chunk distribution, after an object is ingested, the chunks for the object are calculated and distributed among the systems in the erasure coding topology. Only the system where the object was ingested has a full copy of the object data.

After all the chunks have been successfully distributed and a specified amount of time has elapsed, the HCP Geo-distributed Erasure Coding service reduces the object on the ingest system to a chunk. The amount of time that the service waits before reducing an object to a chunk is called the erasure coding delay.

For more information about the Geo-distributed Erasure Coding service, see Geo-distributed Erasure Coding service processing.

Full-copy distribution

With full-copy distribution, after an object is ingested, full copies of the object data are distributed to the systems in the erasure coding topology. After the erasure coding delay expires, the Geo-distributed Erasure Coding service on each system reduces the full copy of the object data on that system to a chunk.

Metadata-only considerations for objects subject to erasure coding

For an erasure-coded object to be properly protected, each system in the erasure coding topology must have a chunk for the object. Therefore, even if the object should be metadata-only on one or more systems, those systems must each store a chunk for the object:

With chunk distribution, each system receives a chunk for an erasure-coded object, regardless of whether the object should be metadata-only on that system.

With full-copy distribution, each system receives a full copy of the object data unless the object should be metadata-only on that system. After the erasure coding delay expires, each system where the object should be metadata-only receives a chunk for the object, and the Geo-distributed Erasure Coding service on each system with a full copy of the object data reduces that full copy to a chunk.

For information about metadata-only objects, see Metadata-only storage tiersMetadata-only storage tiers.

Object counts

An HCP system keeps track of the total number of objects stored on it and, separately, the number of erasure-coded objects stored on it:

The total number of objects stored on a system includes whole objects, metadata-only objects, and objects that are subject to erasure coding, regardless of whether the system currently has a full copy of the object data or a chunk for the object.

The number of erasure-coded objects stored on a system includes only objects for which the system currently has a chunk.

HCP also tracks both of these object counts by tenant and namespace. You can view system-level and tenant-level object counts in the System Management Console, through the management API, and in system-level chargeback reports. For tenants to which you have administrative access, you can also view namespace-level object counts through the management API and in chargeback reports.

Tenant administrators can view total object counts for the applicable tenant and its namespaces in the Tenant Management Console, through the management API, and in tenant-level chargeback reports. Tenant administrators cannot see counts of erasure-coded objects.

© 2015, 2020 Hitachi Vantara LLC. All rights reserved.