Metadata storage

To fully understand how certain services work, you need to know how HCP manages metadata.

When you add an object, upload part of a multipart object, or add a directory or symbolic link to a namespace, the following events occur.

1.HCP creates primary metadata for the item being added. This metadata consists of information HCP already knows, such as the creation date, and for objects and parts only, the data size, hash algorithm, and cryptographic hash value generated by that algorithm. It also includes metadata that was either inherited or specified in the write request, such as retention setting, UID, and GID.

2.HCP creates a second copy of the primary metadata. HCP then distributes both copies of the primary metadata among the HCP general nodes.

3.For objects and parts of objects:

oHCP creates the number of copies of the object data or part data required to satisfy the ingest tier data protection level (DPL) in the service plan associated with the namespace. If the ingest tier is primary running storage, HCP distributes all copies of the data among the HCP storage nodes. If the ingest tier is S Series storage, HCP writes all copies of the data to that storage.

Each copy of the primary metadata for the object or part points to all copies of the data for that object or part. However, object or part data in primary running storage is not necessarily stored on the same nodes as the primary metadata for the object or part.

oIn primary running storage, HCP stores the number of copies of the object or part metadata required to satisfy the ingest tier metadata protection level (MPL) in the service plan. These copies, called secondary metadata, let HCP reconstruct the primary metadata should that become necessary.

If the ingest tier is primary running storage, the MPL is the same as the DPL.

oHCP stores MPL copies of any custom metadata for the object in primary running storage. For multipart objects, HCP stores MPL copies of the custom metadata for the object as a whole instead of MPL copies for each part.

oIf the ingest tier is S Series storage, HCP stores one copy of the secondary metadata along with each copy of the object data on that tier.

The next figure shows the data and primary and secondary metadata that result from storing an object in a namespace with a service plan that sets both the DPL and MPL for the ingest tier to 2. The figure assumes that the ingest tier is primary running storage.

When an object or part is moved from the ingest tier to another storage tier, HCP stores one copy of the secondary metadata along with each copy of the object or part data on that tier. Regardless of which tier the object or part data is on, HCP always keeps in primary running storage the number of copies of the secondary metadata and custom metadata required to satisfy the MPL for that tier.

© 2015, 2020 Hitachi Vantara LLC. All rights reserved.