Kerchunk References

virtualizarr.parsers.KerchunkJSONParser

__call__

__call__(url: str, registry: ObjectStoreRegistry) -> ManifestStore

Parse the metadata and byte offsets from a given Kerchunk JSON to produce a VirtualiZarr ManifestStore.

Parameters:

• url (str) –

  The URL of the input Kerchunk JSON (e.g., "s3://bucket/kerchunk.json").

• registry (ObjectStoreRegistry) –

  An ObjectStoreRegistry for resolving urls and reading data.

Returns:

• ManifestStore –

  A ManifestStore that provides a Zarr representation of the parsed Kerchunk JSON.

__init__

__init__(
    group: str | None = None,
    fs_root: str | None = None,
    skip_variables: Iterable[str] | None = None,
)

Instantiate a parser with parser-specific parameters that can be used in the __call__ method.

Parameters:

• group (str | None, default: None ) –

  The group within the Kerchunk JSON to be used as the Zarr root group for the ManifestStore.

• fs_root (str | None, default: None ) –

  The qualifier to be used for Kerchunk chunk references containing relative paths.

• skip_variables (Iterable[str] | None, default: None ) –

  Variables in the Kerchunk JSON that will be ignored when creating the ManifestStore.

virtualizarr.parsers.KerchunkParquetParser

__call__

__call__(url: str, registry: ObjectStoreRegistry) -> ManifestStore

Parse the metadata and byte offsets from a given Kerchunk Parquet directory to product a VirtualiZarr ManifestStore.

Parameters:

• url (str) –

  The URL of the input parquet directory (e.g., "s3://bucket/my-kerchunk-references.parq").

• registry (ObjectStoreRegistry) –

  An ObjectStoreRegistry for resolving urls and reading data.

Returns:

• ManifestStore –

  A ManifestStore which provides a Zarr representation based on the parsed Kerchunk Parquet directory.

__init__

__init__(
    group: str | None = None,
    fs_root: str | None = None,
    skip_variables: Iterable[str] | None = None,
    reader_options: dict | None = None,
)

Instantiate a parser for virtualizing Kerchunk's Parquet references into a Virtual Zarr store using the __call__ method.

Parameters:

• group (str | None, default: None ) –

  The group within the input Kerchunk Parquet references to be used as the Zarr root group for the ManifestStore.

• fs_root (str | None, default: None ) –

  The qualifier to be used for chunk references containing relative paths.

• skip_variables (Iterable[str] | None, default: None ) –

  Variables in the Kerchunk Parquet references that will be ignored when creating the ManifestStore.

• reader_options (dict | None, default: None ) –

  Configuration options used internally for the fsspec backend.