zarr_metadata.v3.definition

zarr_metadata.v3.definition

Metadata fields read against their definitions: the public door.

Every Zarr v3 extension point -- codecs, data types, chunk grids, chunk key encodings, storage transformers -- is a metadata field: a name, and a configuration whose JSON the extension defines. A definition says what that JSON is and what is allowed in it. It is a value, not a class to subclass:

• name, the name the metadata carries;
• configuration, the TypedDict the configuration's JSON is, which is the one declaration of it: the type checker is compiled from it, and a checked configuration has it as its static type. It reads as the typing spec defines a TypedDict -- total, Required, NotRequired, closed and extra_items mean what they mean to a type checker, whether or not its module postpones annotations -- and a member's type carries its bounds, as pydantic reads them: a gzip level is Annotated[int, Interval(ge=0, le=9)];
• rules, a function over that TypedDict yielding what the spec disallows that a type cannot say -- members read together -- located in the configuration.

What kind of metadata a definition defines is its type: CodecDefinition (with the codec's kind, and its size: whether the size of what it gives out is fixed by the size of what it is handed, "static", or depends on the values, "dynamic"), DataTypeDefinition, ChunkGridDefinition, ChunkKeyEncodingDefinition, StorageTransformerDefinition.

Reading JSON. Three steps, each feeding the next, and each usable on its own by a caller that holds nothing but JSON:

1. check(value, SomeTypedDict), from zarr_metadata.typed_json, type-checks JSON against a TypedDict and needs nothing else: a value of the TypedDict or None, and every problem, each located. The value holds what the TypedDict admits and nothing else. A member typed with a field alias is checked as the JSON a metadata field is.
2. definition.read_configuration(configuration) is the configuration as that definition reads it: type-checked, each nested field's envelope judged -- a stray member, a must_understand of false -- and then the rules, for one configuration: GZIP_CODEC.read_configuration({"level": 12}).

3. resolve(field, CodecDefinition, CORE_AND_EXTENSIONS) reads a whole field in a scope: its envelope judged, its name related to a definition, its configuration judged, and each nested field read the same way. It returns what the scope made of the field, and every problem: AcceptedField by the definition that claims its name, with the configuration it checked and allowed and the fields it holds, each read the same way; UnclaimedField, a name nothing in scope claims, left unjudged, which is what keeps the format open; or RefusedField, whose problems say why. Each has the field's JSON, the name it is written with, the kind it was read as, read_as, the definition that claims its name -- None for UnclaimedField -- and the fields it holds as the scope read them, nested. The two a model holds, AcceptedField and UnclaimedField, have a configuration and to_json(), the field as a document writes it; two of them are equal when they read the same, however each was spelled. ResolvedField is the three, for match. A field is read as one of the five kinds, with or without type arguments; resolve(field, Definition, scope) is a TypeError, since nothing is filed under it. A whole v3 array document is read by read_array_metadata_v3, in zarr_metadata.model, whose reading gives each field with where it sits in the document.

   from zarr_metadata.v3.codec.gzip import GZIP_CODEC from zarr_metadata.v3.definition import CORE_AND_EXTENSIONS, CodecDefinition, resolve

   resolved, problems = resolve({"name": "gzip", "configuration": {"level": 12}}, CodecDefinition, CORE_AND_EXTENSIONS) resolved # RefusedField(..., definition=CodecDefinition(name='gzip'), ...) problems[0].loc # ('configuration', 'level') problems[0].input # 12 dict(problems[0].ctx) # {'ge': 0, 'le': 9}

   resolved, problems = resolve({"name": "gzip", "configuration": {"level": 5}}, CodecDefinition, CORE_AND_EXTENSIONS) resolved.definition is GZIP_CODEC # True resolved.configuration # {'level': 5}

Problems are values, not exceptions: ValidationProblem(loc, message, kind), with kind one of invalid_type, invalid_value, missing_key, unknown_key and invalid_json. A field with an unknown key is still read -- the key reported, the configuration judged without it -- so a consumer that tolerates one filters by kind and uses what was read; the field is valid only when there is no problem at all. Each problem carries what its message says as data, as pydantic's errors and zod's issues do: input, what was found at loc -- the 12 above -- and ctx, what was expected, where that is more than a type: the bounds {"ge": 0, "le": 9}, or the values of a closed set.

Writing an extension. A TypedDict, which says what the configuration's JSON is, bounds and all; a function for the rules a type cannot say; and a definition; then a scope that holds it. The TypedDict is a typing_extensions.TypedDict: closed and extra_items are PEP 728's, which typing.TypedDict does not take on the versions this package supports. The rules are handed the configuration and the fields it holds as the scope read them: a field that is read keeps what it read inside it as AcceptedField.nested, a Nested mapping by where each sits, so a struct's rules reach its field types. read_configuration, which reads in no scope, hands them none. A rule's message shows a value as the package's own messages do, as JSON, with shown: null, [1, 2], "C". A rule reports where a problem is; what is found there is the problem's input without the rule saying so. Define each function at a module's top level: a model holds the definitions that read its fields, so it pickles, and compares equal once loaded, only when they do -- a lambda or a closure does not pickle, and a functools.partial pickles but compares unequal to itself loaded.

from collections.abc import Iterator
from typing import Annotated, NotRequired

from annotated_types import Ge
from typing_extensions import TypedDict

from zarr_metadata.v3.definition import (
    CORE_AND_EXTENSIONS,
    CodecDefinition,
    Nested,
    ValidationProblem,
)


class AcmeLz4Configuration(TypedDict, closed=True):
    acceleration: Annotated[int, Ge(1)]
    dictionary: NotRequired[str]
    dictionary_size: NotRequired[Annotated[int, Ge(1)]]


def acme_lz4_rules(
    configuration: AcmeLz4Configuration, nested: Nested
) -> Iterator[ValidationProblem]:
    if "dictionary" in configuration and "dictionary_size" not in configuration:
        yield ValidationProblem(
            ("dictionary_size",), "a dictionary needs its size", "missing_key"
        )


ACME_LZ4 = CodecDefinition(
    name="acme.lz4",
    configuration=AcmeLz4Configuration,
    kind="bytes_bytes",
    size="dynamic",
    rules=acme_lz4_rules,
)
SCOPE = CORE_AND_EXTENSIONS.extended_with(ACME_LZ4)

A scope reads whole documents as well as fields: validate_array_metadata_v3(document, context=SCOPE), from zarr_metadata.model, reads each extension point of a v3 array document through the definitions in SCOPE, and so do the model's from_json and from_key_value: a fill value is judged against the data type it names, by that data type's definition, the chunk grid against the shape, by the grid's definition, and the codecs as a pipeline, each by its definition against the chunk it is handed.

The TypedDict says what a key it does not declare is: with closed=True, a problem, as above; with extra_items=, a key holding that type; with closed=False, anything at all. One that says none of these is open by default, and would take a misspelled key without a word, so a definition refuses it. Its members are the shapes JSON takes: int, float, bool, str, None, JSONValue, a Literal, tuple[T, ...] and tuple[T1, T2], a union, a TypedDict, Mapping[str, V], a NewType and a type alias. A number's type may carry bounds, as annotated-types spells them and pydantic reads them -- Gt, Ge, Lt, Le and Interval, one from each side -- at any depth: tuple[Annotated[int, Ge(1)], ...] bounds each element. A value out of them is a problem, invalid_value, whose message says what the type admits, "expected an integer >= 1, got 0", and whose ctx holds the bounds. The rules are asked only of a configuration within its bounds, so a rule relies on them, as pydantic's after-validators and zod's refinements do: until a value out of bounds is fixed, it is the one problem reported of the configuration. Annotated may also carry a note, a string or a Doc. Any other metadata -- a MinLen, a Predicate, pydantic's Field -- is refused when the definition is built, since a type the checker does not hold its values to would say what is not so.

A member holding another metadata field is annotated with the field alias of its kind -- a shard's codecs: tuple[CodecField, ...] -- and read in the scope its field is read in. What is wrong with the field it holds is that field's own, reported where it sits: the field holding it is still read, as a document holding it would be. A member that takes codecs of static size only is annotated StaticCodecField -- a shard's index_codecs, since a reader finds the index by a size it knows before reading it -- and a codec of dynamic size there is a problem at its place, where the field is read in a scope; a name nothing claims is left unjudged, its size unknown with the rest of it. ZarrV3MetadataFieldJSON is the same JSON, but checks as JSON and nothing more, so a definition refuses a member typed with it. An extension with nothing to configure takes EmptyConfiguration, and is written with its name alone.

A data type also says what its fill value is: fill_value, the JSON shape of one as an annotation the checker reads -- Int8FillValue, whose type carries the range -- and fill_value_rules, a function yielding what the spec disallows in a fill value of that shape that the type cannot say: a hex string of another width, a number of byte values the size does not take. The rules are handed the configuration, the fields it holds as the scope read them, and the typed fill value, so a struct judges each field's fill value by that field's own type. fill_value_problems(data_type, value) judges a fill value against a data type field the scope read; one nothing in scope claims leaves it unjudged. A data type that says nothing of its fill value takes any JSON. A fill value may be spelled more ways than one -- "NaN" and "0x7fc00000" are one float32 -- so a data type says which spelling is its value's own: fill_value_canonical, handed what the rules are handed and a fill value they allow. Two fill values are one value of the type exactly when their canonical spellings are written alike: the same JSON, as json.dumps writes it, which == is not -- it takes -0.0, a float32 of its own, for 0.0. canonical_fill_value(data_type, value) spells one, and gives UNSET for a fill value with a problem; a data type that says nothing of it spells each value as written.

A data type says how its values are stored, too: storage, a function of its configuration and the fields it holds, giving a StorageClass -- in single bytes, in several bytes at a time, or each in as many as it needs. A struct's is its fields'. storage_of(data_type) asks it of a data type field the scope read: the bytes codec takes an endian for numbers of several bytes, and a struct refuses a field whose values vary in size. A data type that says nothing of it leaves it unknown.

A chunk grid says which arrays it fits: shape_rules, a function yielding what the spec disallows in a grid of its configuration over an array of a given shape -- a dimension with no chunk length, chunks that fall short of one -- located in the configuration. A grid that says nothing of the shape fits every one. It also says the lengths its chunks take along each axis of an array it fits, chunk_lengths: a set per axis, since a rectilinear grid's chunks differ. A reading holds both of the grid it read: an entry for each dimension of the shape, None where nothing says the lengths.

A codec is judged against what it is handed. The array hands its first codec a Chunk: the lengths of its grid's chunks along each of the array's dimensions, and its data type field, with None for what nothing says. A codec handed an array says what the spec disallows in it handed a chunk: chunk_rules, located in its configuration -- a transpose whose order has another number of axes. An array -> array codec says what it hands the next, whatever its chunk rules found: transition -- transpose permutes the axes. A reading reads the codec fields as a pipeline: their order -- array -> array codecs, one array -> bytes codec, bytes -> bytes codecs -- and then each against the chunk it is handed, giving each codec's Stage with that chunk. A codec that holds pipelines of its own says what each is handed: pipelines, by the member of its configuration that holds each -- a shard's inner codecs its inner chunks, its index codecs the shard index -- and each is read the same way, its stages kept as the codec's Stage.inner. Nothing is guessed: the codec after one the scope did not read, or after one that says nothing of what it hands on, is handed a chunk nothing is known of, Chunk(), which is refused nothing; a codec after that hands on only what it says of its own accord.

Raw bits are the one data type whose name carries its configuration: a document writes r and the size in bits, and r16 reads as r*, as the specification's table writes raw bits, with {"bits": 16}. A reader that reads raw bits its own way defines r*; a data type named r16 is refused, since that name reads as r*. r* itself is notation, and a document that writes it names nothing in any scope.

The simplest spelling. A field without problems has a simplest equivalent spelling, which is what two fields are compared by: each nested field in its own simplest spelling, then the definition's canonical -- blosc drops a typesize that noshuffle ignores, a rectilinear grid run-length encodes its chunk shapes -- and the envelope in the fewest words every reader takes: a data type with nothing to configure is its bare name, any other field an object, {"name": ...}, as a Zarr v3.0 reader takes no short-hand name in codecs; raw bits write their size back into the name, in decimal, so r008 is r8. A field with any problem, an unknown key included, has none: a simpler spelling of it would erase what its author wrote. What canonical gives is judged again: one that does not hold is a ValueError, a fault in the definition. canonical_fill_value spells a fill value the same way, as the data type that read it spells one.

JSON Schema. node_metadata_json_schema_v3, in zarr_metadata.model, writes a whole zarr.json as a JSON Schema, draft 2020-12, for a validator in another language or an editor, its fields as the scope reads them: each definition's field -- its name, its configuration as its TypedDict says, bounds and all, a must_understand of true, and its bare name when it needs no configuration -- and a name nothing in scope claims, with any configuration. A field a configuration holds is written in the same scope, and a member taking codecs of static size only takes those. The rules are not in it, so a field it accepts may still have a problem; one resolve reads without a problem, it accepts, as JSON: arrays as lists, as a parser gives them. Each configuration TypedDict, and each field alias, is written once, in $defs, under its name; the fill value is held to its data type's.

Scopes as values. Two scopes are equal when they file the same definitions, and equal scopes hash alike. Context.joined(*scopes) is the least scope above each, or a ScopeConflictError naming each name filed two ways; extended_with remains the way to take a name over on purpose. A model's refined_in moves it to a scope that claims more and contradicts nothing, and refines orders two models by information: a name nothing claimed, read by a definition, is a gain; the reverse a loss; one name read by two definitions a conflict.

A definition checks itself when it is built, and each of these is a TypeError saying what is wrong: a configuration that is not a TypedDict, says nothing of the keys it does not declare, or has a member no checker reads, named down to the TypedDict that holds it; a name that is not a string; a member declared as a function -- rules, canonical, fill_value_rules, storage, shape_rules, chunk_lengths, chunk_rules, transition, pipelines -- that is not one; a data type's fill_value no checker reads, or one holding a metadata field, which a value of the data type never is; a codec kind that is not one of the three, or a size that is not "static" or "dynamic"; a function no codec of its kind is asked -- chunk rules or pipelines of a bytes -> bytes codec, which is handed bytes, or a transition of a codec that hands on bytes; a data type named as raw bits of one size are written. A scope refuses a definition of no kind. Nothing happens at class creation.

CORE module-attribute

CORE: Final = Context.of(*_CORE)

Only what the Zarr v3 specification defines.

CORE_AND_EXTENSIONS module-attribute

CORE_AND_EXTENSIONS: Final = Context.of(
    *_CORE, *_EXTENSIONS
)

What the specification defines, plus what zarr-extensions registers.

ChunkGridField module-attribute

ChunkGridField = TypeAliasType(
    "ChunkGridField", ZarrV3MetadataFieldJSON
)

A member holding a chunk grid: a document's chunk_grid.

ChunkKeyEncodingField module-attribute

ChunkKeyEncodingField = TypeAliasType(
    "ChunkKeyEncodingField", ZarrV3MetadataFieldJSON
)

A member holding a chunk key encoding: a document's chunk_key_encoding.

ClaimKey module-attribute

ClaimKey: TypeAlias = tuple[type[Definition[Any]], str]

A kind and the name a definition is filed under: what a scope answers claimant for.

Claims module-attribute

Claims: TypeAlias = Mapping[
    ClaimKey, Definition[Any] | None
]

What a reading claims of each name a document writes: the definition that read it, or None where nothing claimed it.

CodecField module-attribute

CodecField = TypeAliasType(
    "CodecField", ZarrV3MetadataFieldJSON
)

A member holding a codec: a document's codecs is tuple[CodecField, ...], and so is a shard's.

CodecKind module-attribute

CodecKind = Literal[
    "array_array", "array_bytes", "bytes_bytes"
]

What a codec does to what it is handed: the three positions a pipeline orders.

CodecSize module-attribute

CodecSize = Literal['static', 'dynamic']

Whether the size of what a codec gives out is fixed by the size of what it is handed.

static: it is -- bytes writes each element in its width, crc32c adds four bytes. dynamic: it depends on the values -- every compressor.

DataTypeField module-attribute

DataTypeField = TypeAliasType(
    "DataTypeField", ZarrV3MetadataFieldJSON
)

A member holding a data type: a document's data_type, or a struct field's; read in the scope what holds it is read in.

JSONValue module-attribute

JSONValue = TypeAliasType(
    "JSONValue",
    int
    | float
    | bool
    | str
    | list["JSONValue"]
    | tuple["JSONValue", ...]
    | Mapping[str, "JSONValue"]
    | None,
)

A recursive type alias for JSON-encodable values.

Defined via TypeAliasType (rather than a plain TypeAlias) so the self-reference is a named recursion point that pydantic can resolve when building a TypeAdapter; a bare recursive TypeAlias raises PydanticUserError/RecursionError at validation time.

Lengths module-attribute

Lengths: TypeAlias = tuple[frozenset[int] | None, ...]

Per axis, every length chunks take along it -- a set, since a rectilinear grid's differ -- or None where unknown.

Loc module-attribute

Loc: TypeAlias = tuple[str | int, ...]

Where in a document a value sits: the keys and indices down to it.

Nested module-attribute

Nested: TypeAlias = Mapping[Loc, ResolvedField[Any]]

The fields a configuration holds, each as the scope read it, by where it sits in the configuration.

ProblemKind module-attribute

ProblemKind = Literal[
    "missing_key",
    "invalid_type",
    "invalid_value",
    "invalid_json",
    "unknown_key",
]

Machine-readable classification of a ValidationProblem.

• missing_key: a required key (document key or store key) is absent.
• invalid_type: a value has the wrong structural type (e.g. a string where a mapping is required, a non-JSON-serializable object).
• invalid_value: a value has an acceptable type but an invalid content (e.g. zarr_format: 2 in a v3 document, order: "Q").
• invalid_json: bytes that do not decode as JSON.
• unknown_key: a key an object's type does not declare, where the type says it is closed, as a closed TypedDict does. Whether a Zarr configuration is closed is rarely said (zarr-developers/zarr-specs#270 has been open since 2023), and many readers refuse such a key. It gets a kind of its own so that a caller who tolerates it can tell it from a wrong value, and so that it never masks the other findings about the same object.

ResolvedField module-attribute

ResolvedField = TypeAliasType(
    "ResolvedField",
    "AcceptedField[D] | UnclaimedField | RefusedField[D]",
    type_params=(D,),
)

One metadata field as a scope read it: read by the definition that claims its name, claimed by nothing, or refused.

StaticCodecField module-attribute

StaticCodecField = TypeAliasType(
    "StaticCodecField", ZarrV3MetadataFieldJSON
)

A member holding a codec of static size: a shard's index_codecs is one, since a reader finds the index by a size it knows before reading it.

StorageClass module-attribute

StorageClass = Literal[
    "single_byte", "multi_byte", "variable_length"
]

How a data type's values are stored: in single bytes, in several bytes at a time, or each in as many as it needs.

A number of several bytes is stored in a byte order, which the bytes codec's endian says. A value made of single bytes -- a uint8, or a struct of int8 fields -- has no byte order, and a value whose size varies takes a codec of its own.

StorageTransformerField module-attribute

StorageTransformerField = TypeAliasType(
    "StorageTransformerField", ZarrV3MetadataFieldJSON
)

A member holding a storage transformer: a document's storage_transformers is tuple[StorageTransformerField, ...].

AcceptedField dataclass

Bases: Generic[D]

A field a definition in scope read: the name it is written with, the definition, and the configuration it allowed.

A problem with the envelope around it -- a stray member, a must_understand of false -- is reported with the field and leaves it read; so is a problem of a field its configuration holds, which is that field's own, as nested says. Two fields are equal when they read the same, however each was spelled, as field_key compares them: "bytes" and {"name": "bytes"} are one field, and so are a blosc with and without the typesize that noshuffle ignores, which the definition's canonical folds. Equal fields hash alike.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, slots=True, kw_only=True)
class AcceptedField(Generic[D]):
    """A field a definition in scope read: the name it is written with, the definition, and the configuration it allowed.

    A problem with the envelope around it -- a stray member, a
    `must_understand` of `false` -- is reported with the field and leaves
    it read; so is a problem of a field its configuration holds, which is
    that field's own, as `nested` says. Two fields are equal when they
    read the same, however each was spelled, as `field_key` compares
    them: `"bytes"` and `{"name": "bytes"}` are one field, and so are a
    blosc with and without the `typesize` that `noshuffle` ignores, which
    the definition's `canonical` folds. Equal fields hash alike.
    """

    json: JSONValue
    """The field as written, refined: arrays as tuples; it takes no part in equality."""
    name: str
    """The name it is written with: `"r16"`, though its definition is filed under `r*`."""
    definition: D
    """The definition that read it."""
    configuration: Mapping[str, JSONValue]
    """The configuration, type-checked and allowed by the rules; for raw bits, what the name carries.

    Each field it holds is written as a document writes it, as that
    field's `to_json` writes it, so the configuration says what was read
    however it was spelled: a shard's `"crc32c"` and `{"name": "crc32c"}`
    are one index codec.
    """
    nested: Nested = dataclasses.field(default_factory=_nothing_nested)
    """The fields the configuration holds, each as the scope read it, by where it sits in the configuration.

    A struct's field types at `("fields", 0, "data_type")`, a shard's
    codecs at `("codecs", 0)`: what a definition's functions consult about
    the fields inside its own.
    """
    read_as: type[Definition[Any]] = dataclasses.field(init=False, repr=False)
    """The kind of metadata it was read as: its definition's."""

    def __eq__(self, other: object) -> bool:
        if not is_field(other):
            return NotImplemented
        return field_key(self) == field_key(other)

    def __hash__(self) -> int:
        return hash(field_key(self))

    def __post_init__(self) -> None:
        # The runtime half of the annotations: a field read by hand, as an
        # extension's may be, fails here rather than where a function trusts it.
        definition = cast("object", self.definition)
        kind = kind_of(cast("Definition[Any]", definition))
        refusal = (
            f"a field read is read by a definition of a kind, got {definition!r}"
            if kind is None
            else _misread(definition, kind, self.name)
        )
        if refusal is not None:
            raise TypeError(refusal)
        object.__setattr__(self, "read_as", kind)

    def to_json(self) -> JSONValue:
        """The field as a document writes it, for every reader: its configuration as read, sharing nothing with the field.

        The envelope takes the fewest words every reader takes: a data type
        with nothing to configure is its bare name, as core data types have
        been written since Zarr v3.0; any other field is an object,
        `{"name": ...}`, since a Zarr v3.0 reader takes no bare name in
        `codecs`
        (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L585-L592).
        A name that carries its configuration, as raw bits' does, is
        written alone.
        """
        return copied(written_json(self))

configuration instance-attribute

configuration: Mapping[str, JSONValue]

The configuration, type-checked and allowed by the rules; for raw bits, what the name carries.

Each field it holds is written as a document writes it, as that field's to_json writes it, so the configuration says what was read however it was spelled: a shard's "crc32c" and {"name": "crc32c"} are one index codec.

definition instance-attribute

definition: D

The definition that read it.

json instance-attribute

json: JSONValue

The field as written, refined: arrays as tuples; it takes no part in equality.

name instance-attribute

name: str

The name it is written with: "r16", though its definition is filed under r*.

nested class-attribute instance-attribute

nested: Nested = dataclasses.field(
    default_factory=_nothing_nested
)

The fields the configuration holds, each as the scope read it, by where it sits in the configuration.

A struct's field types at ("fields", 0, "data_type"), a shard's codecs at ("codecs", 0): what a definition's functions consult about the fields inside its own.

read_as class-attribute instance-attribute

read_as: type[Definition[Any]] = dataclasses.field(
    init=False, repr=False
)

The kind of metadata it was read as: its definition's.

to_json

to_json() -> JSONValue

The field as a document writes it, for every reader: its configuration as read, sharing nothing with the field.

The envelope takes the fewest words every reader takes: a data type with nothing to configure is its bare name, as core data types have been written since Zarr v3.0; any other field is an object, {"name": ...}, since a Zarr v3.0 reader takes no bare name in codecs (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L585-L592). A name that carries its configuration, as raw bits' does, is written alone.

Source code in src/zarr_metadata/v3/_definition.py

def to_json(self) -> JSONValue:
    """The field as a document writes it, for every reader: its configuration as read, sharing nothing with the field.

    The envelope takes the fewest words every reader takes: a data type
    with nothing to configure is its bare name, as core data types have
    been written since Zarr v3.0; any other field is an object,
    `{"name": ...}`, since a Zarr v3.0 reader takes no bare name in
    `codecs`
    (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L585-L592).
    A name that carries its configuration, as raw bits' does, is
    written alone.
    """
    return copied(written_json(self))

Chunk dataclass

What a codec is handed: chunks of some lengths along each axis, of a data type.

What nothing says is None: the lengths along an axis the grid does not say, and every part of the chunk handed on by a codec that says nothing of what it hands on. A data type field the scope did not read is held as written, and says nothing of the values either. A codec's chunk rules judge what is known and leave the rest, so a chunk nothing is known of, Chunk(), is refused nothing.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, slots=True)
class Chunk:
    """What a codec is handed: chunks of some lengths along each axis, of a data type.

    What nothing says is None: the lengths along an axis the grid does not
    say, and every part of the chunk handed on by a codec that says nothing
    of what it hands on. A data type field the scope did not read is held
    as written, and says nothing of the values either. A codec's chunk
    rules judge what is known and leave the rest, so a chunk nothing is
    known of, `Chunk()`, is refused nothing.
    """

    lengths: Lengths | None = None
    """Per axis, the lengths the chunks take along it; None when not even the number of axes is known."""
    data_type: ResolvedField[DataTypeDefinition[Any]] | None = None
    """The data type field of the values, as a scope read it; None when no field says what they are: a document naming none, which its reading holds as `UNSET`, hands the pipeline a chunk of no known type."""

    def __post_init__(self) -> None:
        lengths = cast("object", self.lengths)
        if lengths is not None and not _is_lengths(lengths):
            msg = f"a chunk's lengths are a frozenset of integers or None per axis, got {lengths!r}"
            raise TypeError(msg)
        data_type = cast("object", self.data_type)
        if data_type is not None and not _is_data_type_field(data_type):
            msg = f"a chunk's data type is a data type field a scope read, got {data_type!r}"
            raise TypeError(msg)

    @property
    def rank(self) -> int | None:
        """The number of axes; None when unknown."""
        return None if self.lengths is None else len(self.lengths)

data_type class-attribute instance-attribute

data_type: ResolvedField[DataTypeDefinition[Any]] | None = (
    None
)

The data type field of the values, as a scope read it; None when no field says what they are: a document naming none, which its reading holds as UNSET, hands the pipeline a chunk of no known type.

lengths class-attribute instance-attribute

lengths: Lengths | None = None

Per axis, the lengths the chunks take along it; None when not even the number of axes is known.

rank property

rank: int | None

The number of axes; None when unknown.

ChunkGridDefinition dataclass

Bases: Definition[C]

A chunk grid, and the arrays it fits.

shape_rules is what the spec disallows in a grid of this configuration over an array of a given shape: a dimension with no chunk length, chunks that fall short of one. It is handed the configuration, the fields it holds as the scope read them, and the shape, and locates its problems in the configuration. A grid that says nothing of the shape fits every one.

chunk_lengths is what the first codec of the array's pipeline is handed: the lengths the grid's chunks take along each axis of an array of a shape it fits -- one for each axis of a regular grid, every length a rectilinear grid lists. It is asked only of a grid its shape rules accept. A grid that says nothing of it leaves the lengths along every axis unknown.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class ChunkGridDefinition(Definition[C], kind=True):
    """A chunk grid, and the arrays it fits.

    `shape_rules` is what the spec disallows in a grid of this
    configuration over an array of a given shape: a dimension with no
    chunk length, chunks that fall short of one. It is handed the
    configuration, the fields it holds as the scope read them, and the
    shape, and locates its problems in the configuration. A grid that
    says nothing of the shape fits every one.

    `chunk_lengths` is what the first codec of the array's pipeline is
    handed: the lengths the grid's chunks take along each axis of an
    array of a shape it fits -- one for each axis of a regular grid, every
    length a rectilinear grid lists. It is asked only of a grid its shape
    rules accept. A grid that says nothing of it leaves the lengths along
    every axis unknown.
    """

    label: ClassVar[str] = "chunk grid"
    field_aliases: ClassVar[tuple[TypeAliasType, ...]] = (ChunkGridField,)

    shape_rules: Callable[[C, Nested, tuple[int, ...]], Iterable[ValidationProblem]] = no_rules
    """What the spec disallows in this grid over an array of a shape, located in the configuration."""
    chunk_lengths: Callable[[C, Nested, tuple[int, ...]], Lengths] = unknown_lengths
    """The lengths its chunks take along each axis of an array of a shape it fits, None where unknown."""

chunk_lengths class-attribute instance-attribute

chunk_lengths: Callable[
    [C, Nested, tuple[int, ...]], Lengths
] = unknown_lengths

The lengths its chunks take along each axis of an array of a shape it fits, None where unknown.

field_aliases class-attribute

field_aliases: tuple[TypeAliasType, ...] = (ChunkGridField,)

The field aliases a configuration member holding a field of this kind is annotated with: CodecField and StaticCodecField for a codec.

label class-attribute

label: str = 'chunk grid'

The kind as a message names it: "codec".

shape_rules class-attribute instance-attribute

shape_rules: Callable[
    [C, Nested, tuple[int, ...]],
    Iterable[ValidationProblem],
] = no_rules

What the spec disallows in this grid over an array of a shape, located in the configuration.

ChunkKeyEncodingDefinition dataclass

Bases: Definition[C]

A chunk key encoding.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class ChunkKeyEncodingDefinition(Definition[C], kind=True):
    """A chunk key encoding."""

    label: ClassVar[str] = "chunk key encoding"
    field_aliases: ClassVar[tuple[TypeAliasType, ...]] = (ChunkKeyEncodingField,)

field_aliases class-attribute

field_aliases: tuple[TypeAliasType, ...] = (
    ChunkKeyEncodingField,
)

The field aliases a configuration member holding a field of this kind is annotated with: CodecField and StaticCodecField for a codec.

label class-attribute

label: str = 'chunk key encoding'

The kind as a message names it: "codec".

CodecDefinition dataclass

Bases: Definition[C]

A codec: what it does to what it is handed, and whether the size of what it gives out is static.

A codec handed an array -- array -> array, array -> bytes -- says what the spec disallows in it handed a Chunk: chunk_rules, handed the configuration, the fields it holds as the scope read them, and the chunk, and locating its problems in the configuration -- a bytes codec without endian, handed a multi-byte data type. An array -> array codec also says what it hands on: transition, the chunk the next codec is handed, given the one it is handed -- transpose permutes the axes, cast_value changes the data type. The two are the spec's pair: a codec computes what it gives from the shape and data type it is handed, and "If the decoded_representation_type is not supported, this algorithm must fail with an error" (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L987-L994). The transition is asked of every chunk the codec is handed, whatever its chunk rules found, so it gives only what holds either way: a transpose whose order has another number of axes hands on lengths nothing is known of. A codec that says nothing of what it hands on hands the next a chunk nothing is known of.

A codec that holds pipelines of its own says what each is handed: pipelines, by the member of its configuration that holds each, the chunk its first codec is handed, given the chunk the codec is handed -- a shard's inner codecs are handed its inner chunks, and its index codecs the shard index. Like the transition, it is asked whatever the chunk rules found, and gives only what holds either way. A function no codec of its kind is asked -- the chunk rules of a bytes -> bytes codec, which is handed bytes -- is refused.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class CodecDefinition(Definition[C], kind=True):
    """A codec: what it does to what it is handed, and whether the size of what it gives out is static.

    A codec handed an array -- array -> array, array -> bytes -- says what
    the spec disallows in it handed a `Chunk`: `chunk_rules`, handed the
    configuration, the fields it holds as the scope read them, and the
    chunk, and locating its problems in the configuration -- a `bytes`
    codec without `endian`, handed a multi-byte data type. An array ->
    array codec also says what it hands on: `transition`, the chunk the
    next codec is handed, given the one it is handed -- `transpose`
    permutes the axes, `cast_value` changes the data type. The two are the
    spec's pair: a codec computes what it gives from the shape and data
    type it is handed, and "If the decoded_representation_type is not
    supported, this algorithm must fail with an error"
    (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L987-L994).
    The transition is asked of every chunk the codec is handed, whatever
    its chunk rules found, so it gives only what holds either way: a
    `transpose` whose `order` has another number of axes hands on lengths
    nothing is known of. A codec that says nothing of what it hands on
    hands the next a chunk nothing is known of.

    A codec that holds pipelines of its own says what each is handed:
    `pipelines`, by the member of its configuration that holds each, the
    chunk its first codec is handed, given the chunk the codec is handed
    -- a shard's inner codecs are handed its inner chunks, and its index
    codecs the shard index. Like the transition, it is asked whatever the
    chunk rules found, and gives only what holds either way. A function
    no codec of its kind is asked -- the chunk rules of a bytes -> bytes
    codec, which is handed bytes -- is refused.
    """

    label: ClassVar[str] = "codec"
    field_aliases: ClassVar[tuple[TypeAliasType, ...]] = (CodecField, StaticCodecField)

    kind: CodecKind
    size: CodecSize
    chunk_rules: Callable[[C, Nested, Chunk], Iterable[ValidationProblem]] = no_rules
    """What the spec disallows in this codec handed a chunk, located in the configuration."""
    transition: Callable[[C, Nested, Chunk], Chunk] = unknown_chunk
    """The chunk the next codec is handed, given the one this array -> array codec is handed."""
    pipelines: Callable[[C, Nested, Chunk], Mapping[str, Chunk]] = no_pipelines
    """The pipelines it holds, by the member of its configuration that holds each, and the chunk each is handed."""

    def _refusal(self) -> str | None:
        kind: object = self.kind
        if kind not in get_args(CodecKind):
            return f"{self.name!r}: kind is one of {get_args(CodecKind)!r}, got {kind!r}"
        size: object = self.size
        if size not in get_args(CodecSize):
            return f"{self.name!r}: size is one of {get_args(CodecSize)!r}, got {size!r}"
        defaults = {member.name: member.default for member in dataclasses.fields(CodecDefinition)}
        for member in _UNASKED[self.kind]:
            if getattr(self, member) is not defaults[member]:
                return f"{self.name!r}: {member}, which no codec of kind {kind!r} is asked"
        return None

chunk_rules class-attribute instance-attribute

chunk_rules: Callable[
    [C, Nested, Chunk], Iterable[ValidationProblem]
] = no_rules

What the spec disallows in this codec handed a chunk, located in the configuration.

field_aliases class-attribute

field_aliases: tuple[TypeAliasType, ...] = (
    CodecField,
    StaticCodecField,
)

The field aliases a configuration member holding a field of this kind is annotated with: CodecField and StaticCodecField for a codec.

label class-attribute

label: str = 'codec'

The kind as a message names it: "codec".

pipelines class-attribute instance-attribute

pipelines: Callable[
    [C, Nested, Chunk], Mapping[str, Chunk]
] = no_pipelines

The pipelines it holds, by the member of its configuration that holds each, and the chunk each is handed.

transition class-attribute instance-attribute

transition: Callable[[C, Nested, Chunk], Chunk] = (
    unknown_chunk
)

The chunk the next codec is handed, given the one this array -> array codec is handed.

Conflict dataclass

One place two readings of a name disagree: the key, what one claimed, what the other found, and where in a document when known.

Source code in src/zarr_metadata/v3/_scope.py

@dataclass(frozen=True, slots=True)
class Conflict:
    """One place two readings of a name disagree: the key, what one claimed, what the other found, and where in a document when known."""

    key: ClaimKey
    claimed: Definition[Any] | None
    found: Definition[Any] | None
    loc: Loc | None = None

    def __str__(self) -> str:
        kind, name = self.key
        where = "" if self.loc is None else f" at {self.loc!r}"
        return f"{kind_name(kind)} {name!r}{where}: claimed {self.claimed!r}, found {self.found!r}"

Context dataclass

The definitions in scope while metadata is read.

A value with no reading of its own: resolve reads a field in it, and claimant is the one question it answers, which definition a name belongs to. Built from definitions with Context.of, extended with more by extended_with; two scopes are equal when they file the same definitions, and equal scopes hash alike.

Source code in src/zarr_metadata/v3/_registry.py

@dataclass(frozen=True, slots=True, eq=False)
class Context:
    """The definitions in scope while metadata is read.

    A value with no reading of its own: `resolve` reads a field in it,
    and `claimant` is the one question it answers, which definition a
    name belongs to. Built from definitions with `Context.of`, extended
    with more by `extended_with`; two scopes are equal when they file the
    same definitions, and equal scopes hash alike.
    """

    tables: Tables

    @classmethod
    def of(cls, *definitions: Definition[Any]) -> Context:
        """A scope of exactly these definitions; a later one takes a name over from an earlier.

        `TypeError` for a definition of no kind, which no position in a
        document could hold.
        """
        tables: dict[type[Definition[Any]], dict[str, Definition[Any]]] = {}
        for definition in definitions:
            kind = kind_of(definition)
            if kind is None:
                msg = (
                    f"{definition.name!r} is a definition of no kind; build it as a "
                    "CodecDefinition, DataTypeDefinition, ChunkGridDefinition, "
                    "ChunkKeyEncodingDefinition or StorageTransformerDefinition, or as a "
                    "kind of your own"
                )
                raise TypeError(msg)
            tables.setdefault(kind, {})[definition.name] = definition
        return cls(
            MappingProxyType({kind: MappingProxyType(table) for kind, table in tables.items()})
        )

    def extended_with(self, *definitions: Definition[Any]) -> Context:
        """This scope, plus definitions of your own.

        A name already filed under the same kind is taken over by what is
        passed here, which is how a reader substitutes its own reading of a
        codec the package already defines -- or of raw bits, by defining
        `r*`.
        """
        return Context.of(*self.definitions(), *definitions)

    def definitions(self) -> tuple[Definition[Any], ...]:
        """Every definition in scope, kind by kind."""
        return tuple(entry for table in self.tables.values() for entry in table.values())

    def __repr__(self) -> str:
        # Short, as a default argument shows it: in full, a scope's repr is
        # every definition's, and `help` of a validator runs to pages.
        return f"Context(<{len(self.definitions())} definitions>)"

    def __reduce__(self) -> tuple[Callable[..., Context], tuple[Definition[Any], ...]]:
        # A scope is its definitions, so it pickles as them, and goes to
        # another process with the documents it is to read there.
        return (Context.of, self.definitions())

    def __copy__(self) -> Context:
        return self

    def __deepcopy__(self, memo: dict[int, object]) -> Context:
        # A scope never changes, so a copy of it is itself.
        return self

    def __eq__(self, other: object) -> bool:
        if not isinstance(other, Context):
            return NotImplemented
        return self._filed() == other._filed()

    def __hash__(self) -> int:
        return hash(self._filed())

    def _filed(self) -> frozenset[tuple[type[Definition[Any]], str, Definition[Any]]]:
        """Every definition in scope with the kind and name it is filed under: what two scopes are compared by."""
        return frozenset(
            (kind, name, definition)
            for kind, table in self.tables.items()
            for name, definition in table.items()
        )

    def disagreements(self, claims: Claims) -> Disagreements:
        """Where this scope reads `claims`, a reading's, otherwise: what it would gain, and what it conflicts with, as `Disagreements` says.

        A claim is keyed by the name its definition is filed under -- raw
        bits under `r*` -- so it is looked up as filed, not as a document
        writes it.
        """
        return disagreements_of(lambda kind, name: self.tables.get(kind, {}).get(name), claims)

    @classmethod
    def joined(cls, *contexts: Context) -> Context:
        """The least scope that files everything each of `contexts` files: their join.

        `ScopeConflictError` when two of them file different definitions
        under one name of one kind; `extended_with` is for taking a name
        over on purpose.
        """
        filed: dict[tuple[type[Definition[Any]], str], Definition[Any]] = {}
        conflicts: list[Conflict] = []
        for context in contexts:
            for kind, table in context.tables.items():
                for name, definition in table.items():
                    held = filed.get((kind, name))
                    if held is not None and held != definition:
                        conflicts.append(Conflict((kind, name), held, definition))
                        continue
                    filed[kind, name] = definition
        if len(conflicts) != 0:
            raise ScopeConflictError(conflicts)
        return cls.of(*filed.values())

    def claimant(self, kind: type[D], name: str) -> D | None:
        """The definition of `kind` in scope that reads `name`, a name a document writes; None if none does.

        The one filed under the name `spelled` reads it as: itself, but
        for raw bits, `r16` read by the definition of `r*`.
        """
        asked = as_kind(kind)
        filed, _ = spelled(asked, name)
        if filed is None:
            return None
        return cast("D | None", self.tables.get(asked, {}).get(filed))

claimant

claimant(kind: type[D], name: str) -> D | None

The definition of kind in scope that reads name, a name a document writes; None if none does.

The one filed under the name spelled reads it as: itself, but for raw bits, r16 read by the definition of r*.

Source code in src/zarr_metadata/v3/_registry.py

def claimant(self, kind: type[D], name: str) -> D | None:
    """The definition of `kind` in scope that reads `name`, a name a document writes; None if none does.

    The one filed under the name `spelled` reads it as: itself, but
    for raw bits, `r16` read by the definition of `r*`.
    """
    asked = as_kind(kind)
    filed, _ = spelled(asked, name)
    if filed is None:
        return None
    return cast("D | None", self.tables.get(asked, {}).get(filed))

definitions

definitions() -> tuple[Definition[Any], ...]

Every definition in scope, kind by kind.

Source code in src/zarr_metadata/v3/_registry.py

def definitions(self) -> tuple[Definition[Any], ...]:
    """Every definition in scope, kind by kind."""
    return tuple(entry for table in self.tables.values() for entry in table.values())

disagreements

disagreements(claims: Claims) -> Disagreements

Where this scope reads claims, a reading's, otherwise: what it would gain, and what it conflicts with, as Disagreements says.

A claim is keyed by the name its definition is filed under -- raw bits under r* -- so it is looked up as filed, not as a document writes it.

Source code in src/zarr_metadata/v3/_registry.py

def disagreements(self, claims: Claims) -> Disagreements:
    """Where this scope reads `claims`, a reading's, otherwise: what it would gain, and what it conflicts with, as `Disagreements` says.

    A claim is keyed by the name its definition is filed under -- raw
    bits under `r*` -- so it is looked up as filed, not as a document
    writes it.
    """
    return disagreements_of(lambda kind, name: self.tables.get(kind, {}).get(name), claims)

extended_with

extended_with(*definitions: Definition[Any]) -> Context

This scope, plus definitions of your own.

A name already filed under the same kind is taken over by what is passed here, which is how a reader substitutes its own reading of a codec the package already defines -- or of raw bits, by defining r*.

Source code in src/zarr_metadata/v3/_registry.py

def extended_with(self, *definitions: Definition[Any]) -> Context:
    """This scope, plus definitions of your own.

    A name already filed under the same kind is taken over by what is
    passed here, which is how a reader substitutes its own reading of a
    codec the package already defines -- or of raw bits, by defining
    `r*`.
    """
    return Context.of(*self.definitions(), *definitions)

joined classmethod

joined(*contexts: Context) -> Context

The least scope that files everything each of contexts files: their join.

ScopeConflictError when two of them file different definitions under one name of one kind; extended_with is for taking a name over on purpose.

Source code in src/zarr_metadata/v3/_registry.py

@classmethod
def joined(cls, *contexts: Context) -> Context:
    """The least scope that files everything each of `contexts` files: their join.

    `ScopeConflictError` when two of them file different definitions
    under one name of one kind; `extended_with` is for taking a name
    over on purpose.
    """
    filed: dict[tuple[type[Definition[Any]], str], Definition[Any]] = {}
    conflicts: list[Conflict] = []
    for context in contexts:
        for kind, table in context.tables.items():
            for name, definition in table.items():
                held = filed.get((kind, name))
                if held is not None and held != definition:
                    conflicts.append(Conflict((kind, name), held, definition))
                    continue
                filed[kind, name] = definition
    if len(conflicts) != 0:
        raise ScopeConflictError(conflicts)
    return cls.of(*filed.values())

of classmethod

of(*definitions: Definition[Any]) -> Context

A scope of exactly these definitions; a later one takes a name over from an earlier.

TypeError for a definition of no kind, which no position in a document could hold.

Source code in src/zarr_metadata/v3/_registry.py

@classmethod
def of(cls, *definitions: Definition[Any]) -> Context:
    """A scope of exactly these definitions; a later one takes a name over from an earlier.

    `TypeError` for a definition of no kind, which no position in a
    document could hold.
    """
    tables: dict[type[Definition[Any]], dict[str, Definition[Any]]] = {}
    for definition in definitions:
        kind = kind_of(definition)
        if kind is None:
            msg = (
                f"{definition.name!r} is a definition of no kind; build it as a "
                "CodecDefinition, DataTypeDefinition, ChunkGridDefinition, "
                "ChunkKeyEncodingDefinition or StorageTransformerDefinition, or as a "
                "kind of your own"
            )
            raise TypeError(msg)
        tables.setdefault(kind, {})[definition.name] = definition
    return cls(
        MappingProxyType({kind: MappingProxyType(table) for kind, table in tables.items()})
    )

DataTypeDefinition dataclass

Bases: WithFillValue[C]

A data type, and the fill value an array of it takes.

fill_value is the JSON shape of a fill value -- Int8FillValue, an annotation the checker reads as it reads a configuration's members, its range among it -- and fill_value_rules is what the spec disallows in a fill value of that shape that the type cannot say: a hex string of another width. The rules are handed the configuration, the fields it holds as the scope read them (a struct's field types), and the typed fill value. A data type that says nothing of its fill value takes any JSON.

fill_value_canonical spells a fill value that has no problem -- well typed, and allowed by the rules -- in the one spelling its value has, so two fill values are one value of the type exactly when their canonical spellings are written alike: "NaN" and "0x7fc00000" are one float32, and 0.0 and -0.0 two. It is handed what the rules are handed. A data type that says nothing of it spells each of its values one way: as written.

storage says how its values are stored -- in single bytes, in several bytes at a time, or each in as many as it needs -- which is what the bytes codec asks of the data type it is handed: an endian, for numbers of several bytes. A struct's is its fields', so it is handed the fields the configuration holds as the scope read them. A data type that says nothing of it leaves it unknown.

One named as a document writes raw bits of one size -- r16 -- is refused: that name reads as r*, so nothing would ever read it with this definition.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class DataTypeDefinition(WithFillValue[C], kind=True):
    """A data type, and the fill value an array of it takes.

    `fill_value` is the JSON shape of a fill value -- `Int8FillValue`, an
    annotation the checker reads as it reads a configuration's members,
    its range among it -- and `fill_value_rules` is what the spec
    disallows in a fill value of that shape that the type cannot say: a
    hex string of another width.
    The rules are handed the configuration, the fields it holds as the
    scope read them (a struct's field types), and the typed fill value. A
    data type that says nothing of its fill value takes any JSON.

    `fill_value_canonical` spells a fill value that has no problem --
    well typed, and allowed by the rules -- in the one spelling its value
    has, so two fill values are one value of the type exactly when their
    canonical spellings are written alike: `"NaN"` and `"0x7fc00000"` are
    one `float32`, and `0.0` and `-0.0` two. It is handed what the rules
    are handed. A data type that says nothing of it spells each of its
    values one way: as written.

    `storage` says how its values are stored -- in single bytes, in
    several bytes at a time, or each in as many as it needs -- which is
    what the `bytes` codec asks of the data type it is handed: an
    `endian`, for numbers of several bytes. A struct's is its fields', so
    it is handed the fields the configuration holds as the scope read
    them. A data type that says nothing of it leaves it unknown.

    One named as a document writes raw bits of one size -- `r16` -- is
    refused: that name reads as `r*`, so nothing would ever read it with
    this definition.
    """

    label: ClassVar[str] = "data type"
    field_aliases: ClassVar[tuple[TypeAliasType, ...]] = (DataTypeField,)

    @classmethod
    def spelled(cls, name: str) -> tuple[str | None, dict[str, JSONValue] | None]:
        if name == RAW_BYTES_NAME:
            return None, None
        written = RAW_BYTES_NAME_PATTERN.fullmatch(name)
        if written is not None:
            return RAW_BYTES_NAME, {"bits": int(written.group(1))}
        return name, None

    def carrying_name(self, configuration: Mapping[str, JSONValue]) -> str | None:
        if self.name != RAW_BYTES_NAME:
            return None
        return f"r{configuration['bits']}"

    @classmethod
    def envelope_json(cls, name: str, configuration: Mapping[str, JSONValue]) -> JSONValue:
        # A data type with nothing to configure is its bare name, as core
        # data types have been written since Zarr v3.0.
        if len(configuration) != 0:
            return {"name": name, "configuration": configuration}
        return name

    storage: Callable[[C, Nested], StorageClass | None] = unknown_storage
    """How its values are stored, given the configuration and the fields it holds; None when unknown."""

    def _refusal(self) -> str | None:
        if RAW_BYTES_NAME_PATTERN.fullmatch(self.name) is not None:
            return (
                f"{self.name!r} is how a document writes raw bits of one size, which read as "
                f"{RAW_BYTES_NAME!r}; to read raw bits your own way, define {RAW_BYTES_NAME!r}"
            )
        # Named, not a bare `super()`: a dataclass with slots is rebuilt.
        return super(DataTypeDefinition, self)._refusal()

field_aliases class-attribute

field_aliases: tuple[TypeAliasType, ...] = (DataTypeField,)

The field aliases a configuration member holding a field of this kind is annotated with: CodecField and StaticCodecField for a codec.

label class-attribute

label: str = 'data type'

The kind as a message names it: "codec".

storage class-attribute instance-attribute

storage: Callable[[C, Nested], StorageClass | None] = (
    unknown_storage
)

How its values are stored, given the configuration and the fields it holds; None when unknown.

carrying_name

carrying_name(
    configuration: Mapping[str, JSONValue],
) -> str | None

The name that carries configuration for this definition, the inverse of spelled: r16 for r* with {"bits": 16}; None when its names carry nothing.

Source code in src/zarr_metadata/v3/_definition.py

def carrying_name(self, configuration: Mapping[str, JSONValue]) -> str | None:
    if self.name != RAW_BYTES_NAME:
        return None
    return f"r{configuration['bits']}"

envelope_json classmethod

envelope_json(
    name: str, configuration: Mapping[str, JSONValue]
) -> JSONValue

A field of name and configuration as a document of the format writes it, in the fewest words every reader takes: for v3, an object.

Source code in src/zarr_metadata/v3/_definition.py

@classmethod
def envelope_json(cls, name: str, configuration: Mapping[str, JSONValue]) -> JSONValue:
    # A data type with nothing to configure is its bare name, as core
    # data types have been written since Zarr v3.0.
    if len(configuration) != 0:
        return {"name": name, "configuration": configuration}
    return name

spelled classmethod

spelled(
    name: str,
) -> tuple[str | None, dict[str, JSONValue] | None]

How a name a document writes reads: the name its definition is filed under, and the configuration the name carries.

A name is filed as itself and carries nothing, (name, None). A kind whose names carry configuration says otherwise: a v3 data type r16 is filed under r* with {"bits": 16}. A name no document writes, which only files a definition, is (None, None).

Source code in src/zarr_metadata/v3/_definition.py

@classmethod
def spelled(cls, name: str) -> tuple[str | None, dict[str, JSONValue] | None]:
    if name == RAW_BYTES_NAME:
        return None, None
    written = RAW_BYTES_NAME_PATTERN.fullmatch(name)
    if written is not None:
        return RAW_BYTES_NAME, {"bits": int(written.group(1))}
    return name, None

Definition dataclass

Bases: Generic[C]

One extension's metadata, as JSON: its name, the TypedDict its configuration is, its rules.

configuration is the TypedDict, and so the one declaration of the JSON: the checker is compiled from it, the static type of a checked configuration is it, and a document's author writes to it. It reads as the typing spec defines it -- total, Required, NotRequired, closed and extra_items mean what they mean to a type checker -- and it says what a key it does not declare is: with closed=True, a problem, reported and left out; with extra_items=, a key of that type; with closed=False, anything. rules yields what the spec disallows in a configuration of that type, as it finds each; it is handed only a configuration that has passed the check, holding what the TypedDict admits and nothing else, each member within the bounds its type carries, and the fields it holds as the scope read them -- a struct's field types -- which is nothing when no scope read it. read_configuration is the two, for a caller holding JSON.

Each function is handed the configuration as a read-only view, no dict: copy.deepcopy and json.dumps refuse it, and a function that folds a spelling builds a new mapping, {**configuration} without the member, rather than editing what it was handed.

canonical is where two spellings of the configuration that mean the same thing are made one.

Built by hand, a definition refuses what it could not read with: a configuration that is not a TypedDict, says nothing of the keys it does not declare, or has a member no checker reads, which is named; and a name or rules that are not what they say.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True)
class Definition(Generic[C]):
    """One extension's metadata, as JSON: its name, the TypedDict its configuration is, its rules.

    `configuration` is the TypedDict, and so the one declaration of the
    JSON: the checker is compiled from it, the static type of a checked
    configuration is it, and a document's author writes to it. It reads
    as the typing spec defines it -- `total`, `Required`, `NotRequired`,
    `closed` and `extra_items` mean what they mean to a type checker --
    and it says what a key it does not declare is: with `closed=True`, a
    problem, reported and left out; with `extra_items=`, a key of that
    type; with `closed=False`, anything. `rules` yields what the spec
    disallows in a configuration of that type, as it finds each; it is
    handed only a configuration that has passed the check, holding what
    the TypedDict admits and nothing else, each member within the bounds
    its type carries, and the fields it holds as the scope read them -- a
    struct's field types -- which is nothing when no scope read it.
    `read_configuration` is the two, for a caller holding JSON.

    Each function is handed the configuration as a read-only view, no
    `dict`: `copy.deepcopy` and `json.dumps` refuse it, and a function that
    folds a spelling builds a new mapping, `{**configuration}` without the
    member, rather than editing what it was handed.

    `canonical` is where two spellings of the configuration that mean the
    same thing are made one.

    Built by hand, a definition refuses what it could not read with: a
    `configuration` that is not a TypedDict, says nothing of the keys it
    does not declare, or has a member no checker reads, which is named;
    and a `name` or rules that are not what they say.
    """

    _is_kind: ClassVar[bool] = False
    """Whether this class is a kind, what a scope files definitions by: set by `class Kind(Definition, kind=True)`.

    Read from a class's own namespace, never inherited: a subclass of a
    kind is a definition of that kind, and declares no `kind`.
    """
    label: ClassVar[str] = "definition"
    """The kind as a message names it: "codec"."""
    field_aliases: ClassVar[tuple[TypeAliasType, ...]] = ()
    """The field aliases a configuration member holding a field of this kind is annotated with: `CodecField` and `StaticCodecField` for a codec."""

    name: str
    """The name the metadata carries, which a scope files the definition under."""
    configuration: type[C]
    """The TypedDict the configuration is."""
    rules: Callable[[C, Nested], Iterable[ValidationProblem]] = no_rules
    """What the spec disallows in a well-typed configuration and the fields it holds, located in it."""
    canonical: Callable[[C], C] = unchanged
    """A well-typed, allowed configuration in its simplest equivalent spelling.

    Only the definition's own members: a nested field is put in its own
    canonical form by `canonicalize`, which knows where each one sits.
    """

    @classmethod
    def name_problem(cls, name: str, at: Loc) -> ValidationProblem | None:
        """The problem `name`, at `at`, is when no document of the format writes it for a field of this kind; None when one may: for Zarr v3, when the spec gives an extension such a name."""
        return name_problem(name, at)

    @classmethod
    def well_named(cls, name: str) -> bool:
        """Whether a document of the format may write `name` for a field of this kind, as `name_problem` says."""
        return cls.name_problem(name, ()) is None

    @classmethod
    def spelled(cls, name: str) -> tuple[str | None, dict[str, JSONValue] | None]:
        """How a name a document writes reads: the name its definition is filed under, and the configuration the name carries.

        A name is filed as itself and carries nothing, `(name, None)`. A
        kind whose names carry configuration says otherwise: a v3 data
        type `r16` is filed under `r*` with `{"bits": 16}`. A name no
        document writes, which only files a definition, is `(None, None)`.
        """
        return name, None

    def carrying_name(self, configuration: Mapping[str, JSONValue]) -> str | None:
        """The name that carries `configuration` for this definition, the inverse of `spelled`: `r16` for `r*` with `{"bits": 16}`; None when its names carry nothing."""
        return None

    @classmethod
    def named_configuration(
        cls, value: object
    ) -> tuple[str | None, Mapping[str, object] | None, Problems]:
        """`value`, a field as a document of the format writes it, split into `(name, configuration, problems)`, as the module's `named_configuration` splits a v3 field."""
        return named_configuration(value)

    @classmethod
    def envelope_problems(cls, value: object) -> Problems:
        """Every reason `value` is not a field's envelope as the format writes one for this kind, what the configuration holds left unjudged."""
        return envelope_problems(value, allow_must_understand_false=False)

    @classmethod
    def envelope_json(cls, name: str, configuration: Mapping[str, JSONValue]) -> JSONValue:
        """A field of `name` and `configuration` as a document of the format writes it, in the fewest words every reader takes: for v3, an object."""
        if len(configuration) != 0:
            return {"name": name, "configuration": configuration}
        return {"name": name}

    @classmethod
    def configuration_loc(cls, loc: Loc) -> Loc:
        """Where the configuration of a field at `loc` sits: under `configuration` for v3; at the field for a format that writes the parameters beside the name."""
        return (*loc, "configuration")

    @classmethod
    def name_loc(cls, loc: Loc) -> Loc:
        """Where the name of a field at `loc`, written as an object, sits: under `name` for v3."""
        return (*loc, "name")

    def __init_subclass__(cls, *, kind: bool = False, **kwargs: object) -> None:
        """Files a subclass: `kind=True` declares a kind, as `typing.Protocol` and SQLAlchemy's `__abstract__` mark a class and not its subclasses."""
        # Named, not `super()`: a dataclass with slots is rebuilt, and the
        # cell a bare `super()` reads names the class that was thrown away.
        super(Definition, cls).__init_subclass__(**kwargs)
        # A dataclass with slots is built twice, the second time without
        # the class keywords but with the first class's namespace, and the
        # class built last is the one a document is read with: the mark is
        # kept in the namespace, and the aliases are filed last.
        if kind:
            cls._is_kind = True
        for alias in cls.__dict__.get("field_aliases", ()):
            _FIELD_KINDS[alias] = cls

    def __post_init__(self) -> None:
        refusal = _malformed(self) or self._refusal()
        if refusal is not None:
            raise TypeError(refusal)
        try:
            _vet(self.configuration)
        except TypeError as error:
            msg = f"{self.name!r}: {error}"
            raise TypeError(msg) from error

    def __repr__(self) -> str:
        # Short, as a reading that holds definitions shows them: in full, a
        # definition's repr is each function it holds, at its address.
        return f"{type(self).__name__}(name={self.name!r})"

    def _refusal(self) -> str | None:
        """What is wrong with the members a kind adds; None when nothing is, or it adds none."""
        return None

    @property
    def requires_configuration(self) -> bool:
        """Whether a document must write a configuration: whether the TypedDict has a required key."""
        return len(typeddict_keys(self.configuration).required) != 0

    def _check_configuration(self, value: object, loc: Loc = ()) -> tuple[C | None, Problems]:
        """`value` type-checked as this definition's configuration, each nested field's envelope judged.

        `zarr_metadata.typed_json.check` is the type check alone; this also
        judges the envelope of each metadata field a member holds.
        """
        configuration, problems = _configuration_checked(value, self.configuration, loc)
        return configuration, with_input(problems, value, loc)

    def read_configuration(self, value: object, loc: Loc = ()) -> tuple[C | None, Problems]:
        """`value` type-checked, then judged by the rules: the configuration if it holds, and every problem.

        The rules are asked only of a configuration that type-checked,
        its bounds kept, and whose nested fields are well formed, holding
        what its TypedDict admits and nothing else, so a caller holding
        JSON never reaches a rule with a member of the wrong type, out of
        its bounds, or one the type says cannot be there. No scope reads the fields it holds, so the rules see
        none of them read, and a rule about one -- a struct's field of a
        type whose values vary in size -- finds nothing to judge: `resolve`
        reads the field in a scope, and asks every rule.
        """
        configuration, problems = self._check_configuration(value, loc)
        if configuration is None:
            return None, problems
        refused = ruled(self, lambda: self.rules(read_only(configuration), _nothing_nested()), loc)
        return (configuration if len(refused) == 0 else None), (
            *problems,
            *with_input(refused, value, loc),
        )

canonical class-attribute instance-attribute

canonical: Callable[[C], C] = unchanged

A well-typed, allowed configuration in its simplest equivalent spelling.

Only the definition's own members: a nested field is put in its own canonical form by canonicalize, which knows where each one sits.

configuration instance-attribute

configuration: type[C]

The TypedDict the configuration is.

field_aliases class-attribute

field_aliases: tuple[TypeAliasType, ...] = ()

The field aliases a configuration member holding a field of this kind is annotated with: CodecField and StaticCodecField for a codec.

label class-attribute

label: str = 'definition'

The kind as a message names it: "codec".

name instance-attribute

name: str

The name the metadata carries, which a scope files the definition under.

requires_configuration property

requires_configuration: bool

Whether a document must write a configuration: whether the TypedDict has a required key.

rules class-attribute instance-attribute

rules: Callable[
    [C, Nested], Iterable[ValidationProblem]
] = no_rules

What the spec disallows in a well-typed configuration and the fields it holds, located in it.

__init_subclass__

__init_subclass__(
    *, kind: bool = False, **kwargs: object
) -> None

Files a subclass: kind=True declares a kind, as typing.Protocol and SQLAlchemy's __abstract__ mark a class and not its subclasses.

Source code in src/zarr_metadata/v3/_definition.py

def __init_subclass__(cls, *, kind: bool = False, **kwargs: object) -> None:
    """Files a subclass: `kind=True` declares a kind, as `typing.Protocol` and SQLAlchemy's `__abstract__` mark a class and not its subclasses."""
    # Named, not `super()`: a dataclass with slots is rebuilt, and the
    # cell a bare `super()` reads names the class that was thrown away.
    super(Definition, cls).__init_subclass__(**kwargs)
    # A dataclass with slots is built twice, the second time without
    # the class keywords but with the first class's namespace, and the
    # class built last is the one a document is read with: the mark is
    # kept in the namespace, and the aliases are filed last.
    if kind:
        cls._is_kind = True
    for alias in cls.__dict__.get("field_aliases", ()):
        _FIELD_KINDS[alias] = cls

carrying_name

carrying_name(
    configuration: Mapping[str, JSONValue],
) -> str | None

The name that carries configuration for this definition, the inverse of spelled: r16 for r* with {"bits": 16}; None when its names carry nothing.

Source code in src/zarr_metadata/v3/_definition.py

def carrying_name(self, configuration: Mapping[str, JSONValue]) -> str | None:
    """The name that carries `configuration` for this definition, the inverse of `spelled`: `r16` for `r*` with `{"bits": 16}`; None when its names carry nothing."""
    return None

configuration_loc classmethod

configuration_loc(loc: Loc) -> Loc

Where the configuration of a field at loc sits: under configuration for v3; at the field for a format that writes the parameters beside the name.

Source code in src/zarr_metadata/v3/_definition.py

@classmethod
def configuration_loc(cls, loc: Loc) -> Loc:
    """Where the configuration of a field at `loc` sits: under `configuration` for v3; at the field for a format that writes the parameters beside the name."""
    return (*loc, "configuration")

envelope_json classmethod

envelope_json(
    name: str, configuration: Mapping[str, JSONValue]
) -> JSONValue

A field of name and configuration as a document of the format writes it, in the fewest words every reader takes: for v3, an object.

Source code in src/zarr_metadata/v3/_definition.py

@classmethod
def envelope_json(cls, name: str, configuration: Mapping[str, JSONValue]) -> JSONValue:
    """A field of `name` and `configuration` as a document of the format writes it, in the fewest words every reader takes: for v3, an object."""
    if len(configuration) != 0:
        return {"name": name, "configuration": configuration}
    return {"name": name}

envelope_problems classmethod

envelope_problems(value: object) -> Problems

Every reason value is not a field's envelope as the format writes one for this kind, what the configuration holds left unjudged.

Source code in src/zarr_metadata/v3/_definition.py

@classmethod
def envelope_problems(cls, value: object) -> Problems:
    """Every reason `value` is not a field's envelope as the format writes one for this kind, what the configuration holds left unjudged."""
    return envelope_problems(value, allow_must_understand_false=False)

name_loc classmethod

name_loc(loc: Loc) -> Loc

Where the name of a field at loc, written as an object, sits: under name for v3.

Source code in src/zarr_metadata/v3/_definition.py

@classmethod
def name_loc(cls, loc: Loc) -> Loc:
    """Where the name of a field at `loc`, written as an object, sits: under `name` for v3."""
    return (*loc, "name")

name_problem classmethod

name_problem(
    name: str, at: Loc
) -> ValidationProblem | None

The problem name, at at, is when no document of the format writes it for a field of this kind; None when one may: for Zarr v3, when the spec gives an extension such a name.

Source code in src/zarr_metadata/v3/_definition.py

@classmethod
def name_problem(cls, name: str, at: Loc) -> ValidationProblem | None:
    """The problem `name`, at `at`, is when no document of the format writes it for a field of this kind; None when one may: for Zarr v3, when the spec gives an extension such a name."""
    return name_problem(name, at)

named_configuration classmethod

named_configuration(
    value: object,
) -> tuple[
    str | None, Mapping[str, object] | None, Problems
]

value, a field as a document of the format writes it, split into (name, configuration, problems), as the module's named_configuration splits a v3 field.

Source code in src/zarr_metadata/v3/_definition.py

@classmethod
def named_configuration(
    cls, value: object
) -> tuple[str | None, Mapping[str, object] | None, Problems]:
    """`value`, a field as a document of the format writes it, split into `(name, configuration, problems)`, as the module's `named_configuration` splits a v3 field."""
    return named_configuration(value)

read_configuration

read_configuration(
    value: object, loc: Loc = ()
) -> tuple[C | None, Problems]

value type-checked, then judged by the rules: the configuration if it holds, and every problem.

The rules are asked only of a configuration that type-checked, its bounds kept, and whose nested fields are well formed, holding what its TypedDict admits and nothing else, so a caller holding JSON never reaches a rule with a member of the wrong type, out of its bounds, or one the type says cannot be there. No scope reads the fields it holds, so the rules see none of them read, and a rule about one -- a struct's field of a type whose values vary in size -- finds nothing to judge: resolve reads the field in a scope, and asks every rule.

Source code in src/zarr_metadata/v3/_definition.py

def read_configuration(self, value: object, loc: Loc = ()) -> tuple[C | None, Problems]:
    """`value` type-checked, then judged by the rules: the configuration if it holds, and every problem.

    The rules are asked only of a configuration that type-checked,
    its bounds kept, and whose nested fields are well formed, holding
    what its TypedDict admits and nothing else, so a caller holding
    JSON never reaches a rule with a member of the wrong type, out of
    its bounds, or one the type says cannot be there. No scope reads the fields it holds, so the rules see
    none of them read, and a rule about one -- a struct's field of a
    type whose values vary in size -- finds nothing to judge: `resolve`
    reads the field in a scope, and asks every rule.
    """
    configuration, problems = self._check_configuration(value, loc)
    if configuration is None:
        return None, problems
    refused = ruled(self, lambda: self.rules(read_only(configuration), _nothing_nested()), loc)
    return (configuration if len(refused) == 0 else None), (
        *problems,
        *with_input(refused, value, loc),
    )

spelled classmethod

spelled(
    name: str,
) -> tuple[str | None, dict[str, JSONValue] | None]

How a name a document writes reads: the name its definition is filed under, and the configuration the name carries.

A name is filed as itself and carries nothing, (name, None). A kind whose names carry configuration says otherwise: a v3 data type r16 is filed under r* with {"bits": 16}. A name no document writes, which only files a definition, is (None, None).

Source code in src/zarr_metadata/v3/_definition.py

@classmethod
def spelled(cls, name: str) -> tuple[str | None, dict[str, JSONValue] | None]:
    """How a name a document writes reads: the name its definition is filed under, and the configuration the name carries.

    A name is filed as itself and carries nothing, `(name, None)`. A
    kind whose names carry configuration says otherwise: a v3 data
    type `r16` is filed under `r*` with `{"bits": 16}`. A name no
    document writes, which only files a definition, is `(None, None)`.
    """
    return name, None

well_named classmethod

well_named(name: str) -> bool

Whether a document of the format may write name for a field of this kind, as name_problem says.

Source code in src/zarr_metadata/v3/_definition.py

@classmethod
def well_named(cls, name: str) -> bool:
    """Whether a document of the format may write `name` for a field of this kind, as `name_problem` says."""
    return cls.name_problem(name, ()) is None

Disagreements dataclass

Where a scope reads a reading's claims otherwise: the names it would gain a meaning for, and those it conflicts with, a lost meaning among them.

Source code in src/zarr_metadata/v3/_scope.py

@dataclass(frozen=True, slots=True)
class Disagreements:
    """Where a scope reads a reading's claims otherwise: the names it would gain a meaning for, and those it conflicts with, a lost meaning among them."""

    gains: tuple[ClaimKey, ...]
    conflicts: tuple[Conflict, ...]

    @property
    def agrees(self) -> bool:
        """Whether the scope reads every claim identically."""
        return len(self.gains) == 0 and len(self.conflicts) == 0

agrees property

agrees: bool

Whether the scope reads every claim identically.

EmptyConfiguration

Bases: TypedDict

The configuration of a definition with nothing to configure: its field is written with its name alone.

Source code in src/zarr_metadata/v3/_definition.py

class EmptyConfiguration(TypedDict, closed=True):
    """The configuration of a definition with nothing to configure: its field is written with its name alone."""

MetadataValidationError

Bases: ValueError

Raised when a value fails validation, by the entry points that raise rather than report.

Carries every problem found (not just the first) in .problems, as an immutable tuple: a raised error is a finished report, and a caller inspecting it must not be able to edit the record.

Source code in src/zarr_metadata/_json.py

class MetadataValidationError(ValueError):
    """Raised when a value fails validation, by the entry points that raise rather than report.

    Carries every problem found (not just the first) in `.problems`, as an
    immutable tuple: a raised error is a finished report, and a caller
    inspecting it must not be able to edit the record.
    """

    problems: tuple[ValidationProblem, ...]

    def __init__(self, problems: Sequence[ValidationProblem]) -> None:
        self.problems = tuple(problems)
        for entry in cast("tuple[object, ...]", self.problems):
            # The runtime half of the annotation: a caller that is not
            # type-checked, and hands over anything else, fails here
            # rather than far away, where a `loc` is read off it.
            if not isinstance(entry, ValidationProblem):
                msg = (
                    "MetadataValidationError takes ValidationProblem values, "
                    f"got {type(entry).__name__}"
                )
                raise TypeError(msg)
        super().__init__("\n".join(str(problem) for problem in self.problems))

    def __reduce__(
        self,
    ) -> tuple[
        type[MetadataValidationError], tuple[tuple[ValidationProblem, ...]], dict[str, object]
    ]:
        # An exception pickles and copies as its class called with its
        # `args`, which here are the message; it is built from its problems,
        # and the rest of its state -- its notes among it -- follows.
        return (type(self), (self.problems,), self.__dict__)

RefusedField dataclass

Bases: Generic[D]

A field that could not be read -- not a field at all, not JSON, or refused by the definition that claims its name -- as its problems say.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, slots=True, kw_only=True)
class RefusedField(Generic[D]):
    """A field that could not be read -- not a field at all, not JSON, or refused by the definition that claims its name -- as its problems say."""

    json: JSONValue | UNSET
    """The field as written, refined: arrays as tuples; `UNSET` when it is not JSON, which no document holds."""
    name: str | None
    """The name it is written with; None when it names none."""
    read_as: type[Definition[Any]]
    """The kind of metadata it was read as."""
    definition: D | None = None
    """The definition that claims its name and refused it; None when nothing in scope claims it, or it names none."""
    nested: Nested = dataclasses.field(default_factory=_nothing_nested)
    """The fields its configuration holds, each as the scope read it; empty when its configuration was not checked against its TypedDict."""

    def __eq__(self, other: object) -> bool:
        if not is_field(other):
            return NotImplemented
        return field_key(self) == field_key(other)

    def __hash__(self) -> int:
        return hash(field_key(self))

    def __post_init__(self) -> None:
        # The runtime half of the annotations; `read_as` with its type
        # arguments dropped, as `resolve` drops them.
        kind = as_kind(self.read_as)
        object.__setattr__(self, "read_as", kind)
        definition = cast("object", self.definition)
        refusal = None if definition is None else _misread(definition, kind, self.name)
        if refusal is not None:
            raise TypeError(refusal)

definition class-attribute instance-attribute

definition: D | None = None

The definition that claims its name and refused it; None when nothing in scope claims it, or it names none.

json instance-attribute

json: JSONValue | UNSET

The field as written, refined: arrays as tuples; UNSET when it is not JSON, which no document holds.

name instance-attribute

name: str | None

The name it is written with; None when it names none.

nested class-attribute instance-attribute

nested: Nested = dataclasses.field(
    default_factory=_nothing_nested
)

The fields its configuration holds, each as the scope read it; empty when its configuration was not checked against its TypedDict.

read_as instance-attribute

read_as: type[Definition[Any]]

The kind of metadata it was read as.

ScopeConflictError

Bases: ValueError

Raised where two scopes, or a scope and a reading, give one name two meanings, or one would lose a meaning the other has.

Carries every conflict in .conflicts, as MetadataValidationError carries every problem.

Source code in src/zarr_metadata/v3/_scope.py

class ScopeConflictError(ValueError):
    """Raised where two scopes, or a scope and a reading, give one name two meanings, or one would lose a meaning the other has.

    Carries every conflict in `.conflicts`, as `MetadataValidationError`
    carries every problem.
    """

    def __init__(self, conflicts: Sequence[Conflict]) -> None:
        self.conflicts: tuple[Conflict, ...] = tuple(conflicts)
        super().__init__("; ".join(str(conflict) for conflict in self.conflicts))

Stage dataclass

One codec of a pipeline, and the chunk it is handed.

Source code in src/zarr_metadata/v3/_pipeline.py

@dataclass(frozen=True, slots=True)
class Stage:
    """One codec of a pipeline, and the chunk it is handed."""

    codec: ResolvedField[CodecDefinition[Any]]
    """The codec, as the scope read it."""
    incoming: Chunk | None
    """The chunk it is handed.

    None for a codec handed bytes -- a bytes -> bytes codec, or any codec
    after the array -> bytes codec -- and for a codec nothing in scope
    claims when what it is handed is not known to be an array.
    """
    inner: Mapping[str, tuple[Stage, ...]] = dataclasses.field(default_factory=_no_stages)
    """The pipelines it holds, by the member of its configuration that holds each: each codec with the chunk it is handed."""

codec instance-attribute

codec: ResolvedField[CodecDefinition[Any]]

The codec, as the scope read it.

incoming instance-attribute

incoming: Chunk | None

The chunk it is handed.

None for a codec handed bytes -- a bytes -> bytes codec, or any codec after the array -> bytes codec -- and for a codec nothing in scope claims when what it is handed is not known to be an array.

inner class-attribute instance-attribute

inner: Mapping[str, tuple[Stage, ...]] = dataclasses.field(
    default_factory=_no_stages
)

The pipelines it holds, by the member of its configuration that holds each: each codec with the chunk it is handed.

StorageTransformerDefinition dataclass

Bases: Definition[C]

A storage transformer.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class StorageTransformerDefinition(Definition[C], kind=True):
    """A storage transformer."""

    label: ClassVar[str] = "storage transformer"
    field_aliases: ClassVar[tuple[TypeAliasType, ...]] = (StorageTransformerField,)

field_aliases class-attribute

field_aliases: tuple[TypeAliasType, ...] = (
    StorageTransformerField,
)

The field aliases a configuration member holding a field of this kind is annotated with: CodecField and StaticCodecField for a codec.

label class-attribute

label: str = 'storage transformer'

The kind as a message names it: "codec".

UnclaimedField dataclass

A field nothing in scope claims: an extension the scope leaves unjudged, which is what keeps the format open.

Equal to another when it is written with the same name and configuration, however each was spelled: nothing in scope interprets its configuration, so it compares as JSON text, as field_key says.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, slots=True, kw_only=True)
class UnclaimedField:
    """A field nothing in scope claims: an extension the scope leaves unjudged, which is what keeps the format open.

    Equal to another when it is written with the same name and
    configuration, however each was spelled: nothing in scope interprets
    its configuration, so it compares as JSON text, as `field_key` says.
    """

    json: JSONValue
    """The field as written, refined: arrays as tuples; it takes no part in equality."""
    name: str
    """The name nothing in scope claims."""
    read_as: type[Definition[Any]]
    """The kind of metadata it was read as: what a definition that claimed it would be."""
    configuration: Mapping[str, JSONValue] = dataclasses.field(init=False)
    """The configuration as written, which nothing judged; empty when none is written."""

    def __eq__(self, other: object) -> bool:
        if not is_field(other):
            return NotImplemented
        return field_key(self) == field_key(other)

    def __hash__(self) -> int:
        return hash(field_key(self))

    def __post_init__(self) -> None:
        # The runtime half of the annotations; `read_as` with its type
        # arguments dropped, as `resolve` drops them.
        object.__setattr__(self, "read_as", as_kind(self.read_as))
        name = cast("object", self.name)
        if not isinstance(name, str) or not self.read_as.well_named(name):
            msg = (
                "a field nothing in scope claims is named as a document names a "
                f"{self.read_as.label}, got {name!r}"
            )
            raise TypeError(msg)
        _, written, _ = self.read_as.named_configuration(self.json)
        configuration: Mapping[str, object] = {} if written is None else written
        object.__setattr__(self, "configuration", cast("Mapping[str, JSONValue]", configuration))

    @property
    def definition(self) -> None:
        """The definition that read it: none did."""
        return None

    @property
    def nested(self) -> Nested:
        """The fields its configuration holds as the scope read them: none, since nothing read its configuration."""
        return _nothing_nested()

    def to_json(self) -> JSONValue:
        """The field as a document writes it, sharing nothing with the field: its configuration as written, in the envelope every reader takes, as `AcceptedField.to_json` writes one."""
        return copied(written_json(self))

configuration class-attribute instance-attribute

configuration: Mapping[str, JSONValue] = dataclasses.field(
    init=False
)

The configuration as written, which nothing judged; empty when none is written.

definition property

definition: None

The definition that read it: none did.

json instance-attribute

json: JSONValue

The field as written, refined: arrays as tuples; it takes no part in equality.

name instance-attribute

name: str

The name nothing in scope claims.

nested property

nested: Nested

The fields its configuration holds as the scope read them: none, since nothing read its configuration.

read_as instance-attribute

read_as: type[Definition[Any]]

The kind of metadata it was read as: what a definition that claimed it would be.

to_json

to_json() -> JSONValue

The field as a document writes it, sharing nothing with the field: its configuration as written, in the envelope every reader takes, as AcceptedField.to_json writes one.

Source code in src/zarr_metadata/v3/_definition.py

def to_json(self) -> JSONValue:
    """The field as a document writes it, sharing nothing with the field: its configuration as written, in the envelope every reader takes, as `AcceptedField.to_json` writes one."""
    return copied(written_json(self))

ValidationProblem dataclass

A single problem found in a value: where it is, what is wrong, what kind of wrong, and the data the message is made of.

loc is the path from the root of what was judged to the offending value, e.g. ("codecs", 0, "name") in a document, and an empty loc refers to that root. kind classifies the failure mode for programmatic dispatch; message is the human-readable description.

input and ctx are what the message says, as data, as pydantic's ErrorDetails and zod's issues carry theirs. input is the JSON found at loc -- 12, for a gzip level of 12 -- and UNSET where nothing is there, as zod has it for a key that is missing (pydantic gives the object missing it), or where what is there is not JSON, which the message shows. It is the object the caller handed in, as pydantic's is, not a copy: a caller that changes its document afterwards changes what input shows. ctx is what was expected, where that is more than a type:

• gt, ge, lt and le: the bounds the value's type carries, as pydantic names them -- {"ge": 0, "le": 9} for a gzip level, whose type is Annotated[int, Interval(ge=0, le=9)] -- or a rule says.
• expected: the values of a closed set, as zod's values holds them, in the order the message lists them -- a Literal's, node_type's, zarr_format's.

Neither takes part in equality or the repr: a problem is the same problem when it is found at the same place and says the same thing. Every function that returns or raises problems fills input from the value its caller handed it, so a rule says only where a problem is.

Source code in src/zarr_metadata/_json.py

@dataclass(frozen=True, slots=True)
class ValidationProblem:
    """A single problem found in a value: where it is, what is wrong, what kind of wrong, and the data the message is made of.

    `loc` is the path from the root of what was judged to the offending
    value, e.g. `("codecs", 0, "name")` in a document, and an empty `loc`
    refers to that root.
    `kind` classifies the failure mode for programmatic dispatch; `message`
    is the human-readable description.

    `input` and `ctx` are what the message says, as data, as pydantic's
    `ErrorDetails` and zod's issues carry theirs. `input` is the JSON found
    at `loc` -- `12`, for a gzip `level` of 12 -- and `UNSET` where nothing
    is there, as zod has it for a key that is missing (pydantic gives the
    object missing it), or where what is there is not JSON, which the
    message shows. It is the object the caller handed in, as pydantic's
    is, not a copy: a caller that changes its document afterwards changes
    what `input` shows. `ctx` is what was expected, where that is more
    than a type:

    - `gt`, `ge`, `lt` and `le`: the bounds the value's type carries, as
      pydantic names them -- `{"ge": 0, "le": 9}` for a gzip `level`,
      whose type is `Annotated[int, Interval(ge=0, le=9)]` -- or a rule
      says.
    - `expected`: the values of a closed set, as zod's `values` holds
      them, in the order the message lists them -- a `Literal`'s,
      `node_type`'s, `zarr_format`'s.

    Neither takes part in equality or the repr: a problem is the same
    problem when it is found at the same place and says the same thing.
    Every function that returns or raises problems fills `input` from the
    value its caller handed it, so a rule says only where a problem is.
    """

    loc: tuple[str | int, ...]
    message: str
    kind: ProblemKind
    input: JSONValue | UNSET = dataclasses.field(
        default=UNSET, kw_only=True, compare=False, repr=False
    )
    ctx: Mapping[str, JSONValue] = dataclasses.field(
        default_factory=_no_ctx, kw_only=True, compare=False, repr=False
    )

    def __post_init__(self) -> None:
        # The runtime half of the annotations: a rule written without a type
        # checker, as an extension's may be, fails where it builds a problem
        # rather than reporting one at a location that is not one.
        loc = cast("object", self.loc)
        if not isinstance(loc, tuple) or not all(
            isinstance(part, str) or (isinstance(part, int) and not isinstance(part, bool))
            for part in cast("tuple[object, ...]", loc)
        ):
            msg = f"a ValidationProblem's loc is a tuple of keys and indices, got {loc!r}"
            raise TypeError(msg)
        message = cast("object", self.message)
        if not isinstance(message, str):
            msg = f"a ValidationProblem's message is a string, got {message!r}"
            raise TypeError(msg)
        kind = cast("object", self.kind)
        if not isinstance(kind, str) or kind not in get_args(ProblemKind):
            msg = f"a ValidationProblem's kind is one of {get_args(ProblemKind)!r}, got {kind!r}"
            raise TypeError(msg)
        ctx = cast("object", self.ctx)
        if isinstance(ctx, _Ctx):
            # A problem's own, checked when it was made: what `replace`
            # hands a copy.
            return
        if (
            not isinstance(ctx, Mapping)
            or not all(isinstance(key, str) for key in cast("Mapping[object, object]", ctx))
            or not is_json(dict(cast("Mapping[str, object]", ctx)))
        ):
            msg = f"a ValidationProblem's ctx is an object of JSON values, got {ctx!r}"
            raise TypeError(msg)
        # Held as a view of a copy of its own at every level, arrays as
        # tuples, so a raised error, a finished report, cannot be edited
        # through it, nor through what was handed in.
        held = cast(
            "dict[str, JSONValue]",
            copied(cast("JSONValue", arrays_to_tuples(dict(cast("Mapping[str, object]", ctx))))),
        )
        object.__setattr__(self, "ctx", _Ctx(held))

    def __str__(self) -> str:
        location = ".".join(str(part) for part in self.loc) if self.loc else "<root>"
        return f"{location}: {self.message}"

    def __reduce__(
        self,
    ) -> tuple[Callable[..., ValidationProblem], tuple[object, ...]]:
        # Pickled and copied as its constructor called again: the view
        # `ctx` is held as does not pickle, and the dict it views does.
        return (_problem, (self.loc, self.message, self.kind, self.input, dict(self.ctx)))

canonical_fill_value

canonical_fill_value(
    data_type: ResolvedField[F], value: object
) -> JSONValue | UNSET

value, a fill value of data_type, a data type field a scope read, in the one spelling its value has; UNSET when it has a problem.

As the data type's fill_value_canonical spells it, so two fill values of a data type are one value exactly when their canonical spellings are written alike -- the same JSON, as json.dumps writes it, which == is not: it takes -0.0 for 0.0. A fill value fill_value_problems finds a problem with has no canonical spelling, as canonical_of gives a field with a problem none: UNSET, since None is the JSON null, a fill value of a data type the scope did not read, which spells a fill value as written.

Source code in src/zarr_metadata/v3/_definition.py

def canonical_fill_value(data_type: ResolvedField[F], value: object) -> JSONValue | UNSET:
    """`value`, a fill value of `data_type`, a data type field a scope read, in the one spelling its value has; `UNSET` when it has a problem.

    As the data type's `fill_value_canonical` spells it, so two fill
    values of a data type are one value exactly when their canonical
    spellings are written alike -- the same JSON, as `json.dumps` writes
    it, which `==` is not: it takes `-0.0` for `0.0`. A fill value
    `fill_value_problems` finds a problem with has no canonical spelling,
    as `canonical_of` gives a field with a problem none: `UNSET`, since
    `None` is the JSON `null`, a fill value of a data type the scope did
    not read, which spells a fill value as written.
    """
    if len(fill_value_problems(data_type, value)) != 0:
        return UNSET
    refined, _ = refine_json(value, ())
    return spelled_canonically(data_type, refined)

fill_value_problems

fill_value_problems(
    data_type: ResolvedField[F],
    value: object,
    loc: Loc = (),
) -> Problems

What is wrong with value as a fill value of data_type, a data type field a scope read.

value is refined to JSON first: not JSON is the first verdict, whatever the data type. It is then checked against the JSON shape the data type's definition declares, and judged by its fill value rules, as read_configuration reads a configuration: a key the shape does not declare is reported and left out, and the rules still judge the rest. The rules see the fields the configuration holds as the scope read them: a struct judges each field's fill value by that field's own type. A data type the scope did not read, out of scope or invalid, leaves a JSON fill value unjudged. loc prefixes every problem.

Source code in src/zarr_metadata/v3/_definition.py

def fill_value_problems(data_type: ResolvedField[F], value: object, loc: Loc = ()) -> Problems:
    """What is wrong with `value` as a fill value of `data_type`, a data type field a scope read.

    `value` is refined to JSON first: not JSON is the first verdict,
    whatever the data type. It is then checked against the JSON shape the
    data type's definition declares, and judged by its fill value rules, as
    `read_configuration` reads a configuration: a key the shape does not declare is
    reported and left out, and the rules still judge the rest. The rules
    see the fields the configuration holds as the scope read them: a
    struct judges each field's fill value by that field's own type. A data
    type the scope did not read, out of scope or invalid, leaves a JSON fill
    value unjudged. `loc` prefixes every problem.
    """
    refined, problems = refine_json(value, loc)
    if len(problems) != 0 or not isinstance(data_type, AcceptedField):
        return with_input(problems, value, loc)
    definition, configuration = data_type.definition, data_type.configuration
    typed, problems = _fill_value_parser(definition.fill_value)(refined, loc)
    if not _usable(problems):
        return with_input(problems, value, loc)
    refused = ruled(
        definition,
        lambda: definition.fill_value_rules(read_only(configuration), data_type.nested, typed),
        loc,
    )
    return with_input((*problems, *refused), value, loc)

resolve

resolve(
    data: object,
    kind: type[D],
    context: Context,
    loc: Loc = (),
) -> tuple[ResolvedField[D], Problems]

data, one metadata field, read as a kind in context: what the scope made of it, and every problem.

All three steps for one field. data is refined to JSON and its envelope judged -- an extra member, a configuration that is not an object, a must_understand that is not a boolean or is false, each a problem. The name is related to a definition in context; the configuration is checked against its TypedDict and judged by its rules; each nested field the check met is read the same way, in the same scope, and what is wrong with one is its own, reported where it sits, as with a document's fields. What comes back is AcceptedField by the definition that claims the name; UnclaimedField when nothing in scope claims it, an unmodelled extension left unjudged, which is what keeps the format open; or RefusedField, with the problems that say why. loc prefixes every problem. kind is one of the five kinds -- CodecDefinition, DataTypeDefinition, ChunkGridDefinition, ChunkKeyEncodingDefinition, StorageTransformerDefinition -- with or without type arguments; anything else is a TypeError.

Source code in src/zarr_metadata/v3/_definition.py

def resolve(
    data: object, kind: type[D], context: Context, loc: Loc = ()
) -> tuple[ResolvedField[D], Problems]:
    """`data`, one metadata field, read as a `kind` in `context`: what the scope made of it, and every problem.

    All three steps for one field. `data` is refined to JSON and its
    envelope judged -- an extra member, a `configuration` that is not an
    object, a `must_understand` that is not a boolean or is `false`, each
    a problem. The name is related to a definition in `context`; the
    configuration is checked against its TypedDict and judged by its
    rules; each nested field the check met is read the same way, in the
    same scope, and what is wrong with one is its own, reported where it
    sits, as with a document's fields. What comes back is `AcceptedField` by the
    definition that claims the name; `UnclaimedField` when nothing in scope
    claims it, an unmodelled extension left unjudged, which is what keeps
    the format open; or `RefusedField`, with the problems that say why. `loc`
    prefixes every problem. `kind` is one of the five kinds --
    `CodecDefinition`, `DataTypeDefinition`, `ChunkGridDefinition`,
    `ChunkKeyEncodingDefinition`, `StorageTransformerDefinition` -- with
    or without type arguments; anything else is a `TypeError`.
    """
    asked = as_kind(kind)
    refined, problems = refine_json(data, loc)
    if len(problems) != 0:
        # Not JSON, so not read; its name, if it has one, still says what
        # claims it, and one the spec does not give an extension is a
        # problem here as on the other path, asked of no definition.
        name = asked.named_configuration(data)[0]
        bad = None if name is None else asked.name_problem(name, asked.name_loc(loc))
        claimant = None if name is None or bad is not None else context.claimant(asked, name)
        refused = RefusedField(json=UNSET, name=name, read_as=asked, definition=claimant)
        found = problems if bad is None else (bad, *problems)
        return cast("ResolvedField[D]", refused), with_input(found, data, loc)
    resolved, found = _resolve_field(refined, asked, context, loc)
    return cast("ResolvedField[D]", resolved), with_input(found, data, loc)

shown

shown(value: object) -> str

value as a problem's message shows it: as the JSON a document writes, null and [1, 2], or by its repr when it is not JSON; what the interpreter will not write, an integer of too many digits or a value nested too deep, by saying so.

Source code in src/zarr_metadata/_json.py

def shown(value: object) -> str:
    """`value` as a problem's message shows it: as the JSON a document writes, `null` and `[1, 2]`, or by its repr when it is not JSON; what the interpreter will not write, an integer of too many digits or a value nested too deep, by saying so."""
    refined, problems = _refine(value, (), finite=False)
    if any(problem.message == _PAST_THE_LEVELS for problem in problems):
        # Nested past the levels a reader walks: said so, not left to the
        # repr, which overflows at a depth the interpreter and platform set.
        return "a value nested too deep to show"
    if len(problems) != 0:
        # Not JSON.
        return shown_by_python(value)
    try:
        return json.dumps(refined, ensure_ascii=False)
    except ValueError:
        # An integer of more digits than the interpreter converts to text:
        # the value itself, by its size, or one a container holds, which
        # is then what Python will not write either.
        if isinstance(value, int):
            return f"an integer of {value.bit_length()} bits"
        return shown_by_python(value)

storage_of

storage_of(
    data_type: ResolvedField[DataTypeDefinition[Any]],
) -> StorageClass | None

How the values of data_type, a data type field a scope read, are stored; None when unknown.

Unknown when the scope did not read it, or its definition does not say. Its storage is the extension author's code: what it gives is checked to be a storage class, and an error it raises says which data type's storage raised it.

Source code in src/zarr_metadata/v3/_definition.py

def storage_of(data_type: ResolvedField[DataTypeDefinition[Any]]) -> StorageClass | None:
    """How the values of `data_type`, a data type field a scope read, are stored; None when unknown.

    Unknown when the scope did not read it, or its definition does not
    say. Its `storage` is the extension author's code: what it gives is
    checked to be a storage class, and an error it raises says which data
    type's storage raised it.
    """
    if not isinstance(data_type, AcceptedField):
        return None
    definition, configuration = data_type.definition, data_type.configuration
    found = asked(
        definition,
        "storage",
        lambda: cast("object", definition.storage(read_only(configuration), data_type.nested)),
    )
    if found is not None and found not in get_args(StorageClass):
        msg = (
            f"{definition.name!r}: its storage gives one of {get_args(StorageClass)!r} or None, "
            f"got {found!r}"
        )
        raise TypeError(msg)
    return cast("StorageClass | None", found)