`astero.grammar`¶

The declaration and every structural query over it.

astero.grammar ¶

Declared grammars, and the structural facts derived from them.

"Grammar" here is not a parser's grammar. astero does not parse. Three different things carry the name, and only the third is this one:

Python's PEG grammar (Grammar/python.gram) turns text into a tree.
CPython's ASDL (Parser/Python.asdl) declares the shape of the nodes that tree is made of. The ast module is generated from it, and from 3.13 ast._field_types exposes it at run time.
A grammar in this library is that node declaration plus a role per field, which is the part neither of the other two records.

The distinction matters because the roles are the only authored part. A grammar is sorts, productions, and fields; each field carries a shape, a sort, and a role. Shapes and sorts are read from whatever already declares them — the interpreter for Python, the annotations for a dataclass IR. The roles are what ASDL does not have, and they are what the derivations below need:

child      structural containment, drives traversal
attr       plain data, never traversed
def(ns)    introduces a name in namespace `ns`
use(ns)    refers to a name in namespace `ns`
defuse(ns) both, for read-modify-write positions
del(ns)    removes a name

A role belongs to the parent's field, not to the child node. That is what makes ctx derivable: ctx describes the position a Name occupies, so the position owns it.

Nothing here parses or rewrites. It answers questions about a declared grammar, and every answer replaces a table that is written by hand somewhere.

Present `dataclass` ¶

The field exists and is neither None nor empty.

Absent `dataclass` ¶

The field is missing, None, or empty.

Field `dataclass` ¶

applies ¶

applies(node: Any) -> bool

Whether this field's role is in force for node.

Production `dataclass` ¶

field ¶

field(name: str) -> Field | None

The field called name, or None if this production has none.

For a caller asking whether a field exists. A caller that knows it does wants production[name], which raises instead of handing back a Field | None every use has to re-check.

Grammar `dataclass` ¶

A declared language: productions, and the namespaces its names live in.

positions ¶

positions(
    kind: Kind | Iterable[Kind], ns: str | None = None
) -> dict[str, tuple[str, ...]]

Production name -> field names holding kind in namespace ns.

This is the query that replaces a hand-written operand table. It spans every production, so a sort the author forgot cannot be missed.

operands ¶

operands(ns: str) -> dict[str, tuple[str, ...]]

Fields that read a name from ns. The def-use graph's edges.

definitions ¶

definitions(ns: str) -> dict[str, tuple[str, ...]]

Fields that introduce a name in ns.

reads ¶

reads(node: Any, ns: str) -> tuple[Any, ...]

The contents of every slot of node that reads a name in ns.

binds ¶

binds(node: Any, ns: str) -> tuple[Any, ...]

The contents of every slot of node that introduces a name in ns.

reference_slots ¶

reference_slots(
    ns: str | None = None,
) -> dict[str, tuple[tuple[str, str], ...]]

Fields holding a reference object, optionally only in one namespace.

The sibling of ident_slots. A language spells a use one of three ways: a node in expression position (Python's Name), a bare identifier in a field (PL/0's Assign.name), or an object carrying the name (postpile's Value). The first two are identifier slots; the third is here, and needs reference_sorts declared to be visible at all — without it a rename over such a grammar is a silent no-op.

A pair per slot where ident_slots gives a name, because a reference slot is two facts: the field, and where in the object the identifier sits. reference_sorts already states the second, and a caller that has to look it up again re-derives the filter this method just applied — which is how both readers of it grew the same five lines re-checking a state that cannot occur here.

ident_slots ¶

ident_slots(
    ns: str | None = None,
) -> dict[str, tuple[str, ...]]

Fields holding a bare identifier, optionally only in one namespace.

Renaming has to reach every one of these. Enumerating them by hand is what dropped vararg, kwarg, lambda parameters and async def parameters in the pass this replaces.

Pass ns when renaming means renaming something in particular. Without it the answer spans namespaces, so it includes Attribute.attr and keyword.arg, and renaming a variable foo would rewrite x.foo into a different attribute.

subset ¶

subset(
    names: Iterable[str], *, name: str | None = None
) -> Grammar

This grammar restricted to names, for declaring a sublanguage.

A compiler for a Python subset declares its language by naming the productions it admits, and everything else — the roles, sorts, shapes and conditions — comes from the grammar it is a subset of. Nothing is authored twice.

Productions the subset does not name are simply absent, so a rewrite that tries to build one fails at the rule rather than downstream, and concrete() reports the smaller language.

children ¶

children(prod_name: str) -> tuple[Field, ...]

Fields a structural traversal descends into.

with_trait ¶

with_trait(trait: str) -> frozenset[str]

Productions carrying trait. Replaces per-instruction enumerations.

constructor ¶

constructor(name: str) -> type

The class that builds production name.

Matching can work off CPython's classes for any host whose nodes subclass them, since every dispatch is an isinstance. Construction cannot: prescrypt-ng's Assign carries a mixin its later passes read, so a rewrite has to build that class rather than ast.Assign.

A production the host does not have is an error here rather than a fallback, because falling back would build a node the host cannot compile and defer the failure to somewhere less obvious.

concrete ¶

concrete(base: str | None = None) -> frozenset[str]

Productions that can appear in a tree, optionally only under base.

A grammar read off a class hierarchy contains abstract entries: stmt and expr are productions with no instances. A consumer's dispatch table is not obliged to cover those, and is obliged to cover the rest, so this is the set a coverage check should compare against.

Answered from the classes, so it needs a grammar that carries them. prescrypt-ng compiles async def and has no handler for await, async for or async with, which is the shape this finds.

check ¶

check() -> list[str]

Internal consistency. Returns complaints, empty when well-formed.

GrammarBuilder `dataclass` ¶

Assembles a Grammar. The three front doors all end up here.

from_dataclasses ¶

from_dataclasses(
    name: str,
    classes: Iterable[type],
    *,
    roles: Mapping[str, Mapping[str, Any]] | None = None,
    namespaces: Iterable[str] = (),
    ident_sorts: Iterable[str] = ("ident",),
    reference_sorts: Mapping[str, str] | None = None,
    conditions: Mapping[tuple[str, str], Condition]
    | None = None,
    traits: Mapping[str, Iterable[str]] | None = None,
    data_sorts: Iterable[str] = (
        "str",
        "int",
        "bool",
        "float",
        "object",
    ),
) -> Grammar

A grammar read off annotated dataclasses.

The third front door, beside astero.python.build reading a module of ast classes and lang_ssa written out by hand. A compiler whose IR is already dataclasses declares it in place: the field names, sorts and shapes come from the annotations, and only the roles are authored.

roles maps a production name to its field roles. A field with no role is an attr when its sort is plain data and a child otherwise, which is the answer that keeps traversal complete.

A field's entry may be a sequence of (role, condition) pairs when one slot holds two roles. postpile's AssignValue.target is defined when declare is set and used when it is not, so it is declared once each way.

field_of ¶

field_of(
    name: str,
    role: Role = CHILD,
    shape: Shape = ONE,
    sort: str | None = None,
) -> Field

Terser Field constructor, for declarations written by hand.

sort_of ¶

sort_of(annotation: Any) -> str

The sort an annotation names, with its container and optionality gone.

list[Value] and Value | None are both the sort Value. What varies is the shape, which shape_of reads from the same text.

shape_of ¶

shape_of(annotation: Any) -> Shape

Read a shape off an ast._field_types-style annotation.

astero.grammar¶