astero.python.hygiene¶
This module substitutes and renames without capturing or corrupting names.
astero.python.hygiene ¶
Fresh names, renaming, and capture-avoiding substitution.
A pass that moves code from one place to another has to answer three questions about names, and every one of them is a question the grammar already answers.
Which occurrences of a name may be replaced? Only the ones that use it.
p2w's inliner substituted at every Name node, so inlining f(99) into a body
containing [x for x in ...] produced [99 for 99 in ...], which is not a
program. Whether a position uses or binds is what roles say.
Which slots does renaming have to reach? Every slot holding an identifier,
not the ones a pass happens to remember. latexify's renamer listed them by hand
and missed vararg, kwarg, lambda parameters and async def parameters, so
it renamed references to parameters it left alone.
When is a substitution unsafe? When the expression being moved has a free name that the destination binds, so moving it would capture that name. That is computable from the same two answers.
Nothing here decides policy. A caller that cannot supply fresh names gets a refusal naming the obstruction, which is what p2w's inliner does today by declining to inline at all.
CaptureError ¶
Bases: Exception
A substitution could not be made without capturing a name.
Fresh
dataclass
¶
A supply of names that avoids a set of names already in use.
Every name handed out is added to taken, so two calls never collide even
when the caller does not record the first one.
binding_occurrences ¶
binding_occurrences(
node: Any, grammar: Grammar, ns: str
) -> set[int]
id() of every node that binds rather than uses a name.
A binding field may hold a target that is not itself a binding: b[i] = v
binds nothing, and reads both b and i. bound_names already draws that
line, so this walks the same shapes it does.
all_names ¶
all_names(node: Any, grammar: Grammar) -> set[str]
Every identifier appearing anywhere in node, bound or used.
No namespace argument: an identifier slot is a slot whatever namespace its role names, and renaming has to reach all of them.
bound_here ¶
bound_here(
node: Any, grammar: Grammar, ns: str
) -> set[str]
Names node introduces anywhere inside itself.
Accepts a node or a list of them, so a caller holding one field's value does not have to wrap it.
free_names ¶
free_names(
node: Any,
grammar: Grammar,
ns: str,
*,
scopes: Mapping[str, tuple[Scope, ...]] | None = None,
) -> set[str]
Names node uses without introducing.
Without scopes, node counts as one scope, so a name bound anywhere in
it is bound everywhere. That can miss a free name, which is the unsafe
direction for a capture check: the first t of t + sum(t for t in xs)
is free, and moving the expression under a binder of t captures it.
With scopes the answer is exact: a name is free where a Name uses it
outside every region of node that binds it. That needs Python's Name,
as substitution does.
rename ¶
rename(
node: Any,
mapping: Mapping[str, str],
grammar: Grammar,
ns: str,
) -> Any
Rename identifiers of namespace ns, in every slot the grammar declares.
Both binding and using occurrences, because a rename that reaches only one of them changes what the program means.
The namespace matters. Attribute.attr and keyword.arg are identifier
slots too, in a different namespace, and renaming a variable foo must not
rewrite x.foo into a different attribute.
A dotted identifier is skipped. import a.b as c holds a module path in
alias.name, and while that is an identifier position the renaming surface
has to know about, it names a module rather than a local. Renaming a
variable a must not rewrite the import it happens to share a spelling
with. Renaming the binding of a bare import a is a different operation,
since it has to become import a as ... to keep meaning the same thing.
shadowed_at ¶
shadowed_at(
node: Any,
grammar: Grammar,
ns: str,
scopes: Mapping[str, tuple[Scope, ...]],
) -> dict[int, frozenset[str]]
id() of every node, mapped to the names shadowed around it.
A name bound by an inner scope hides the outer one inside that scope's own fields and nowhere else, which is why this cannot be a flat set. A function's parameters shadow inside its body, and its decorators are evaluated outside where they do not.
What a scope binds stops at the scopes nested in it. The v of
lambda: [v for v in xs] + [v] belongs to the comprehension, and counting
it as the lambda's hid the lambda's own free v from substitution.
A declared name is not the declaring scope's: global g and nonlocal x
bind nothing where they are written, and a global name is not any
enclosing function's either. A class body's names are visible in the body
and in the annotation scopes inside it, and not in a function or generator
nested in it: class C: x = 1; def m(self): return x reads the global x.
symtable is the oracle for all of it, in test_scopes_symtable.py.
substitute ¶
substitute(
node: Any,
mapping: Mapping[str, AST],
grammar: Grammar,
ns: str,
*,
scopes: Mapping[str, tuple[Scope, ...]] | None = None,
fresh: Fresh | None = None,
) -> Any
Replace uses of the names in mapping, avoiding capture.
node is copied, so the argument is left alone. Each replacement is copied
too, so two occurrences of one name do not share a subtree.
Raises CaptureError when the substitution cannot be made safely and
fresh was not supplied:
nodebinds one of the names being substituted, by assigning it or by shadowing it in an inner scope. Not every occurrence then means the same thing, so replacing them all is wrong.- a replacement has a free name that
nodebinds, so moving it in would capture that name.
With fresh, the second is repaired by renaming the offending binders. The
first is always refused. A repair renames a name where the binding that
captures gives it its meaning, which takes scopes to know; without it the
tree counts as one scope and the name is renamed throughout.
With scopes, shadowing stops counting as rebinding: a name bound by an
inner scope hides the outer one inside that scope and is left alone there,
rather than making the whole substitution impossible. Pass the table that
says what a name binds over, astero.python.BINDING_SCOPES for Python, which is
not the same as the one saying what symtable calls a block.