nodestep.middleware

Hooks around graphs, nodes, models and tools, and the built-in middleware.

Subclass Middleware and override the hooks you need. Built in: tool approvals, tool-call limits, summarization, to-do lists, skills and memory. See Middleware.

Base class and hooks

Middleware

Base class for hooks around graphs, nodes, model calls and tools.

Override the hooks you need. The order they run in and what each may return: Middleware.

Raises

TypeError

If a subclass defines a before_*, after_* or on_* method that is not a hook, such as after_tools.

Methods

before_graph(ctx)

Observe the state before the first superstep, also on a resume.

before_graph(ctx) -> Awaitable[None] | None

Returns

None	Returning a Replacement raises GraphExecutionError; any other value raises TypeError.

after_graph(ctx)

Observe the final state; called only when the run completes.

after_graph(ctx) -> Awaitable[None] | None

Returns

None	Returning a Replacement raises GraphExecutionError; any other value raises TypeError.

on_run_end(ctx, outcome)

Observe the end of a run, however it ended.

on_run_end(ctx, outcome) -> Awaitable[None] | None

Called once per run for each middleware whose before_graph was called; every hook runs, even when one raises. See on_run_end.

Parameters

ctx	GraphMiddlewareContext
outcome	RunOutcome

Returns

None	Returning a Replacement raises GraphExecutionError; any other value raises TypeError.

before_node(ctx)

Called with a copy of the node's input before the node runs.

before_node(ctx) -> Replacement | Awaitable[Replacement | None] | None

Return ctx.replace(value) to give the node a different input; it is not stored.

Returns

Replacement or None

A replacement value, or None to keep the current one.

after_node(ctx)

Called with the node's update before it is merged.

after_node(ctx) -> Replacement | Awaitable[Replacement | None] | None

Returns

Replacement or None

A replacement value, or None to keep the current one.

before_tool(ctx)

Called with validated tool arguments; may raise ToolDeniedError.

before_tool(ctx) -> Replacement | Awaitable[Replacement | None] | None

Returns

Replacement or None

A replacement value, or None to keep the current one.

after_tool(ctx)

Called with the validated tool result.

after_tool(ctx) -> Replacement | Awaitable[Replacement | None] | None

Returns

Replacement or None

A replacement value, or None to keep the current one.

before_model(ctx)

Called by model_node with the request before every model call.

before_model(ctx) -> Replacement | Awaitable[Replacement | None] | None

Returns

Replacement or None

ctx.replace(request) with a ChatRequest to send instead, or None to keep the current one.

after_model(ctx)

Called by model_node with the response after every model call.

after_model(ctx) -> Replacement | Awaitable[Replacement | None] | None

With model_node(stream=True) it runs after the stream ended; the "tokens" events carried the model's own chunks.

Returns

Replacement or None

ctx.replace(response) with a ChatResponse the node uses instead, or None to keep the current one.

on_interrupt(ctx, payload)

Answer an interrupt without pausing.

on_interrupt(ctx, payload) -> Any | None

Must be sync: returning an awaitable raises TypeError. A subgraph child's unanswered interrupts come here with the subgraph node's context. See Answering in code.

Parameters

ctx	NodeMiddlewareContext	state is a copy of the node's input, after before_node replacements.
payload	Any	Payload passed to interrupt.

Returns

Any or None

The answer, or None to pass to the next middleware and finally pause the run.

on_error(ctx, error)

Recover from a node error.

on_error(ctx, error) -> Any | Awaitable[Any | None] | None

Parameters

ctx	NodeMiddlewareContext	Its state is a private copy of the node's input.
error	Exception	What the node raised.

Returns

Any or None

A result to use instead, in any form a node may return (ctx.state counts as the received state), or None to fail the run. Changing ctx.state and returning anything else raises GraphExecutionError.

on_tool_error(ctx, error)

Observe a tool that raised, or returned a value its output model rejects.

on_tool_error(ctx, error) -> Awaitable[None] | None

It cannot recover: the tool's error is raised after every hook ran, with each hook's own error added as a note. See on_tool_error.

Parameters

ctx	ToolMiddlewareContext	value holds the arguments the tool ran with.
error	Exception	What the tool raised, or the validation error of its result.

Returns

None	A Replacement becomes a GraphExecutionError note on the error, any other value a TypeError note.

tools()

Return the tools this middleware provides.

tools() -> Iterable[Tool]

No node gets them implicitly; see Middleware tools.

HookName

HookName: one of "before_graph", "after_graph", "before_node", "after_node", "before_tool", "after_tool", "before_model", "after_model", "on_interrupt", "on_error", "on_tool_error", "on_run_end"

Names of the hooks a Middleware subclass may define.

Replacement

Replacement(value)

Value returned by a hook to replace the state, update or tool value.

Attributes

value

Any

What the hook puts in place of the value it received.

Hook contexts

GraphMiddlewareContext

GraphMiddlewareContext(graph_name, state, run_id, thread_id, branch_id, root_run_id='', resuming=False)

Context of before_graph, after_graph and on_run_end.

Attributes

graph_name	str
state	S	A copy of the run's state.
run_id	str
thread_id	str
branch_id	str
root_run_id	str	Id of the outermost run when the graph runs inside another one.
resuming	bool	Whether the run was started with resume=.

NodeMiddlewareContext

NodeMiddlewareContext(graph_name, node_name, state, stored_state, run_id, thread_id, step, task_id='', root_run_id='', superstep_size=1, scratch=dict(), state_schema=None)

Context of the node hooks.

Attributes

graph_name	str
node_name	str
state	S	The node input in before_node, after earlier replacements; the node's update in after_node.
stored_state	S	A copy of the input as stored, before any before_node replacement.
run_id	str
thread_id	str
step	int	Superstep, counted from 0.
task_id	str	Id of the node task.
root_run_id	str	Id of the outermost run when the graph runs inside another one.
superstep_size	int	Number of node tasks in this superstep.
scratch	dict[str, Any]	A dict shared by the hooks of one node task.
state_schema	StateSchema | None	The graph's state schema.

Methods

replace(value)

Wrap a new node input or update as the hook's return value.

replace(value) -> Replacement

ModelMiddlewareContext

ModelMiddlewareContext(graph_name, node_name, run_id, thread_id, step, task_id, root_run_id, model, request, response=None)

Context of before_model and after_model.

Attributes

graph_name	str
node_name	str
run_id	str
thread_id	str | None
step	int	Superstep, counted from 0.
task_id	str	Id of the node task that calls the model.
root_run_id	str	Id of the outermost run when the graph runs inside another one.
model	str	The chat model's model name.
request	ChatRequest	The request; in after_model, what was sent.
response	ChatResponse | None	The response, None in before_model.

Methods

replace(value)

Wrap a new request or response as the hook's return value.

replace(value) -> Replacement

ToolMiddlewareContext

ToolMiddlewareContext(graph_name, node_name, tool_name, value, state=None, run_id=None, thread_id=None, tool_call_id=None, root_run_id=None, task_id='')

Context of before_tool, after_tool and on_tool_error.

Attributes

graph_name	str | None
node_name	str | None
tool_name	str
value	Any	The arguments, or the result in after_tool.
state	S | None	State the calling node received; None outside a graph run.
run_id	str | None
thread_id	str | None
tool_call_id	str | None
root_run_id	str | None	Id of the outermost run when the graph runs inside another one.
task_id	str	The calling node's task, "" outside a graph run.

Methods

replace(value)

Wrap new tool arguments or a new result as the hook's return value.

replace(value) -> Replacement

RunOutcome

RunOutcome(status, interrupts, error)

How a run ended, passed to Middleware.on_run_end.

Attributes

status	Literal['completed', 'paused', 'failed', 'cancelled']	"cancelled" also covers a stream closed before its last event.
interrupts	Mapping[str, Interrupt]	Pending interrupts by key, read-only; empty unless the run paused.
error	BaseException | None	What the run raised; set only when status is "failed".

S

S = TypeVar('S')

The state type of the graph a middleware context belongs to.

Helpers

run_before_hooks

async run_before_hooks(middleware, hook_name, value, make_context) -> Any

Run a before_* hook of each middleware in order.

Parameters

middleware	Iterable[Middleware]
hook_name	HookName	The before_* hook to run, such as "before_node".
value	Any	Current value, replaced by each hook's Replacement.
make_context	Callable[[Any], Any | Awaitable[Any]]	Builds the hook context for the current value.

Raises

ValueError	If hook_name is "before_graph" or "after_graph".
TypeError	If a hook returns anything but None or a Replacement.

run_after_hooks

async run_after_hooks(middleware, hook_name, value, make_context) -> Any

Run an after_* hook of each middleware in reverse order.

Parameters

middleware	Iterable[Middleware]
hook_name	HookName	The after_* hook to run, such as "after_node".
value	Any	Current value, replaced by each hook's Replacement.
make_context	Callable[[Any], Any | Awaitable[Any]]	Builds the hook context for the current value.

Raises

ValueError	If hook_name is "before_graph" or "after_graph".
TypeError	If a hook returns anything but None or a Replacement.

collect_middleware_tools

collect_middleware_tools(middleware) -> list[Any]

Return the tools of a middleware list, in list order.

Parameters

middleware

Iterable[Middleware]

Raises

TypeError

If an item does not subclass Middleware.