nodestep.chat

Chat messages, requests and responses, and the Chat protocol for models.

A chat model is any object with model, complete and stream. See Agents and tools for the chat integrations, and Write a chat integration.

Chat models

Chat

Protocol of a chat model: any object with model, complete and stream.

Attributes

model

str

Name of the model; middleware sees it as ModelMiddlewareContext.model.

Methods

async complete(request)

Send a request and return the full response.

async complete(request) -> ChatResponse

Parameters

request

ChatRequest

Messages, tool definitions and an optional output schema.

Returns

ChatResponse

The model's reply.

stream(request)

Send a request and yield the response as it is generated.

stream(request) -> AsyncIterator[ChatStreamChunk]

Parameters

request

ChatRequest

Messages, tool definitions and an optional output schema.

Returns

AsyncIterator[ChatStreamChunk]

Content, tool-call, finish and usage chunks in arrival order.

Requests and responses

ChatRequest

ChatRequest(*, messages, tools=[], tool_choice=None, output_schema=None)

Request to a chat model, the same for every chat integration.

Attributes

messages	list[Message]
tools	list[ToolDefinition]
tool_choice	Literal['auto', 'required', 'none'] | str | None	Whether the model may, must or must not call tools; another string names the tool to call, and None sends no choice.
output_schema	dict[str, Any] | None	JSON schema the answer must follow.

ChatResponse

ChatResponse(*, content=None, refusal=None, tool_calls=[], finish_reason=None, usage={}, model=None, raw=None)

Reply from a chat model, the same for every chat integration.

Attributes

content	str | None	Answer text; None when the model only calls tools or refuses.
refusal	str | None	The model's refusal text, when it declined to answer.
tool_calls	list[ToolCall]
finish_reason	str | None	Why the model stopped, as the chat integration reports it.
usage	dict[str, Any]	Token counts as the chat integration reports them.
model	str | None	Model that answered, as the chat integration names it.
raw	Any	The response object the chat integration received, for fields nodestep does not map.

ChatStreamChunk

ChatStreamChunk(*, content_delta=None, refusal_delta=None, tool_call_delta=None, finish_reason=None, usage=None, model=None, raw=None)

One incremental piece of a streamed response.

Attributes

content_delta	str | None	Next part of the answer text.
refusal_delta	str | None	Next part of the refusal text.
tool_call_delta	dict[str, Any] | None	Part of one tool call, shaped as in Streamed tool calls.
finish_reason	str | None	Why the model stopped; set on a final chunk.
usage	dict[str, Any] | None	Token counts; set on the chunk that reports them.
model	str | None	Model that answered, as the chat integration names it.
raw	Any	The event object the chat integration received.

assemble_stream

async assemble_stream(chunks) -> ChatResponse

Collect streamed chunks into a single response.

Parameters

chunks

AsyncIterator[ChatStreamChunk]

Chunks produced by Chat.stream.

Returns

ChatResponse

Joined content, assembled tool calls, finish reason, usage and the model named by the chunks.

Raises

ModelProviderError

If a streamed tool call has no id or no name.

Messages

Message

Message: SystemMessage or HumanMessage or AIMessage or ToolMessage

Any chat message, told apart by its type field when validated.

BaseMessage

BaseMessage(*, id=None, content=None)

Base class of all chat messages.

Attributes

id	str | None	Optional message id, for example to remove the message later.
content	str | None
role	str	Role of the message as a chat integration sends it.

SystemMessage

SystemMessage(*, id=None, content=None, type='system')

Instruction message for the model.

Subclass of BaseMessage.

Attributes

type	Literal['system']
role	str	Always "system".

HumanMessage

HumanMessage(*, id=None, content=None, type='human')

Message written by the user.

Subclass of BaseMessage.

Attributes

type	Literal['human']
role	str	Always "user".

AIMessage

AIMessage(*, id=None, content=None, type='ai', refusal=None, tool_calls=[], finish_reason=None, usage={})

Reply produced by the model: text, a refusal, or tool calls.

Subclass of BaseMessage.

Attributes

type	Literal['ai']
refusal	str | None	The model's refusal text, when it declined to answer.
tool_calls	list[ToolCall]
finish_reason	str | None	Why the model stopped, as the chat integration reports it.
usage	dict[str, Any]	Token counts as the chat integration reports them.
role	str	Always "assistant".

ToolMessage

ToolMessage(*, id=None, content=None, type='tool', name=None, tool_call_id=None, arguments=None)

Result of a tool call; arguments are the ones it ran with when they differ from the call.

Subclass of BaseMessage.

Attributes

type	Literal['tool']
name	str | None	Name of the tool that ran.
tool_call_id	str | None	Id of the ToolCall this message answers.
arguments	dict[str, Any] | None
role	str	Always "tool".

Tool calls

ToolCall

ToolCall(*, id, name, arguments={}, arguments_error=None)

A tool invocation requested by the model.

Attributes

id	str	Id the model gave the call; its ToolMessage answers with it.
name	str
arguments	dict[str, Any]	Parsed arguments; empty when arguments_error is set.
arguments_error	str | None	Why the argument text could not be parsed, or None.
arguments_json	str	Arguments serialized as a JSON string.

Methods

from_arguments_json(id, name, arguments_json)

class method Build a tool call from the JSON argument text a model produced.

from_arguments_json(id, name, arguments_json) -> ToolCall

Parameters

id	str	Tool call id.
name	str	Tool name.
arguments_json	str | None	Argument text, parsed with parse_tool_arguments.

Returns

ToolCall

The call, with arguments_error set when the text is invalid.

ToolDefinition

ToolDefinition(*, name, description, input_schema={})

Tool description sent to the model.

Attributes

name	str
description	str
input_schema	dict[str, Any]	JSON schema of the arguments.

parse_tool_arguments

parse_tool_arguments(raw) -> tuple[dict[str, Any], str | None]

Parse the JSON arguments of a tool call.

Parameters

raw

str | None

Argument text produced by the model.

Returns

tuple[dict, str or None]

The parsed arguments and an error description when they are invalid. Empty text means no arguments; anything but a JSON object, null included, is invalid.