Skip to main content

Streaming

client.chat.stream() returns an AsyncGenerator<string> that yields text deltas as they arrive from the model. Use this for real-time UIs where you want to display the response word by word.

Basic streaming​

const stream = client.chat.stream({
  model:    'auto',
  messages: [{ role: 'user', content: 'Write a haiku about the sea.' }],
});

for await (const chunk of stream) {
  process.stdout.write(chunk); // or append to UI
}

Collecting the full text​

let fullText = '';

for await (const chunk of client.chat.stream({
  model:    'auto',
  messages: [{ role: 'user', content: 'Tell me a story.' }],
})) {
  fullText += chunk;
}

console.log(fullText);

React example​

async function streamToState(
  messages: ChatMessage[],
  setText: (t: string) => void,
) {
  let accumulated = '';

  for await (const chunk of client.chat.stream({ model: 'auto', messages })) {
    accumulated += chunk;
    setText(accumulated);
  }
}

Parameters​

stream() accepts the same parameters as complete(). See Chat completions.

How it works​

AICP uses server-sent events (SSE). The generator reads the data: … lines from the response body and yields the choices[0].delta.content field from each parsed JSON object. The [DONE] sentinel ends the stream automatically.