Skip to content

Stream a response

Most chat interfaces go through the same steps. Each one maps to a state:

  1. The user asks. Show thinking straight away, before the request has even left.
  2. The AI uses a tool, such as a web search, if your API tells you. Show searching.
  3. The first words arrive. Show writing, and stream the text in next to it.
  4. The stream ends. Show done. Or, if anything failed along the way, show error.

Press “Ask” to start.

These examples read a plain text stream from /api/chat. Swap in your own endpoint; only the four state changes matter.

<mulling-status id="status" hidden></mulling-status>
<p id="answer"></p>
<script>
async function ask(question) {
const status = document.getElementById('status');
const answer = document.getElementById('answer');
answer.textContent = '';
status.hidden = false;
status.setAttribute('state', 'thinking');
try {
const res = await fetch('/api/chat', { method: 'POST', body: JSON.stringify({ question }) });
if (!res.ok) throw new Error(res.statusText);
const reader = res.body.pipeThrough(new TextDecoderStream()).getReader();
for (;;) {
const { value, done } = await reader.read();
if (done) break;
status.setAttribute('state', 'writing');
answer.textContent += value;
}
status.setAttribute('state', 'done');
} catch {
status.setAttribute('state', 'error');
}
}
</script>

Setting writing for every chunk is fine: nothing changes, and nothing is announced again, until the state does.

A plain text stream doesn’t say when the AI is searching. Some APIs send events, such as a tool-use or search event, before the text starts. Set searching when one arrives, and writing when the text begins. Without such events, go straight from thinking to writing.

You can swap the loader for the answer straight away, or leave its check mark as a sign the answer is complete. See Endings.