Streaming

How it works

With stream: true the answer arrives as text/event-stream: each data: line is a chat.completion.chunk, and the stream ends with data: [DONE]. The OpenAI library reads this for you.

stream = client.chat.completions.create(    model="MODEL_ID",    messages=[{"role": "user", "content": "Write a two-line poem."}],    stream=True,    stream_options={"include_usage": True},)for chunk in stream:    if chunk.choices:        print(chunk.choices[0].delta.content or "", end="")

With include_usage the last chunk carries usage and its choices is empty; the code above allows for that.

Errors during a stream

Errors before the answer starts (key, model, credit) come back like a normal request, with an error status code. If the connection drops mid-stream, you have the text up to that point; decide whether to send again or keep what you have.