Token Streaming Support in SDK

  • Problem: Users perceive latency because initial tokens are delayed visually even when backend completion is fast.

  • Use case: Improve UX for long-running tasks in web apps by showing progress immediately.

  • Functionality: Dual token streams (output and process), a "processing response" JSON key for intermediate node updates, and an async streaming API that can pipe data directly into React components.

Please authenticate to join the conversation.

Upvoters
Status

Planned

Board

πŸ’‘ Feature Requests

Date

About 2 months ago

Author

Aman Sharma

Subscribe to post

Get notified by email when there are changes.