Token Streaming Support in SDK

  • Problem: Users perceive latency because initial tokens are delayed visually even when backend completion is fast.

  • Use case: Improve UX for long-running tasks in web apps by showing progress immediately.

  • Functionality: Dual token streams (output and process), a "processing response" JSON key for intermediate node updates, and an async streaming API that can pipe data directly into React components.

Please authenticate to join the conversation.

Upvoters
Board

💡 Feature Requests

Date

9 months ago

Author

Aman Sharma

Subscribe to request

Get notified by email when there are changes.