Semantic Caching

Add built in semantic caching that saves on request costs and also make the response more predicatable.


Issue reported by the user -

Chat response took 12+ seconds to respond to a user question (while also consuming tokens and giving a non-deterministic answer) to answer the identical question that someone else has asked.

Is it possible to enable semantic caching for my Lamatic Chat Widget?

Please authenticate to join the conversation.

Upvoters
Status

Planned

Board

πŸ’‘ Feature Requests

Date

17 days ago

Author

AlexBozanic

Subscribe to post

Get notified by email when there are changes.