Skip to main content

Semantic Caching

Feature RequestsEngineering

Add built in semantic caching that saves on request costs and also make the response more predicatable.


Issue reported by the user -

Chat response took 12+ seconds to respond to a user question (while also consuming tokens and giving a non-deterministic answer) to answer the identical question that someone else has asked.

Is it possible to enable semantic caching for my Lamatic Chat Widget?

Posted by AlexBozanic ·
1 comment

Log in to comment and vote

Comments1

  • cwhiteman

    •

    Sep 10, 2025

    @Dhruv Pamneja is testing an interim approach of building exact match vectors with the answer stored as metadata in the VDB.