Skip to main content

Test Case Expected Results

When creating Test Cases, capture expected results (in addition to capturing the name and test conditions as Lamatic does currently). Consider also,

  • Using code checks and AI to find deviation and grade result quality.

  • Use this quality score as an input for deciding whether to deploy an update.

  • Assist in recommending the best option when A|B testing prompts, agents, workflows, models and other settings such as temperature.

  • Using these results to create benchmarks.

  • Measuring and alerting about “model drift” over time.

  • Integrating “human-in-the-loop” strategies to expand Test Cases and quality checking.

Added Later (based on feedback from another user):

  • Make it easier to see the history of Test Case executions (without leaving Experiments). Marc Greenberg didn’t realize he could find the results in Logs, but also didn’t like having to dig through logs to find the execution details for a given test.

17 comments

Log in to comment and vote

Comments17

  • Aman Sharma

    Team•

    Dec 2, 2024

    AKA LLM as Judge, Inspiration -

    https://langfuse.com/docs/scores/model-based-evals

  • Aman Sharma

    Team•

    Sep 18, 2025

    Maintain history of evals scores over tests and versions

  • [Deleted User]

    •

    Dec 12, 2025

  • [Deleted User]

    •

    Dec 12, 2025

  • [Deleted User]

    •

    Dec 12, 2025

  • [Deleted User]

    •

    Dec 12, 2025

  • [Deleted User]

    •

    Dec 12, 2025

  • [Deleted User]

    •

    Dec 14, 2025

    > hello

  • [Deleted User]

    •

    Dec 14, 2025

    javascript;alert(1) hello

  • [Deleted User]

    •

    Dec 14, 2025

    <<<<<<<<<<<<<