Test Case Expected Results

When creating Test Cases, capture expected results (in addition to capturing the name and test conditions as Lamatic does currently). Consider also, Using code checks and AI to find deviation and grade result quality. Use this quality score as an input for deciding whether to deploy an update. Assist in recommending the best option when A|B testing prompts, agents, workflows, models and other settings such as temperature. Using these results to create benchmarks. Measuring and alerting about “model drift” over time. Integrating “human-in-the-loop” strategies to expand Test Cases and quality checking. Added Later (based on feedback from another user): Make it easier to see the history of Test Case executions (without leaving Experiments). Marc Greenberg didn’t realize he could find the results in Logs, but also didn’t like having to dig through logs to find the execution details for a given test.

cwhiteman Over 1 year ago

15

💡 Feature Requests

Full Refresh option in Force Sync

Users currently struggle and have to perform multiple steps when indexing a new schema in the vectorDB. Current Steps - Update the metadata schema Change the mode the full refresh deploy the flow Schedule Runs Data indexed Proposed Steps - Option in Jobs > Force Sync > Full Refresh Automatically updates flows, Deploys it, Trigger a Job Return back the flow to Incremental

Aman Sharma 4 months ago

12

💡 Feature Requests

Integration to ZeroEntropy

Integrate with ZeroEntropy API for Vectorization, Ingestion and Query

Ghita 10 months ago

Engineering

🔌 Integration

Spaces for static file uploads and usage

Ability to upload files and use them in flows (to, among other things, eliminate the need to find a place to host files either for testing or production purposes). For example, PDF upload for training or CSV for instructions.

Aman Sharma 7 months ago

1

Engineering

💡 Feature Requests

Dispay Test Run Stats in Debugger

Problem: Developers must manually compute speed and token metrics from logs. Use case: Quickly benchmark and optimize workflows against performance targets. Functionality: Real-time display of execution time, token usage, accuracy indicators, TTFT, throughput, and benchmark comparisons within the debugger.

Aman Sharma About 2 months ago

1

💡 Feature Requests

Autocomplete Widget

Feature : An input box like feature, which post some characters entered provides suggestions (Gen-AI based or RAG based) and allow users to complete the input box. This could basically take a collection of objects to show the output Why : Can allow for quick widget suggestions based on user documents and/or basic gen-ai suggestions Example(s) :

Dhruv Pamneja 6 months ago

💡 Feature Requests

Break Node and End Node

Problem: No clear way to terminate workflows early or break out of loops based on conditions, forcing users to create workarounds with maximum iteration counts. Use Case: When API calls fail or specific conditions are met, users want to cleanly exit workflows or break out of polling loops without continuing unnecessary iterations. Proposed Solution: Implement dedicated break and end nodes that allow conditional workflow termination and loop breaking.

Aman Sharma About 2 months ago

💡 Feature Requests

Vector Search as Tool

Support Inbuilt vector search of vectorDB as tool call for AI Node

Aman Sharma About 2 months ago

💡 Feature Requests

A/B Testing Analytics and Reporting in Logs

Problem: A/B experiments lack clear labeling and result visualization. Use case: Make data-driven decisions about prompts, flows, and model choices. Functionality: Experiment labels on runs, statistical analysis with confidence intervals, dashboards for variant comparison, and filtering/grouping by experiment tags.

Aman Sharma About 2 months ago

💡 Feature Requests

Automated Response Evaluation System (Graders)

Problem: Manual grading of responses does not scale and is inconsistent. Use case: Systematically assess response quality across large test suites. Functionality: Grader agents with configurable rubrics, batch “grade all” execution, and integration with logs for historical evaluation.

Aman Sharma About 2 months ago

💡 Feature Requests

Persistent Test Run History and Analytics

Problem: Debug runs are not persisted, hindering regression and performance tracking. Use case: Enable long-term testing workflows with historical comparisons. Functionality: Store all runs with timestamps and versions, searchable and filterable history, and side-by-side comparison of different runs and versions.

Aman Sharma About 2 months ago

💡 Feature Requests

11Labs

Problem: Basic TTS/STT is insufficient for production-quality voice applications. Use case: Deliver high-quality, natural-sounding voice interactions for customer-facing apps. Functionality: A dedicated 11Labs node with full API coverage, streaming voice generation, configurable voices, and secure credential management.

Aman Sharma About 2 months ago

🔌 Integration

Integrated Voice Processing Capabilities

Problem: No native voice model support, requiring external services for voice features. Use case: Build voice companions, real-time call analysis, and other voice-first applications. Functionality: Dedicated STT and TTS nodes with model selection, streaming audio support, real-time transcription/synthesis, and voice-based triggers and responses.

Aman Sharma About 2 months ago

💡 Feature Requests

Granular Async Control for Sub-workflows

Problem: Async behavior is currently dictated by the trigger node only, limiting flexibility. Use case: Run deep analysis or enrichment sub-flows in the background independent of the main trigger. Functionality: Async mode selection on the execute-flow node, support for fire-and-forget sub-workflows, and decoupling from parent workflow async settings.

Aman Sharma About 2 months ago

💡 Feature Requests

Classifier Node - Chat History Support

Problem: Classifier lacks a Chat history field, forcing context to be injected via prompt hacks. Use case: Enable context-aware classification in multi-turn conversations. Functionality: Add a standard memory field, automatically reference chat history, and align the classifier API with other context-aware nodes.

Aman Sharma About 2 months ago

💡 Feature Requests

First-Party Chat History Management

Problem: Developers must manually manage JSON and memory fields across nodes, with inconsistent support. Use case: Simplify building and testing conversational flows with a toggle-based experience. Functionality: Checkbox-style chat history enablement, consistent memory field support on all nodes, and automatic user-based history storage and retrieval.

Aman Sharma About 2 months ago

💡 Feature Requests

Token Streaming Support in SDK

Problem: Users perceive latency because initial tokens are delayed visually even when backend completion is fast. Use case: Improve UX for long-running tasks in web apps by showing progress immediately. Functionality: Dual token streams (output and process), a "processing response" JSON key for intermediate node updates, and an async streaming API that can pipe data directly into React components.

Aman Sharma About 2 months ago

💡 Feature Requests

Billing: implement a Billing Manager permissions option

Customer reports: “Only the person that opens the account right now can enter billing information. Even if you’re an admin, you cannot enter billing information for what I’ve seen. Is there a way to solve this in a future product feature?” While best practice is to restrict the ability to edit billing info in a multi-user SaaS product, the ability to assign this role is desireable and will support the use case where a user signs up with a free account, but needs someone else to input billing info to upgrade. Will also support the ability for users to transition off the team and to have multiple billing managers for backup.

cwhiteman 5 months ago

3

👏🏻 Feedback

Native integration to mySQL

Requirement from Pat at co-dev to connect to mySQL (on PlanetScale).

cwhiteman 7 months ago

High Priority

🔌 Integration

PostgreSQL Read-Only User Setup Guide

Add comprehensive documentation for creating a dedicated read-only PostgreSQL user before connecting to Lamatic, following security best practices similar to Airbyte's integration guide. Problem Currently, users may connect to Lamatic using admin/superuser credentials, which poses security risks. A clear guide for creating properly scoped read-only users is needed to ensure secure database connections. Proposed Solution Add a pre-setup section to the PostgreSQL integration documentation with step-by-step instructions for creating a dedicated read-only user. Suggested Documentation Structure: Step 1: Create a Dedicated Read-Only PostgreSQL User 1. Connect to your PostgreSQL database as a superuser: psql -h -p -U -d 2. Create a dedicated user for Lamatic: CREATE USER lamatic_user WITH PASSWORD 'your_secure_password'; 3. Grant connection privileges: GRANT CONNECT ON DATABASE TO lamatic_user; 4. Grant schema usage: GRANT USAGE ON SCHEMA TO lamatic_user; 5. Grant read permissions on tables: -- For existing tables GRANT SELECT ON ALL TABLES IN SCHEMA TO lamatic_user; -- For future tables (optional) ALTER DEFAULT PRIVILEGES IN SCHEMA GRANT SELECT ON TABLES TO lamatic_user; 6. Grant read permissions on sequences (if needed): GRANT SELECT ON ALL SEQUENCES IN SCHEMA TO lamatic_user; Step 2: Configure Lamatic Connection Use the newly created lamatic_user credentials in your Lamatic PostgreSQL connection settings. Step 3: Verify Connection Test the connection to ensure the read-only user has appropriate access without write permissions. Benefits Enhanced Security: Limits database access to read-only operations Principle of Least Privilege: Grants only necessary permissions Audit Trail: Clear separation between admin and integration users Risk Mitigation: Reduces potential impact of credential compromise Acceptance Criteria [ ] Documentation includes complete SQL commands [ ] Examples cover common schema configurations [ ] Verification steps are provided [ ] Security considerations are highlighted [ ] Troubleshooting section for common permission issues This addition would significantly improve the security posture of PostgreSQL integrations with Lamatic while providing users with clear, actionable guidance.

Aman Sharma 7 months ago

👏🏻 Feedback