In Progress
Test Case Expected Results
When creating Test Cases, capture expected results (in addition to capturing the name and test conditions as Lamatic does currently). Consider also, Using code checks and AI to find deviation and grade result quality. Use this quality score as an input for deciding whether to deploy an update. Assist in recommending the best option when A|B testing prompts, agents, workflows, models and other settings such as temperature. Using these results to create benchmarks. Measuring and alerting about “model drift” over time. Integrating “human-in-the-loop” strategies to expand Test Cases and quality checking. Added Later (based on feedback from another user): Make it easier to see the history of Test Case executions (without leaving Experiments). Marc Greenberg didn’t realize he could find the results in Logs, but also didn’t like having to dig through logs to find the execution details for a given test.
cwhiteman Over 1 year ago
💡 Feature Requests
In Progress
Test Case Expected Results
When creating Test Cases, capture expected results (in addition to capturing the name and test conditions as Lamatic does currently). Consider also, Using code checks and AI to find deviation and grade result quality. Use this quality score as an input for deciding whether to deploy an update. Assist in recommending the best option when A|B testing prompts, agents, workflows, models and other settings such as temperature. Using these results to create benchmarks. Measuring and alerting about “model drift” over time. Integrating “human-in-the-loop” strategies to expand Test Cases and quality checking. Added Later (based on feedback from another user): Make it easier to see the history of Test Case executions (without leaving Experiments). Marc Greenberg didn’t realize he could find the results in Logs, but also didn’t like having to dig through logs to find the execution details for a given test.
cwhiteman Over 1 year ago
💡 Feature Requests
Planned
Full Refresh option in Force Sync
Users currently struggle and have to perform multiple steps when indexing a new schema in the vectorDB. Current Steps - Update the metadata schema Change the mode the full refresh deploy the flow Schedule Runs Data indexed Proposed Steps - Option in Jobs > Force Sync > Full Refresh Automatically updates flows, Deploys it, Trigger a Job Return back the flow to Incremental
Aman Sharma 5 months ago
💡 Feature Requests
Planned
Full Refresh option in Force Sync
Users currently struggle and have to perform multiple steps when indexing a new schema in the vectorDB. Current Steps - Update the metadata schema Change the mode the full refresh deploy the flow Schedule Runs Data indexed Proposed Steps - Option in Jobs > Force Sync > Full Refresh Automatically updates flows, Deploys it, Trigger a Job Return back the flow to Incremental
Aman Sharma 5 months ago
💡 Feature Requests
Support for Local Development
An open-source SDK to address developer adoption barriers and vendor lock-in concerns that are currently limiting growth. Problem Developers resist Lamatic's no-code/low-code platform due to limited library support, lack of local development capabilities, missing CLI tools, and weak version control integration Enterprise and startup customers worry about vendor lock-in and business continuity if Lamatic changes pricing or discontinues service Competitors offering open-source alternatives have an advantage Proposed Solution Build a standalone, open-source SDK enabling developers to: Develop AI agents/systems locally in their preferred IDE Test and debug offline without cloud dependency Use Git for version control across all project components Create custom integrations and capabilities Deploy anywhere (on-premise, private/public cloud, or Lamatic-managed) Compile flows into executable code via CI/CD pipelines Core Architecture YAML-based configuration system with: Pre-built node libraries for AI components Containerized runtime (Pod + Core) executing configurations Compiler for optimizing YAML into executable workers Local web editor accessible via localhost CLI for all development operations Multiple invocation methods (API, webhooks, widgets)
Aman Sharma About 3 hours ago
💡 Feature Requests
Support for Local Development
An open-source SDK to address developer adoption barriers and vendor lock-in concerns that are currently limiting growth. Problem Developers resist Lamatic's no-code/low-code platform due to limited library support, lack of local development capabilities, missing CLI tools, and weak version control integration Enterprise and startup customers worry about vendor lock-in and business continuity if Lamatic changes pricing or discontinues service Competitors offering open-source alternatives have an advantage Proposed Solution Build a standalone, open-source SDK enabling developers to: Develop AI agents/systems locally in their preferred IDE Test and debug offline without cloud dependency Use Git for version control across all project components Create custom integrations and capabilities Deploy anywhere (on-premise, private/public cloud, or Lamatic-managed) Compile flows into executable code via CI/CD pipelines Core Architecture YAML-based configuration system with: Pre-built node libraries for AI components Containerized runtime (Pod + Core) executing configurations Compiler for optimizing YAML into executable workers Local web editor accessible via localhost CLI for all development operations Multiple invocation methods (API, webhooks, widgets)
Aman Sharma About 3 hours ago
💡 Feature Requests
In Progress
Spaces for static file uploads and usage
Ability to upload files and use them in flows (to, among other things, eliminate the need to find a place to host files either for testing or production purposes). For example, PDF upload for training or CSV for instructions.
Aman Sharma 8 months ago
Engineering
💡 Feature Requests
In Progress
Spaces for static file uploads and usage
Ability to upload files and use them in flows (to, among other things, eliminate the need to find a place to host files either for testing or production purposes). For example, PDF upload for training or CSV for instructions.
Aman Sharma 8 months ago
Engineering
💡 Feature Requests
Support For dynamic File References in Flow Config
The nodes are embedded inline with formatting artifacts (hello\\nhello), apparent test content, and no versioning metadata. Editing it requires understanding the entire config file, and changes produce noisy diffs that obscure actual prompt improvements. Before: prompts: - id: 187c2f4b-c23d-4545-abef-73dc897d6b7b role: assistant content: >- hello hello Important:please provide the response in wrap text format... After: prompts: - id: rag_system_prompt_v1 role: assistant # Edit prompts in /prompts/rag_system.md — supports preview & versioning contentRef: ./prompts/rag_system.md version: "1.2.0" lastUpdated: "2025-01-15" Developer Impact: Prompts become first-class artifacts — reviewable in PRs, A/B testable, and editable without touching pipeline config.
Aman Sharma About 3 hours ago
💡 Feature Requests
Support For dynamic File References in Flow Config
The nodes are embedded inline with formatting artifacts (hello\\nhello), apparent test content, and no versioning metadata. Editing it requires understanding the entire config file, and changes produce noisy diffs that obscure actual prompt improvements. Before: prompts: - id: 187c2f4b-c23d-4545-abef-73dc897d6b7b role: assistant content: >- hello hello Important:please provide the response in wrap text format... After: prompts: - id: rag_system_prompt_v1 role: assistant # Edit prompts in /prompts/rag_system.md — supports preview & versioning contentRef: ./prompts/rag_system.md version: "1.2.0" lastUpdated: "2025-01-15" Developer Impact: Prompts become first-class artifacts — reviewable in PRs, A/B testable, and editable without touching pipeline config.
Aman Sharma About 3 hours ago
💡 Feature Requests
In Progress
Dispay Test Run Stats in Debugger
Problem: Developers must manually compute speed and token metrics from logs. Use case: Quickly benchmark and optimize workflows against performance targets. Functionality: Real-time display of execution time, token usage, accuracy indicators, TTFT, throughput, and benchmark comparisons within the debugger.
Aman Sharma 2 months ago
💡 Feature Requests
In Progress
Dispay Test Run Stats in Debugger
Problem: Developers must manually compute speed and token metrics from logs. Use case: Quickly benchmark and optimize workflows against performance targets. Functionality: Real-time display of execution time, token usage, accuracy indicators, TTFT, throughput, and benchmark comparisons within the debugger.
Aman Sharma 2 months ago
💡 Feature Requests
Planned
Autocomplete Widget
Feature : An input box like feature, which post some characters entered provides suggestions (Gen-AI based or RAG based) and allow users to complete the input box. This could basically take a collection of objects to show the output Why : Can allow for quick widget suggestions based on user documents and/or basic gen-ai suggestions Example(s) :
Dhruv Pamneja 7 months ago
💡 Feature Requests
Planned
Autocomplete Widget
Feature : An input box like feature, which post some characters entered provides suggestions (Gen-AI based or RAG based) and allow users to complete the input box. This could basically take a collection of objects to show the output Why : Can allow for quick widget suggestions based on user documents and/or basic gen-ai suggestions Example(s) :
Dhruv Pamneja 7 months ago
💡 Feature Requests
Planned
Connect with External Vector DB
Support Vector DB Weaviate PG_Vector PineCone Additional MongoDB ChromaDB Qdrant
Aman Sharma 11 months ago
🔌 Integration
Planned
Connect with External Vector DB
Support Vector DB Weaviate PG_Vector PineCone Additional MongoDB ChromaDB Qdrant
Aman Sharma 11 months ago
🔌 Integration
Show Only Compatible Test Cases for Current Flow
Automatically filter test cases and only show the ones that match the current flow’s input schema. @cwhiteman
Ramandeep Singh About 1 month ago
💡 Feature Requests
Show Only Compatible Test Cases for Current Flow
Automatically filter test cases and only show the ones that match the current flow’s input schema. @cwhiteman
Ramandeep Singh About 1 month ago
💡 Feature Requests
Break Node and End Node
Problem: No clear way to terminate workflows early or break out of loops based on conditions, forcing users to create workarounds with maximum iteration counts. Use Case: When API calls fail or specific conditions are met, users want to cleanly exit workflows or break out of polling loops without continuing unnecessary iterations. Proposed Solution: Implement dedicated break and end nodes that allow conditional workflow termination and loop breaking.
Aman Sharma 2 months ago
💡 Feature Requests
Break Node and End Node
Problem: No clear way to terminate workflows early or break out of loops based on conditions, forcing users to create workarounds with maximum iteration counts. Use Case: When API calls fail or specific conditions are met, users want to cleanly exit workflows or break out of polling loops without continuing unnecessary iterations. Proposed Solution: Implement dedicated break and end nodes that allow conditional workflow termination and loop breaking.
Aman Sharma 2 months ago
💡 Feature Requests
Planned
A/B Testing Analytics and Reporting in Logs
Problem: A/B experiments lack clear labeling and result visualization. Use case: Make data-driven decisions about prompts, flows, and model choices. Functionality: Experiment labels on runs, statistical analysis with confidence intervals, dashboards for variant comparison, and filtering/grouping by experiment tags.
Aman Sharma 2 months ago
💡 Feature Requests
Planned
A/B Testing Analytics and Reporting in Logs
Problem: A/B experiments lack clear labeling and result visualization. Use case: Make data-driven decisions about prompts, flows, and model choices. Functionality: Experiment labels on runs, statistical analysis with confidence intervals, dashboards for variant comparison, and filtering/grouping by experiment tags.
Aman Sharma 2 months ago
💡 Feature Requests
In Progress
Automated Response Evaluation System (Graders)
Problem: Manual grading of responses does not scale and is inconsistent. Use case: Systematically assess response quality across large test suites. Functionality: Grader agents with configurable rubrics, batch “grade all” execution, and integration with logs for historical evaluation.
Aman Sharma 2 months ago
💡 Feature Requests
In Progress
Automated Response Evaluation System (Graders)
Problem: Manual grading of responses does not scale and is inconsistent. Use case: Systematically assess response quality across large test suites. Functionality: Grader agents with configurable rubrics, batch “grade all” execution, and integration with logs for historical evaluation.
Aman Sharma 2 months ago
💡 Feature Requests
In Progress
Persistent Test Run History and Analytics
Problem: Debug runs are not persisted, hindering regression and performance tracking. Use case: Enable long-term testing workflows with historical comparisons. Functionality: Store all runs with timestamps and versions, searchable and filterable history, and side-by-side comparison of different runs and versions.
Aman Sharma 2 months ago
💡 Feature Requests
In Progress
Persistent Test Run History and Analytics
Problem: Debug runs are not persisted, hindering regression and performance tracking. Use case: Enable long-term testing workflows with historical comparisons. Functionality: Store all runs with timestamps and versions, searchable and filterable history, and side-by-side comparison of different runs and versions.
Aman Sharma 2 months ago
💡 Feature Requests
Planned
11Labs
Problem: Basic TTS/STT is insufficient for production-quality voice applications. Use case: Deliver high-quality, natural-sounding voice interactions for customer-facing apps. Functionality: A dedicated 11Labs node with full API coverage, streaming voice generation, configurable voices, and secure credential management.
Aman Sharma 2 months ago
🔌 Integration
Planned
11Labs
Problem: Basic TTS/STT is insufficient for production-quality voice applications. Use case: Deliver high-quality, natural-sounding voice interactions for customer-facing apps. Functionality: A dedicated 11Labs node with full API coverage, streaming voice generation, configurable voices, and secure credential management.
Aman Sharma 2 months ago
🔌 Integration
Planned
Integrated Voice Processing Capabilities
Problem: No native voice model support, requiring external services for voice features. Use case: Build voice companions, real-time call analysis, and other voice-first applications. Functionality: Dedicated STT and TTS nodes with model selection, streaming audio support, real-time transcription/synthesis, and voice-based triggers and responses.
Aman Sharma 2 months ago
💡 Feature Requests
Planned
Integrated Voice Processing Capabilities
Problem: No native voice model support, requiring external services for voice features. Use case: Build voice companions, real-time call analysis, and other voice-first applications. Functionality: Dedicated STT and TTS nodes with model selection, streaming audio support, real-time transcription/synthesis, and voice-based triggers and responses.
Aman Sharma 2 months ago
💡 Feature Requests
Planned
Granular Async Control for Sub-workflows
Problem: Async behavior is currently dictated by the trigger node only, limiting flexibility. Use case: Run deep analysis or enrichment sub-flows in the background independent of the main trigger. Functionality: Async mode selection on the execute-flow node, support for fire-and-forget sub-workflows, and decoupling from parent workflow async settings.
Aman Sharma 2 months ago
💡 Feature Requests
Planned
Granular Async Control for Sub-workflows
Problem: Async behavior is currently dictated by the trigger node only, limiting flexibility. Use case: Run deep analysis or enrichment sub-flows in the background independent of the main trigger. Functionality: Async mode selection on the execute-flow node, support for fire-and-forget sub-workflows, and decoupling from parent workflow async settings.
Aman Sharma 2 months ago
💡 Feature Requests
Planned
Classifier Node - Chat History Support
Problem: Classifier lacks a Chat history field, forcing context to be injected via prompt hacks. Use case: Enable context-aware classification in multi-turn conversations. Functionality: Add a standard memory field, automatically reference chat history, and align the classifier API with other context-aware nodes.
Aman Sharma 2 months ago
💡 Feature Requests
Planned
Classifier Node - Chat History Support
Problem: Classifier lacks a Chat history field, forcing context to be injected via prompt hacks. Use case: Enable context-aware classification in multi-turn conversations. Functionality: Add a standard memory field, automatically reference chat history, and align the classifier API with other context-aware nodes.
Aman Sharma 2 months ago
💡 Feature Requests
Planned
First-Party Chat History Management
Problem: Developers must manually manage JSON and memory fields across nodes, with inconsistent support. Use case: Simplify building and testing conversational flows with a toggle-based experience. Functionality: Checkbox-style chat history enablement, consistent memory field support on all nodes, and automatic user-based history storage and retrieval.
Aman Sharma 2 months ago
💡 Feature Requests
Planned
First-Party Chat History Management
Problem: Developers must manually manage JSON and memory fields across nodes, with inconsistent support. Use case: Simplify building and testing conversational flows with a toggle-based experience. Functionality: Checkbox-style chat history enablement, consistent memory field support on all nodes, and automatic user-based history storage and retrieval.
Aman Sharma 2 months ago
💡 Feature Requests