When creating Test Cases, capture expected results (in addition to capturing the name and test conditions as Lamatic does currently). Consider also, Using code checks and AI to find deviation and grade result quality. Use this quality score as an input for deciding whether to deploy an update. Assist in recommending the best option when A|B testing prompts, agents, workflows, models and other settings such as temperature. Using these results to create benchmarks. Measuring and alerting about “model drift” over time. Integrating “human-in-the-loop” strategies to expand Test Cases and quality checking. Added Later (based on feedback from another user): Make it easier to see the history of Test Case executions (without leaving Experiments). Marc Greenberg didn’t realize he could find the results in Logs, but also didn’t like having to dig through logs to find the execution details for a given test.

cwhiteman almost 2 years ago

17

💡 Feature Requests

A dedicated UI to choose model preferences on project level. There will be different categories of models ( Tabs ) for Text | Embedding| Chat | Image | Audio | Transcript For Each category customer can create 5 Settings and reorder them. In each settings can choose 4 tabs ( Basic | Fallback | Best | A/B Test ) When using model selection in a workflow node, choose a preferred setting options which will populate the UI with credentials and model selection. ( By Default the first preference will be chosen or create a new preferences

Aman Sharma almost 2 years ago

2

💡 Feature Requests

Users currently struggle and have to perform multiple steps when indexing a new schema in the vectorDB. Current Steps - Update the metadata schema Change the mode the full refresh deploy the flow Schedule Runs Data indexed Proposed Steps - Option in Jobs > Force Sync > Full Refresh Automatically updates flows, Deploys it, Trigger a Job Return back the flow to Incremental

Aman Sharma 9 months ago

12

💡 Feature Requests

Use web API to give real time TTS and STT on search and chat widget

Aman Sharma about 1 year ago

💡 Feature Requests

An open-source SDK to address developer adoption barriers and vendor lock-in concerns that are currently limiting growth. Problem Developers resist Lamatic's no-code/low-code platform due to limited library support, lack of local development capabilities, missing CLI tools, and weak version control integration Enterprise and startup customers worry about vendor lock-in and business continuity if Lamatic changes pricing or discontinues service Competitors offering open-source alternatives have an advantage Proposed Solution Build a standalone, open-source SDK enabling developers to: Develop AI agents/systems locally in their preferred IDE Test and debug offline without cloud dependency Use Git for version control across all project components Create custom integrations and capabilities Deploy anywhere (on-premise, private/public cloud, or Lamatic-managed) Compile flows into executable code via CI/CD pipelines Core Architecture YAML-based configuration system with: Pre-built node libraries for AI components Containerized runtime (Pod + Core) executing configurations Compiler for optimizing YAML into executable workers Local web editor accessible via localhost CLI for all development operations Multiple invocation methods (API, webhooks, widgets)

Aman Sharma 5 months ago

💡 Feature Requests

Integrate with ZeroEntropy API for Vectorization, Ingestion and Query

Ghita over 1 year ago

Engineering

🔌 Integration

Problem: Developers must manually compute speed and token metrics from logs. Use case: Quickly benchmark and optimize workflows against performance targets. Functionality: Real-time display of execution time, token usage, accuracy indicators, TTFT, throughput, and benchmark comparisons within the debugger.

Aman Sharma 7 months ago

1

💡 Feature Requests

Develop a dedicated Prompt IDE that allows for direct editing, improved support for variables, and clear variable mapping—including documentation on data mapping. Introduce the ability to organize prompts into folders for better manageability and to support logical grouping of related prompts. Provide tools to make prompt versioning and comparison modes more intuitive, reducing visual and workflow clutter.

Aman Sharma 11 months ago

High Priority

👏🏻 Feedback

Ability to upload files and use them in flows (to, among other things, eliminate the need to find a place to host files either for testing or production purposes). For example, PDF upload for training or CSV for instructions.

Aman Sharma 12 months ago

1

Engineering

💡 Feature Requests

Multi-tab Support and Flow Comparison Enable right-click to open flows in new tabs for easy comparison Support multiple tabs within the main window (similar to how Linear handles native apps) Display full flow names - currently get cut off/truncated in the UI, making it difficult to distinguish between flows Allow side-by-side visual comparison of different flow versions Consider implementing an Electron wrapper for native app experience with better tab support Cross-Environment Flow Navigation Ability to change environments within a single flow view - users want to see what the same flow looks like in staging while viewing production Maintain flow state when switching environments instead of returning to flow home screen Use flow slug associations to link flows across environments (dev/staging/production) Reduce manual effort required to promote workflows from dev → staging → production

Aman Sharma 4 months ago

💡 Feature Requests

The nodes are embedded inline with formatting artifacts (hello\\nhello), apparent test content, and no versioning metadata. Editing it requires understanding the entire config file, and changes produce noisy diffs that obscure actual prompt improvements. Before: prompts: - id: 187c2f4b-c23d-4545-abef-73dc897d6b7b role: assistant content: >- hello hello Important:please provide the response in wrap text format... After: prompts: - id: rag_system_prompt_v1 role: assistant # Edit prompts in /prompts/rag_system.md — supports preview & versioning contentRef: ./prompts/rag_system.md version: "1.2.0" lastUpdated: "2025-01-15" Developer Impact: Prompts become first-class artifacts — reviewable in PRs, A/B testable, and editable without touching pipeline config.

Aman Sharma 5 months ago

💡 Feature Requests

Improve the branch merge interface with better file naming (showing actual flow names instead of random IDs) and debugger like node structure showing changes between versions. Visual Flow Comparison Implementation Create a mini preview version of flows for comparison Implement side-by-side comparison with shaded out portions to show differences Show nodes that don't exist in previous flow versions with visual indicators Support similar sync scroll between compared flows

Aman Sharma 6 months ago

UI/UX

💡 Feature Requests

Problem: No clear way to terminate workflows early or break out of loops based on conditions, forcing users to create workarounds with maximum iteration counts. Use Case: When API calls fail or specific conditions are met, users want to cleanly exit workflows or break out of polling loops without continuing unnecessary iterations. Proposed Solution: Implement dedicated break and end nodes that allow conditional workflow termination and loop breaking.

Aman Sharma 7 months ago

💡 Feature Requests

Problem: Manual grading of responses does not scale and is inconsistent. Use case: Systematically assess response quality across large test suites. Functionality: Grader agents with configurable rubrics, batch “grade all” execution, and integration with logs for historical evaluation.

Aman Sharma 7 months ago

💡 Feature Requests

Problem: Developers must manually manage JSON and memory fields across nodes, with inconsistent support. Use case: Simplify building and testing conversational flows with a toggle-based experience. Functionality: Checkbox-style chat history enablement, consistent memory field support on all nodes, and automatic user-based history storage and retrieval.

Aman Sharma 7 months ago

💡 Feature Requests

Customer reports: “Only the person that opens the account right now can enter billing information. Even if you’re an admin, you cannot enter billing information for what I’ve seen. Is there a way to solve this in a future product feature?” While best practice is to restrict the ability to edit billing info in a multi-user SaaS product, the ability to assign this role is desireable and will support the use case where a user signs up with a free account, but needs someone else to input billing info to upgrade. Will also support the ability for users to transition off the team and to have multiple billing managers for backup.

cwhiteman 11 months ago

5

👏🏻 Feedback

Feature : An input box like feature, which post some characters entered provides suggestions (Gen-AI based or RAG based) and allow users to complete the input box. This could basically take a collection of objects to show the output Why : Can allow for quick widget suggestions based on user documents and/or basic gen-ai suggestions Example(s) :

Dhruv Pamneja 11 months ago

💡 Feature Requests

Requirement from Pat at co-dev to connect to mySQL (on PlanetScale).

cwhiteman 12 months ago

High Priority

🔌 Integration

Add functionality to version control within an environment - Projects Flows Prompts Version control should be visible via - logs API requests Reports

Aman Sharma about 1 year ago

1

👏🏻 Feedback

Support Vector DB Weaviate PG_Vector PineCone Additional MongoDB ChromaDB Qdrant

Aman Sharma over 1 year ago

🔌 Integration