Update info buttons within the UI

Customers wonder what the difference between Incremental & Full Refresh is on the Google Trigger Node. The info button doesn’t really answer this question, so users are left to try /ask in the product-help Slack channel. Based on my test, the answer (shown below) is partially incorrect. I tested removing a document and the subsequent job did not remove the related records. However, updates were correctly reflected in the DB.

Please update the Sync Mode info button for the Google Trigger as follows:
“Use Full Refresh mode to clear the DB and re-embed all content. Use Incremental mode to re-embed updated files (will not remove records for deleted files).”

FOR INFORMATION ONLY: Slack Channel: help_product /ask
Lamatic.aiLamatic.aiAPP 12:01 PM
Please wait a moment while I answer your question: On the Google Drive trigger node, what is the difference between selecting "Full Refresh" versus "Incremental"? Can you explain how each setting would handle the following 2 use cases:
1. A document that was previously indexed is updated. Are the related records in the vector db updated when the Incremental setting is selected?
2. A document that was previously indexed is deleted. Are the related records in the vector db removed when the Incremental setting is selected?

Lamatic.aiLamatic.aiAPP 12:01 PM
On the Lamatic platform, the Google Drive trigger node comes with two main settings for handling documents: "Full Refresh" and "Incremental."1. Full Refresh: This setting implies that every time the trigger is activated, it reprocesses all documents in the specified Google Drive folder. This means that records in the vector database related to all documents will be completely refreshed, including updates, deletions, and additions. It is akin to reindexing your entire dataset.2. Incremental: This setting focuses on processing only changes made since the last execution of the trigger. It is designed for efficiency by processing only new or modified documents and detecting deletions since the last run.For the two use cases mentioned:
1. A document that was previously indexed is updated:
With "**Incremental**" setting, when the Google Drive trigger runs, it will recognize the update to the document and update the related records in the vector database accordingly. This ensures that the index reflects the most current version of the document without reprocessing unchanged documents. CORRECT
2. A document that was previously indexed is deleted:
With "**Incremental**" setting, when the trigger runs, it should detect the deletion of the document and subsequently remove the related records in the vector database. This keeps your vector database clean and aligned with the current state of your Google Drive contents. INCORRECT
These settings help in managing efficiency and data consistency, especially in scenarios where document updates and deletions are frequent.

Please authenticate to join the conversation.

Upvoters
Status

Completed

Board

👏🏻 Feedback

Date

About 1 year ago

Author

cwhiteman

Subscribe to post

Get notified by email when there are changes.