Design the ingestion pipeline for a RAG system with frequent document updates.
Instruction: Explain how you would ingest, update, and retire documents for a rapidly changing knowledge base. Current index I1 uses embedding model E1 with 768 dimensions. New index I2 uses E2 with 1,024 dimensions and is only 70% built. A source deletion occurs during migration. Specify serving, activation and deletion behavior.
Context: Assesses whether the candidate can design a practical architecture and explain the main tradeoffs. Explain how you would ingest, update, and retire documents for a rapidly changing knowledge base.
Updated
Official answer available
Read the opening below, then unlock the full answer and practical guidance.
I would give each source document a stable ID and version, then process updates idempotently. Duplicate events should not create duplicate chunks, and an old event should not replace a newer active version...
Related Questions
-
easy
-
easy
-
easy
-
easy
-
easy
-
easy