Cortex

#The problem

What was actually wrong, in terms someone outside the team would recognise. Not the solution phrased as a problem.

Several product teams needed to answer questions over their own document sets. Each had started building retrieval separately, and each had independently arrived at a slightly different, slightly wrong version of the same pipeline. The duplicated effort was the visible cost. The invisible one was that none of them could tell whether their retrieval was good, because none of them had an evaluation harness.

#Context and constraints

The constraints are the interesting part of any system. Without them any architecture looks arbitrary.

• Multi-tenant, with hard isolation. Tenants must never see each other's documents, and that has to be enforced somewhere a single query cannot bypass.
• Ingestion is bursty. A tenant onboarding uploads a corpus at once; steady state is near zero. Provisioning for the peak wastes most of the year.
• Latency budget is set by the product, not by us. The retrieval stage has a fixed share of the end-to-end budget, and a reranker that exceeds it is not worth its accuracy.
• Operated by one team, used by several. Anything requiring the platform team in the loop for a routine change becomes the bottleneck.

#Architecture

One diagram, then the reasoning. The diagram should show the flow, not every component.

1. Ingestasync, bursty
2. Chunkper-type policy
3. Indexdense + lexical
4. Retrievehybrid
5. Reranklatency-bound

Describe each stage in a sentence or two, and say what it is allowed to be bad at. A component with no stated weakness has not been designed, only drawn.

#Decisions

The point of a case study. Each one gets the alternative it beat and the reason.

#Isolation at the storage layer, not the query layer

Options considered, what was chosen, and the tradeoff that came with it.

#One index per tenant, or one index with a filter

The cost model of each, and where the crossover is.

#Hybrid retrieval from the start

Why lexical matching was not deferred, and what class of query it rescues.

#What broke

The most valuable section, and the one usually missing. Real failures, described plainly. A case study with no failures reads as marketing.

#What I would do differently

Written with hindsight, not defensiveness.

#What I learned

The transferable part: the model that now applies to the next system, not the specifics of this one.