Updated on
To answer a question about a customer contract, an AI application needs the contract details and applicable terms. The software must retrieve this information and supply it to the language model. How this happens, and which information may be used, are questions of context architecture.
Required data and access rights
I use context architecture to describe the technical design of this information supply. It includes the sources, how they are updated and the instructions to the model. The application needs to retrieve the relevant content while respecting users’ access rights.
In customer service, general product information and an individual customer’s contract details are different kinds of information. Both may be needed for an answer, but only authorized people may see contract data. The application must check that permission before passing content to the model.
Supply context within the workflow
I consider four connected tasks:
- Instructions: Specify the task, response format and boundaries, including how to handle missing information.
- Retrieval: Fetch suitable documents and data, for example through retrieval-augmented generation (RAG), accounting for their source and currency.
- Process state: Keep track of what has been established and which step comes next.
- System connections: Design access to business systems around permissions and permitted actions.
In the example, the system could retrieve product information from a knowledge base and case status from a ticketing system. If an answer needs approval, the application forwards it to the responsible person. Access controls are implemented in software and must not depend solely on instructions to the language model.
Test answers and workflows
Domain specialists and developers test the application together using typical enquiries. They examine the sources used, answer accuracy and forwarding to responsible staff. Cases include conflicting information, outdated documents and missing permissions.
This also helps diagnose errors. With a wrong answer, the team can investigate whether suitable sources were retrieved, instructions were clear, or the model processed the information incorrectly. This requires a record of the content the model received.
Responsibilities during operation
Before introduction, someone must be responsible for updating sources, managing permissions and reviewing answers. These tasks require staff capacity after launch as well. If contract terms change, for example, the new information needs to reach the application in time.
Assessing this type of application therefore includes examining how its information will remain current. Defined responsibilities are needed to plan the ongoing operational effort.