The phrase "data sovereignty" is everywhere right now. Vendors use it in their pitch decks. IT managers use it in budget requests. Compliance teams use it in risk assessments. Almost nobody agrees on what it means in practice.
Here is a working definition: data sovereignty means your organization retains legal, physical, and operational control over its data at every stage of processing. Not just at rest. Not just in transit. At every stage - including when an AI model is actively reading it to answer a question.
That last part is where most AI tools fail the test.
The problem with cloud AI tools
When your staff types a question into a cloud AI product - Claude, ChatGPT, Copilot, Gemini - what happens to that question?
It leaves your network. It is tokenized, embedded, and processed on servers you do not own, in jurisdictions you may not have approved, by infrastructure you have no visibility into. The vendor's terms of service govern what happens to it from that point forward. In many cases, those terms include the right to use your inputs to improve the model.
The diagram below shows the actual processing path:
Every step inside the dashed red boundary happens on infrastructure you do not control. Your query, your documents, and the embeddings derived from them are all processed there. If your organization operates under PIPEDA, Quebec Law 25, HIPAA, or any sector-specific regulation that restricts where data is processed, this is a compliance problem - not a theoretical one.
What sovereignty actually requires at each step
Most organizations focus on data storage. That is necessary, but insufficient. Real data sovereignty requires control at four distinct points in the AI processing pipeline:
Tokenization - the process of converting your text into fragments the model can read. This happens on the model's inference server. If that server is in another country, your raw text has crossed a border.
Embedding - converting tokens into numerical vectors that encode meaning. These vectors are derived from your content. In a cloud system, the embedding model runs on vendor hardware, meaning your content is processed there.
Storage and retrieval - where your document index and conversation history live, and who can query them. Cloud products store this on vendor infrastructure by default. Retention policies, access controls, and deletion rights are governed by the vendor's terms, not yours.
Inference - the LLM generating an answer. In a cloud product, this runs on vendor hardware. In a private system, it runs on your hardware, inside your network, under your operational control.
Sovereignty is not achieved by checking a box that says "enterprise tier." It requires that all four of these steps happen within infrastructure your organization controls.
What the private alternative looks like
A private AI system processes the same query through the same four steps - but nothing crosses your network boundary at any point. The diagram below shows what that looks like in practice:
Two things are worth noting in the second diagram. First, the green boundary stays intact across all four processing steps. Second, the answer includes a source citation - [SOP p.2] - which means users can verify every answer against the original document. There is no blend of external training knowledge. The model works only from documents you have put in front of it.
The Canadian context
For organizations operating in Canada, data sovereignty is not just a technical preference. Quebec's Law 25 requires that organizations conducting a privacy impact assessment before sending personal information outside the province. PIPEDA governs cross-border transfers. Several regulated sectors - financial services, healthcare, legal - have additional obligations that make it difficult or impossible to use AI tools that route data through foreign infrastructure.
Running AI inference on Canadian soil, on infrastructure you control, is not a niche requirement. For mid-market organizations in regulated industries, it is a precondition for adoption.
What to ask your current AI vendor
If you are evaluating an AI product for your organization, these are the four questions worth asking before you sign anything:
- Where is my data processed during inference — which country, which provider, which data centre?
- Do you use my inputs or documents to train or fine-tune your models, and under what conditions?
- What is your data retention policy for queries, and how do I exercise a right to deletion?
- Can you provide a signed data processing agreement that specifies jurisdiction and limits on data use?
If the answers are vague, that is the answer.
Worth a look
Discreetly AI is a working example of this architecture in practice - a private AI assistant that runs entirely on Canadian infrastructure, with no data leaving the client's environment at any point in the processing pipeline. If you are working through these questions for your own organization, it is worth a look.
