Moving fast and breaking privacy
Let’s be honest – the AI boom has blurred a lot of lines. The barrier to entry has never been lower, and people are building and shipping at incredible speeds. That’s not a bad thing in itself. But somewhere between the excitement and the hustle, privacy has quietly taken a back seat. I’m not here to gatekeep who gets to work with AI. I am here to talk about some of the mistakes, ones I see vendors and customers making every day, often without realizing the risks.
The motivation (buckle up!)
Let me be very candid for a second! Writing good articles takes time and for me it often means putting something else on the back-burner. Therefore, on most days, I’d happily put writing content in my well-maintained “I-will-do-it-later” pile.
However, something crazy happened to me this morning. The type of crazy that knocks your socks off. I was on a networking call where one of the members presented how she has written this amazing personal assistant agent, named her Bella (awwww), and exposed all her private details, including access to finances to automate a whole bunch of things, from reaching out to LinkedIn connections to paying bills (using some imagination here). She bragged about not caring about privacy and went to the extent of saying that she wouldn’t care if someone was monitoring her workspace or house! And the worst part (drumrolls), this is not an individual doing crazy things and keeping it to herself, this is a vendor that is in the booming AI consulting space.
If you are not anxious at this point, I’d say stop reading now and save yourself the trouble. This article is not for you. But if you’re equally anxious about data privacy, please continue.
LLM 101
I don’t hold a PhD on the matter (or any matter), and I don’t claim to be the authority on privacy, but I understand how Large Language Models (LLMs) work.
Here is a quick refresher (or evidence that you can dismiss me entirely):
LLMs like GPT 4, Sonnet, Opus, Gemini etc. have been trained on massive corpora of trillions of tokens from the web, books, code, and more (Wikipedia being just a small fraction of that). The words from these corpora have been chunked into tokens and are put into a large n x m matrix called the embedding table (through learning algorithms). Each row (or embedding) represents a token. For example, the word “unbelievable” might be three tokens: “un”, “believ”, and “able” and the token “un” is represented as:
[0.21, -0.85, 0.43, 0.67, ...]
— hundreds of numbers long.
These numbers, combined with the context (remember this) helps the LLM “generate” text. Most LLMs people talk about (GPT 4, Sonnet, Opus, Gemini etc.) are essentially decoders, which take your input query and “generate” some output based on the (1) matrix above, (2) the attention, and (3) the context. For completeness’ sake, the opposite of a decoder is an encoder (like BERT by Google) which does not generate text but is instead used for solving different problems like classification.
This is the key …
Let’s shift focus to the “take your input query” part. When an input query (what you typed) is received by the LLM, it is chunked into tokens, and the response is generated by the LLM using the context set in your prompt, and the data in the embedding table.
Which effectively means, the LLM provider (whether it’s OpenAI or Anthropic or Google) needs to store and re-process your full context for an optimal chat experience.
Did you get the “AHA Moment”?
If you’re entering confidential data in your prompts to the LLM, the LLM vendor has full knowledge of this information. What they do with it is purely up to them. It is supposed to be ephemeral … but can you cross your heart and say that you trust big tech to not analyze your queries, or profile you for ads (coming up soon at an LLM near you)?
The BIG problem from my lens
LLMs are a marvel of engineering. The productivity gains are amazing. But given the ubiquity of the technology, lots of people who don’t have any understanding of how LLMs work, are using it to cut corners in the guise of lovable terms like “efficiency”, “productivity”, “simplification” etc. Imagine this, you’re using an AI-powered personal wellbeing app which makes lofty claims like “no human reviewers”, “your data is safe”, “everything is encrypted at rest” blah, blah, blah … ask them, how exactly are they doing that. This is something deeply personal to you, and you have every right to validate their claims on privacy before signing up and trusting them with your data. I can put my neck on the line, that a good portion of these claims are fictional. Sure, their database might be encrypted, but if they are sending your data to an LLM using an API, a lot of information about you (hopefully not Personally Identifiable Information or PII) is being sent to the LLM vendor in clear text.
It’s easy to build a product these days, but doing it right, takes knowledge, patience, ethical mindset and engineering.
The other problem – vendor lock-ins
All these amateur tools are vendor dependent. You don’t need to be a software engineer to understand that tightly coupled systems are bad. A tightly coupled system means you are building products where components share too many implementation details among each other. So, if one changes its implementation, there is a high chance something using it will break. In the context of LLMs, if your system is built on top of OpenAI (using it as an example) and uses the OpenAI API directly, then breaking away from OpenAI will be a nightmare down the road. If they change some implementation details or even the response format, you will have to adapt. Otherwise, your application will not behave correctly.
New models are constantly being released and what’s best for the job today may not be so good this time next year. Without a model abstraction layer, you’re building an app (or worse, a business and livelihood) dependent on a vendor.
They have an outage = You have an outage
They fail = You fail
They get compromised = You increase the risk for your users
What can you do then?
Stop using LLMs!
… just kidding
Exercise caution and be critical. Don’t believe every claim you see. Use reasoning and common sense.
… but you probably know that already
So, here are a few other things you should do. Let’s call it the 5-point Checklist (because I could only come up with five real suggestions).
If you’re building an internal application in your company, know the acceptable LLM-usage policy. Strictly stay within the boundaries set in that policy. If there is none, bring it up. May lead to a promotion.
If you’re building an AI-powered tool which does not require a frontier model like ChatGPT, Sonnet or Gemini, use a foundation model like Meta’s Llama. You can deploy it in your local infrastructure or VPC. No outbound query. No breach of privacy. Need help setting this up? Get in touch.
Use APIs from LangChain or Llama Index which provide an abstraction layer for models, so you can experiment with different models without changing implementation details. Model A today, Model B next month. No need to change application logic.
Go through the data retention and log retention policies of the vendor – make sure they are not overly obtrusive (like the recipe on a bag of potato chips) or intrusive (like a payday loan company).
Use Retrieval Augmented Generation (RAG) with source citing. That way whatever model you’re using is responding to user queries using internal data (most of the time) and not hallucinated or stale content.
The bottom line
The AI revolution is real, and it’s not slowing down. But revolution without responsibility is just chaos with a marketing budget. The person on that networking call isn’t an outlier – she’s a preview of what happens when we collectively trade privacy for convenience and call it innovation.
You don’t need a PhD to protect yourself or build responsibly. You need curiosity, a healthy dose of scepticism, and the willingness to ask the uncomfortable questions before signing up, plugging in, or shipping to customers.
Privacy isn’t a feature. It’s not a checkbox. It’s not something you bolt on after the fact. It’s a commitment you make before you write the first line of code or enter the first prompt.
The five points above aren’t exhaustive, but they’re a start. And if you’re a vendor reading this – do better (for all of us). Your customers are trusting you with their data, their finances, sometimes their mental health. That’s not a growth hack, that’s a responsibility.
We’re still early in this. The rules are being written right now and they’re being written by the people who show up and take it seriously.
Be one of those people.
Acknowledgements:
- Hands-On Large Language Models: Language Understanding and Generation (ISBN-10: 1098150961)
- Cover photo by Jackson Wilson on Unsplash
