A model running on your own machine can still send your data halfway around the world before it finishes answering.
The pitch is clean. Download the weights. Run the inference locally. No API call, no cloud round trip, no chance of your prompt ending up in someone else’s training set. The pitch is also incomplete. Local inference is one piece of the privacy story. It is not the whole story, and treating it as the whole story is how companies end up exposing the exact data they thought they were protecting.
Three ideas get constantly confused in the public conversation. They are not the same idea. Conflating them is what makes “we are using open source AI” meaningless as a privacy claim.
Open weights, open source, and the gap in between
“Open source” used to mean you could read the source code, modify it, and redistribute your changes. That definition is from the Open Source Initiative and it is roughly 25 years old. It works well for compilers, web servers, and most things written before 2020.
It works less well for large language models. A model has weights, training data, training code, fine tuning data, fine tuning code, and the inference runtime. The Llama models from Meta release the weights under a community license. The training data is not released. The training code is not released. The fine tuning recipes are not released. By the strict definition, the model is not open source. It is “open weights.”
That distinction matters for privacy because the parts you cannot see are the parts that determine what the model has memorised, what it tends to leak, and what biases it carries. The weights are the artefact. The data behind the weights becomes the substance.
Locally executed, with cloud based everything else
The most common deployment pattern we see in 2026 looks like this. The model runs on a developer laptop or a private GPU box. The model is fed context from a vector database. The vector database is hosted in a managed cloud. The embeddings for that vector database are generated by a third party API. The agent uses a browser tool that is a wrapper around a remote headless Chrome service. Errors are reported to a hosted Sentry equivalent. Telemetry is on by default.
Every one of those hops is a place where the prompt, the response, or the underlying data can leave the machine. The inference is local. The system is not.
Remote embeddings are the most common leak
When you embed a document for semantic search, the text of the document is sent to the embedding API. The API returns a vector. If the API is hosted by OpenAI, Cohere, Voyage or any of the other major providers, the text of the document has just been sent to a third party. The vector you store locally is not the leak. The call that created it is.
Self hosted embedding models exist. They are not as fast as the hosted ones and the quality gap is real but narrowing. If privacy becomes the requirement, the embedding model runs on the same machine as the inference model. There is no remote call.
Telemetry and error reporting are on by default
Almost every AI developer tool ships with telemetry on by default. LangSmith. Helicone. Langfuse. Weights & Biases. The convenience serves as the product. The product is also a record of every prompt, every completion, every tool call, every model parameter, sometimes including the full text. Most of them have a “self host” option. Most of them are not self hosted by default.
Same story for crash reporting. Sentry, Bugsnag, Rollbar. The default config sends stack traces, which often contain variable dumps, which often contain the prompt the agent was running. Disable the default. Read what is being sent. Then decide.
Docker images and unverified dependencies
Running a local model usually means pulling a Docker image. The image is built from a Dockerfile that runs `pip install` and `npm install` and a few shell scripts. You do not see those scripts run. You see the resulting container, which works.
The container has a base image. The base image has a package manager. The package manager pulls from a registry. The registry has a long history of compromises. This amounts to the same software supply chain that hit SolarWinds, that hit 3CX, that hit xz utils in 2024. The model running inside the container is local. The supply chain that delivered the container is global.
Pin image digests, not tags. Run a vulnerability scanner. Rebuild from source for anything that touches production data.
Local models with externally hosted agent memory
Persistent memory is one of the headline features of agent frameworks in 2026. LangMem, Mem0, Letta, Zep, the list is long. Most of them offer a hosted option. The hosted option means the agent’s memory, which is by definition the things the agent has decided are worth remembering, lives on someone else’s infrastructure.
The privacy story for agent memory sits as the privacy story for any other production database. Where is it stored, who can read it, how long is it retained, what amounts to the deletion story, what serves as the backup story, what happens when the vendor goes down. The fact that the data is “memories” rather than “customer records” does not change the answers. It just makes the questions less likely to be asked.
How to conduct an AI data flow audit
Once a quarter, take a single end to end agent task. Trace every byte. The prompt. The retrieved documents. The embedding call. The vector search. The model inference. The tool calls. The response. The telemetry. The error reports. The logs. The backups. Map each step to a network endpoint. The endpoints that are inside your VPC are private. The endpoints that are not, are not.
You will be surprised. The model is local. The rest is somewhere else. The work is to make the rest local too, or to accept that the privacy boundary is not where you thought it was.
The bottom line
Privacy is not a product setting. It is an architecture that someone has to inspect and continuously maintain. Open weights give you a model. Local execution gives you inference. Neither gives you a private system. A private system is a chain of decisions about embeddings, vector stores, agent memory, telemetry, error reporting, dependency provenance, and network egress. The decisions are not made by downloading a model. They are made by an engineer who has the time and authority to ask “where does this data actually go.”
If that engineer is you, congratulations. If that engineer is a vendor, get the answers in writing. If nobody is asking the question, the answer is no, your AI is not private, no matter where the weights live.
Sources & Further Reading
All claims in this article are sourced from primary documentation, vendor advisories, and reputable security researchers.
Spotted an error? Email the editor. Corrections are issued with a visible correction note.
Editorial standards. Every article on humanrequired.org is reviewed by a human editor before publication. AI may assist with drafting or research; final editorial control is human. Read the full standards.
