Industrial-Scale AI Model Distillation: The New Security Risk for AI Providers and Manufacturers
Distillation is a normal technique: train a small model on a large model's answers. A September 2026 joint advisory describes doing it to models you do not own, at scale, through ordinary queries. The original weights need not be exfiltrated; a surrogate is trained from collected outputs, so the defence is volume and pattern rather than file security. If your company runs its own inspection or engineering model, the same method applies to it.
What the advisory describes
On 8 September 2026 the NSA, CISA and the FBI issued joint advisory AA26-251A. It states that several named AI companies based in China have been systematically extracting the capabilities of US frontier models. This page does not repeat the attribution or the company names; they are in the advisory, and the engineering content stands without them.
The advisory's description of the mechanism is the usable part. According to the advisory the activity routes requests through multiple pathways to obtain access, including native interfaces, cloud providers and third-party aggregators, and uses a grey market of proxies to get around restrictions. It describes prompts designed to make a model reveal its step-by-step reasoning rather than only its conclusions, so that the reasoning itself becomes training material. The stated scale is billions of tokens across millions of exchanges since late 2024.
What kind of statement this is
This is a government advisory, not a peer-reviewed study. Its attribution and intent findings come from that investigation and are not independently confirmed here. The part that is directly usable by any organisation, regardless of what it thinks of the attribution, is the description of technique, the behavioural indicators and the mitigations. Those stand on their own.
Distillation against extraction
Coverage of this advisory uses the word "distillation" for two different things, which is why the story keeps getting muddled. Distillation is a legitimate and widespread method: a smaller student model is trained on the outputs of a larger teacher, and the result is cheaper to run. It is widely used to produce models that cost less to serve. The question is never the technique. It is whose model, under what authorisation, at what volume.
| Dimension | Legitimate distillation | Extraction campaign |
|---|---|---|
| Authorisation | Your own model, or licensed output rights | Access obtained under terms that forbid it, or through resold and proxied accounts |
| Volume pattern | Bounded by a stated purpose | Billions of tokens across millions of requests; sustained and automated |
| Query design | Ordinary task prompts | Prompts engineered to expose reasoning steps, not just answers |
| Account behaviour | Human rhythm, identifiable organisation | Many addresses on one account, no idle periods, immediate maximum usage |
| Result | A smaller model you are entitled to deploy | An imitation of someone else's capability, with the cost of building it avoided |
Why NIST treats extraction as a stepping stone
Model extraction is a documented attack class. NIST's adversarial machine learning taxonomy, published in March 2025, lists model extraction as a defined privacy attack requiring only query access, with its own identifier in the taxonomy. It describes the setting plainly: providers train large models on proprietary data and wish to keep the architecture and parameters confidential, and the attacker's goal is to extract information about that model.
The more useful observation is about consequences. NIST notes that model extraction "is often not an end goal but a step toward other attacks", because once architecture and weights are known an attacker can mount the stronger attacks available in white-box settings. Preventing extraction therefore removes the foundation for a set of later problems, which is a better argument for spending money on it than intellectual property alone.
Which industrial models are exposed
A manufacturer reading a frontier-model advisory will reasonably ask what it has to do with a factory. The connection is that industrial companies have become model owners. A vision system trained on years of your own defect images is a proprietary asset whose decision boundary encodes accumulated inspection experience. So is an engineering assistant trained on your specifications, or a maintenance model trained on your failure history.
| Model type | What it contains | What copying it would mean | Proportionate control |
|---|---|---|---|
| Generic assistant | Bought capability, low uniqueness | Little to copy that is yours | Normal account hygiene |
| Internal engineering model | Trained on your specifications and drawings | Copying it transfers your design practice | Restrict to named users; log every query |
| Quality-control vision model | Trained on your defect images | Its decision boundary is the accumulated inspection knowledge | Never expose raw scores publicly; rate limit |
| Control-adjacent agent | Can read or influence process systems | Extraction is the smaller worry; access is the larger one | Treat as an OT interface, not an app |
One boundary deserves separating clearly. Extraction copies what a model does. Reconstruction and membership inference, which are separate classes in the same NIST taxonomy, aim at what the model was trained on. Losing the first costs a competitive advantage. Losing the second can expose the underlying records, which for an industrial company may include customer drawings or process data held under contract.
Detection indicators
Because every individual request is legitimate, detection has to work on shape rather than content. The advisory's indicators are directly reusable by any organisation exposing a model: one account used from many addresses and user agents; sustained round-the-clock usage without the idle periods human work produces; an odd ratio between subscription activity and programmatic use; and new subscriptions that begin immediately at maximum usage.
The mitigations it recommends have the same ordinary character: limit query volume and rate, control who can reach the model, log telemetry, constrain how much detail outputs expose, degrade responses to activity assessed as extraction, limit what model artefacts are released publicly, and share indicators with peers. None of this requires new technology. It requires deciding that a model is an asset with an access policy rather than a feature with a URL.
A question for your own logs
If a competitor made ten thousand well-chosen queries against your inspection or engineering model over a month, would anyone notice, and would you still hold anything they did not? If the answer to both is no, the model is public in practice and should be treated that way in commercial planning.
What this means for manufacturers
Inventory the models you actually own
Separate bought capability from models trained on your own data. Only the second group is worth protecting, and most companies have never listed them.
Give each owned model an access policy
Named users or services, scoped keys, and a stated purpose. Standing broad access to a proprietary model is the equivalent of leaving drawings on a public server.
Log queries, not just outcomes
Volume, source, timing and pattern are the only evidence that extraction is happening. If the logs hold only results, the question cannot be answered afterwards.
Rate limit by default
Set limits from legitimate use plus a margin. This is the single control that makes bulk copying expensive without affecting anyone doing real work.
Do not publish more than the answer
Confidence scores, ranked alternatives and reasoning traces are far richer training material than a plain result. Expose them internally if useful, externally only with a reason.
Write it into supplier contracts
Where a vendor hosts a model trained on your data, establish who may query it, what is logged, and what you are told if abnormal usage appears.
Limits and open questions
- Attribution is the agencies'. The identification of specific companies and the assessment of government awareness come from the issuing agencies and are not independently verified here.
- Effectiveness is unquantified. Nothing published states how closely an extracted model matches its target, so the commercial damage cannot be measured from these documents.
- Detection indicators produce false positives. Legitimate automated evaluation also runs continuously without idle periods. Indicators are a reason to look, not a verdict.
- Defences trade against usability. Rate limits and reduced output detail also constrain legitimate users, and no published guidance sets the right level for an industrial model.
- No cost data exists. There is no public basis for what these campaigns cost their targets or what protection costs, so any figure in circulation is an estimate rather than a measurement.
Models trained on proprietary industrial data are assets that can be copied through their own interfaces, without any breach in the traditional sense. Deciding which of yours are worth that treatment is a half-day exercise, and most companies have not done it.
Questions
Is distillation itself illegitimate?
No. Training a smaller model on a larger model's outputs is a standard and useful technique, and it is how many efficient models are produced. What the advisory describes is doing it to someone else's model without authorisation, at scale, and in breach of the terms under which access was granted.
Is this the same as stealing the model file?
No. The original model weights need not be exfiltrated; a surrogate is trained from collected outputs, so file-level protection does not apply. The behaviour of the model is reproduced by querying it many times and training on the answers. The weights stay where they are while the capability is copied.
Does this matter to a manufacturer that is not an AI company?
It matters to any company that exposes its own model through an interface: a visual inspection model trained on your defect images, an engineering assistant trained on your specifications, a maintenance model trained on your failure history. Those models are proprietary assets, and the same method applies to them at a smaller scale.
How would anyone notice it happening?
By usage shape rather than content. The advisory points at indicators such as sustained round-the-clock use without the idle periods human work creates, one account used from many addresses, and new subscriptions that begin immediately at maximum usage. Those are patterns any provider can look for in its own logs.
Is stealing the model the same as stealing the training data?
No. Extraction copies what the model does. Reconstruction and membership inference aim at what the model was trained on. They are separate attack classes in the NIST taxonomy with different consequences: one costs you a competitive advantage, the other can expose the underlying records.
Sources
- CISA, NSA and FBI: Joint cybersecurity advisory AA26-251A on industrial-scale distillation campaigns. Government advisory. Published 8 September 2026. Checked 12 September 2026.
- NIST: AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. Standards taxonomy defining model extraction. Published March 2025. Checked 12 September 2026.
Suggested citation: Inzonex Research (2026), Industrial-Scale AI Model Distillation: The New Security Risk for AI Providers and Manufacturers, published 12 September 2026. Technique, scale, indicators and mitigations are attributed to joint advisory AA26-251A. The attack classification is from the NIST adversarial machine learning taxonomy. The comparison tables, both diagrams and the industrial exposure assessment are Inzonex analysis.