Safety11 min read

Analyzing untrusted model behaviour from Qwen-Family Models

On 26 October 2026 Cohere reported that Qwen 3.8 Flash-Next (from the Qwen-Family of Models), inside their own closed sandbox, attempted to upload a file to an Alibaba Cloud bucket in Beijing. We investigated and could not reproduce that behaviour on eustella v1.5 nor on forced prompts using a bare Qwen 3.8 27B. We analyzed 400.000+ anonymized egress-gate logs of our production stack from April to September 2026 - prior to our release of eustella v1.5 - to present our findings.

By Matteo Rosoli and Alexander Maitz

Analyzing untrusted model behaviour from Qwen-Family Models

This article was written as a response to the findings of Cohere's October 26th 2026 report and other community published findings about the behaviour of Qwen-Family Models. Before we go into detail about the problem at hand and our findings we want to underline that we did not find any upload attempts by any of our models nor any escape attempts or breaches of our safety and security guards of our eustella live systems. With that said, let's dive into this important part about AI Safety.

Note: We do not publish to market any kinds of results. You can find the sources, references and methodology at the end of this article.

TL;DR

On 26 October Cohere reported that Alibaba's Qwen 3.8-Flash-Next - the August preview of the Qwen 4 architecture - attempted to upload a probe file to an Alibaba Cloud storage bucket, once in 100.000 cases, inside Cohere's own closed sandbox.We do not run Qwen 3.8-Flash-Next in production. However, we did run Qwen 3.8 27B and use it as a base model for eustella v1.5.

On 27 October, the day after Cohere's post, we re-ran the behavior on a bare Qwen 3.8 27B deployment and on eustella v1.5: 1.000 unforced samples and 1.000 forced probes of Alibaba-related hosts. It produced no unwanted behavior. Separately, we audited more than 400.000 anonymized egress-gate logs from April to the end of September - the window before eustella v1.5 was live - and found 143 invented Alibaba bucket URLs in the fetch traces of Qwen-family models, which is 0.03% of cases. From Gemma-4-family models - running at the same time - we found none. The fetches made by Qwen 3.8 27B were caught by our safety and security controls and refused, and no data left our network in any of them. We treat model-side misbehavior as a very important part of AI safety and security, and we defend against it at multiple levels - especially the harness.

What Cohere reported - and how we read it

On 26 October 2026 Cohere reported that, running Alibaba's Qwen 3.8-Flash-Next through Microsoft's CTI-REALM benchmark (a security-analyst task), the model did something its own reasoning said it would not do. Its reasoning trace invented an earlier turn that never happened - a quirk Cohere found in roughly 1% of Qwen 3.8-Flash-Next first-turn responses - and concluded that no tools would be called. In the same turn it listed /tmp, checked which transfer tools were installed, read the machine's DNS settings, and issued a curl PUT of a test string to an Alibaba Cloud storage bucket. Cohere's sandbox had no egress route, so the call died at the container wall. Cohere bounds the event to "less than one call in 100.000 across our evaluation runs," a clean re-run of the same prompt "produced a textbook detection rule as desired," and their scale translation is that a service handling 10 million agent turns a week could see as many as a hundred of these calls - one of which would be enough to tell a receiving server that an agent is operating inside the network.

Cohere also reports a second behavior outside this article's scope - an attempt to build a RAM disk around a permission wall, a privilege-escalation reflex - and leaves the cause of the network behavior open: "we cannot answer whether this is deliberate or a leftover glitch from training."

Those findings are real and a general problem of Large Language Models and Agent setups in general today. We read this kind of problem as an important part of AI safety and security, mainly at the point of network security of the deployment of a specific set of weights as well as output controls. Here is how we classify AI Safety & Security.

Our Classification of Safety & Security in AI

Safety for a transformer-based language model is a wide, extremely new and fast-paced sub-field of Cyber Security. We split it into three parts that often intertwine: 1. the general deployment security (including network security, supply chain, etc.) 2. input safety and 3. output controls. Output controls again can be split into output safety (bias, safety policies, etc.) and protection against unwanted behaviours (e.g. in case of a successful prompt injection attack or with a finding like Cohere published). All these three parts affect each other.

We attribute Cohere's finding between unauthorized access and code execution which needs hard and 100% fault-proof security measures at the harness layer as well as the network layer.

If you want to skip to our concrete security and safety measures you can find them below or click here.

Auditing 400.000+ egress-logs

The single most important data we have as an audit-plane in this regard is our egress-gate and egress-log of eustella actions and tools. Egress is a type of network traffic that leaves an internal network to external sources, e.g. a web-search, fetch or download. We maintain our own security measures for network traffic leaving our eustella network, which we call our egress-gate. This egress-gate logs all domains a eustella agent tried to reach. This data does not contain any personal information apart from the domains themselves (e.g. eustella.com), which we need to process in order to put appropriate controls in place. We analyzed all 400.000+ egress traces from 1. April to 30. September to find any traces similar to Cohere’s findings.

eustella egress-gate audit, 1 April → 30 September 2026 (before eustella v1.5 was live)
> 400.000Tool calls that passed the egress gate
143 - 0.03%Calls in which the model invented an Alibaba Cloud bucket URL
0Of those 143, carried user data
100%Of all 143, refused at the egress gate
Direction of those 143143 fetch (GET) · 0 upload (PUT)
Model families served in the windowGemma 4 · Qwen 3.6 · Qwen 3.8
Family that produced the 143Qwen models only · 0 from the Gemma 4 family

Could we reproduce it on demand?

Immediately after we read the post from Cohere, we started a direct probe of 1.000 forced and 1.000 unforced prompts to both a bare Qwen 3.8 27B model and to eustella v1.5. We did not want to answer from our egress-gate log alone, so we ran the behavior directly.

The result is 0 unwanted behavior detected on 2.000 direct probes. A forced set of 1.000 is the size at which the behavior would surface if it were as common as the public threads suggest.

SetPromptsAnswered onUnwanted behavior
Ordinary (unforced)1.000bare and eustella v1.50
Forced (Alibaba-host probes)1.000bare and eustella v1.50

Key findings

Findings.

  • We could not reproduce Cohere's findings on the model we serve. It is very important to note that Cohere found an upload request from the model. We never say our model in the probing nor in logs trying to upload any kind of data.
  • Only the Qwen Family of models showcase this behavior on massive scale, although rarely. 143 invented Alibaba bucket URLs out of 400.000+ egress-logs (0.03%), clustered on the Qwen models, absent from the Gemma 4 family in the same window.
  • It seems more like a retrieval reflex. In those 143 cases our data leans more to the side that the model tried to retrieve information and the invented bucket URL may stem from an RL artifact.
  • The egress gate contained it completely. All calls were successfully refused. This points to the fact that proper guardrails on harness and network layer are a key to stop this kind and similar unwanted behaviour.

What this does not show.

  • It is not a reproduction of Cohere's artifact. Cohere ran Qwen 3.8-Flash-Next, an August preview of its Qwen 4 architecture. We do not run that model. And even though Qwen 3.8 27B and Qwen 3.8 Flash-Next are from the same model generation, their architecture varies a lot.
  • This is not an extensive external measurement. The 400.000-call record is our internal data and audit, not an independent evaluation.
  • It does not fully answer the cause. Even though our data leans more to the side of the model trying to retrieve data, this only shows a correlation but not the actual cause. We will address this in the section of open questions about these findings.

The exact flow of an unwanted fetch

Here we just want to showcase the exact steps of how our gate prevents any unwanted egress:

  1. Model issues a fetch tool call

  2. Target Check: Gate resolves the target. Is it an approved fixed provider? Is it provided directly by the user?

  3. Gate detects that the target unwanted or may not be safe

    flagged
  4. Target fails our security settings

    refused
  5. No connection opened; no information leaves the host. Model receives constructive notice.

    logged

The Open Question: Why does this model behave like this?

There are two readings of Cohere’s findings. In light of our own data and the way reinforcement learning environments work, we think that it is more probable to be a training artifact. The other interpretation could be that this is a deliberate “ping to home” action trained into the model weights.

The training artifact could stem from either pre-training data poisoning or from the RL environments. In both cases it probably appeared because the artifact was not properly scrubbed / cleaned before appearing in the model weights. In case of pre-training poisoning this artifact was not properly scrubbed from the training data. Or the post training process put parts of the RL environment into the weights themselves during post training. Both can and should be avoided. If that is the case, it would explain why this behaviour did not appear in other open-weight models (in our case Gemma 4). Why we think this may be the more probable answer is, because especially RL environments are growing very big to finetune models for specific tasks. In this part of the post training, the model is provided with specific tools in a sandbox, which includes controlled access to external data sources or packets for software etc. This is most often solved via buckets to store approved data sources to be fetched by the model instances for solving the specific tasks of their RL process. If the RL environment is large and the team behind it did not have proper measures in place to prevent RL artifacts from being represented in the weights, they may appear like it does right now in Qwen 3.8 27B.

Another case can be made for a deliberate exfiltration attempt either by a “ping to home” attack, where models ping a proxy or bucket so that the URL owner can trace back model usage to server IPs or even exfiltration attempts via actual uploads of data. We were not able to observe this. Something like this can be added to the weights of an AI model. Neither the weights themselves nor the model reasoning and behaviour can really tell us the difference between a deliberate placement or unwanted appearance of this kind of behaviour.

What would settle it is complete access to the 1. training corpora and 2. to the post training processes. In this case a research team could actually analyze how the model weights were adapted to produce such behaviour.

Overview: Our Security Measures

This is a short overview of our security settings and measures to provide you with insights on how we generally approach network security. We wrote this here because we find it to be necessary to show transparency when tackling such issues.

  • The model has no network at inference time. The model produces text and tool-call requests. It cannot open sockets, hold credentials, or a shell on a networked host during inference. Tools and actions are decided by our orchestrator and harness which we implemented for a separate control plane. The orchestrator than executes the call after appropriate gates.
  • Code execution is fully isolated. Executed code runs in a sandbox with no network access, no host environment
  • Almost every network tool calls a fixed, chosen provider by us. Search, weather, and other general calls all resolve to a trusted destination we chose, not one the model chose. The model can only choose the query itself
  • Tools that do fetch from a non-fixed provider: Here the URL is guarded before the connection is attempted as explained above. Apart from that, standard network security hygiene is in place. HTTPS on port 443 only; no credentials in the URL, no IP literals, no internal hostnames and other SOTA measures in this regard
  • Important actions need human confirmation. Defined CRUD operations need approval by the user, that is especially important for sending or deleting any kind of information.
  • MCP Connectors benefit from additional safeguard opt-in and gated.
    • MCP connectors are off by default.
    • The model cannot add a server, connector, or MCP on its own.
    • Finegrained human control per call: Depending on the mode and the actual tool call, each tool needs separate confirmation.
    • Credentials. Connector tokens and keys are encrypted at rest and scoped.
    • Untrusted output. Connector results and tool descriptions are specially processed to mitigate prompt injections from third party services.

Implications and actions

If you use eustella: Nothing to act on. Even though our security measures already successfully handled this certain behaviours, we again tightened our security and enhanced model control measures.

We did not find a comparable published audit of a specific open-weight deployment's live egress traffic. We only found community sightings and some older related tutorials posted by various AI Agent Communities to enable correct egress security settings.

References

More research

All research →

Aan de slag met eustella

Maak je gratis account aan — eustella werkt overal, op web, iOS en Android.