BotPsy-0.8B: a 0.8-billion-parameter agent model that runs entirely on-device — and calls tools better than a model 2.5 times its size

The real problem: cloud agents cost money and leak data

A normal morning: your agent reads your calendar, checks disk space on your homelab, and reminds you about a dentist appointment at 3 p.m. — each small chore is another API call to the cloud, billed per token. By month’s end, the bill comes from hundreds of trivial tasks that never needed a large model.

Most daily agent work is not deep reasoning — it is deciding which tool to call, with which parameters, then reading the result. On 11/11/2026, Tether AI Research announced on the Hugging Face blog a model of just 0.8 billion parameters that runs entirely on-device yet calls tools better than a model from the same family 2.5 times its size: BotPsy-0.8B.

What is BotPsy?

BotPsy-0.8B is a language model specialized in function calling and agentic workflows, developed by Tether AI Research and post-trained from Qwen3.5-0.8B. It handles text only, under the Apache 2.0 license. The weights are already available: qvac/BotPsy-0.8B (BF16, 1.52 GB) and qvac/BotPsy-0.8B-GGUFs (BF16 / Q8_0 0.81 GB / Q4_K_M 0.53 GB) for llama.cpp and the QVAC SDK. The model runs on stock Transformers, vLLM, and llama.cpp — no custom code required.

The clearest audience is self-hosters and homelab users, developers building agents, coding-agent users who want an agent that runs locally, and IT managers who care about cost and data privacy. If your data must not leave your machine, or you want the marginal cost of each tool call to drop to near zero, BotPsy is worth trying. The developer states plainly, though, that BotPsy is not for high-stakes tasks or decisions involving money, health, or sensitive personal information.

How BotPsy works: architecture and mechanism

A four-stage training pipeline

The post-training pipeline has four stages: SFT on tool-calling data; multi-teacher on-policy distillation, in which three Qwen3.5-4B teachers train the 0.8B student; then two rounds of RL using the DAPO algorithm. The training code and data were not released. RL taught the model a decision discipline: stop reasoning at the right moment and call a tool, instead of reasoning in circles. Average reasoning tokens on GPQA Diamond fell about 3 times (11,563 down to 3,690); the reasoning-loop rate dropped from 22.9% to 1.4%. In my view, this is the key insight: the gain comes from the model knowing when to stop and act — distillation transferred tool-calling skill, not knowledge.

The reasoning and tool-calling flow

The model reasons in thinking mode, then decides whether it needs to call a tool. If it does, it produces a Qwen3.5 XML tool call, the tool executes on the machine, the result returns to the model for further reasoning; if no suitable tool exists, the model refuses instead of inventing a tool call. Two settings determine quality: thinking mode must be on (enable_thinking=True) — the default template is non-thinking, and forgetting to enable it leaves the model running without any error but with much worse answers; and sampling parameters temperature 0.3, top_p 0.95, top_k 20, presence_penalty 1.5. The server must use the right parsers — qwen3_xml for tool calls, qwen3 for reasoning on vLLM. When no suitable tool exists, BotPsy refuses in 86.1% of cases (BFCL v4), the highest in the tested group.

Architecture diagram

flowchart TD
    A[User Request<br/>e.g. check disk space] --> B[BotPsy Thinking<br/>enable_thinking=True]
    B --> C{Decision:<br/>Need to call a tool?}
    C -- Yes --> D[Tool Call<br/>Qwen3.5 XML<br/>parser qwen3_xml]
    D --> E[Tool Execution local<br/>read-only / sandbox<br/>data never leaves the machine]
    E --> F[Tool Result]
    F --> B
    C -- No --> G[Final Answer<br/>Reply to the user]
    C -- No suitable tool --> H[Refuse / Ask Clarification<br/>hallucination refusal 86.1%]
    H --> G

Dependencies and technical limits

BotPsy needs Transformers with Qwen3.5 support (tested with 5.19.0); you can optionally install flash-linear-attention and causal-conv1d. The model prioritizes English; no figures exist for Vietnamese — test it yourself before trusting it. The roadmap is a plan, not a shipped feature.

Performance, according to the developer’s numbers

All benchmarks below were run by Tether itself (evaluated 03/11/2026, thinking mode) and have not been independently verified — test it with your own tools before trusting them.

On agentic benchmarks, BotPsy leads in 5 of 6 tests, including against a model 2.5 times larger: BFCL v4 44.0 (base 23.4; 2B-Think 37.7); ACEBench 58.7 (30.1; 55.4); τ-bench 51.5 (23.2; 46.5); ToolSandbox 64.6 (30.5; 41.2); average 45.0 (24.7; 38.8). The only loss: Claw-Eval 46.5 versus 48.9. On multi-turn tests, τ²-bench telecom reached 82.5 versus 69.3; multi-turn BFCL v4 rose from 7.5 to 46.1.

General knowledge still follows scale: GPQA Diamond 38.1 versus 59.9 for the 2B model, MMLU-Pro 48.9 versus 65.5. Instruction following improved sharply: IFEval 90.8 (base 59.7), IFBench 55.9 (base 22.0) — the highest in the tested group. Weaknesses the developer admits itself: BrowseComp-Plus only 4.5; BFCL web search 14.0, memory 16.3; τ³-banking no model passed 5. On safety, the vendor’s own red-teaming found an unsafe rate of 21% and prompt injection of 31% — improved over the base, but still high.

How to deploy BotPsy on your machine

Prerequisites

You need one of three things: Python with Transformers supporting Qwen3.5 (tested with 5.19.0), vLLM, or llama.cpp with a GGUF file. Sizes: BF16 about 1.52 GB, Q8_0 about 0.81 GB, Q4_K_M about 0.53 GB — the Q4_K_M build runs on a laptop CPU via llama.cpp (inferred from the file size, not tested).

Method 1 — vLLM server, OpenAI-compatible

This is the best option if you are building an agent, because the server is OpenAI API-compatible:

vllm serve qvac/BotPsy-0.8B --language-model-only \
    --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3

Each flag: --language-model-only serves only the language model; --enable-auto-tool-choice lets the server decide when to call a tool; --tool-call-parser qwen3_xml parses tool calls in Qwen3.5 XML format; --reasoning-parser qwen3 separates the reasoning part from the reply content.

Method 2 — llama.cpp with GGUF

This is the best option for a laptop without a strong GPU. The GGUF repo recommends the Q8_0 build (average score 50.7, almost the same as BF16 at 50.4).

hf download qvac/BotPsy-0.8B-GGUFs botpsy-0.8b-q8_0.gguf --local-dir .
llama-server -m botpsy-0.8b-q8_0.gguf --jinja -c 32768 \
    --temp 0.3 --top-p 0.95 --top-k 20 --presence-penalty 1.5 \
    --chat-template-kwargs '{"enable_thinking": true}' --port 8080

Test the server with curl

One curl command works for both servers (llama-server on port 8080, vLLM on port 8000):

curl -s http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "Add the dentist on Friday at 3 pm."}],
  "tools": [{"type": "function", "function": {
    "name": "create_event",
    "parameters": {"type": "object", "properties": {
      "title": {"type": "string"}, "day": {"type": "string"},
      "time": {"type": "string"}},
    "required": ["title", "day", "time"]}}}],
  "max_tokens": 8192
}'

Expected result: reasoning_content contains the reasoning, tool_calls contains the create_event function (title Dentist, day Friday, time 15:00). If tool_calls is empty, recheck the parser and enable_thinking.

Method 3 — Transformers (Python)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "qvac/BotPsy-0.8B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
device = "cuda" if torch.cuda.is_available() else "cpu"
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).to(device)

Remember to enable enable_thinking=True in apply_chat_template and use sampling temperature 0.3, top_p 0.95, top_k 20, presence_penalty 1.5.

Common errors and fixes

Forgetting to enable thinking mode is the most common error: the model runs much worse without reporting an error — recheck enable_thinking=True everywhere the model is called. The second error is the wrong tool-call parser on vLLM: the agent “talks about” a tool instead of calling it — recheck --tool-call-parser qwen3_xml and --reasoning-parser qwen3. An old Transformers version will error when loading the model — upgrade to 5.19.0.

Backing up and rolling back

Rolling back to your old model is just changing the model id in the config — back to Qwen3.5-0.8B-Think, Qwen3.5-2B-Think, or back to a cloud API — no agent code changes are needed because BotPsy runs behind an OpenAI-compatible API.

Technical assessment

BotPsy calls tools better than a Qwen model 2.5 times larger in the same family on BFCL v4 (44.0 versus 37.7). It runs fully on-device: data never leaves the machine, there is no API dependency, and operating costs are near zero after downloading the weights. Apache 2.0 and a standard stack, with no runtime lock-in; about 3 times fewer reasoning tokens.

On the other hand, general knowledge is far weaker than the 2B model (GPQA 38.1 versus 59.9; MMLU-Pro 48.9 versus 65.5). The model is weak at web search, long-term memory, and banking tasks, and prioritizes English. The benchmarks come from the vendor only; the training code and data were not released, so the process cannot be audited. In the local spectrum, BotPsy is the smallest end: Colibri runs a large MoE model on a personal machine in exchange for capability, while BotPsy trades capability for size.

The real security strength is that data never leaves the machine when running locally. But the real weakness must also be stated plainly: an unsafe rate of 21% and prompt injection of 31% according to the vendor’s own red-teaming is still high, because a small model is easily prompt-injected through content returned by tools. Operate with least-privilege tools: start with read-only tools, do not grant write, delete, or money-transfer tools until the model’s behavior is under control, sandbox tools, log every tool call, and never put secrets in prompts.

On performance and scaling, use only the figures available: weights of 0.53–1.52 GB, reasoning tokens down about 3 times, loop rate 1.4%. This article gives no throughput or latency figures, because there are no independent numbers and I have not run it myself.

Comparing BotPsy with the alternatives

The table uses exactly the vendor-published figures, in thinking mode. BotPsy wins at tool calling per parameter and loses at general knowledge. Muse Ember is a small model for agents but runs through a cloud API — the same philosophy, except it still depends on an API. Jev cuts AI costs by routing; BotPsy is the extreme answer: move the task entirely to the local machine.

Real use cases

Homelab and self-hosting. A local agent controls the homelab — checking container status, reading logs, creating a ticket when a service fails — and infrastructure data never leaves the home network. Combined with the Orkas article on the local-first ecosystem, BotPsy can serve as the tool-calling brain of a desktop agent system.

Developers building agents and local coding agents. Use BotPsy as the tool-calling brain inside your own agent framework, via the OpenAI-compatible vLLM API. Routine tasks run locally at zero cost; hard cases escalate to a larger model.

Businesses and IT managers. An internal agent handles sensitive data without sending it outside, with near-zero marginal cost — provided tools are limited to low privileges, fully logged, and not used for high-stakes decisions. BotPsy does not replace large models — it replaces the API calls that were never necessary.

Conclusion

Use BotPsy if you need a local, private, cheap agent for routine tasks of low to medium difficulty, or if you are a developer who wants a small tool-calling model to embed in your own system. Not yet if you need deep knowledge, open web search, financial or medical tasks, or independently verified figures before deciding.

Next step: download the Q4_K_M 0.53 GB build and try it with llama.cpp, or set up a vLLM server with the four flags above, then test it with a read-only tool. BotPsy is not a good all-round model — it is proof that tool-calling skill can be compressed into 0.8 billion parameters.

Lessons learned

1. Tool-calling skill is separate from knowledge: BotPsy beats the 2B model at tool calling but trails far behind on GPQA and MMLU-Pro.
2. “Thinking mode must be on” is a deployment requirement, not an option: the default template is non-thinking.
3. The parser is part of the model: Qwen3.5 XML demands the exact --tool-call-parser qwen3_xml.
4. Vendor numbers are not verified truth — test it yourself.
5. On-device safety is not absolute safety: unsafe 21% and prompt injection 31%.
6. Small models enable a cascade architecture: BotPsy locally for routine work.

Verification scope

Checked on 11/11/2026, based on the Tether AI Research blog on Hugging Face, the qvac/BotPsy-0.8B model card, and the qvac/BotPsy-0.8B-GGUFs repo. Verified against official documentation: model specs, license, training pipeline, vLLM command, Transformers 5.19.0 requirement, tool-call format, and sampling parameters. Thinking mode is off by default and must be enabled with enable_thinking=True. File sizes are release specs, not performance benchmarks. Benchmarks and safety figures are the developer Tether’s own and have not been independently verified. This article did not install or run BotPsy. The llama.cpp and curl test commands were cross-checked against the GGUF repo card at the time of writing.

References

• Tether AI Research — BotPsy: On-Device Personal Assistant with a Sub-Billion-Parameter Agentic Language Model: https://huggingface.co/blog/qvac/botpsy
• Model card qvac/BotPsy-0.8B: https://huggingface.co/qvac/BotPsy-0.8B
• GGUF weights qvac/BotPsy-0.8B-GGUFs: https://huggingface.co/qvac/BotPsy-0.8B-GGUFs