Union Command v10 · Test guide

Say it in English or Hinglish. Get a function call your agent can run.

Union Command v10 is a small offline model that turns one spoken or typed command into a structured action with typed arguments, for laptops and Raspberry Pi. This page shows what it does, how to download and test it, and how it scored.

How it fits in an agent

It covers 371 actions across 17 categories. Every prediction includes a confidence score, the top-3 alternatives and schema validation errors, so an agent can execute, ask the user (clarify) or hand the text to a larger LLM (unknown or low confidence).

{"action": "set_timer", "args": {"amount": 5, "unit": "min"}, "confidence": 0.9998,
 "alternatives": [...], "validation_errors": [], "latency_ms": 6.1}

Uses, with examples in ENGLISH and HINGLISH

Categories, sample commands, outputs and executor tools

One intent per category: all ENGLISH commands first, then the same commands in HINGLISH. Model output is the real v10 prediction. Executor tool is what the sample executor would run on each platform; the bracket shows whether that adapter was ready on the test machine. Result appears only for read-only commands that were actually executed.

Test results

Metric: exact match. The action and every typed argument must equal the label; errors stay in the denominator. Same checkpoint on both devices (SHA-256 c91d44e4a687152c…).

Accuracy

The held-out test split contains whole template families never used for training and was evaluated once for this release. Earlier development results on the same checkpoint: Grouped dev (reserved template groups) 96.50%, Validation 98.22%.

By language (held-out test)

By category (held-out test)

Speed

Temperature

Raspberry Pi 5 notes. The first attempt ran the held-out test in one go with 4 threads. After 202 s and 15,040 commands the SoC reached about 85 °C with the fan near 9,900 RPM, and the Pi dropped off the network until it was power-cycled. The reported Pi results come from a second, gentler run: 2 threads, the held-out test split into five chunks of 5,000 rows, and a cool-down below 65 °C before each chunk. Temperature, the 5 V rail, throttle flags and fan speed were logged every 5 seconds (test_results/rpi5/sensors.log). That run finished with throttle flags at 0x0 throughout, a peak of 73.2 °C and a 5 V rail between 5.05 V and 5.20 V. The Pi made exactly the same predictions as the laptop: the same 1,314 commands were wrong on both devices across the three sets. On the Pi, 1 thread gives the lowest single-command latency; extra threads mainly help batched throughput. For long jobs on a Pi 5, use 1 or 2 threads, keep the active cooler and a 5 V / 5 A supply, and watch `vcgencmd measure_temp` and `vcgencmd get_throttled`.

Test devices

Download and test

git clone https://github.com/sraivante/tiny-agentic-home-robotic-for-edge-device-v10.git
cd tiny-agentic-home-robotic-for-edge-device-v10
python download_model.py

python -m venv .venv
# Windows: .venv\Scripts\activate
# Pi/Linux: . .venv/bin/activate
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt

python quickstart.py
python quickstart.py "turn off the wifi" "wifi band karo"
python quickstart.py --json "set a timer for 5 minutes"

python scripts/evaluate.py --data examples/sample_commands.jsonl
python scripts/benchmark.py --threads 1,2,4

run.bat          # Windows
bash run.sh      # Linux / Raspberry Pi
# opens http://127.0.0.1:8770

from tinyagent.runtime import Predictor
m = Predictor("models/a100_minilm_v10_quoted/best.pt", threads=2)
print(m.predict("set the volume to 40"))
print(m.predict("volume 40 kar do"))

Limitations

• Synthetic training data with no real speech-to-text output; test with your own ASR transcripts.
• Small Devanagari coverage; English and Roman-script Hinglish dominate.
• One command per utterance; it parses, it does not plan.
• Weakest spots: near-duplicate GPIO/I2C actions, a few near-synonyms and some free-text argument boundaries.
• The model never refuses; always keep a validator and risk gate in front of an executor.