DizyDiz Homelab · 2026-07-21

RainbowAI2 Rebuild & LLM Fleet Audit

One night: dead server → rebuilt, benchmarked, audited. Charts first; details folded at the bottom.
2.3×
faster 30B inference
16–33 → 58.3 tok/s, same model
3.7×
more work per GPU-watt
~745W → ~360W for the same job
256K
context unlocked
27B multimodal — impossible on 6GB shards
~$1.1k
net cost (est.)
~$1.5k parts − 1660 array resale

The rebuild, one bar each

Same 30B coding model, same test — the retired array vs the new silicon.

1660 array (retired)
≤33
5060 Ti pool · new
58.3
Hommer 7900 XTX
90.5
active fleet (tok/s)retired hardware

Audit: every model, one bar, one verdict

Speed only earns a spot if tool-calls are 3/3 — dropped tool calls are what killed OpenClaw.

Hommer · 24GB (Peepers only)
coder:30b-peepers
90.5✓ KEPT
gpt-oss:20b
122.3✗ 2/3
qwen3.6:27b
6.2✗ slow
RainbowAI · 5060 Ti pool 32GB
coder:30b · default
58.3✓ KEPT
qwen3.6:27b · deep
23.6➕ ADDED
qwen3:14b
43.1✓ KEPT
gpt-oss:20b
86.0✗ 2/3
RTX 3060 · 12GB (chat + vision lane)
qwen3.5:9b · chat
59.8✓ KEPT
gemma4:12b
36.5✗ vision
kept (tok/s)addedrejected / removed

The elimination bracket

Twelve candidates researched → three worth downloading → one earned a slot.

12
researched
July 2026 leaderboards + Ollama library
3
pulled & benchmarked
qwen3.6:27b · gpt-oss:20b · gemma4:12b
1
earned a slot
qwen3.6:27b → the pool's deep model

The VRAM wall — why 9 never got downloaded

Our biggest pool is 32GB. Weight size decides who even gets a tryout; the frontier darlings aren't runnable in this house.

gemma4:12b
32GB pool ceiling
7.6GB
gpt-oss:20b
14GB
qwen3.6:27b
17GB
gemma4:26b MoE
18GB
qwen3.6:35b
24GB
Llama 4 Scout
~65GB
GLM-5.2 · DeepSeek V4 · Kimi K2.6/K3
≫200GB
fits — benchedfits — addedcut without downloading

Small-class paper cuts (fit, but no case): DeepSeek R1 distills & Llama 3.3 8B — qwen3.5:9b already beats their profile at 3/3 tools; Dolphin 3.0 (uncensored) — noted for the guardrails article, no fleet role. Frontier sizes are estimates.

Why gemma4 lost the vision job

The driveway watcher samples every 20 seconds — a model slower than the poll can never catch up.

qwen2.5vl · kept
20s poll budget
1.6s
gemma4:12b
22.9s
qwen3.6:27b (pool)
46s
Details for the weeds — full numbers
ModelHosttok/sTools 3×Strict JSONVisionVerdict
qwen3-coder:30b-peepersHommer90.53/3kept — Peepers primary
gpt-oss:20bHommer122.32/3removed — tool drops
qwen3.6:27bHommer6.23/3removed — ROCm speed
qwen3-coder:30bPool58.33/3kept — default + fallback
qwen3.6:27bPool23.63/346sadded — deep/256K/vision
qwen3:14bPool43.13/3kept — mid batch
gpt-oss:20bPool86.02/3removed
qwen3.5:9b306059.83/3kept — chat
gemma4:12b306036.53/322.9sremoved — vision too slow
qwen2.5vl:7b306075.51.6skept — Farm Watch
nomic-embed-text3060instantkept — reindex cost blocks swap

Method: fleet fully quiesced (farm-watch paused, OpenDiz timer + chat bridge stopped, all models unloaded) per Jess. Harness: warm-speed on a 40-number listing · native tool-calls ×3 trials via /api/chat · strict-JSON extraction · vision on a live farm snapshot. Cost figures are estimates pending receipts.