ginipick/Qwen-Image-Edit-Rapid-AIO
Text-to-Image โข Updated โข 1
ginipick
ginipick
AI & ML interests
None yet
Recent Activity
reacted to SeaWolf-AI's post with ๐ฅ 2 days ago
๐ต VKUE โ No GPU? Runs anyway.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound โ sweep 34B params/token and an 8 GB card dies at 1โ2 tok/s.
So we ran ONE 34.7B reasoning model โ Ourbox-35B-JGOS, a sparse Mixture-of-Experts โ as the identical weights across the whole hardware spectrum. All measured:
โข B200: 18,057 tok/s (aggregate)
โข 1ร A10G: 126 tok/s
โข 8 GB laptop (RTX 5060): 20 tok/s
โข GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB โ ~11ร less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop โ or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 โ 3.7ร from sparsity alone, ~2ร the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
๐ Article: https://huggingface.co/blog/FINAL-Bench/vkue
๐ต GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
๐ต CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
๐ VKUE leaderboard: https://huggingface.co/spaces/FINAL-Bench/VKUE
๐ค Model: https://huggingface.co/FINAL-Bench/Ourbox-35B-JGOS-GGUF
โก VKAE (speed): https://huggingface.co/spaces/VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere. updated a Space 4 days ago
ginigen/AI liked a model 12 days ago
FINAL-Bench/Darwin-36B-Opus-VKAEOrganizations
ginipick/GLM-4.6
Text Generation โข 357B โข Updated โข 3
ginipick/neutts-air
Text-to-Speech โข 0.7B โข Updated โข 14 โข 1
ginipick/MiniMax-M2
Text Generation โข 229B โข Updated โข 5
ginipick/PaddleOCR-VL
Image-Text-to-Text โข 1.0B โข Updated โข 6
ginipick/DeepSeek-OCR
Image-Text-to-Text โข 3B โข Updated โข 4
ginipick/Gemma-3-R1984-4B
Image-Text-to-Text โข 4B โข Updated โข 5 โข โข 52
ginipick/QwQ-32B-NF4
Text Generation โข 33B โข Updated โข 2 โข 30
ginipick/wan-lora-cat
Text-to-Video โข Updated โข 1
ginipick/c-bag
Updated
ginipick/flux-lora-eric-cat
Text-to-Image โข Updated โข 26 โข โข 230