VKIE β VIDRAFT Kernel Inference Engine
π
1
VKIE (λΉν€) β VKAE accelerates, VKUE saves, VKIE maximizes
π In a Training Loop
VIDRAFT_LAB
SeaWolf-AI
AI & ML interests
Contact: arxivgpt@gmail.com
Recent Activity
liked a Space about 4 hours ago
FINAL-Bench/VKIE updated a Space about 5 hours ago
FINAL-Bench/VKIE published a Space about 5 hours ago
FINAL-Bench/VKIEOrganizations
liked a
Space about 4 hours ago
updated a
Space about 5 hours ago
published a
Space about 5 hours ago
updated a
model about 6 hours ago
updated a
Space about 6 hours ago
replied to their post about 6 hours ago
The 35B model currently released for testing is being served through Hugging Face Spaces in an on-premise-style, GPU-free CPU-only environment.
The free CPU version provided by Hugging Face is running on a virtualized 2 vCPU + 16GB RAM instance, delivering approximately 5.5 to 6 tokens per second.
replied to their post about 6 hours ago
The 35B model currently released for testing is being served on Hugging Face Spaces in a fully on-premise-style environment with no GPU β CPU only.
It is running on an upgraded CPU instance with 8 virtual CPUs and 32GB of RAM, delivering approximately 20 tokens per second.
reacted to their post with π€βππβ€οΈ 2 days ago
Post
5092
π΅ VKUE β No GPU? Runs anyway.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound β sweep 34B params/token and an 8 GB card dies at 1β2 tok/s.
So we ran ONE 34.7B reasoning model β Ourbox-35B-JGOS, a sparse Mixture-of-Experts β as the identical weights across the whole hardware spectrum. All measured:
β’ B200: 18,057 tok/s (aggregate)
β’ 1Γ A10G: 126 tok/s
β’ 8 GB laptop (RTX 5060): 20 tok/s
β’ GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB β ~11Γ less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop β or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 β 3.7Γ from sparsity alone, ~2Γ the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
π Article: https://huggingface.co/blog/FINAL-Bench/vkue
π΅ GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
π΅ CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
π VKUE leaderboard: FINAL-Bench/VKUE
π€ Model: FINAL-Bench/Ourbox-35B-JGOS-GGUF
β‘ VKAE (speed): VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound β sweep 34B params/token and an 8 GB card dies at 1β2 tok/s.
So we ran ONE 34.7B reasoning model β Ourbox-35B-JGOS, a sparse Mixture-of-Experts β as the identical weights across the whole hardware spectrum. All measured:
β’ B200: 18,057 tok/s (aggregate)
β’ 1Γ A10G: 126 tok/s
β’ 8 GB laptop (RTX 5060): 20 tok/s
β’ GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB β ~11Γ less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop β or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 β 3.7Γ from sparsity alone, ~2Γ the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
π Article: https://huggingface.co/blog/FINAL-Bench/vkue
π΅ GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
π΅ CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
π VKUE leaderboard: FINAL-Bench/VKUE
π€ Model: FINAL-Bench/Ourbox-35B-JGOS-GGUF
β‘ VKAE (speed): VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere.
- 4 replies
upvoted an article 2 days ago
Article
VKUE: No GPU? Runs Anyway β a 34.7B Reasoner on a Laptop and on Bare CPU
FINAL-Bench
β’ β’ 15 reacted to their post with π₯ 2 days ago
Post
5092
π΅ VKUE β No GPU? Runs anyway.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound β sweep 34B params/token and an 8 GB card dies at 1β2 tok/s.
So we ran ONE 34.7B reasoning model β Ourbox-35B-JGOS, a sparse Mixture-of-Experts β as the identical weights across the whole hardware spectrum. All measured:
β’ B200: 18,057 tok/s (aggregate)
β’ 1Γ A10G: 126 tok/s
β’ 8 GB laptop (RTX 5060): 20 tok/s
β’ GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB β ~11Γ less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop β or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 β 3.7Γ from sparsity alone, ~2Γ the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
π Article: https://huggingface.co/blog/FINAL-Bench/vkue
π΅ GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
π΅ CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
π VKUE leaderboard: FINAL-Bench/VKUE
π€ Model: FINAL-Bench/Ourbox-35B-JGOS-GGUF
β‘ VKAE (speed): VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound β sweep 34B params/token and an 8 GB card dies at 1β2 tok/s.
So we ran ONE 34.7B reasoning model β Ourbox-35B-JGOS, a sparse Mixture-of-Experts β as the identical weights across the whole hardware spectrum. All measured:
β’ B200: 18,057 tok/s (aggregate)
β’ 1Γ A10G: 126 tok/s
β’ 8 GB laptop (RTX 5060): 20 tok/s
β’ GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB β ~11Γ less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop β or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 β 3.7Γ from sparsity alone, ~2Γ the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
π Article: https://huggingface.co/blog/FINAL-Bench/vkue
π΅ GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
π΅ CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
π VKUE leaderboard: FINAL-Bench/VKUE
π€ Model: FINAL-Bench/Ourbox-35B-JGOS-GGUF
β‘ VKAE (speed): VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere.
- 4 replies
posted an update 2 days ago
Post
5092
π΅ VKUE β No GPU? Runs anyway.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound β sweep 34B params/token and an 8 GB card dies at 1β2 tok/s.
So we ran ONE 34.7B reasoning model β Ourbox-35B-JGOS, a sparse Mixture-of-Experts β as the identical weights across the whole hardware spectrum. All measured:
β’ B200: 18,057 tok/s (aggregate)
β’ 1Γ A10G: 126 tok/s
β’ 8 GB laptop (RTX 5060): 20 tok/s
β’ GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB β ~11Γ less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop β or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 β 3.7Γ from sparsity alone, ~2Γ the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
π Article: https://huggingface.co/blog/FINAL-Bench/vkue
π΅ GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
π΅ CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
π VKUE leaderboard: FINAL-Bench/VKUE
π€ Model: FINAL-Bench/Ourbox-35B-JGOS-GGUF
β‘ VKAE (speed): VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound β sweep 34B params/token and an 8 GB card dies at 1β2 tok/s.
So we ran ONE 34.7B reasoning model β Ourbox-35B-JGOS, a sparse Mixture-of-Experts β as the identical weights across the whole hardware spectrum. All measured:
β’ B200: 18,057 tok/s (aggregate)
β’ 1Γ A10G: 126 tok/s
β’ 8 GB laptop (RTX 5060): 20 tok/s
β’ GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB β ~11Γ less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop β or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 β 3.7Γ from sparsity alone, ~2Γ the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
π Article: https://huggingface.co/blog/FINAL-Bench/vkue
π΅ GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
π΅ CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
π VKUE leaderboard: FINAL-Bench/VKUE
π€ Model: FINAL-Bench/Ourbox-35B-JGOS-GGUF
β‘ VKAE (speed): VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere.
- 4 replies
liked a
model 2 days ago
published an article 2 days ago
Article
VKUE: No GPU? Runs Anyway β a 34.7B Reasoner on a Laptop and on Bare CPU
FINAL-Bench
β’ β’ 15 liked a
Space 2 days ago
published a
model 2 days ago
published a
Space 2 days ago