explore

Pick a thread.

Every story we have published, by topic. Tap a tile to filter — no reloads, just the thread you want to pull.

ai

27 stories

Quantization Explained: What Q4, Q8, and FP16 Actually Do to a Local Model

Quantization shrinks AI model weights from 16-bit floats to 4- or 8-bit. Here is exactly what that trade-off costs you, and how to pick between Q4_K_M, Q8, and FP16 for the model you actually want to run, with an interactive memory calculator.

BitByteCore Silicon Desk · Aug 6, 2026 · 8 min read

Fresh

Article · aiDeep read

Mixture-of-Experts Models: How They Work and Why They Cut Inference Costs

MoE models activate only a fraction of their parameters per token: DeepSeek-V3 fires 37B of 671B, delivering large-model quality at small-model compute cost. How routers, experts, and load balancing actually work, and where the savings are and are not real. With an interactive dense-vs-MoE view.

BitByteCore Silicon Desk · Aug 6, 2026 · 10 min read

Article · aiDeep read

The Real Difference Between MCP, Function Calling, and Agent Loops

They're not competitors — they're three layers of one system. Function calling is what the model emits, MCP is how tools reach it, and the agent loop is what keeps it running. Confuse them and you'll spend an afternoon debugging the wrong layer.

BitByteCore AI Desk · Aug 5, 2026 · 3 min read

Article · aiDeep read

How Much VRAM You Actually Need to Run a Local LLM

VRAM is the hard constraint on running a local LLM. Here's the real math — parameters, precision, quantization, KV cache — what fits on 8GB, 24GB, 32GB, and unified-memory machines, plus where quality and speed actually break.

BitByteCore AI Desk · Aug 5, 2026 · 7 min read

Article · aiDeep read

How to Evaluate a Local LLM for a Real Task: A Repeatable Testing Framework

A leaderboard score is a population statistic, measured on a full-precision model you'll never run. Whether a local model can handle your invoice pipeline is a different question — here's the repeatable discipline for answering it on your own hardware.

BitByteCore AI Desk · Aug 5, 2026 · 11 min read

Pick a thread.

ai

Quantization Explained: What Q4, Q8, and FP16 Actually Do to a Local Model

Mixture-of-Experts Models: How They Work and Why They Cut Inference Costs

The Real Difference Between MCP, Function Calling, and Agent Loops

How Much VRAM You Actually Need to Run a Local LLM

How to Evaluate a Local LLM for a Real Task: A Repeatable Testing Framework

More stories

Browse by tag