Skip to content

Category

ai

Everything we have published on ai, newest first.

Quantization Explained: What Q4, Q8, and FP16 Actually Do to a Local Model
Article · aiDeep read

Quantization Explained: What Q4, Q8, and FP16 Actually Do to a Local Model

Quantization shrinks AI model weights from 16-bit floats to 4- or 8-bit. Here is exactly what that trade-off costs you, and how to pick between Q4_K_M, Q8, and FP16 for the model you actually want to run, with an interactive memory calculator.

BitByteCore Silicon Desk · Aug 6, 2026 · 8 min read

Fresh
Mixture-of-Experts Models: How They Work and Why They Cut Inference Costs
Article · aiDeep read

Mixture-of-Experts Models: How They Work and Why They Cut Inference Costs

MoE models activate only a fraction of their parameters per token: DeepSeek-V3 fires 37B of 671B, delivering large-model quality at small-model compute cost. How routers, experts, and load balancing actually work, and where the savings are and are not real. With an interactive dense-vs-MoE view.

BitByteCore Silicon Desk · Aug 6, 2026 · 10 min read

The Real Difference Between MCP, Function Calling, and Agent Loops
Article · aiDeep read

The Real Difference Between MCP, Function Calling, and Agent Loops

They're not competitors — they're three layers of one system. Function calling is what the model emits, MCP is how tools reach it, and the agent loop is what keeps it running. Confuse them and you'll spend an afternoon debugging the wrong layer.

BitByteCore AI Desk · Aug 5, 2026 · 3 min read

How Much VRAM You Actually Need to Run a Local LLM
Article · aiDeep read

How Much VRAM You Actually Need to Run a Local LLM

VRAM is the hard constraint on running a local LLM. Here's the real math — parameters, precision, quantization, KV cache — what fits on 8GB, 24GB, 32GB, and unified-memory machines, plus where quality and speed actually break.

BitByteCore AI Desk · Aug 5, 2026 · 7 min read

How to Evaluate a Local LLM for a Real Task: A Repeatable Testing Framework
Article · aiDeep read

How to Evaluate a Local LLM for a Real Task: A Repeatable Testing Framework

A leaderboard score is a population statistic, measured on a full-precision model you'll never run. Whether a local model can handle your invoice pipeline is a different question — here's the repeatable discipline for answering it on your own hardware.

BitByteCore AI Desk · Aug 5, 2026 · 11 min read

More stories