Tag
#local-llm
Every story tagged local-llm, newest first.
Quantization Explained: What Q4, Q8, and FP16 Actually Do to a Local Model
Quantization shrinks AI model weights from 16-bit floats to 4- or 8-bit. Here is exactly what that trade-off costs you, and how to pick between Q4_K_M, Q8, and FP16 for the model you actually want to run, with an interactive memory calculator.
BitByteCore Silicon Desk · Aug 6, 2026 · 8 min read
How to Evaluate a Local LLM for a Real Task: A Repeatable Testing Framework
A leaderboard score is a population statistic, measured on a full-precision model you'll never run. Whether a local model can handle your invoice pipeline is a different question — here's the repeatable discipline for answering it on your own hardware.
BitByteCore AI Desk · Aug 5, 2026 · 11 min read
The best smartphones for on-device AI in 2026
For most people in mid-2026, the best on-device AI phone is the Samsung Galaxy S26 Ultra — it runs the widest range of AI tasks locally, no server round-trip, at a price you can justify. Here's how it compares to the iPhone 17 Pro Max, Pixel 10, OnePlus 13 and the ROG Phone 10 Pro.
Silicon Desk · Aug 3, 2026 · 11 min read
How to choose the right quantization for a local LLM
Decode the Q4, Q5, and Q8 labels on model files, understand what bits-per-weight actually costs you, and pick a quantization that fits your RAM without wrecking quality.
Signal Desk · May 24, 2026 · 4 min read