When to Use Structured Output (JSON Mode) in LLMs
JSON mode forces models to output valid JSON. When it's a lifesaver vs. when it's overkill and makes the model worse.
Tag: ai
All the articles with the tag "ai".
Using AI to Find Security Bugs in Your Code
Claude Code found a Linux vulnerability hidden for 23 years. You can use the same AI code auditing approach to find bugs in your own projects before attackers do.
LLM Temperature and top_p Explained Without the Math
Temperature and top_p control randomness in LLMs. No probability theory needed. Just practical intuition and how to tune them.
LLM Backends: vLLM vs llama.cpp vs Ollama
vLLM, llama.cpp, and Ollama all run local LLMs — compare throughput, memory use, GPU support, and which fits your hardware.
RAG Chunking: Why Chunk Size Is Everything
RAG breaks documents into chunks. But what chunk size? Too small and context is lost. Too large and semantic search fails. Here's how to pick.
LiteLLM & vLLM: One API to Rule All Your Models
Stop juggling 17 different LLM SDKs. LiteLLM and vLLM give you a unified OpenAI-compatible API for every model — local or cloud, fast and production-ready.
System Prompts: The LLM Feature Most People Ignore
System prompts are your secret weapon. How they work, why they matter more than you think, and 5 patterns that actually change model behavior.
LLM Quantization: Q4_K_M Isn't Always the Best Choice
Q4_K_M is the default, but it's not magic. When Q3, Q5, or Q6 makes sense. How to benchmark quantization tradeoffs on your hardware.
Running Multiple Ollama Models Without Running Out of RAM
Ollama can load one model at a time on limited hardware. How to switch between models, use CPU offloading, and manage VRAM intelligently.
Piper vs Coqui: Text-to-Speech on Your Own Hardware (Because AWS Polly Charges Per Character Like It's 1999 SMS)
Run local TTS with Piper or Coqui on Linux, Docker, or Home Assistant. Fast, private, offline text-to-speech — no cloud fees, no data leaks, no surprises.
Context Window vs Token Limit: Not the Same Thing
What's the actual difference between context window and token limit? Why one model says 8K and another says 128K. A practical breakdown.
Ollama Memory Management: Why Models Keep Loading
Why your GPU fills up with Ollama. How to inspect VRAM, tune keep-alive, force-unload models with a single request, and stop the reload pain in 2026.