local inference

AI ExLlamaV3 makes local AI infrastructure more practical for founders

5 min 1.2K views

AI A Single RTX 5000 PRO Is Running Qwen3 27B at 200k Context and 80 Tokens Per Second and That Number Should Change How Founders Think About Local Inference Economics

6 min 614 views

AI APEX MoE Quants Just Added 25 Models and an I-Nano Tier That Runs Frontier-Class MoE on 8GB of VRAM and the Community Reception Tells You Something Useful

7 min 350 views

AI Google's Gemma 4 just outscored ChatGPT and Gemini Chat and you can run it yourself

4 min 253 views

AI llama.cpp is becoming the Linux of large language models and the cloud AI giants should be paying attention

4 min 221 views

AI The open-source AI ecosystem keeps treating llama.cpp like a second-class citizen and developers are tired of it

4 min 223 views

AI llama.cpp merges speculative checkpointing and local AI inference takes a significant leap forward

4 min 1.6K views