2 Comments
User's avatar
Popular AI's avatar

For CPU-only local AI, what matters more to you: faster replies from a smaller model, or better answers from a slower one?

elewon's avatar
1dEdited

I’ve been running Qwen3.5 4B on an old laptop with 16GB RAM and honestly it’s been way better than I expected for just chatting and summarizing stuff. Grabbed the extra RAM stick pretty cheap from etech devices, which helped a lot before I even tried bigger models. Tried bumping up to 8B once out of curiosity and yeah, the wait kinda kills the vibe lol. Sticking with 4K context like you said made a huge difference too.