PicoLM is an LLM inference engine written in C99. It currently supports llama-2, GPT-2, Qwen 3.6/3.8(+MoE) and Gemma-3n models. Significant amount of work went into CPU SIMD acceleration/testing/correctness, and wide cross-platform availability with constant testing to never lose portability (from DOS through OS/X 10.4 to modernity). CUDA/HIP is supported, and accelerated IMMA kernels are available. More work needs to be done on prompt processing speed, but text generation is quite fast already. GGUFs are mmap()'ed, not preloaded, so it's much more friendly to RAM usage than llama.cpp. Extern…

#hackernews #showhn #projects #llama #qwen #picolm #cpp #gemma3n