BigMoeOnEdge outperforms Strata and colibri on 8-16 GB cards and Mac and you can run any model
BigMoeOnEdge, a tool for running mixture-of-experts models, works with any MoE GGUF that llama.cpp can load on 8-16 GB cards and Mac, handling NVIDIA, AMD, Intel, and CPU inference.
toolsopen source reddit.com ·