27.4B agent decision model now runs local from 17.67 GB instead of being trapped on giant servers.

ggml-org/OpenJev-GGUF
https://huggingface.co/ggml-org/OpenJev-GGUF

#AI #LocalLLM #OpenSource #GGUF #Quantization

yzma 1.29 is out! Run high performance local AI models from @golang.org without CGo. Support for the new @hf.co llama.cpp v0.6.0, typed decisions w/ System One models, auto-threading, new extended batch API, & much more!

Go get it right now!

yzma.ai #golang #llamacpp #gguf #tinygo #localAI

#selfhosting #batchinference #gpuoptimization #gguf #quantization #llm 2/2

You can now run a 117B open-weight reasoning model on a single H100: 5.1B active params, 80GB, Apache-2.0.

unsloth/gpt-oss-120b
https://huggingface.co/unsloth/gpt-oss-120b

#AI #LocalLLM #OpenSource #GGUF #Quantization

LLM Quantization คืออะไร

อ่านต่อ : www.blockdit.com/posts/6ab8af...

#ShoperGamer #LLMQuantization #LLM #Ai #ModelAi #FP32 #FP16 #GGUF #GPTQ #Knowledge #Study #Feed

LLM Systems: Operational Reference for Self-Hosted Inference

#AICoding #LLM #AI #Dev #OpenSource #SelfHosted #llama.cpp #GGUF

https://llmsystems.dev/

If you see IQ, it stands for 'importance-aware' quantization, which generally performs better than standard K-quants when you're forced to go to very low bit rates like 3-bit. #selfhosting #batchinference #llm #quantization #gguf #localai 2/2

100% local chat with a GGUF model in your browser using WebAssembly.

Written in @golang.org compiled with @tinygo.org. Runs in @developer.chrome.com / @firefox.com, supports WebGPU when available.

Try it!
hybridgroup.github.io/yzma-wasm-ex...

#golang #wasm #llama #tinygo #yzma #gguf #webgpu