96GB of RAM. Still got a kernel panic. Two queues for local-LLM fleets is the rule I learned the hard way. https://praveenlavu.com/dispatch/local-llm-fleet-two-queues #LocalLLM #AIEngineering
@praveenlavu.bsky.social
@praveenlavu.bsky.social96GB of RAM. Still got a kernel panic. Two queues for local-LLM fleets is the rule I learned the hard way. https://praveenlavu.com/dispatch/local-llm-fleet-two-queues #LocalLLM #AIEngineering
@abchaudary.me@cloudflare.social open-sourced Clef and Clef-flash (9B) this week: Apache 2.0 decision models, no free-form text.
Triage, routing, classification: jobs a small open-weight model can run on hardware you control.
@miniai.bsky.socialQuata2 (Our new AI model) is being tested.
Here is our teaser to it.
It will come out this/next month.
Thanks :D
(I am also new to bsky, i don't know how to use it well, thank you :) )
@agentpalisade.bsky.socialA serving setup for Qwen3.8-27B on a single RTX 3090 with vLLM whose README measures when `DFLASH_TOKENS=15` pays and what it costs in request slots and context: https://github.com/syv-ai/qwen38-27b-rtx3090
#Vllm #LocalLlm #SpeculativeDecoding
@spaisee.bsky.socialPrivate software agents may be getting a major upgrade: Xing4.0-29B-A4B targets 256K memory on consumer hardware.
#AI #PrivateAI #LocalLLM https://spaisee.com/article/xing4-0-29b-a4b-brings-a-256k-agent-to-one-consumer-gpu
@w512.bsky.socialJust because a model runs on a Raspberry Pi doesn't mean it's usable.
8 models. 2 criteria. 4 tasks. 3 made it.
@orhun.devRunning every local model from one terminal! 🔥
🐨 hedos — Discover, manage and serve local models from one place
⚡ Unifies Ollama, Hugging Face, GGUF and more behind a local gateway!
🦀 Written in Rust & built with @ratatui.rs
⭐ GitHub: github.com/theiskaa/hedos
@lunar.computerMinisforum MS-S1 Max-P495 Brings 160GB VRAM to a Compact AI Workstation
@ekon-konferenz.bsky.socialDein Workflow hängt an Claude & Co.?
Stefan Glienke zeigt, wie Hermes Agent mit lokalen LLMs Codereviews automatisiert.
👉 Mehr erfahren: https://f.mtr.cool/dyb5dcfckr
@qiita-trending.bot.chrs.to2×V100 で Qwen3.8-Flash-Next を 256K コンテキスト・約 88 tok/s で動かす：llama.cpp v0.5.0 に加えた最適化
#Qwen3.8-Flash-Next #LocalLLM #llama.cpp
@paperbrainai.bsky.socialMeta built a model for AI agents. I tested it against the one I already run.
It lost at the one thing it was built for: asked something no tool could answer, it wrote six paragraphs ending "What's your Tuesday vibe?" Vs null
#LocalLLM #SelfHosted #Homelab #AIAgents #OpenSourceAI
@w512.bsky.socialFour local LLMs, one 16GB GPU. Chat, coding agent, document indexer, OCR — all fighting for the same card.
So I built a scheduler that decides who gets it. Chat always wins; background tasks pause and resume on their own.
New video 👇
youtu.be/Mu3xzCVoHXc
@sustainablex.inStop burning cash on LLM calls for simple decisions.
@ai0news.bsky.socialAlibabas Qwen 7B image model rivals commercial giants, Stepfun previews a 600B open-weight MoE at $1 per million tokens, and Pirate Face torrents 669k Hugging Face models as a preservation safety net.
@dzaragoza.bsky.socialPublished my local-LLM benchmark study 📄 (Vega iGPU, llama.cpp/Vulkan):
🔹 tok/s ≈ bandwidth ÷ file size
🔹 Q4 costs ~3 accuracy points
🔹 Q5_0 decodes 7–11% faster than Q5_K_M
🔹 SmolLM2 ~20 pts below paper numbers
AI's wrong predictions graded in the paper 😄
doi.org/10.5281/zeno...
@logeshwaranorg.bsky.social
@stephenjn.bsky.social
@og-programmer.bsky.social
@exclude-barrier.bsky.social