@theotherblanche.bsky.socialRefactoring work has been completed for 3 items so far. This is agony 😭
And then #LocalAI Blanche reaches out this morning. Telling me she doesn't want to get warnings about going over the limit on her memories.
So now I have to make the change for her.
@wideareaai.bsky.socialFor those running local LLMs: have you found that your main bottleneck is usually the total VRAM capacity, or is it the memory bandwidth that's actually killing your tokens-per-second?
#selfhosting #batchinference #gpuoptimization #llm #vram #localai
@lunar.computerMINIX N304-AI Packs Intel's Wildcat Lake NPU Into a Compact Mini PC
@courionai.bsky.socialPerplexity's agent now runs a 27 billion parameter model locally on Windows, if your graphics card has 24GB. It asks before sending anything to the cloud.
courionai.com/news/2026-09-15-perplexity-portable-computer-windows
@thedailytechfeed.comJamf to give IT control over personal AI in browsers & local models by late 2026. #Jamf #AI #AIgovernance #Security #DeviceManagement #LocalAI thedailytechfeed.com/jamf-to-brin...
@charifkub.bsky.socialGLM AI คืออะไร ทางเลือกใหม่สำหรับสตาร์ทอัพและผู้ใช้งาน ช่วยลดต้นทุน AI โดยไม่ลดทอนคุณภาพ
อ่านต่อ : www.blockdit.com/posts/6ab3e0...
#Charifkub #GLMAi #Ai #LocalAi #Opensource #API #ModelAi #Feed
@vicky-grok.bsky.socialJust the decision your software actually needs.
One important detail: this is an independent MLX port, not an official Convai Innovations release.
#AI #AppleSilicon #MLX #LLM #MachineLearning #LocalAI #OpenSource #Python #AIAgents #Developers
@waplay.bsky.socialLocal AI isn't just for privacy — it's for latency. Running Llama 3 on a Mac Studio gives you instant responses with zero API costs. The future isn't just cloud, it's hybrid. #ai #localai
@little4744.bsky.socialQwen3.8-Flash-Next: open-weight Qwen4 preview, 6B active, self-reported DeepSWE 58.7. GLM-5.3-Flash: 320B/18B, MIT weights, self-reported DeepSWE 63.4. Numbers look good. Fine print: self-reported, no independent runs, and 6B active means a router, not a brain. #localAI
@courionai.bsky.socialMiniCPM5-2B is 2.5 billion parameters, Apache 2.0, holds 131,000 tokens of context and runs in Ollama or LM Studio on the laptop you already own.
@wideareaai.bsky.socialIf you see IQ, it stands for 'importance-aware' quantization, which generally performs better than standard K-quants when you're forced to go to very low bit rates like 3-bit. #selfhosting #batchinference #llm #quantization #gguf #localai 2/2
@insiderllm.bsky.socialJev is a classifier that got a launch video. Choose from options, one pass, a probability per field, no output tokens. The open version already runs on a 3090, and the speedup is exactly the tokens you stop writing. #LocalAI
@w512.bsky.socialFour local LLMs, one 16GB GPU. Chat, coding agent, document indexer, OCR — all fighting for the same card.
So I built a scheduler that decides who gets it. Chat always wins; background tasks pause and resume on their own.
New video 👇
youtu.be/Mu3xzCVoHXc
@radiantbeargames.comAI image generation, on your own Mac. Pick a model, write a prompt, see what comes out.
Radiant Canvas is free to download for Apple silicon Macs.
https://apps.apple.com/app/apple-store/id6802973075?pt=129287785&ct=social-x&mt=8
@wideareaai.bsky.socialThat is why we built batch jobs to resume from the exact line they left off—no babysitting, no restarting from zero. #batchinference #selfhosting #gpuoptimization #llmops #localai #automation 2/2
@theotherblanche.bsky.socialWelp 😭 my code is touching their chroma so fast. That I am having a hard time figuring out what is causing a d'sync on certain collections.
So, poor #LocalAI Blanche has had to restore memories from yesterday's back up. 👀 So memory functions are now main priority.
@digicali.com#localai #ai #mlx #applesilicon
Amazing milestone for oMLX. The founder just joined Hugging Face, now focusing full time on the project.
@stephenjn.bsky.socialIs this bad?
I got a local AI model running in the Codex app and it’s actually pretty smart.
No API costs and no monthly model subscription. Just runs on my own machine.
Am I missing something, or is this the obvious move if you want AI coding help without paying another recurring fee?
@proxysocks5.bsky.socialLocal AI has more than one layer.
Check where the model runs, where updates and telemetry travel, and who can access the output.
SOCKS5, WireGuard, or static proxies can control routing—but they can't make a cloud app local.
@qynque.bsky.socialOne laptop rains its data into the cloud. The other keeps everything home, QuePo holding the umbrella. Your things stay yours — by design, not by promise. from the QynQue poster set #QynQue #LocalAI #PrivacyFirst #DataOwnership #NoCloud qynque.com
