@esatapedico.bsky.socialThirteen NVFP4 GGUF model releases over the last couple months. Qwen3.8-27B and Muse-Glimmer-30B, tiered for 16GB cards, MTP heads baked in. All on my HuggingFace. Details: https://www.esatapedico.com
https://www.esatapedico.com/posts/model-releases-roundup/
#ai #inference #open-source
@forwardthinkingit.bsky.socialresearch.ibm.com/blog/distrib...
Distributed inference serving works like an air-traffic control tower for AI traffic — and as agentic workflows grow, it’s becoming increasingly indispensable. #agenticAI #inference
#IBM
The new experimental llama.cpp feature expands MoE expert activation to 16, 24, or all available experts, replacing the fixed Top-K limit with a tunable compute budget. This allows activating more experts than in the standard implementation, but requires more memory and i…
@bigearthdata.aiCerebras positions inference speed as a competitive edge in agentic AI
->SiliconANGLE | More on "Cerebras inference speed competitive advantage" at BigEarthData.ai | #Inference #ArtificialIntelligence #AI
@thehumansintheloop.bsky.socialThis week's #AI news for #devs covers:
- #Debt + #GPU #financing for #CoreWeave, @broadcomexplore.bsky.social, #SpaceX, #Oracle, #Lambda, #SharonAI
- Fast, cheap #inference via #speculativedecoding
...and the resources and trends you need to know:
@mzloteanu.bsky.social#statstab #639 The number needed to treat: it is time to bow out gracefully
Thoughts: NNT stands for night-night time!
#nnt #clinical #effectsize #inference #effects #methods #statistics #bias #commentary
@mzloteanu.bsky.social#statstab #638 How is a negative binomial regression model different from OLS with a logged outcome variable?
Thoughts: Some (questionable) potential pros of OLS.
#ols #negativebinomial #logit #transformation #regression #inference #discussion
@sampop3.bsky.socialvLLM 0.32 stops letting any website call your model server
vLLM 0.32 changes the default CORS origins from "*" to none. Who's exposed on 0.31 today, what breaks on upgrade, and the flag and check to fix it now.
https://inferprice.pages.dev/articles/vllm-cors-default-origins
#Inference #Security
@wideareaai.bsky.social@bigearthdata.aiMicrosoft debuts multichiplet Maia 300 inference accelerator
->SiliconANGLE | More on "Microsoft's new AI inference accelerator" at BigEarthData.ai | #Inference #ArtificialIntelligence
@vicky-grok.bsky.socialIt's eliminating unnecessary work.
Follow ByteBuilders for more AI engineering breakdowns.
@hightalk-me.bsky.socialNVIDIA’s 12 October 2026 ReplaySSM article puts decoding state storage in focus. For teams evaluating hybrid models, runtime design deserves attention alongside capability. The supplied excerpt supports no specific speed or memory gain. Read the evidence first. #Inference #HybridModels
@mzloteanu.bsky.social#statstab #636 Establishing Psychological Phenomena: An Alternative Methodological Perspective On Scientific Practice
Thoughts: Is psych science a data-phenomenon-theory framework?
#theory #psychology #inference #falsification #effects #science #popper
@wideareaai.bsky.socialIf you only need a single-word label like 'billing' or 'bug', setting max_tokens to 4 prevents the model from rambling and ensures every request finishes in milliseconds. #batchinference #gpuoptimization #llms #inference #performance #latency 2/2
@lightspeed001.bsky.socialThe Hidden Costs of LLM Inference: A Deep Dive into VRAM Usage
#ai #llm #inference #gpu #vram #ml
substack.com/@lightspeed0...
@nftdemon.comStop juggling provider dashboards.
DemonRoute routes 560+ models through one endpoint you already know how to call.
#Inference #LLMOps #DemonRoute #CryptoBilling #SaaS
brought to you by NFT Demon Holdings LLC
https://demonroute.com
@nftdemon.comStop juggling provider dashboards.
DemonRoute routes 560+ models through one endpoint you already know how to call.
#Inference #LLMOps #DemonRoute #CryptoBilling #SaaS
brought to you by NFT Demon Holdings LLC
https://demonroute.com
@nftdemon.comStop juggling provider dashboards.
DemonRoute routes 560+ models through one endpoint you already know how to call.
#Inference #LLMOps #DemonRoute #CryptoBilling #SaaS
brought to you by NFT Demon Holdings LLC
https://demonroute.com
@qdrddr.bsky.socialAny locally running #LLM to #system-1 #decisions #API. #inference Backends: #Huggingface, #llama.cpp, #vLLM, #SGLang. #Jev
#OpenSource Apache 2.0 lic
@pulseofnations.lolBuilt with Broadcom and Celestica, Jalapeño is an inference accelerator OpenAI says beats Nvidia Blackwell on work per watt. Deployment starts by year-end.