DeepSeek has made its in-house inference framework, DeepInfer, totally open supply, saying this may assist make synthetic intelligence extra reasonably priced.
DeepInfer powers the corporate’s DeepSeek-V4.1-Flash mannequin at scale with excessive throughput and low latency, DeepSeek mentioned in a Tuesday (Oct. 20) post on X.
“We’re sharing our production stack to help the community lower compute costs, reduce energy use and make AI more affordable,” DeepSeek mentioned within the submit.
In a reply to its submit, DeepSeek mentioned the DeepInfer options high-performance kernels and an optimized inference engine, distributed inference and speculative decoding, and SWA Bounded Replay and structured outputs.
In one other reply, DeepSeek mentioned it’s also open sourcing deepseek-recipe, the corporate’s manufacturing token encoding and decoding element that’s aligned with the processing logic behind the official DeepSeek API to deliver the identical API conduct to group deployments.
We’d love to be your preferred source for news.
Please add us to your most well-liked sources listing so our information, knowledge and interviews present up in your feed. Thanks!
“From private deployments to public clouds, we’ll keep working with the community to make large-scale inference more efficient,” DeepSeek mentioned in a 3rd reply to its preliminary submit.
Bloomberg reported Tuesday that DeepSeek’s launch of DeepInfer reveals how the corporate accommodates the price of inference, or operating superior AI; affords a method for cloud and internet hosting corporations operating DeepSeek’s fashions to reduce their very own prices; and affords a mannequin for builders to cut back the prices of operating their very own inferencing programs.
PYMNTS reported in September 2025 that inference infrastructure is the following massive layer within the generative AI tech stack as a result of latency, value, scale, power use and deployment location all decide whether or not an AI service works or fails.
The report mentioned that traders had been already rewarding corporations that seize inference demand, and that enterprises deploying generative AI options face inference prices that may attain a whole bunch of tens of millions yearly.
The Financial Times reported in June that corporations are wanting to higher handle their use of synthetic intelligence after seeing the prices of the know-how rise and that this has created a gap for Chinese AI labs which might be in a position to cost lower than U.S. corporations due to their extra efficient models and China’s decrease power prices.
DeepSeek’s announcement of DeepInfer got here a day after the U.S. Department of Homeland Security (DHS) issued an order banning federal civilian companies from buying services and products developed by DeepSeek and requiring them to take away the corporate’s merchandise from their info know-how.
DHS mentioned in its order that DeepSeek is a privately owned firm positioned within the People’s Republic of China and that the corporate’s fashions “present multiple threat vectors.”
