programme access backs routing, deployment and tuning work

GoodVision AI has joined NVIDIA Connect, a move the company says will help it lower AI inference costs and speed up performance across its platform.

The Redwood City, California, company builds compute infrastructure for AI inference. It runs that system through three linked parts: cloud services, a real-time Smart Routing Engine and a global network of purpose-built, immersion-cooled AI Factories.

NVIDIA Connect gives solution providers and service companies access to NVIDIA compute platforms, software and technical resources. GoodVision AI plans to use that access to improve inference performance, accelerate AI Factory deployment and sharpen the routing algorithms at the centre of its system.

David Wang on Smart Routing

The Smart Routing Engine sits at the core of the platform and is the main focus of the NVIDIA Connect work. In milliseconds, it checks four factors for each inference request: the model needed for the task, the sensitivity of the data, the cost limit and the latency target.

After that, the system sends the job to the right model version and the right compute location. That approach helps companies avoid paying frontier-model prices when a smaller model can do the work.

In GoodVision AI’s own deployments, the Smart Routing Engine has reduced AI inference costs by about 60%. It has also cut network latency by roughly 50% and lifted gross margin on the related business by around 50%.

Chief executive David Wang said many companies still buy more compute than their AI workloads need. He said the routing system matches each request with the right model and hardware, which has helped cut costs by about 60% and latency by about half.

Through NVIDIA Connect, GoodVision AI will also get earlier access to NVIDIA GPU platforms and AI software. That should help it tune inference workloads, roll out AI Factories faster and keep improving how it allocates compute across models and locations.

GoodVision AI describes its platform as a way for enterprise customers to run AI in production with more control, lower cost and faster response times. The company is led by Wang and operates from Redwood City.