Nasdaq improves generative AI response times 30% with NVIDIA NeMo Retriever and NIM

Overview

Nasdaq built a generative AI platform for internal chatbots and search, running NVIDIA NeMo Retriever and NVIDIA NIM microservices as part of NVIDIA AI Enterprise. After running four global hackathons to prototype AI applications, Nasdaq deployed self-hosted GPU embeddings on NVIDIA L40 GPUs, achieving 30% faster response times and 30% improved chatbot accuracy while reducing embedding costs.

The challenge

Nasdaq's generative AI platform faced slow embedding operations and high operational costs, and as the platform gained more users it needed higher model accuracy while remaining accessible to all skill levels, scalable and secure.

The solution

Nasdaq built a generative AI platform using NVIDIA NeMo Retriever and NVIDIA NIM microservices, part of NVIDIA AI Enterprise. The team ran four global hackathons (three days each, across regions) as a proof of concept to prototype chatbots and other AI applications, then prioritized the most impactful hacks for production. It implemented self-hosted GPU embeddings using NVIDIA NIM on NVIDIA L40 GPUs to cut costs and optimize resource usage.

Reported business value

The platform delivered 30% faster response times and a 30% improvement in chatbot/conversational-interface accuracy, reduced embedding costs through self-hosted GPU embeddings, and NIM's real-time performance insights let the team quickly identify issues like slow data indexing and inaccurate responses.