Table of Contents
- The Changing Economics of Generative AI
- Capitalizing on Cloud and Data
- Navigating Data Reliability Challenges
- Enhancing Customer Service with Generative AI
- Priming Data for Success
- The Future of Generative AI in IT
Key Highlights:
- Generative AI costs have decreased dramatically, leading to miscalculations in ROI projections by CIOs.
- Capital One's embrace of public cloud computing and open-source AI models underscores the importance of data governance and accessibility.
- Effective use of generative AI relies heavily on data quality and discoverability, impacting the success of AI initiatives.
Introduction
The rise of generative AI has ushered in a new era for information technology, one that is reshaping how organizations approach data, analytics, and customer engagement. As the head of AI strategies at Capital One, Prem Natarajan has highlighted a critical issue facing many Chief Information Officers (CIOs): the miscalculation of generative AI costs. This misjudgment can lead to "irreversibly bad decisions" that hinder organizational progress in an environment characterized by rapidly changing economic models. As generative AI technology evolves, so too must the strategies that organizations employ to leverage its capabilities effectively.
The Changing Economics of Generative AI
In the past two years, the costs associated with AI inference—essentially the processing of data through AI models—have plummeted by more than a factor of 1,000. Natarajan cites that what once cost $10 for inference can now be accomplished for just one cent. This staggering decrease in costs presents both an opportunity and a challenge for CIOs. Traditional methods of calculating return on investment (ROI) are becoming obsolete, as Natarajan asserts that relying on these outdated frameworks may lead to significant strategic missteps.
Jason Andersen, a principal analyst tracking AI for Moor Insights & Strategy, agrees with Natarajan's assessment. He emphasizes that the speed and complexity of technological advancements in AI are unprecedented, making conventional investment strategies less relevant. As enterprises strive to deploy AI effectively, they must adapt to a landscape where old norms no longer apply.
Capitalizing on Cloud and Data
When Natarajan took the helm at Capital One in March 2023, the generative AI landscape was rapidly shifting, especially following the introduction of OpenAI's ChatGPT. The immediate need to align AI strategies with corporate objectives led to intense collaboration within the AI team. Natarajan highlighted that a strong data foundation is crucial for AI success. The proprietary data that Capital One possesses is invaluable in creating generative AI applications that distinguish the company in the competitive financial services sector.
Capital One's decision to use Meta's open-source Llama language model is a testament to the organization's commitment to flexibility and customization. By transitioning to a fully cloud-based infrastructure and shutting down its data centers, Capital One has positioned itself as a leader in leveraging public cloud capabilities. This shift not only enhances the company's data management but also allows for a more agile approach to AI development.
The importance of data cannot be overstated. Natarajan emphasizes that creating a cutting-edge technology organization is futile without a rich data history. The depth and quality of data can significantly influence the outcomes of AI initiatives. Organizations often overlook the challenges presented by unstructured data, which has remained largely inaccessible to traditional machine learning algorithms. Generative AI offers new avenues for extracting insights from this vast reservoir of unstructured information.
Navigating Data Reliability Challenges
Generative AI deployments often encounter issues such as hallucinations—instances where AI produces inaccurate information. Capital One addresses these challenges by imposing strict limitations on the types of queries that can be made, ensuring that responses are drawn exclusively from curated internal data sources. By focusing on specific customer inquiries and leveraging trusted knowledge bases, Capital One minimizes the risk of misinformation.
Natarajan notes the importance of human oversight in AI outputs. The company employs a "human-in-the-loop" strategy, where AI-generated results are rigorously reviewed by human experts. This practice not only enhances the quality of the information provided to customers but also facilitates continuous learning from corrections made by human operators. The goal is to reduce the frequency of human intervention as the AI system matures and improves its accuracy.
Enhancing Customer Service with Generative AI
The application of generative AI at Capital One extends beyond backend processes; it is also crucial for enhancing customer interactions. The company's agent servicing model has shown promising results, enabling customer service representatives to resolve inquiries more efficiently. For instance, when a customer inquires about a declined transaction, agents can quickly access relevant information in real-time, resulting in a significant increase in the accuracy of search results compared to legacy systems.
Additionally, Capital One has introduced an innovative AI tool known as Chat Concierge, designed to assist car dealerships in facilitating auto loans. This tool utilizes multiple logical agents to simulate human reasoning, offering a seamless experience for customers. Chat Concierge not only provides information but can also take action based on customer requests, such as scheduling test drives or comparing vehicle options. Reports indicate that car dealers using this tool have seen customer engagement soar by up to 55%, while latency has been reduced fivefold since its launch.
Priming Data for Success
Natarajan emphasizes that the path to successful generative AI implementation hinges on robust data practices. Organizations aiming for AI transformation must prioritize data governance, quality, reliability, and discoverability. Generative AI thrives on vast datasets, allowing it to recognize subtle patterns and insights that drive business value. Without sufficient data, the efficacy of AI tools is severely limited.
Moor's Andersen reinforces this notion, asserting that sound architectural decisions significantly influence the success of AI projects. Capital One's choice to adopt an open-source model highlights its forward-thinking approach, showcasing a willingness to innovate and adapt to the evolving technology landscape. By maintaining flexibility in its AI models and leveraging the capabilities of AWS cloud services, Capital One is well-positioned to navigate the complexities of generative AI.
The Future of Generative AI in IT
As the generative AI landscape continues to evolve, organizations must remain vigilant in adapting their strategies. The rapid pace of technological advancement demands that CIOs rethink their approaches to investment and ROI calculations. A failure to do so could result in missed opportunities and strategic misalignment.
The insights provided by Natarajan and Andersen serve as a reminder of the transformative potential of generative AI when harnessed correctly. By focusing on data quality, governance, and innovative applications, organizations can unlock new avenues for growth and customer engagement.
FAQ
What is generative AI? Generative AI refers to a branch of artificial intelligence that focuses on creating new content or data, such as text, images, and other forms of media, based on existing information.
How has the cost of generative AI changed in recent years? The cost of generative AI inference has dramatically decreased, dropping by more than a factor of 1,000 in just two years, making it more accessible for organizations to deploy AI solutions.
Why is data quality important for generative AI? High-quality data is crucial for generative AI because it enables the AI to identify meaningful patterns and insights. Poor data can lead to inaccurate outputs and hinder the effectiveness of AI applications.
What are the benefits of using open-source models for AI? Open-source models provide flexibility, customization, and community support, allowing organizations to tailor AI solutions to their specific needs and leverage the latest advancements in the field.
How does Capital One utilize generative AI in customer service? Capital One uses generative AI tools to enhance customer service by providing representatives with real-time access to information, enabling them to resolve inquiries more efficiently and accurately.