Enabling Data Access, Scalability and Protection for Generative AI

00:00 [This transcript was auto-generated.]
00:08
Thank you. I am Sathish Muthukrishnan, Chief Information data and digital officer for Ally financial and thrilled to be here, sharing our data story. And our technology transformation story that has led us to a place where we can unlock our AI ambitions sounds very complicated, but it really is not. It's a series of building blocks, and everything is driven and defined by software. As you see in your everyday life technology is omnipresent. So why not take advantage of that to build your technology ecosystem, or transform your existing technology ecosystem. So you can take advantage of all of the tools that you have to ultimately serve your customer, like we always do. That's where we start at ally. We are customer focused and obsessed. We want to drive differentiated customer experience that drives business results. But we also want to bring in the best technology to do both of those. So let me start talking about these building blocks. Couple of those will sound boring, but believe me, they're very critical. And absolutely important for this transformation. Infrastructure is the number one thing that you have to stay focused on. By infrastructure, I mean that the place where your applications run, where your data resides, and how you leverage both of them have to be modernized. Cloud probably is coming to your mind, I am not biased towards public or private cloud. But I am biased towards driving high resiliency, significant scalability, ease of integration, and efficiency of use when you have your applications running on your cloud. So we embarked on a journey to build and learn from our own private cloud. And then we started exposing and running our applications on public cloud. Once you have your applications running on public cloud, you also have a clear understanding of where your data has to reside. And it's important to give them the right platform to when you have this infrastructure platform ready to go. The next thing that you should focus on is your network. Again, sounds very boring, almost 90% of the companies have the network built for the 1980s. That's when internet was born. And your network. Still, as cables and wires and switches and data centers. And all of this data, even though they're on cloud, all of this application, even though they're on Cloud, their access through these aged networks. So to take full advantage of this modern infrastructure, you have to modernize your network. And by modernizing my your network, I'm talking about moving from a hardware defined network to a software defined network. Those two are the fundamental building blocks. If you have these two, then you're able to define your applications to be event an API driven. Those are software buzzwords, but what it means is you only use your application when you need to. There is a specific customer event that requires this application to be triggered. So you can use it when it's needed. And that awareness resides on your cloud infrastructure. That awareness is also present in your network. When you have these applications API enabled, you can tag them easily use their functionality, don't worry about the complexity. And then it also makes it easy for you to start collecting data around how these applications work. Now that we have talked about three different buckets of this ecosystem, the fourth one, which is what we're going to talk about, mostly for this presentation is data. Centralizing data, and then figuring out how to use them was how data transmission happened. But now, you have to centralize data that is essential. That is driving a unique customer experience and leveraging your cutting edge, modern network and infrastructure. And by the way, they are being triggered to these events and API's. Once you have centralize the data Make sure that it's managed and protected. So you can further leverage. The ability to centralize data also gives you the freedom to use it both internally and externally. And once you find 360 degree use of data is when you can drive unique customer experiences. Having data that is centralized, gives you the visual aid, to start to tell you how your applications are performing. But more importantly, it tells you how your customers are taking advantage of your experiences, it's important for your entire organization to have access to the data, it's important for them to understand data flows freely, from how your customers behave, how your applications behave, how resilient they have been, and how effectively they have been used. You have collected all this data you have stored it. You also have to provide tools and training. So this data can be found easily. If data is not found. It's no point collecting the data and storing it. Now that you've collected, and you've allowed people to use it, make sure that it's fortified, meaning it's managed, and the lifecycle of the data, every element of the data is defined and secure. And data has to be fit for purpose. In the past, we used to collect all kinds of data, without clearly defining the permissible and effective use of the data. You don't need all of your data to drive a unique customer experience. So figure out what percentage or what type of data is required to invoke this application that will drive a unique customer experience and derive the results. The business results that you're looking for. Here are the key tenants required for us to scale. And to take the next step. I am recapping what I talked about future proofing your infrastructure, modernizing your network, modernizing your platform, so they are trigger based. So you're not collecting cloud bills that are exorbitant, you're not running cloud, when they are not required to be run, you have centralized your data, you've protected it. And you have managing each data element to the lifecycle. And you have robust governance across all of this to explain how they work, and why they work and when they should work. That gives you scale. And once you have that scale, now you can power the engine to accelerate your AI ambitions, or so called the innovation flywheel. Once everybody across the enterprise starts to understand, I have access to this data, I can easily find them. I understand how this data drives customer behaviors, and how this data drives unique customer experiences, then give them the tools to amplify that thinking through AI make their everyday job easier. And that's what we have done here at ally. I also want to caution that it this journey is not easy. Once you have gone through the journey, the arduous journey of defining all of this driving transformation and getting to the other side feels amazing. And you can't wait to unleash the potential of your company, the data, the talent that you have. But you also have to watch out for some of the downsides. There is privacy and security risks, you have to understand that you have to bring along your control groups, the ones that have customer benefit, top of mind, your risk, your audit, your compliance, your legal teams, they all have to be part of this development to understand the risk of exposing or using the data in a certain way.
 
09:14
You have to worry about bias and fairness. You have to worry about exposing your IP, you have to worry about accuracy of the data being used. That is why robust governance and taking things one step at a time is going to significantly help you. So we pay equal attention to security, privacy and governance as we do for some of the other cool cutting edge things like data transformation using AI being on Cloud, having even driven architecture. Now with all of this foundation, we were able to launch ally.ai This is a platform that brings together the power of the journey that I just walked through. Now that we have north of 70% of our applications cloud enabled, about 80% of our data centralized, we have a software defined network, we are able to take advantage of that, and define ally.ai, where all of the AI driven initiatives are taking place in a common single platform, makes it easy for us to learn makes it easy for us to protect makes it easy for us to scale and derive that customer experience and business value in accelerated fashion. As you might imagine, generative AI plays a significant role in our ally AI platform. Generative AI for those of you that may not know, and I'm sure that if none of you, generative AI is became critical because you're moving from analytics to creation. You're moving from technology, making it difficult for humans to understand and adapt to a common person being able to use advanced concepts like AI. We see staying power, we see a lot of value that can be unlocked through ally.ai. This is the platform that we created. But before I dive into the platform, I'll tell you with the help of our CEO, we created three simple principles for us to use generative AI.
 
11:38
The launch of chat GPT in 2022, uncovered the transformative potential of generative AI and financial services technology specialists are exploring and testing commercially available generative AI and large language models. Very few of them are building their own proprietary platforms that have the speed and functionality of known generative AI tools. But with security protections, and the human touch that inspires confidence, elevates productivity and reimagines the customer experience. The launch of ally AI maps precisely to our one ally technology strategy that will enable customers to have a truly personalized banking experience. Allies generative AI platform is fully hosted in our cloud environment to protect our data. we've ensured all ally enterprise level security controls are in place to allow for safe exploration of this new capability. We're in the process of enabling a few use cases leveraging this platform. Our first use of this technology is in the Customer Care platform. Today, agents spend valuable minutes after each customer call manually summarizing the conversation. Here we see our ally AI platform taking raw audio from a customer call to generate a high fidelity transcript along with a summary of the interaction. Topics are extracted, and the summary is shown to the agent. The agent then has an opportunity to correct any mistakes or add more context. This shaves minutes of the agent's time per customer call while improving the accuracy of the call summary. Using the platform, our customer care associates will be more productive and able to focus on meaningful engagement with customers. Allies leadership and the exploration and real world use of generative AI allows us to adapt to a rapidly changing environment and serve our customers even better.
 
13:40
The first one was use it on internal ally customers first, the second always have a human in the middle because the technology is maturing. And number three, eliminate PII, I personally identifiable information being exposed to these large language models, which power these generative AI models, and not allow these large language models to learn from ally data. So what we did was create a bridge between internal ally and external large language models. Even though these large language models are hosted privately for ally in a very secure environment, we decided not to share ally data to train these large language models. We want to understand how this technology works. We want to understand the value that it brings. But we want to do it in a way that makes sense for our internal and external customers. As we learn more, and as we strengthen our controls. We are excited to start unlocking the long list of already prioritized use cases that are waiting to be executed. So our first use case Was it was a unique and interesting one, we have our call center associates capture the conversation that they have with their customers after every call, capture that conversation and summary, understand how the call went, document the steps, the next steps they provided. And also understand the sentiment behind the call. What this does is during the call, the call center associates have to document what they're doing, which takes away from singular focus on customers. And then it also takes valuable time from servicing the next customer. We thought this would be a great use case for generative AI. After all, generative AI is excelling in creation. So generative AI was going to create that summary for us. But it came with unique challenges. How do you transform the conversation into text. So generative AI can provide the summary. So the team had to build that into the ally AI platform. And in real time, transcribe the conversation happening between the customer care associate and the customer. When the call was complete, that text was fed into the large language model to create a summary and show that summary to our call center associates, they are able to edit change, and also able to provide feedback on whether this was accurate or not. And then file that summary. We learned a lot through this process. At times, there were calls with the customer playing some show on TV with loud volume that intermingled with their conversation and somehow found a way to the summary, we had to strip that out, we had to make sure that we are not capturing the personally identifiable information our customers are sharing with our associates. We also found hallucination when the calls are short, the large language models are very confident. And they tend to behave confidently saying I do understand the summary even though it is 30 seconds. Let me provide you with that summary. That's not what we were looking for. So there is constant tuning happening with the help of our customer care associates. You may have heard, RL HF, reinforced learning human feedback. That is how open AI and BART train these large language models. We use that same methodology to train the dedicated model for ally. And now north of 85% of the summaries, gets a thumbs up from our customer care associates. This is Use Case Number one, we are very excited with the value that this is adding. And we are continuing to work on other use cases, slowly but steadily. Hopefully, this entire journey of transformation. Some of it hard, some of it fun, some of it already done, some of it new that Allah has gone through to unlock value for our customers resonates with you. I am so glad I was able to share hard work of several associates across all of ally, the partnership with our control groups like I talked about, and with a lot of pride, showing how we're continuing to advance our company by leaning forward with the right technology that creates magical customer experiences and also great business outcomes. Thank you