Implementing AI In The Real World — With Kyndryl's Antoine Shagoury
Channel: Alex Kantrowitz
Published at: 2025-01-31
YouTube video id: hYB7Yfy2kiw
Source: https://www.youtube.com/watch?v=hYB7Yfy2kiw
kindrell Chief technology officer antoan shagui joins us to talk about implementing AI in the real world covering everything from Agents to open source in a YouTube exclusive brought to you by kindrell Antoine great to see you welcome to the show thanks Alex appreciate the time today oh I'm so excited to speak with you mainly because you're actually seeing what it's like to implement AI in the real world and we hear often companies talking about AI as a philosophy or AI as a distant dream AI 3 to 5 years from now but you're doing it uh and so let's talk immediately about what you're seeing in the agent world we just had Mark Benny off on the show recently talking about how they're bringing agents uh into into reality uh connecting it with the Salesforce data um I still can't quite tell what the reality is for everybody else and exactly you know what that means so you are doing this in the real world you're helping companies Implement agents um what can you tell us about that one the first thing to reflect on it it's it's absolutely amazing at the rate and Pace in which you know the the conversation has grown the opportunity discussions you know have continued to kind of Advance um but one of the things that we we we'd love to kind of talk about is especially with agents it's it's it's you don't start there it doesn't start with an agent there's a whole different process that kind of gets us into that set of capabilities and one of the more difficult things to break when you go from what the expectation may be to how do you want to approach it really starts to lay the groundwork on what we think is the right pattern and practice that that help makes it work and when I the way I refer to that is almost and um you may laugh me for saying it it's kind of that Journey message so you kind of have to take that with a little bit of grain of salt um but as we start to unpack the business a bit that demystifying um it sounds strange how do we demystify a business we start with understanding the information the data sets available how do we really understand what we're working with and um often we get into a little bit into demonstrating what we can do with simple AI introductions some simple automations to step through do we have the right data sets based on what the client's looking to do and as we build into that that's when we start to build the pattern of information and we go from simple automations to predictive capabilities and that is really at the first time there's signs to put an opportunity to how we can deploy an agent to demonstrate going from a predictive opportunity to something prescriptive so we know enough about a business process enough about A System's operation to really an agent step in to orchestrate maybe manage a state of an application a failover process and that really starts to drive it and as as much as it sounds a little bit defeating it really demonstrates that you know we we can really change the framework on how clients approach it and I I'll put one last comment to the answer too which which sometimes gets a raised eyebrow for me ultimately we're saying look how do you want to become autonomous so agents are really just another step getting closer to to really automated systems and automated processes and that's how we really look start to look at and approach it yeah and that's what I wanted to ask you is Agent just a new fancy word for automation I mean we've been doing so the the tech world has been doing automation for a long time so is there anything new and different about this agentic moment as every research house is telling us about that should let us think hey this is something that's actually going to work this time or is this just like we're automating a little bit more and it's rebranded to give you the word answer it can be both I oh no that makes total sense like it's kind of like the proverbial Rat Hole a little bit or the rabbit hole um yeah we can we can go and really just apply it in simple automations but that's not the benefit it's it's really the granularity that we can start to map into to help the business to help kind of provide the outcome and I'd actually look at um this is this is going to make sound strange too it's almost like the impact radius like the impact area there's so many things that agents can impact whether it's looking at a business process I'm looking at you know human involvement like who's involved like how many people have to be involved in a say a business process a service or a transaction so it has the opportunity to really get very deep into can it be the better orchestrator right of an event can it be the better analyst to understand the permutations of a situation can it be a better distributor or a a run agent to distribute when something should run so it has a phenomenal opportunity and we are seeing use cases where the clients are able to get deeper into and that's why I use that term demystifying business a little bit as we get deeper into how business runs it really exposes Legacy systems human intermediation across different processes applications even partners and agents allow us really to start to decompose that in a way in which we can focus on what really matters There's an opportunity no there's definitely an opportunity can you give me like one concrete example of how like you've helped uh and then we're going to get a little bit into I know we're like talking a lot about what you do I want to talk a little bit about what the company is uh but but before we get there just one concrete example of how you've helped a company uh Implement an agent or sure we stop there yeah no it's a good one and maybe simple's best sometimes but you know uh and I'll touch on you know AI coding or AI assisted coding for example we've we've be able to use our our AI agent framework to deploy not only codee assist a agents but also into test and deployment agents so how we can also shorten the time in which where we see errors in coding to be returned back and be refined so we've deployed that across a client for example um International client in Telco space on how they improved the code life cycle the code deployment life cycle and the in the Improvement life cycle so there was enough specifics in what they wanted to deliver within this code Factory that the agents were able to really overlay it in a way in which which we improve quality improve quality that means lower defects klock type of analogy and how we can improve how we deploy more effective releases better functionality faster to the client base so it's it's specific it's it's it's not like it's not dominating and changing the world day one but it does provide efficiencies and a lot of the early gains going back to the basics can I provide more efficiencies right you know back into the business and that's one of the perfect use cases yes so let's just for audience tell them a little bit about what kindrell is because sure you have a pretty interesting history oh yeah we we spun out from IBM a little over three years ago um and the way I can best represent what we were is we were the uh implementers we were the integrators and we were the operators for managed services and the infrastructure in the systems as a part of IBM and their client base um and when we spun off it's a phenomenal Heritage to base our business on so when you think about what we do we are experts in running large complex Global Mission critical systems and as we left IBM we went from being you know say you know running their portfolio and their Integrations and we expanded in our Partnerships so we became again close partners with Microsoft AWS Google Dell now Nvidia think about those time examples sap I mean Oracle relationships so we can now really really extend our integration implement ation experience really to the benefit of our client you know from that community so we really become like what I call this ecosystem player but we are rooted in manage services right and it's these are the companies that we speak about on the show every week so you have pretty good visibility into them I might ask you an Nvidia question later in fact I probably will on the implementation side I think this is one of the big areas of interest for me and for our audience and for anybody working in Tech which is uh in AI in particular there are so many proofs of concept everybody has built some AI po as they call it and then it kind of sits there um and sometimes they make it out but most of them don't I think like something like 10 or 20% of proof of Concepts in AI have haven't really been proofs at all and no sorry only 10 or 20% have made it out the door and 80 or 90% haven't really been proofs of at all they've just been sort of prototypes and you know they look nice you go to a management meeting everyone cheers and they never see the light of day on the implementation side why is it so hard to get AI projects out of the door um wow this is I got to be careful on how I answer this one right I'm not sure I'm going to help the numbers or statistics in the conversation the um I'll probably reflect first a little indirectly and then directly so indirectly it's it's no different than the rate and pace of solutions hunting for a problem I mean the market is amazing that way and the investment is is phenomenal so we have so many things pent up in the opportunity side um the best part we're starting to see is the willingness to adopt the willingness to try the pocs have actually gone up exponentially they haven't gone down they haven't died down at all in that scenario but we often get into is the expectation the expectation that AI is going to naturally solve the problem where we haven't really defined it yet so there's a lot of I'll say missed expectations and although yeah 80% failure rate is not not uncommon or call it you know getting thrown on the shelf right first of a kind is last of a Kind type of scenario um the the the approach we often get into is yeah we we we worked on the POC but it's actually trying to find out what's missing and we continually work on so a lot of the effort we bring is in the approach so how do we understand what you're looking for what's the business problem and we often find many of the poc's turn into they don't require complex AI new model development and things that space they require simple automations so they require more or more data right or more programmatic changes in how the application is operating or the business is operating so a lot of it is just the I guess the gap between the understanding of what technology is and what it can do versus how you need to deliver that value and and we see that and and as much as um as much as we talk about it and it sounds almost like in a negative way it it is a part of the discovery process and the the one thing we always want to encourage though is probably what the business is intending to do how do I uncover how do I evolve what's actually happening here and that's that's usually the best way to keep the focus and actually recraft the thesis and we try and do that more so I think as much uh and I'll I'll I'll leave on this one strange kind of um point with you we are also getting better at what we need to do in the POC requests so how do we help shape it and that's where I think you see a lot of maturity happening not just with kindrell I think you see a lot of us in the market now maturing how we're engaging them right so if I'm reading what you're saying the right way there was Chad CHP came out November 2022 everyone says we need an AI strategy they throw AI at every problem they have and it makes sense for some folks but for a lot of companies what you're saying is basically hey slow down uh you might not need a generative AI solution here you might be able to fix this with uh more standard automation yeah it's it's it's often the case to start now again the journey is important and I do like the chat GB example I mean and that became even in itself a great opportunity for us to show how agents can help continue to refine and filter responses even sources but everyone thought it would have everything they needed just trying the next the next staunch or the next opportunity with with the product so now that we've established that who is generative AI working for I mean what are the 10% that this actually makes sense for um you know we're seeing you know and outside I was going to laugh and and say not but there's some really interesting things that that are maturing so um and depending I'm you know happy to go into more detail on some of this depending on your questions but if I look at things like some of the Telecom clients that we have um a lot of the work in how they're approaching AI approaching information gathering approaching the analysis um is working very well and how we start to apply again gen looking at procedurals product you know capabilities um you know selling opportunities so how do you go into upsell so I think there's a huge kind of gain that we're seeing in certain industries so they're seeing benefits in that so now they've evolved into cart generation so how do I understand what may be better targeted for you personalization so they've gone from understanding what they built within the workflow they built the the the efficiencies within the workflow the product life cycle and now they're using you know gen to assist within that sale process and now agents involved in personalizing baskets and and shopping carts for you so reducing the time in which to actually get you to sign a contract for a new Service Automotive a little bit differently interesting though um big push on understanding personalization so harvesting a lot of the surveys the interactions the chats and really directing that through so they've really started to leverage models and now micro models to really specialize to Target audiences better and that's influencing supply chain quality so I think the I think there may be a really different dichotomy in where the industries are right you know based on the access to information how they can start to address that and how targeted they were either addressing a client need or in theory like for example a an opportunity to a committed contract type of life cycle right and so everyone's scrambling and they're like where's the ROI and Ai and we see the models getting better why isn't that immediately translated and I think what I'm hearing from you is that each industry is going to have to find the specific ways that it makes sense for them and that's a discovery process yeah absolutely well said now you've given a couple interesting examples and you offered to drill down a little bit deeper so I'm going to take you up on it okay you said it's helping in the sales process for telecom companies can you talk a little bit about what that looks like in practice yeah it's a little bit of a double click in and you know kind of through the process in itself um think about even I mean we're all customers right of Telos in some some uh realm or another one as you start to call in you know getting almost the immediate uh analysis or information on our profiles what services do we have how long have we had the service things like a person would normally talk to you when they're trying to upsell you on a new contract or a contract renewal but it starts to build that knowledge base of information the system can very quickly we start to identify opportunities hey we can reduce your bill you know by X because you're not using these services so can I improve your experience all of a sudden suggestions start to come in and you don't need the same human interaction associated with it and as you start to get into that you can now give scenarios to show you this can reduce your price so instead of you going through this human process and I'm not sure everybody is but if like I'm I'm a customer of one large TCO in the United States they'll call me every year with different scenarios how do I reduce that I wind up having to do the analysis myself they've closed the Gap and many of them are starting to close the gap on how they approach and demonstrate the information of your usage how they can improve your experience improve again the economics associated with your service and how fast they can actually change the service for you they really shorten the life cycle of opportunity and the agents in the process are collecting so I gave you the example earlier around specific types of agents that generate or help lead to Value so the orchestration agent type of concept can now start to break up or Shard the the query to start to pull the information the analysis in so where are you what's your demographic what's your usage pattern what's your bills associated with your usage it brings it to the analysis almost immediately then from orchestration you have the calculations of the opportunity agent how do I give scenarios that may equate to your demographic right your income range right opportunity say family size so we can start now put opportunity analysis in play and then we have presentation agents how do I represent that to you in a language in a presentation not so the personalization becomes very apparent so it really it it it is a um an accelerant and it's an augment to helping to really improve the service right or elongate your willingness to keep the service with them the annuity that they have with you and their in their company I don't if that helped characterize enough for you no it's definitely did and uh it's fascinating how it's just going to exist along every single step and yeah it's going to be I mean if this is what we're seeing today I just imagine that as the models get better it's going to be pretty impressive both for companies who are trying to sell to us and maybe for us as well to be able to you know send our agents out in the world and be like all right what's everybody offering and how do I get the best deal oh man and you probably shouldn't comment it's like the agent War goes in my head it's like the war of the agents in the backg going to negotiate with each other they really are yeah I agree go ahead oh no no I was going to say I I I made the comment before that impact kind of statement there's so many impacts that it's there's such a different Evolution like there's a statement I which I I you I probably get t for saying but we are seeing the greatest evolution of business in our lifetime fundamentally we are and it's it's changing business it's changing how we develop product it's changing how we experience you know Services it's changing job roles right there's so many things that this has implications to and it just it's it's mindboggling in concept most definitely I I agree with you 100% I don't think you're going to get in trouble for that statement I think I agree with it for sure and in fact in the in the book you see behind my shoulder here always day one which was about um basically how all the big tech companies use AI when I was reporting on Amazon I found that they were using automation systems to negotiate with the vendors who were supplying their fulfillment centers and my point of the book was like write this book because big tech companies are going to know what's going on before everybody else does does and if you see what they're doing then you'll have a heads up for what comes for the rest of us and the AI negotiating with vendors I I was like that's crazy that's never going to come for everybody else but as you're saying we're getting close which is amazing absolutely so let's talk a little bit more about some of the things that might happen upon implementation and what you're going to have to deal with on a um on a CTO or technology level or even if you're just like you know working in the trenches and trying to implement and there are some considerations that you need to have and I think the first thing and you sort of have hinted at this a couple of times when you talk about how you prepare companies to go about this journey and what you need to get in order is data because there I mean my mind has been blown about all the different scenarios that uh might occur if you don't take care of your data um could your AI agent or chatbot spit out data it shouldn't could employees you know there've been examples of employes seeing the CEO's emails so talk about from your perspective how data uh privacy data I guess segmentation is important yeah I think um I I probably can't underscore important enough it's Paramount it's it's one of those scenarios where um it's probably the investment area that's the most you know unspoken where the proper grammar is right it's the least spoken about but it becomes the most pivotal right CU one you know data inventory data harvesting classification data access really becomes you know critical um if you can tell by my hairline I grew up in highly regulated businesses I mean you couldn't touch a piece of information without 10 people watching it in that type of scenario but you'd be surprised how many Industries don't have that rigor so understanding data lineage within the process becomes absolutely fundamental so understanding the auditability of how information has been one generated collected propagated been manifested that's been somewhat you know what I call that Foundation element within that securing the information within it so there's been a growth and I think probably almost too quiet but a growth within new technologies that are allowing us to tokenize information and uh in theory you know even you know even protect data in in uh in its construct so how do we actually start to look at data in the true sense that it is so we go from this data inventory data lineage management then how do I understand now how do I can actually protect the information or components of it so understanding so may need to protect your name but I may not need to protect your your race or your gender I may need to protect your your geography but I don't need to protect your usage so even that's now growing within that construct um and the security in itself in and around that has grown exponentially so how we look at that kind of data pattern and access the the next area that we often that we often see into is and this is where it becomes another opportunity on how we build automation but you know how do we build transparency within the data usage for models for the analytics and for the usage themselves so it not only puts the visibility within how the data has been used it also then puts a tremendous like say kind of I say magnifying glass on how it's been adopted so a lot of those things are really started to mature also as we're going through that space But it's a part of it it it also lends itself and demonstrates and exposes a lot of the issues with Legacy so you know the Legacy Tech or the tech that situation we're dealing with um and I don't want to say I could probably spend 10 years on the data issues but the the other we get into that's concerning know it doesn't it right it's a hint htin right worth another podcast maybe with someone with more hair than me but the the thre you get into is data duplication like data manipulation a lot of the this is could be a say strange phrase um but data has become so mutable in the environments from made business because the applications are fragile so what they've done is they layer application services but they don't touch them and what they do is they keep modifying Source data and when you start to see that that is a Cascade so when you when you're looking at hallucinations you're looking at errors in calculations these are often the sources models models can be adapted data if you start with a bad ingredient you're going back to square one in regard to the analysis the Assumption right and the and and the metrics associated with it is that the biggest lift for companies trying to work in the AI space right right now getting their data in in order um I think it's the biggest hurdle yeah it's the biggest hurdle not even a lip it's the biggest hurdle to get past or be able to demonstrate there's enough of an area to prove it because a lot of the areas need to be proven still so boards uh a lot of the SE Suite it's this is there's a tremendous visibility not just on tech for tech anymore but can you know can the technology enable a business opportunity or an outcome so that visibility is the first you know really large speed bump right the the the next lift is scale scale is the real lift the way like how do I if I proved it how do I scale it right so how do you do that um so this is where uh this is where it's kind of the concept of you know uh how do I break it up in a way in which become affordable it's very expensive by the way so getting deep in analysis deep in processing continual real-time model operations become very expensive so a lot of things we talk about is what can be what we can do to either run that right through Partners so how do you come more more Cloud friendly more on demand more scale or El capability um even the examples of our Nvidia work is you know how do we can be better at purposely driven workloads so how do we understand quality of service on a on on a workload does it need the highest performance gpus can it run can it run a different type of memory environments can it run on a slow burn so a lot of things we do is we also will size shape and estimate how we can optimize or architect right the analysis and what data is needed and how to run it because I'm sorry cost matters right the taxi can be fairly large right in the over process I've heard some horror stories absolutely some really bad ones um so talk a little bit about who makes the decisions about uh whether to go forward with AI is it the tra is it a traditional Tech decision or are there new folks in involved in it great question oh it's a phenomenal question I um I can tell you without much hesitation there is a material shift we're seeing a lot more Business Leaders involved in the decision process and I think that comes in a couple of different uh a couple different reasons or I think environmental changes and I kind of made that Evolution comment before um but the consumption model is evolving too there's a lot less appetite for build there's a lot less appetite you know for uh creating Solutions into the business it's it's a needed now scenario so we're seeing more and more so the analysis at least we feel it being we feel it in the sense of we're seeing large the majority of investment being directed through Business Leaders and how we're driving into it um it also shows there's a lot less patience in drift and a lot less patience in a thesis being proven wrong but it is it is a shift um within that within that uh within that operating environment yeah my theory is this is coming from the CEO often who's reading about it in the press and seeing the magic and saying we need to harness that and that's and it's driven from the very top of the company usually I don't know if you just to kind of help compliment like like a year ago we started to look at just for fun the growth of AI being mentioned you know in company announcements right so we all started playing that we saw it in the Martin news and everything else and it just it just went exponential I mean it literally went exponential and then when we went from it being in the news to how many meetings did we have with a a seite that discussed what the strategy benefits could be and we start you it went from you know heads of infrastructure data management people an you know an analyst teams things that then it started to get into Coos and then the CFOs started to ask questions on looking at you know like operating Effectiveness scary stuff for technologist you don't want to ever open up the curtain right um you know and then to your point yeah CEOs became engaged it's like how do I there was a great conversation um probably earlier uh early last year and then to Middle last year a CEO basically said I want to understand the pulse of my business yeah right and and and these are the tools that they want to deploy so it's actually on the one side it's exciting it it's but it's a very different audience it's a very different appetite and it is is turning that directional tide definitely so on the build uh the build I guess you said build is important or what what you say about build people don't have patience for that anymore yeah there's a there's a lot there's a lot less appetite to start building Homegrown Solutions anymore but this is what I need to ask you though because you're you're CTO um there's a debate between you go off the shelf or you go open source so I mean open source gets you kind of half the way there but it's still building on top of it so what do you think is the most effective way to to implement AI is it the off the-shelf stuff or the open source models I um I'm going to underwhelm you and it's the worst it's it's a little bit of both you have to kind of balance what the what the use case is there's tremendous opportunities and and I do think there's an interesting split um in the market there's like and and and I'm sorry I haven't really thought about this too much detail like from an explanation standpoint but there's the embedded tooling so like like the things like the Salesforce conversations or the service now conversations there's a lot of AI That's embedded within the workflows and the information being gathered and a lot of that you're not necessar going to replace that fundamentally so there's there's a combination what you start to get into it but infusing and I'll use that word a little bit differently right but infusing different capabilities within your interaction either with data you know data classification the model development even in how you're starting to create micro models so small language model capabilities to Target your business that's where the I think that's where the I think the real linkage starts to come so it's not as if you're creating again your own model your own infrastructure your own analytics the question is is there enough around how do I create the benefit to both and the area that we see a great amount of energy is in the um integration okay so how how can I leverage can I leverage and we did we did some work on on someone to see or compare or or arbitrate the large language models mhm it's like okay well let's let's understand what you're looking for and let's see you know what parameters they need within the models and you know can you now devolve or basically create a micro model that's very targeted to your business that basically was a recipe so how do I look at what's happening within you know within say chat TP versus Gemini um how do I start looking at llama differently and that wasn't as if it was it was brought from scratch but it was a recipe that kind of created that that integration layer and that's where I think that I think the good energy I think there good energy good results that happened from that definitely no I love I think it's the right way to to look at it definitely 100% okay I I don't want to go without talking to you about your Nvidia partnership sure um you guys are partners with Nvidia um talk a little bit about the nature of the partnership and then also what's it like working with a company like Nvidia they are they are fascinating the way that they operate uh yeah it's um I think we learned something new every every turn I I uh so one you you can't hide they are driving a material growth in the market um and when we look at you know kind of the partnership opportunity you kind of first people think initially oh it's all about Revenue it's all about kind of growth turnover in sales but one you know to our Delight working with them it was really around how do we accelerate opportunity how do we identify and co-create adoption work and and I I I see your smile sortly because even I was kind of like wait this sounds too good to be true I mean I'm not I'm this is there's usually a hook that's going to get you somewhere around the corner um but you know from a I'll I'll probably give you from perspective it is probably one of the best engineering Partnerships that that demonstrates you can colearn you can you know develop and Target opportunities and you can you really can think about how you co-create so a lot of the things that we've started to do and the partnership was based on was can we start to natively extend our capabilities to use their for example Ai and their agent Frameworks can we develop it faster can we use like their interfaces like the Nim interface kind of scenarios where we can speed to market right opportunity and can we Shield our customers from some of the complexity so a lot of that was really at the at the root of the partnership that we're working with them so there's a learning part of that partnership There's an opportunity to kind of uh extend into their platform so like the Nemo concepts with them how do we make available to the environments and we even came back and uh we announced a few months back we had a partnership between Dell and Nvidia on private AI infrastructure so can we actually plan build deploy Sovereign kind of controlled environments and it just became a natural extension so it's really an enabler type of a contract and relationship yeah no I was smiling because uh with Nvidia when you talk about Nvidia you hear the word accelerate within the first 10 words and you mentioned it so I said okay there you go this is working I uh okay so let's go back to basics it is it is kids in the candy shop right right it just one the compute capability the power that you start to to see we're also can I I'm not sure how old we all are I'm definitely old given my my appearance in hairline but it's kind of like you know we're not just the president we're also a client right so you know a lot of our things that we do is manage services we you we manage um exabytes of information across our client base we manage you know thousands of customers a lot of things we do around how do we improve operations so again even ourselves we're looking how to improve that when we start to Advantage our workloads our models on their environment it's an exponential Improvement right in time and and turnover velocity a different term but how do we improve and optimize the models and the and the input accuracy so it it really becomes really interesting it's kind of like throwing a lot of octane in your car really fast yeah you it's interesting right yeah yeah for me someone in my position I'm always asking how real is this yeah it's real it's very real very real so anine if folks want to work together with you or work together with kindrell what's the best way to get in touch um so one uh we can do it multip ways so one we're operating in most countries so we have a we we definitely have an ability to directly connect to most operating countries our operating environments and we can forward you uh for something later on the interview you can come to kel.com we can kind of do that kind of capability right and just reagg we use our own tools to help direct right you know clients the right way I won't say AI tools but it's kind of fun when you have one in that space but uh but we we we are operate we we are operating in most countries that way and happy to engage directly with our customers awesome well Antoine great to speak with you hopefully the first of many conversations I learned a ton and as I said before it's real so thanks again for coming on the on the show great session Alex it's really appreciate it thanks a lot thank you anine and thanks everybody for watching we'll be back on the channel soon