When AI goes down your application shouldn't so let's build a feature that fails over it times out fast and it falls back to a queue Reliability is paramount for any application if our AI is slow Then we need to degrade gracefully because users don't care why it's slow They just want the app to work. So we always need to have a plan B We need to fail over or we need to time out or we need to push things to a queue So that we can have predictable behavior because we have to design for failure failure is gonna happen Rate limits outages. These are all normal things that we have to account for

failure is gonna happen Rate limits outages. These are all normal things that we have to account for but our application Shouldn't just crater when these things happen We need to be sure our application can handle those kinds of things So we are going to create an agent that can fail over. Let's call this product description Agent the idea is going to be that we have a product name We have a target audience and then we have features and then it's going to write a description for us that

We have a target audience and then we have features and then it's going to write a description for us that That's it. That's simple. So let's open up the product description agent and let's start with our attributes And first of all we will have our provider now We have been using lab and then open AI throughout this course and we're still going to do that However, we can also pass in an array to the provider Attribute and this array can have as many models that we want. This is going to give us built in

Attribute and this array can have as many models that we want. This is going to give us built in Failover. So our first option is going to be open AI So right now this agent will use open AI unless if for whatever reason it can't access open AI Maybe we hit a rate limit. Maybe it's down Maybe, you know, it's a network issue Well, then it's going to fall back to anthropic and then we could add another here if we wanted to then fall back to Azure It's going to go in the order of the items inside of this array

here if we wanted to then fall back to Azure It's going to go in the order of the items inside of this array So it's always going to start with our first model. It will fall back to the second model Then it will fall back to whatever model that we specify and it will keep going on So that's our failover order and that's all we have to do in order to have fail over Now, of course, we need to go to our config and we need to set the API keys for whatever providers that we are going to use

Now, of course, we need to go to our config and we need to set the API keys for whatever providers that we are going to use I didn't really plan to do it this way But I'm going to change this up so that's our first is going to be anthropic because I don't have API access to anthropic I do for open AI but not anthropic So I don't have any config set for anthropic. We'll see if this is going to fall over or not Maybe it is maybe it's not so we're going to go back to use the cheapest model because well

Maybe it is maybe it's not so we're going to go back to use the cheapest model because well I like saving money. So we're going to do that. We'll have our max tokens once again Let's have something that's I don't know a thousand just to give us a little playroom there And then our next item timeout. So not only can we fall back using different providers But we can also specify a timeout. This is in seconds So if we timeout going to our providers for 15 seconds

But we can also specify a timeout. This is in seconds So if we timeout going to our providers for 15 seconds Then it's going to fall back and it will fall back to a queue. So right here Of course, we have more than two lines But these two lines are there to start providing resiliency to our application as far as this particular agent is concerned We aren't going to have messages or tools. We don't need structured data or anything like that We just want a set of instructions. So just like everything else before we are going to have our prompt and

We just want a set of instructions. So just like everything else before we are going to have our prompt and Our closing prompt and our prompt looks like this. You are a product marketing assistant Write a clear customer friendly product description use short paragraphs and bullet points We're helpful avoid hype and make claims only from the provided details There's a prompt. There's no other methods This is simply an agent that is going to take whatever Information we give it and it is going to spit out some text and now we need a

This is simply an agent that is going to take whatever Information we give it and it is going to spit out some text and now we need a control So let's make a controller. So let's call this AI Product description controller and let's open that up So let me be your AI agent and I will provide the code. We'll go over it It's ultimately straightforward. So we're gonna be good here So our index method is of course going to be the most straightforward method that we have because we get the user We get the team, but then we get this run ID or at least we're gonna see if

that we have because we get the user We get the team, but then we get this run ID or at least we're gonna see if there's a run ID because well We do if there's a run ID then we get the information for that run so that we can display this inside of the view So here's what's gonna happen the user is gonna fill out the form and they're gonna submit that form There are two paths. There's the sync path, which is the best path That's the fastest path that is the path of we take the information from the user was send it to the agent

That's the fastest path that is the path of we take the information from the user was send it to the agent And then we get a response from the agent and voila. We're done that could fail In fact, we are planning for that failure in which case it's going to have the async path Which is essentially queuing up that agent so that it can be sent at a later time So that's what we're doing here if we have a run that we want to view then That's why we are getting that so that we can view the run if we are waiting for that run to complete all because

That's why we are getting that so that we can view the run if we are waiting for that run to complete all because Everything was queued there. So let's take a look at the store method because this is where all of the fun stuff happens So once again, we get the user we get the team ID then we validate the incoming information now This is a form for creating a product description So the form is gonna have the product name It's gonna have the audience is gonna have the tone that we want as well as any features that we want the description to have and

It's gonna have the audience is gonna have the tone that we want as well as any features that we want the description to have and We can see that you know This stuff is getting validated there because then it's going to be used to build a prompt We can look at that where we have an audience line a tone line and then we build that prompt that we are going to send to the agent So basically we're just compiling all of the information that we want the agents to have and then giving it to the agent That's all it is pretty straightforward as far as that is concerned and then we

agents to have and then giving it to the agent That's all it is pretty straightforward as far as that is concerned and then we create our agent now You know we talked about the sync path the synchronous path and the async path So if we are in async mode, that means there was a failure We need to cue our interaction with the AI. So we are going to cue the agent prompt with this cue Generation we pass in the run we pass in the agent we pass in the prompt and then we tell the user that We will finish this in the background and we will look at this cue generation

then we tell the user that We will finish this in the background and we will look at this cue generation here in a moment But that's only if we are in async mode if we are still in synchronous mode Then we want to use our agent to send our prompt and it's going to do its thing However, if it fails, then we update our run saying that it's cute because you know It could have failed because you know one of the models or or both of the providers weren't Accessible or the timeout or or whatever we are hitting this for some reason

providers weren't Accessible or the timeout or or whatever we are hitting this for some reason So we are going to cue it up and then we are going to add that to the cue We would tell the user that the provider was slow or unavailable. This is que ued for background processing however, if everything was fun, then we just Update our run it succeeded. We check for the usage so that we can log that in our AI usage table And that's that but if we look at the cue generation method So it's accepting the run because we want to be able to update the run which we

And that's that but if we look at the cue generation method So it's accepting the run because we want to be able to update the run which we do first thing We change it to say that it is queued, but then our agent has a queue method It's awesome. It's just right there. So we say that Cue this prompt with this agent and then when that's done use this callback So everything about interacting with the queue at least as far as our agent is concerned is just automatically handled for us The SDK gives us that cue method and it's done All we have to do is supply a callback function for whenever the queue has

The SDK gives us that cue method and it's done All we have to do is supply a callback function for whenever the queue has finished its job and then That's it. So we cue the agent then when it's done our callback function Executes to where we will find our AI run so that we can update it to say that it succeeded and then we log the usage However, if it was not successful then we catch that with our callback function So that we can update the run to say that it failed and that that's it So ultimately it's very straightforward the fact that the agent has a queue method so that it can write it to the queue

So ultimately it's very straightforward the fact that the agent has a queue method so that it can write it to the queue It's awesome. It makes it so easy to use So off screen I added all of the view stuff, but let's add our routes Which I'll just paste in we of course need to pull in our controller class name there and we're ready to go Now before we run this we need to fire up our cue I'm not going to admit that I forgot to do that but I forgot to do that All right, so for our copy here. We go. We are going to Start with time saver that's for parents and the tone will be light-hearted key

All right, so for our copy here. We go. We are going to Start with time saver that's for parents and the tone will be light-hearted key features are what it saves time Uh, you can spend more time with kids because that's what we want with a time saver To spend more time with your kids and then get more rest So if we generate this now, we're going to run into an error We can see that it is queued for background processing great If we scroll down we see what the error is it's an authentication error because yeah, however

If we scroll down we see what the error is it's an authentication error because yeah, however I would have hoped it would have fallen over to open AI And I guess it really depends upon the the type of error as to why it would fall over because to me It doesn't matter why the the error it should fall over, but it didn't If we refresh though, let's see what happened. Yeah, it's the same thing. That 's disappointing I was wanting to see the the fail over. Oh, well, it is what it is So that means we need to change this so that our first is going to be open AI

I was wanting to see the the fail over. Oh, well, it is what it is So that means we need to change this so that our first is going to be open AI Our second is going to be anthropic. So with that done, we'll leave everything else But let's stop our queue so that we can restart our queue and let's try this again So we will get a fresh form here once again time saver parents light-hearted There's our features generate now. So everything Well, we're gonna see if it works or not. I don't see why it wouldn't work But we'll find out we can see that the providers slower unavailable and there's

Well, we're gonna see if it works or not. I don't see why it wouldn't work But we'll find out we can see that the providers slower unavailable and there's an error unknown finish reason and I love this error because it No idea why and at least from my experience. This is because of the tokens the tokens are Not enough So let's stop that so that we can change our tokens So let's be very generous with 15,000 So if we go back, let's restart our worker and

So let's be very generous with 15,000 So if we go back, let's restart our worker and We will do this again. It's a product name time saver parents light-hearted. Those are our features We will generate now And this should work unless if we time out and if we time out then it will get queued But it looks like we we beat it So time saver is a practical light-hearted helper for busy parents who wants to be a bit more breathing room in the day

So time saver is a practical light-hearted helper for busy parents who wants to be a bit more breathing room in the day So great. So that works just fine Now we could test this by changing the time out we could come in here and we can change the time out However, all we are going to see is an error because it will time out after one second And then it will of course go to the queue, which we'll try it again But it'll time out because it's one second So there's no sense in trying that at all

But it'll time out because it's one second So there's no sense in trying that at all But what we could do then is go back to the browser and let's do a completely different Product for your mama and it's sarcastic and our key feature one Key feature two and key feature three So what we're going to do then is generate this in the background so that it will be done Asynchronously There's no error or anything like that. We just decided to hey generate this in

Asynchronously There's no error or anything like that. We just decided to hey generate this in the background This is the code that executes Let's go back to our controller, which I don't remember what it was called the AI product description So right up here where we check if the mode is async That's because we clicked that generate in background So if we refresh well, we timed out which that's interesting Why did it time out but it at least queued it up

So if we refresh well, we timed out which that's interesting Why did it time out but it at least queued it up But it timed out after 15 seconds. That's a long time But anyway, we can see that it at least works as far as queuing it up into the queue But unfortunately it it timed out So the Laravel AI SDK gives us all of the tools that we need For resiliency And the resiliency is all based upon you know relying upon a third party Our application relies upon the AI providers that third party

And the resiliency is all based upon you know relying upon a third party Our application relies upon the AI providers that third party So there's fallback features which is built in for us which It's disappointing that we didn't see that but it does work. I know it works We can set timeouts but then and to me this is the best part It has built in queue support so that all we have to do is call the queue method On the agent and it will queue up that agent to execute later It's everything that we need for a resilient application (dramatic music)

It's everything that we need for a resilient application (dramatic music)