We as humans are impatient, and it gets worse over time, because as technology gets better, as computers get faster, we get more and more impatient. And waiting on things like AI feels slow even when it's fast. So in this episode, we're going to make our interaction with AI feel instant, or at least a little better than a lot. Of course, when it comes to working with AI, the way that we make it feel instantaneous or we improve the experience is by streaming the response.

instantaneous or we improve the experience is by streaming the response. In the previous episode, our chatbot, if you want to call it that, it works. But it could be a little bit better because, you know, it just pops the response right in. So basically, we send the prompt off, AI is going to build a response, but that response isn't streamed, the response is generated, and we sit there and we wait until we get

isn't streamed, the response is generated, and we sit there and we wait until we get the entire response back to poor experience, and we want to improve that. So we are going to stream the response. So we need a place to save the final draft text after the stream completes. So we're going to store that in our AI runs database table. That way, we keep a single audit record for every AI interaction as well as streamed interactions. So let's use artisan to make a migration. And let's call this add output text to AI runs table.

So let's use artisan to make a migration. And let's call this add output text to AI runs table. That's a mouthful to say. But all we are going to do is add an output text to the AI runs table. So right here, we are going to add to our table a text field called output text . And this can be null. So we will call nullable and we're going to put this after error message. All right. So if we add that column, then we need to write the code to drop that column.

All right. So if we add that column, then we need to write the code to drop that column. If we need to roll back the migration, so we want to drop column and output text. So there's our migration. Let's go ahead and let's migrate the database. And that's it. We're done with that migration. However, I do want to open up the AI run model, and I want to add the output text field

However, I do want to open up the AI run model, and I want to add the output text field to the fillable array. And so then it's just a matter of creating a new controller or something that we can use to stream the response. So let's make a new controller. Should this be a service? Probably we're going to stick with controllers is because so ticket draft reply stream controller.

Probably we're going to stick with controllers is because so ticket draft reply stream controller. So the idea here is we are going to add a feature that will allow us to essentially generate a response from AI. So it's going to read the conversation, and then it's going to generate a response that we could then respond to or add to the messages if we want to, or we can just leave it, whatever. So we're going to make this invocable.

leave it, whatever. So we're going to make this invocable. And let's open up the ticket draft reply stream controller. But we are going to reuse our agent. So let's new up the ticket assistant constructors so that we can pass in the ticket ID, which we need the tickets, don't we? So let's go ahead and get that there. And our prompt is going to be a little bit different. You know, we could open up our chat controller.

And our prompt is going to be a little bit different. You know, we could open up our chat controller. And in fact, let's go ahead and do that. So we created our agent here, then we had the prompt that includes the user message. And then we prompted with that. Here, we are going to do this. We want to tell AI to draft a concise, friendly reply to the most recent user message. That's our prompt.

message. That's our prompt. Because that's that's really all that we want to do here. So from there, we're going to create our AI run. So I'm just going to copy that, paste it because it's going to be mostly the same except our feature key isn't going to be ticket chat. This is going to be draft reply, status running provider open AI model, everything else is essentially going to be the same.

everything else is essentially going to be the same. We do however need a use statement for AI run. But if we take a look at the chat controller, then we would prompt using our prompt. We aren't going to prompt here. Instead, we are going to stream our agent object has a stream method. And then we pass in the prompt that we want to stream so that then we will get a streamed agent response and we'll just call it response.

a streamed agent response and we'll just call it response. We want to use run because we will be needing to modify that. So we are going to stream passing in our prompt. Then when we get a response, we will do something inside of this callback function, which basically we're just lift the code that we have here. Let's use this where we are going to update our run. The status is going to be succeeded finished at is going to be now, but this is where we

The status is going to be succeeded finished at is going to be now, but this is where we will take that output text because we now have text that comes from the response. And we will also have information about the provider. Our response object has a meta property, which gives us access to the provider. But we also have information about the model, which as you might guess is part of that meta object. We will access the model and now we store all of that information with the run.

object. We will access the model and now we store all of that information with the run. And then, you know, it's going to look very similar to what we've done before. We're going to check the usage. Let's just grab that and let's paste that in because once again, a lot of this is going to be very similar. We need a use statement for AI usage. So if we have usage, then we are going to set the AI run ID, the prompt tokens comes

So if we have usage, then we are going to set the AI run ID, the prompt tokens comes from our usage completion tokens, total tokens, cost us everything is going to be the same. So that then the final thing that we need to return is the stream. I see the dollar sign there and I just want to use that as S. Never mind. That's my problem. That's not yours. Okay. So let's just go back over this.

Okay. So let's just go back over this. It's very simple and straightforward. So we are creating our agents. We are just using our ticket assistant that we created already. Then we're going to pass in the prompt to draft a concise and friendly reply. Then we create our AI run. We then stream with our agents passing in the prompt. When we get a response back, we have this callback function that will execute. Then we can update our run so that we can check if we have any response.

When we get a response back, we have this callback function that will execute. Then we can update our run so that we can check if we have any response. Well, you will have a response. We'll check if we have usage information with that response. And then of course, set our AI usage appropriately. And then as least as far as the controller's concerns, we will return the stream. So with that in place, let's go to our routes because we need to define yet another route. Let's take just one of these.

another route. Let's take just one of these. We'll paste it in because it'll still be tickets and then the ticket that we are working with slash AI, but instead of chat, we're going to have draft reply. And of course, the controller is going to be a little bit different because it 's tickets, draft, reply, stream, controller, but then we will have our name tickets, AI, and we'll just call it, I guess, draft, reply, I misspelled draft, draft, reply, dot

and we'll just call it, I guess, draft, reply, I misspelled draft, draft, reply, dot stream. Well, let's add stream there as well. So tickets slash and then the ticket ID AI draft reply stream. There we go. So for our streaming demo, I originally thought we would rewrite or at least modify our chat demo to stream this, but I want to leave that as is just for the sake of poster ity.

demo to stream this, but I want to leave that as is just for the sake of poster ity. So we're going to create something that's going to tie into our ad message here so that we will stream a generated response. So we are going to add another part here underneath our AI chat demo. We'll have a little section so that we can let AI generate our response and we 're going to use alpine to do that. So instead of our head file, you know, where we defined our ticket chat demo,

to use alpine to do that. So instead of our head file, you know, where we defined our ticket chat demo, we are going to define another component. So we'll start, of course, with alpine data, and let's call it ticket draft demo. And we want the ticket ID, let's also have, you know, like we had an initial response, we'll have an initial draft, because I like the idea of having that. Of course, it's completely optional.

we'll have an initial draft, because I like the idea of having that. Of course, it's completely optional. So we'll just leave that as is. And then we just need to define our component. So we of course need the ticket ID, but then we need the draft and we will initialize that with the initial draft. So that's where that will come into play. But then we will have our prompt, and then we will have a controller. Now the controller is going to be used to essentially stop the stream.

But then we will have our prompt, and then we will have a controller. Now the controller is going to be used to essentially stop the stream. If that's what we want to do, we will initially, or not initially, we will eventually create an abort controller for our controller, thus it's called controller. And then we want our method that we are going to call. And you know, let's just call it stream draft. So the first thing that we are going to do is essentially set draft to an empty stream. And then we will create our abort controller for our stream.

stream. And then we will create our abort controller for our stream. Now before we go any farther, if you've never streamed with JavaScript, well, we're not going to go too deep into the description there, we're just going to get this done, and we're going to go from there. I'm going to explain as we go, but yeah, it's not going to be an exhaustive look at streaming with JavaScript.

look at streaming with JavaScript. So we're going to get our response, and we are going to fetch, you know what, I don't want to type that out. So we're going to copy and paste. Now in today's day and age, I would actually be using AI to create this code for me. I still write code, I use AI to like, or not to like to write a lot more code than I write.

I still write code, I use AI to like, or not to like to write a lot more code than I write. So you know, I thought about using agents for this because it's now part of my workflow. And it's probably part of your workflow. If not, I highly recommend that you start using agents. It doesn't matter if it's codecs, or cloud code, or whatever, use whatever you want. It's fantastic. It's awesome.

It's fantastic. It's awesome. I love it. Yeah, I won't go back. So anyway, I would use AI to write this, but whatever. Anyway, so we're going to make a request, but we don't want to make our request to tickets and then AI and chat instead, we want that new URL, which was what draft reply stream. So we are going to send a post request to this new URL that we just set up.

stream. So we are going to send a post request to this new URL that we just set up. But our headers are going to be a little bit different. We won't have a content type. That's gone. Except is not going to be application slash Jason instead is going to be text event stream. And then we, of course, need the CSRF token, so that's going to be the same there. Now we don't have a body in this case.

there. Now we don't have a body in this case. We have a signal and that is going to be from our abort controller so that if we ever need to abort the stream, we do so by using this abort controllers signal. So there we go. So we make the request and we're going to get a response, but it's going to be a stream. We need to read that stream. So we are going to create a reader by getting the response body get reader, but

We need to read that stream. So we are going to create a reader by getting the response body get reader, but it's not enough to read it. We also need a decoder, which we will create a new text decoder and then we need a buffer. All right. So we aren't going to get Jason back as a response. We are streaming data. So we are going to read the body and the state of comes in chunks.

We are streaming data. So we are going to read the body and the state of comes in chunks. So we need to read the body. We're going to have to decode it and we need a buffer so that we can keep going . And then comes the fun part while true sounds so, so dangerous, but we have an escape hatch. We're going to get a value and we are going to get done as we read from the reader. So if we're done, we break sounds dangerous, but as long as we get done, then

reader. So if we're done, we break sounds dangerous, but as long as we get done, then we can break out of this loop and we're going to be fine. So then we are going to decode our data here and we do this using our decoder to decode the value. The value here is a binary value. Stream is true, which essentially tells the decoder that more chunks are coming .

Stream is true, which essentially tells the decoder that more chunks are coming . So don't assume that this is the end. More is coming because we won't reach the end until we're done until the server either closes the stream or we abort out of the stream. So we are going to build this buffer so that we can break it up into parts because these are coming in chunks and those chunks are going to be separated by two new lines.

are coming in chunks and those chunks are going to be separated by two new lines. So we're going to split by two new lines. So we're going to keep the last incomplete chunk in buffer because we want to process only complete events, which is what we're going to do here. For every part of parts, if the part does not start with data, then we essentially want to continue because there could be some other server sent events that start with event or

to continue because there could be some other server sent events that start with event or ID or retry or some other preamble. We want only the ones that start with data so that then we can get the payload to where we want to replace that data preamble with an empty string. Then we want to trim it. Now, if the payload is equal to done, then we will return. Otherwise, we need to try to parse this payload. So we will get the event by Jason parse payload.

Otherwise, we need to try to parse this payload. So we will get the event by Jason parse payload. And if the event type is equal to text delta, then we are going to build the draft with that delta. However, if we get an error, then let's just write that out to the console. And that's that. So we are streaming events from the server. Those events are separated by two new lines. So it's possible that we end up with an incomplete event.

Those events are separated by two new lines. So it's possible that we end up with an incomplete event. So we want to hold on to that and we want to only work with a full event. We are only concerned with data events. We will ignore all of the others and then we work with that data. If we get an error, we display it otherwise, we just keep reading. All right, so that is our stream draft method. I'm following this line down because we have a lot of curly braces because we 're not done. We need to cancel the stream, possibly, in which case, we will say controller

're not done. We need to cancel the stream, possibly, in which case, we will say controller abort. And then we want the ability to take the draft and insert it here into the add message. So let's define a method called insert into reply. So here we need to let's open up the, not the body. I keep wanting to call it the body, but it's the show. Let's find that message body, let's see. Right here is write a reply.

Let's find that message body, let's see. Right here is write a reply. Do we have anything to identify this? No. So let's do this. Let's add a data ticket reply attribute. That way we will be able to access that here. So that's we'll get the reply box with documents query selector. We want the element with the data ticket reply. Is that what I used?

We want the element with the data ticket reply. Is that what I used? I think that's what I used data ticket reply. Yes, that is our text area element. So that will say reply box value is equal to the draft. And yeah, it should be it. So now we just need to add in our form. And I want to do this completely before this area with the add message. So I guess it's going to be right here. I'm just going to paste that in just like we did before.

So I guess it's going to be right here. I'm just going to paste that in just like we did before. I mean, because a lot of markup, here we have our X data for our component. Here we can see the draft. Here's the button to start the draft reply. It will execute dream draft or a stream draft when we click cancel. It will cancel the stream and then we can insert the reply there. There we go. So let's give it a shot. Let's refresh here.

So let's give it a shot. Let's refresh here. We will have an error, which if we scroll up and handle the match case secondary, we're going to use secondary. Let's look at the code. So somewhere second, oh, that's where I used secondary. Let's use ghost. So a primary ghost and ghost. So if we refresh now, that error should go away.

So a primary ghost and ghost. So if we refresh now, that error should go away. We have our AI chat demo, which we know works, but now we have draft reply. So when I click on draft reply, that kind of came in all at once. It streamed. I promise you it's streamed. And ideally, we, we, if we had more text there, we would see more text, but it worked. We can insert into reply and then we could post that message if we wanted to. And then that would just be added to the conversation.

We can insert into reply and then we could post that message if we wanted to. And then that would just be added to the conversation. So it didn't look like it streamed, but that's only because it's a very short. It's a syn, two sentences, but it's streamed. I promise you it's streamed. And if you have more texts, you will definitely see that.