All right. Welcome to Section 2. Let's turn it up a notch. Have a look. So here's where we left off at the conclusion of Section 1. We have support for custom agent classes like the one you see here. We can specify system instructions and the tools that are relevant to it. Next, we have a variety of these tools for interacting with our file system. That way the agent can say, "Hey, can you read this file for me?" Or, "Hey, I want to update this file like so."

can you read this file for me?" Or, "Hey, I want to update this file like so." And then we have a handful of artisan commands, mostly, for example, that function as our chat GPT of sorts, so that we can talk with our agent. Okay, but now clearly, there are so many things that we didn't account for. And I'd like to start tackling some of them one by one. So here's what I want to do first. Take a look. In our agent class, you'll remember we record the history.

Take a look. In our agent class, you'll remember we record the history. This is the conversation. I said this, then the agent ran that tool, and then the agent responded with that. And then I responded with this. So you can see when I prompted, we pushed to the history, we run the model, and then we update the history with the response, right? And this is fine, but as you can imagine, what about a really long conversation?

And this is fine, but as you can imagine, what about a really long conversation? Could we blow out our context at some point when the history is thousands of items? Well, at the time of this recording, absolutely, yes. So what we should probably do right now is limit it. We need some way to say, okay, look, once the conversation gets too large, we need to compact it. Or we at least need to take everything you talked about earlier in the conversation

Or we at least need to take everything you talked about earlier in the conversation and summarize it into something quite a bit more condensed. So how exactly do we do that? Well, there's a number of ways. What I think we should do, and following along with Laravel AI, which I'm sort of prepping you for, so that when you do use that package, everything will just sort of make sense because there are many similarities. Okay, what I think we could do is use an attribute. So imagine we have our chatbot agent here.

Okay, what I think we could do is use an attribute. So imagine we have our chatbot agent here. You're a bit of a jerk, this is our sarcastic chatbot, and it has access to these tools. Okay, what if we also added an attribute that said, okay, well, once the conversation exceeds maybe 20 messages, we should summarize them. Now, actually, on that note though, before we start, give me 30 more seconds. We have two choices here for compacting. We could compact based upon the number of items in our history, which is somewhat naive, but it'll probably be fine.

We could compact based upon the number of items in our history, which is somewhat naive, but it'll probably be fine. Or we could summarize based upon the token usage. So imagine I could have a single item in our history that is a massive text blob. That's clearly going to be very different than an item in our history that says hello, right? So maybe a more mature approach would be to focus on token usage and base our compacting on that. We're going to keep it simple though.

base our compacting on that. We're going to keep it simple though. We're going to instead base it upon the number of items in the history. If this array is 20 items, then maybe that's going to be our signal to compact it. Cool, let's go. Now, if you've never used a PHP attribute before, it's kind of cool. We can reference it like this, and let's just create a class name. What would be a good name? I believe the Laravel AI package is something like max messages, maybe.

What would be a good name? I believe the Laravel AI package is something like max messages, maybe. I'm not sure. We're going to do something like how about compacts after, and then we specify the number of messages or items effectively. So if I had something like this, which is way too low, but just it'll be good for the illustration, then we can say, all right, well, when there are three items in the history, we need to basically compact those messages down into something much more shorter that summarizes the gist of the

messages down into something much more shorter that summarizes the gist of the conversation. In real life, you might have something like 30 or 20. Once we have 30 items, let's compact. Okay, we'll leave it at three so that we can see the result very quickly. All right, but right now it's saying, hey, this attribute doesn't exist. What do you want to do? All right, let's fix that. Within our AI directory, maybe we could add something like attributes.

All right, let's fix that. Within our AI directory, maybe we could add something like attributes. That would be fine, and let's create this class. I'm going to put it within the attributes directory. And now what's cool about these attributes is they're just PHP classes effectively. So I can have a constructor here, and this is going to accept what we pass through here. So what does three represent? That's going to be our threshold, basically.

So what does three represent? That's going to be our threshold, basically. So public, int, and I'll call it threshold. And if you want, this can even be read only. All right, let's close this out. And then finally, I'm going to add one more thing here. I'm going to say attribute. And I'm basically going to specify here, what is it? I don't do these all the time, attribute, target class. Yeah, this is our way of saying, look, this is an attribute that is intended

I don't do these all the time, attribute, target class. Yeah, this is our way of saying, look, this is an attribute that is intended to be applied at the class level. So attributes can be applied at the method level, the parameter level. In our case, I want it to be applied directly to the top of a class. And we can specify that we want that here. Cool, so now if I were to switch back to our chat bot agent, I can go ahead and import that. And now it's not going to do anything yet, because remember, this is just meta information, right?

And now it's not going to do anything yet, because remember, this is just meta information, right? It's not being executed. But we can read it and then execute based upon what we find there. That's what's cool. So right now, we are signaling that look, when the history gets to three items, which is very, very low, we want to compact. And if we take a look at that, the only thing it does is store what that count is, what that threshold is.

is, what that threshold is. So at this point, it would be three. Okay, so now it sounds like within our agent, we need to read this, apply a default if it's not available. So for example, if this isn't here, we still need a threshold. But if it is here, then we use your default. Cool. So next, let's go into our agent and let's figure out where we should do it. Maybe when we prompt, how about this?

So next, let's go into our agent and let's figure out where we should do it. Maybe when we prompt, how about this? So right here, before we run the prompt, we have to figure out if we need to compact, right? So maybe we need to compact at this point, maybe we don't. So with that in mind, why don't we call it like maybe, maybe compact? That's kind of cool, right? I like sometimes using maybe within a method name. So let's create that method down here at the bottom. And how do we determine whether we need to compact?

So let's create that method down here at the bottom. And how do we determine whether we need to compact? Well, we need to figure out what the threshold is. Then we need to read the items in our history and figure out if those exceed the threshold. And if so, we have to shorten it. We have to chop it up in some way, we have to summarize it in some way. So one thing at a time. First, let's read that class and we can do it via the reflection API. So a new reflection class.

First, let's read that class and we can do it via the reflection API. So a new reflection class. And we're just going to read this because, remember, agent is the parent of our chatbot class, so we'll read that. And then let's get the attributes. But specifically, I care about compacts after our new attribute here. OK, let's name that attributes. And then I'm just going to die and dump this so that you can see the output directly.

And then I'm just going to die and dump this so that you can see the output directly. OK, so we can see now when we prompt our agent, the very first thing we're going to do is just run maybe compact. Maybe compact will read the attributes and instantly kill the execution. Let's go. PHP artisan agent, hey, there. Yeah. So we read our compacts after attribute and we can see that the threshold is

Yeah. So we read our compacts after attribute and we can see that the threshold is set to three. So maybe what we could do is say, well, let's see if you applied that attribute to your subclass. If so, our config is basically going to instantiate that class. Otherwise, we will use it a fault. So do we have any attributes at all? Did you apply it?

So do we have any attributes at all? Did you apply it? If so, and I've already beat me to it, we're going to call a new instance and that's basically going to say, all right, let's instantiate that and make sure we send through the threshold there. Otherwise, I'm just going to do it myself. And maybe we'll hard code 30 or we could extract 30 into a property or something.

And maybe we'll hard code 30 or we could extract 30 into a property or something. Okay, so now at this point, we have an instance of our compacts after so that we know what the threshold should be. Cool. So I could say if config threshold is less than the number of items in our history. Yeah, so imagine our threshold is three. And now at this point, we have four items in the history.

Yeah, so imagine our threshold is three. And now at this point, we have four items in the history. We can say, all right, is three less than or equal to four, if so, then we need to compact this. Because now your history is more than what your threshold is. Okay, so we need to compact. So let's do this. Let's just dine some and say we need to compact. Cool.

Let's just dine some and say we need to compact. Cool. So let's try it again, PHP artisan agent, hey, it's going to respond. So now the history is two items, right? What's up with you? And now it's three and let's try one more. Nice. There we go. So at this point, we have four items, I guess that is now above the threshold. So we need to compact this.

So at this point, we have four items, I guess that is now above the threshold. So we need to compact this. And again, three is way too low. We're just doing this for rapid feedback. So we have a couple of ways to deal with this, maybe for an option kind of rough and dirty, but maybe representative of real life is you just chop off the oldest item in the conversation, right? So let's say the threshold is 10.

conversation, right? So let's say the threshold is 10. And now you have a new item in that conversation that brings it to 11. You would literally just delete the very first item in the conversation. You'd unset it, right? And that way you're constantly just kind of forgetting what you talked about a really long time ago. That would work. Another option would be once you reach this threshold, we're going to compact

That would work. Another option would be once you reach this threshold, we're going to compact it down, summarize everything that happened in that conversation, and then reset the history to exactly one item that contains that summary. And that's what we're going to do in this case, but you could, you could do either one if you want. Okay.

if you want. Okay. So we need to compact and summarize. Now we're going to use AI to summarize this conversation. We'll basically take all of the history up until this threshold and say, all right, here's what we talked about. Can you summarize this down into the important parts? You will give me a response, and then I update the history with that summary, and then we

You will give me a response, and then I update the history with that summary, and then we start again. Cool. All right. So I'm going to paste this in. This is just our standard usage for submitting a post request to open AI. Later, maybe we can extract this into a dedicated client class. That way inside we can just say, all right, this client, and we're going to post to it,

That way inside we can just say, all right, this client, and we're going to post to it, right? But for now we're going to keep it in line. So the instructions are very important. This is where we need to tell it, what is your goal? I want you to summarize this history for me. So maybe something like this, kind of wordy. Summarize the following conversation history concisely, preserve the key facts and decisions,

Summarize the following conversation history concisely, preserve the key facts and decisions, tool results, unresolved questions, omit pleasantries and redundant exchanges, right? Okay. So now I need to send through the messages, and the messages would be the history at this point. Here's all of the messages. I want you to summarize that down into a single text response, and then I will

Here's all of the messages. I want you to summarize that down into a single text response, and then I will receive that here, and then once again, just die and dump the response. And I'm sorry, not messages. We want it to be input. Sorry. I think it used to be messages. All right. Let's give it a shot.

All right. Let's give it a shot. Be a tree artisan agent, hey, call me Jeffrey. And you are Bob for this session, and he'll say, okay, I'm Bob, what are we working on? Probably. Yep. We are writing some code. At this point, when I submit it, it's now going to compact. So if we take a look, all right, summarize the following conversation history.

At this point, when I submit it, it's now going to compact. So if we take a look, all right, summarize the following conversation history. It does, Jeffrey, so notice it just breaks it down into the important bits that it needs to remember. All right. We want to call him Jeffrey. Jeffrey specified that Bob is the name. You get the idea. In this case, it's, I don't even know if we're compacting that much, but you

You get the idea. In this case, it's, I don't even know if we're compacting that much, but you get the idea in real life when you have 20 questions or 30 questions or pieces of dialogue that will be summarized nicely. Okay. So now you'll see we abort. Let's not abort. Instead we can, if we come back up, don't forget, we need to go into output,

Let's not abort. Instead we can, if we come back up, don't forget, we need to go into output, first item, content, zero, text, and I think that's it. Cool. So that can be our summary. Okay. So now at this point, we're just going to update the history. The history is now going to be our summary. So the role is going to be the user and the content and maybe what I'll do here

The history is now going to be our summary. So the role is going to be the user and the content and maybe what I'll do here is make it crystal clear that this is a summary of a previous conversation, so I could do something like this earlier, conversation, summary, and then let's insert the summary there. Cool. So now once again, I just want to keep showing you this so that you can see exactly what's

So now once again, I just want to keep showing you this so that you can see exactly what's happening here. I know maybe I'm being a little verbose, but that's okay. Helps me to keep seeing the output. Hey there, I am Jeff. You are Jane for this session. I need help with fourth grade homework for my daughter. It's starting to get hard helping my daughter to bring some stuff from like, this is not

It's starting to get hard helping my daughter to bring some stuff from like, this is not at all how I learned how to do this math. So I don't know what's going on. She will solve division or multiplication in a way that I cannot comprehend. It just makes no sense. Anyways, and I feel like I'm good at math. Anyways, so notice at this point we are compacting and now the history which was four items has not been replaced with a single item where we can say, all right, they are

was four items has not been replaced with a single item where we can say, all right, they are identified as Jeff, I'm going to be Jane, and that's it. Now remember at this point it does not mention the fourth grade homework because we are dying and dumping before we even get to the point where I am prompting, right? So this is where it would say, I need help with fourth grade homework, cool. So now think about it. We summarize it, we update to the history, and now we continue on just like we

So now think about it. We summarize it, we update to the history, and now we continue on just like we were before. So at this point history would be the summary and then we say, I need help with fourth grade math, kind of choking by the way, and then we would run the model with that new summary and the request for fourth grade math. And then when that reaches the threshold again, we would run this on the next iteration, all

And then when that reaches the threshold again, we would run this on the next iteration, all right? Do we need to compact? Yes, so let's summarize everything, replace the history, and then you just keep doing that over and over. And again, this could be entirely dynamic based upon the maximum token usage she want to reach for.

she want to reach for. But in our case, we're just limiting it to the number of items in our history array. Okay, so now at this point, if you want, some of this could even be a custom class. But what we might do is extract this into, where's my extractor, extract, extracted into a method like should compact, should we compact? I love in the Ruby world, by the way, you can do like question marks as part of

a method like should compact, should we compact? I love in the Ruby world, by the way, you can do like question marks as part of your method names, which reads so much better to me. But anyways, if we should compact this config, then let's do it. So maybe we could say all of this can be extracted into a method, compact. All right, let's see how we're doing here. So compact makes the request, it sends through the history, potentially we just accepts the history here.

accepts the history here. I'm not sure if we need to do that, but maybe it wouldn't hurt. We should handle potential exceptions, of course, as well. But yeah, if we come back, we could send through the history at the top level. All right, so maybe compact, should we come back, if so, then compact and send through the history. Finally, this section right here, this is clearly reading the attributes, right ?

Finally, this section right here, this is clearly reading the attributes, right ? But also, we're just kind of figuring out what threshold is and the pathway to get the threshold is through an attribute, but that doesn't matter. So yeah, maybe can I extract this into a method, get threshold. And then that would give me the threshold value instead, I would send that through should compact simply accepts the threshold, it doesn't need the whole attribute, unless it did.

compact simply accepts the threshold, it doesn't need the whole attribute, unless it did. And right now, I don't think it does, and I would simply do this. Yeah, so let's come back, take a look at this, get threshold should return, let 's clean this up, return config threshold, that's going to return the int and put this on its own line, that looks good to me. All right, maybe tweak some of the names a little bit when I have some more time, but

All right, maybe tweak some of the names a little bit when I have some more time, but that's okay. So let's run through it again. When I prompt, we're first going to check, do we need to compact? Maybe we need to compact. So let's run that method. We're going to figure out what the threshold is, and then we're going to ask, all right, do we need to compact vase upon that threshold?

all right, do we need to compact vase upon that threshold? And actually, on that note, should compact could maybe contain the fetching of the threshold? What do you think about that? This is the fun part of coding that we lose with AI, by the way, is just trying things on and saying, do we like that? Maybe, maybe not, that makes it less flexible. This is the fun part of programming that I think we're losing.

Maybe, maybe not, that makes it less flexible. This is the fun part of programming that I think we're losing. So anyways, we could do that. We could even say, let's inline that, let's run pints. Do we need to compact? I don't know. Let's get the threshold, and are we less than the total number of items? If so, yes, that would allow us to get rid of that. I mean, this is the fun of programming, in my opinion. Little tiny decisions that don't matter, and yet at the same time, very much do

I mean, this is the fun of programming, in my opinion. Little tiny decisions that don't matter, and yet at the same time, very much do matter. Cool. So maybe compact? Well, should we, if so, then go ahead and run the compaction. The compaction is going to make a request where we currently inline the HTTP call, which is fine. We get the output, and then we rewrite the history.

is fine. We get the output, and then we rewrite the history. All right. HTTP artisan agent, but actually, real quick, I just thought within compact, I want to show you the history. Otherwise, it's kind of hard to tell that compaction to place. So yeah, compacted conversation to this, and then if I scroll real quick, should we compact? This is wrong.

should we compact? This is wrong. If the threshold is less than the count, if it's equal to, it shouldn't. Yeah, that makes sense. If we have three items and the threshold is three, that's fine, but on the fourth iteration, we should compact. I think that's right. Maybe not. Let's give it a shot.

Maybe not. Let's give it a shot. We are just an agent. My name, this stuff is hard. My name is Jeffrey. You are Bob for this session. Cool. Help me with fourth grade math. All right, so it compacted the conversation. We have a summary.

All right, so it compacted the conversation. We have a summary. All right. So now we know my name. Remember, at this point, when it replies, it doesn't have call me Jeffrey or you are Bob. It just has a summary that says in a previous conversation, he asked you this, okay? Math is wildly different in fourth grade compared to how I learned in the 90s.

okay? Math is wildly different in fourth grade compared to how I learned in the 90s. Every parent, by the way, if you don't have kids, once you do have kids and they reach fourth grade, they're just going to show you a way of solving division that in no way resembles the way that you have done it in your entire life. It's kind of strange. Yeah, I get it. Fourth grade math can feel a very different, blah, blah, blah.

Yeah, I get it. Fourth grade math can feel a very different, blah, blah, blah. Okay. Ready to help me. I'm just trying to trigger the compaction. All right. We got it at this point. So now history, once again, is reset to a single item where we have the summary . And by the way, if you want, you can even make the summary more condensed,

. And by the way, if you want, you can even make the summary more condensed, where you can say it can be a maximum of not this many characters. So in this case, I think it's more verbose than it needs to be, but once again, that would be fine at the point when we have 30 items in the history and then trigger the compaction. But yeah, all of the important bits are here.

compaction. But yeah, all of the important bits are here. All right. Yes, I'm ready. Tell me what areas of math you need help with. So you get the idea. And now we've effectively avoided the situation where the history can graduate to a thousand items long and blow up our token usage. So that solves that particular problem.

items long and blow up our token usage. So that solves that particular problem. In the next episode, let's move on to something else.