Installing Redis for Horizon0:28
There's also a great book by Mohammed Saeed and his experience with queues. I highly recommend checking out both if you want to learn more about queues. So for Horizon, you have to use Redis as a connection for your queues, so you have to make sure you have Redis installed on your dev machine and also your production server. So on a Mac, you can just do brew install redis if you want to install it, and you can just do redis-cli ping to check if it's running, and you'll see pong if it's working. So we also need Redis to work within Laravel, and there are two ways to do this. So the default way is using phpRedis. So if you look at the config/database.php file, you'll see a section here for Redis right here, and you can see it's defaulting to phpRedis.
So if you look at the config/database.php file, you'll see a section here for Redis right here, and you can see it's defaulting to phpRedis. So on a Mac, you have to use PECL to install it. So if I do pecl list, you'll see that I already have it installed. And you also have to go into your php config and set it up there. So php -m for the modules that are running, and let's just look for Redis. And there you see it there. So just Google how to install it. I did it once, and I've had it working ever since. So I can't really show you here, but like I said, this is the default way of using it.
I did it once, and I've had it working ever since. So I can't really show you here, but like I said, this is the default way of using it within Laravel. Now, if you can't get that working, you can just use the Predis package. So composer require predis/predis, and you can just use that. So once you do that, just change this to Predis, and everything should work. But since I have phpRedis working, I'll just stick to that. So let's just make sure everything is working within Redis. So I'm just going to go to my routes file, and let's just set some Redis keys here. So Redis::set('name', 'Andre'), and let's grab that.
So I'm just going to go to my routes file, and let's just set some Redis keys here. So Redis::set name, Andre, and let's grab that. So $name equals Redis::get name, and there's just dd($name) to make sure everything's working. Let's make sure to import Redis. Okay. And let's see if this works. Okay. So it does work. Okay.
Queueing Email Example2:45
So it does work. Okay. Now that we have Redis working on our machine, let's set up a typical queue example. So let me get rid of this. And I already have it set up here, and it's just sending an email here, because email generally takes a few seconds to send. So I have this OrderShipped mailable OrderShipped, and I'm also sleeping it here to emphasize how long it takes. So in my welcome.blade.php, I have a form right here that sends the email and hits that post route in my routes/web.php file.
So in my welcome.blade.php, I have a form right here that sends the email and hits that post route in my routes file. I'm also using this Mail client, MailHog to see all the local emails I send out in development. So let's go ahead and try this. So back here, sendEmail, and you can see how long it takes after I click. So I'm going to click it now takes about three, three to four seconds. And you see that new mail here, and there it is. So if you want to throw this on the queue, you usually want to make use of jobs. But for the Mail facade, you can just do Mail::queue instead of Mail::send.
So if you want to throw this on the queue, you usually want to make use of jobs. But for the Mail facade, you can just do mail()->queue() instead of mail()->send(). So let's do that. And now you'll see that when I hit sendMail(), the server will respond pretty quickly, but we won't get the email. So I press it. So that did work because I didn't change my queue connection to Redis. So let's change that in our .env file. Let's look for QUEUE_CONNECTION here. And by default, it's sync.
Let's look for queues here. And by default, it's sync. So let's change it to Redis. And now I'm gonna try this again. So let me refresh. And now this should return right away, but we should not get the email. Okay. So we're not getting the email because we threw the task on the queue, but we didn't instruct any workers to go ahead and process the queue. So to do that without Horizon, we can just do php artisan queue:work.
instruct any workers to go ahead and process the queue. So to do that without horizon, we can just do php artisan queue:work. And that will fire up a queue worker and that will process the task. So you should see an email being sent out here. There we go. So that's a very basic example of using queues. So what's the problem with this? The problem is it's hard to monitor all the queue jobs that are occurring in the application, including any failures that can occur. It can also be pretty confusing to set up multiple queue workers and managing all of
Installing and Running Horizon5:05
including any failures that can occur. It can also be pretty confusing to set up multiple queue workers and managing all of that as well. Now with Horizon, you're given a beautiful UI dashboard that you can use to manage all of your queue jobs. So let's go ahead and install Horizon and make use of that. So let's go to installation here and just composer require it. And we also need to install it. And this will generate a config and a service provider, which we'll also take a look at. Okay, so let's go ahead and install it.
And this will generate a config and a service provider, which we'll also take a look at. Okay, so let's go ahead and install it. So we have a service provider, some assets for the UI in the public folder, and a config. So let's take a look at the service provider. So HorizonServiceProvider, and there are only a few things in here. So some notifications to monitor your queues, which we'll take a look at later. I want dark mode. So I'll uncomment that. And this gate is for who can have access to the dashboard in non local environments. And again, we'll take a look at this in a later video.
And this gate is for who can have access to the dashboard in non local environments. And again, we'll take a look at this in a later video. So if we go to our dashboard, let me just go to my app here, let me open a new tab and /horizon. And there is the dashboard. And let me just clear out this stuff, which I had for the demo app. Okay, so I clear everything out. But you can see that the status is inactive here. So to use horizon, we have to do php artisan horizon instead of php artisan queue work. And that's going to fire up our workers and whatever queues we have set up in our config.
So to use horizon, we have to do php artisan horizon instead of php artisan queue:work. And that's going to fire up our workers and whatever queues we have set up in our config. And now you can see that it's active with this many processes, and this many jobs on this specific queue, and some information on the server here. So if I send that email again, let's go ahead and send it, send get thrown on the queue, you can see the app returns right away. And you can see the email come in. And you can see information for that job here, jobs past our completed jobs goes here. And you can see information about that specific job. So let's take a look at some of the config options here.
And you can see information about that specific job. So let's take a look at some of the config options here. So let's go to config/horizon.php. And this is for if you're running on a subdomain, which we're not, this is the endpoint. This is the Redis prefix. This is what classifies as a long time for a queue, which we'll take a look at later on. So this is information about how long you want to persist your jobs. So this is in minutes. So recent jobs is 60 minutes, I like to set this to a day.
Dispatching Jobs to Queue8:32
This is how many snapshots it keeps. This is fast termination memory limit. And we'll take a look at configuring different queues and balancing strategies later on as well. So let's make a new Job and a route that can dispatch a bunch of jobs at once. So let's go ahead and stop this. And let's make a Job and by default jobs are queueable. So php artisan make:job SomeJob. And let's make a few Users. So I'm going to go to my database seeder.
And let's make a few users. So I'm going to go to my database seeder. And let's say let's create 10 users here. And let's go ahead and migrate that. So php artisan migrate, okay. And for the job, I want to take in a User since that's our only model. So let's go ahead and do that. So __construct, say $user. And let's type into here. And let's just fix the indentation.
And let's type into here. And let's just fix the indentation. Okay, and now let's go to our routes file. And let's make a new route here. Just make it a callback. Let's call it jobs. And it's taken a number of jobs that we want to dispatch. And it's also taken the userID we want to pass through with each job. So we have a job and a user. Okay, and let's just grab that user from the database.
So we have a Job and a User. Okay, and let's just grab that User from the database. And I have 10 Users, so we can just do User::find(). And let's do a for loop here. So $i is less than $jobs. And let's just dispatch those jobs. So SomeJob::dispatch(), and we'll pass in the User. And that should work, hopefully. Sorry, this should be $jobs. Okay, so now we can make use of this route to dispatch a bunch of jobs.
So for two seconds. And let's also log something so we can see that it worked. So log info. Let's make sure to import log. Say, job complete with user ID of $userId, or this $userId. Okay, so we can try this. Back to our browser. Actually, let's make sure our queue or horizon is running. So php artisan horizon. Okay, so horizon is running.
So php artisan horizon. Okay, so horizon is running. Take a look at the dashboard here. It's running. Okay. Let's refresh that, actually. Yep, it is running. And I forgot to show you this button, which auto-reloads everything. So you probably want to turn that on. And now we can throw those jobs on the queue. So I'm throwing 50 jobs.
And now we can throw those jobs on the queue. So I'm throwing 50 jobs. And let's see if horizon picks that up. User not found. Let's import that. User not found again. Let's make sure to import it here. And some job as well. Okay, let's try again. Null given.
Okay, let's try again. Null given. I think I did not seed my database. So let me stop this again. Let's php artisan migrate --seed fresh with seeding. And let's run php artisan horizon again. And now this should work, hopefully. Horizon is up and running. And this should work now. So 50 jobs.
And this should work now. So 50 jobs. There we go. I didn't return anything. But if you look here, 51 jobs. And you can see pending jobs here. And you can see them clearing as they're completing. So they will go here as they finish. So again, Horizon gives you a nice dashboard where you can monitor everything and keep an eye on your queued jobs.
Handling Failed Jobs13:03
So again, horizon gives you a nice dashboard where you can monitor everything and keep an eye on your queued jobs. Okay, let's take a look at failed jobs. So sooner or later, one of your jobs is going to fail. And let's take a look at what that will look like. So in our case, I'm just going to force an exception. But obviously, different types of failures can happen. So let me stop horizon here. And let's go ahead and just force an exception in our code. So in some job, I'm just going to force one down here.
And let's go ahead and just force an exception in our code. So in some job, I'm just going to force one down here. Oh, and if you want to see the logs, just check out there about that log. And you can see some errors here, but all the logs for the jobs here. Okay, so back to some job. Let's just throw a new GeneralException here. And say there was an error. Okay. And now, if I run horizon again, this will appear in failed jobs. So let's run horizon.
And now, if I run horizon again, this will appear in failed jobs. So let's run horizon. Go ahead and run this. Let's run it two times. Okay. And now, you'll see failed jobs past seven days, two of them here. And you'll see information for them here and the stack trace. So if you look up here, there was only one attempt. And sometimes, you want to have multiple attempts for your failed jobs. So this is a setting in your queue config, I believe.
And sometimes, you want to have multiple attempts for your failed jobs. So this is a setting in your queue config, I believe. So if we go to config/queue.php and scroll down somewhere here for Redis. Actually, the tries is in your horizon config, if I'm not mistaken. Yeah, it's right here. So tries. And you can override it for different environments, but this is the general one. So let's say let's try three times. And in our queue config, we can specify how long we want to wait. So retry_after, let's say, five seconds.
And in our queue config, we can specify how long we want to wait. So retryAfter, let's say, five seconds. So now, if we try that again, we'll see that we'll have three retries here, and they happen five seconds apart. So make sure you stop your php artisan horizon every time you make a code change. Let's run that again. Okay. And now, let's try that route again. So I'm going to refresh. Okay.
So I'm going to refresh. Okay. And you can see in pending jobs here. And eventually, they will fail. So if you take a look at the dashboard, four jobs failed, failed jobs. And now, you can see it attempted three times here. So another cool feature about failed jobs is that you can retry them. You can see a retry count here, and this is the button you do to retry it. So for example, if it failed initially because you're using a third-party service and that service was down, and now it's back up,
So for example, if it failed initially because you're using a third-party service and that service was down, and now it's back up, you can just press this button, and it should work. But in our case, we're forcing an exception. So let's just take that out. So again, let's stop this. And let's go back to our some job. Let's comment that out. Let's run php artisan horizon again. And now, if I retry this job, it should pass.
Tags, Metrics, Batches, Balancing16:46
you're not monitoring any tags. If you look at the completed jobs, you'll see that there are some default tags here for anything you pass into this job. So in this case, it's just app, models, User, and the userId. So if I throw on some more jobs with a different userId, you'll see it reflected in the tag. Okay. And you can see the two here. So if you want to keep an eye out on certain things that are happening in your application, say, for example, userId two is having some issues,
So if you want to keep an eye out on certain things that are happening in your application, say, for example, User id 2 is having some issues, you can monitor that tag. So let's say app/models/User, colon, the User id. We can monitor that. And you can see the jobs as they appear. So let's add two more jobs. And they should appear here when they're done. Cool. Now, you can also override the tags if you want to name it something else.
Cool. Now, you can also override the tags if you want to name it something else. So we can go back to some Job. And this applies to anything that's queueable. You can add a tags method here. So let me add that. And we can just return an array of tags you want. So return. Let's say we want the someJob tag. And let's say we also want user with lowercase, colon, and then the userId.
Let's say we want the some job tag. And let's say we also want user with lowercase, colon, and then the userId. So this userId. Okay. Again, stop horizon. Restart it. And now, if I run this again, we should see it with new tags. And there they are. Okay. Let's take a look at metrics.
So in our case, I'm not going to do that. I'm just going to run it once. And then you can see the graph php artisan horizon snapshot. Okay. And it's run horizon again. And now you should see a graph with one point plotted on it. So back to the horizon dashboard. And there is the one point. So this will get populated over time as you add more data every time the snapshot runs. So you have information about the queue throughput.
So this will get populated over time as you add more data every time the snapshot runs. So you have information about the queue throughput. So in this case, we only have one queue, the default queue and the runtime for the jobs. And you can also have metrics on the different jobs here. Again, we only have one point. Actually, you know what, let me throw some more jobs on the queue and then I'll wait five minutes. And then I'll run the snapshot command again, and then you'll see two points plotted here. So let's run a bunch of jobs. So let's say userID 3 and let's run 200 jobs maybe.
Let's take another snapshot here. Okay. Let's run php artisan horizon again. And now we should have two plots pointed or two points plotted on the metrics here. So there we go. So you can see how over time this will give you more useful data. Okay, let's take a look at job batches here. So batches were introduced in Laravel 8, I believe. And as the name implies, they allow you to batch a certain amount of jobs. So there's some setup involved in getting batches to work.
And as the name implies, they allow you to batch a certain amount of jobs. So there's some setup involved in getting batches to work. So you need a batch table. So let's run this. So let me stop horizon to run this. And let's go ahead and migrate again. And we also have to add the Batchable trait to the job that we want to batch. So let's use Batchable in our SomeJob here. SomeJob. Let's use Batchable here.
Some job. Let's use batchable here. And let's go ahead and import that. Okay. And now we can make use of it like this. So bus batch, and we can batch as many jobs as we want in here. And we can listen for certain events. For example, if they were all successful, if there was a failure, or if the entire batch is complete. So I'm just going to paste in a block of code, which is almost the same as this.
is complete. So I'm just going to paste in a block of code, which is almost the same as this. So let's put this in our routes file and paste it down here. And let's also make sure to use those two things. So let's import those up here. Okay. And hopefully this works. So I'm just batching these jobs to a batch named batch of some jobs. Okay, so let's try this php artisan horizon. And it's hit that batch endpoint.
Okay, so let's try this php artisan horizon:retry. And it's hit that batch endpoint. And we should be able to see that in our batches here. So say batch. Okay, there's the idea of that batch. And there you see it being worked on here, it's still pending, but this will complete as it's finishing. And there it is. So if you go into it, you'll see more information about that batch. And if you go into the completed jobs, I believe there is a tag for the batch ID here.
So if you go into it, you'll see more information about that batch. And if you go into the completed jobs, I believe there is a tag for the batch ID here. Okay, one more thing I want to take a look at. And that is the balance options. So back to our config here. So config/horizon.php. You can see the default options here for the queue. And you can override them here for each environment. So you can see on production, we have 10 processes. And on local, we only have three.
So you can see on production, we have 10 processes. And on local, we only have three. So in general, you want to have more processes depending on the number of jobs you have. But they come at a cost of CPU cycles and RAM depending on your jobs. So you want to use them accordingly. So you can see the default balancing strategy is set to auto here. Let me just set it to simple for now, so I can show you what it's trying to do. So I'm going to set it to simple. I'm going to add another queue here. Let's say another, because right now we only have one default queue.
I'm going to add another queue here. Let's say another, because right now we only have one default queue. And let's set the processes to 20 on local, okay? And what simple does is it's just going to distribute the amount of processes you have evenly across queues. So in this case, we're just going to have 10 for default and 10 for another, and it just stays that way. So let's see that. I'm going to stop Horizon. I'm going to rerun it.
I'm going to stop Horizon. I'm going to rerun it. And let's go ahead and hit our jobs slash, let's say 200 and userId 1. So if we hit this and go to our dashboard here, you'll see that we have 10 processes. And you can see everything is on the default queue, but the number of processes stays the same because that's how Simple works. So you might think that these processes are not doing anything. They're just sitting idle. So why not donate them here as these jobs are completing? So that's exactly what auto does.
So now right now the jobs are complete and you'll see the processes start to go down because there's nothing on the queue right now. So yeah, I hope that was a good overview of using Horizon for your queues in Laravel. So in the next video, we'll take a look at getting notified when something goes wrong in your queues.
