Delaying Job Execution0:00
In the previous episode we created a SendWelcomeEmail job and learned how to dispatch it to the queue. Let's see the many ways we can configure this job. First we can configure a delay on the job to make workers wait before processing it. So right after calling the dispatch helper we are going to call the delay method and enter 5 as the value. Now whenever we dispatch this job the worker will wait 5 seconds before processing it. To test that let's go to the terminal and start a worker by calling php artisan queue:work. And the worker is now running.
work. And the worker is now running. So let's dispatch the job by visiting the route and it's dispatched. And as you can see the worker didn't pick it up right away. It waited for 5 seconds before picking it up and start processing it. Now let's terminate this worker and go back to php store. Remove the delay method. And let's see what else we can configure. An important feature in the Laravel queue system is that we can set a timeout for the job. To do that we are going to go to the job class SendWelcomeEmail and define a public property.
Setting Job Timeout1:05
An important feature in the Laravel queue system is that we can set a timeout for the job. To do that we are going to go to the job class SendWelcomeEmail and define a public property called timeout. So public timeout. And we're going to add the value of 1. Now when this job runs the worker is going to terminate it if it didn't finish after 1 second. This is useful if we want to prevent our job from getting stuck communicated with a third party library or a third party endpoint that is never responding. So let's test this but before we start remember that we are sleeping for 3 seconds inside.
party library or a third party endpoint that is never responding. So let's test this but before we start remember that we are sleeping for 3 seconds inside the handle method. So the worker should terminate this job before it finishes because we configured it to terminate the job after only 1 second. So let's head to the terminal and start the worker. And let's dispatch the job by visiting the route. And we can see that the worker picked the job but terminated it after only 1 second and marked it as a failure. Perfect.
Configuring Job Retries2:10
and marked it as a failure. Perfect. Another thing we can configure is the number of retries. By default the worker is going to consider a job as a failure if it failed for only one time. But we can change that. By defining a public property called tries we are going to let the worker retry this job 3 times before it considers it a complete failure. And to simulate a failure we are just going to throw an exception. And that's it.
And to simulate a failure we are just going to throw an Exception. And that's it. Let's head to the terminal and start the worker then dispatch the job. And as you can see the worker retried the job for 3 times before considering it as a failure. Instead of retrying the job for a specific number of times we can let the job retry for a period of time. To do that let's add a retryUntil method. So under the handle method we are going to define a public function called retryUntil. And inside this method we are going to configure the job to retry for 1 minute.
So under the handle method we are going to define a public function called retryUntil. And inside this method we are going to configure the job to retry for 1 minute. So now add minute, perfect. Before we test that we are going to set the number of tries to -1 so it retries for an unlimited number of times. Now we go to the terminal and start a worker and then dispatch the job. And as you can see here the job will be retried for an unlimited number of times unless is until it expires. And in our case we configured it to expire in 1 minute. But as you can see retrying happens immediately.
Adding Retry Backoff3:47
And in our case we configured it to expire in 1 minute. But as you can see retrying happens immediately. Once the job fails the worker is going to pick it up and retry it immediately. In some cases it's better to wait between a failed job before retrying it again. To do that we need to configure a backoff. So back to our Job class and we are going to define another public property called backoff. And we are going to back off for let's say 2 seconds between each retry. Now let's go back to the terminal, clear the terminal and start a worker. Let's dispatch the job. And we can see that the worker started picking the job and it waits 2 seconds between each.
Let's dispatch the job. And we can see that the worker started picking the job and it waits 2 seconds between each retry instead of retrying it immediately. Perfect, so far so good. Now we learned about the various configuration options around dispatching and running queue jobs. There is something you should know though, that a worker can only process a single job at any given time. If we have too many jobs stored in our queue the worker will go through them one by one which can be very slow.
Running Multiple Workers4:56
If we have too many jobs stored in our queue the worker will go through them one by one which can be very slow. Let's see what we can do about that. First let's clean our job here, remove the tries and the backoff, remove the retryUntil method and remove the exception. Then we go back to the routes file and instead of dispatching a single job we are going to dispatch 100 jobs. So we create a foreach loop and then put the dispatching inside the foreach loop. Now when we visit this route 100 jobs from the sendWelcomeEmail job are going to be dispatched to the queue.
Now when we visit this route 100 jobs from the sendWelcomeEmail job are going to be dispatched to the queue. Let's go to the terminal and clear it. Then start a worker php artisan queue:work. The worker is running and then let's call the route to dispatch the 100 jobs. And as you can see the worker started picking the jobs and it's processing them one by one. Since we have 100 jobs and we have sleep for 3 seconds inside each job the worker is going to take 5 minutes to finish processing all the sendWelcomeEmail jobs that are in the queue. And that's super slow.
queue. And that's super slow. Let's speed things up by starting one more than one worker. To do that we are going to create a new tab in our terminal and we are going to start another instance of the php artisan queue:work command. So we have an instance running here that's processing jobs and another instance in this tab. And if we jump between the two instances of the command we can see the two workers processing jobs at the same time. And that's how we speed things up.
Prioritizing Queues7:46
a welcome email job. Let's see how we can do that. First let's create our payment processing job and we are going to use the php artisan make:job command and we are going to call our job ProcessPayment. We go back to php store and here is the ProcessPayment job that we just created. We are going to dispatch this job after dispatching the 100 jobs of the SendWelcomeEmail. So ProcessPayment and dispatch. Let's go back to the terminal, clear the terminal, start a worker php artisan queue:work and then dispatch the jobs by calling dispatch. And as you can see the worker started and it's processing all the SendWelcomeEmail job.
dispatch the jobs by calling drought. And as you can see the worker started and it's processing all the sendWelcomeEmail job. It will process the hundred of them before looking at the processPayment job which should have the higher priority. Let's see how we can give this processPayment job a higher priority. Let's stop the worker and go back to php store and we are going to dispatch the processPayment job to a special queue. So we're going to chain the onQueue method after calling the dispatch helper and we're going to dispatch this processPayment job to a queue called payments. Now back to the terminal let's clear it first and then start a worker using the php artisan queue:work command.
going to dispatch this ProcessPayment job to a queue called payments. Now back to the terminal let's clear it first and then start a worker using the php artisan queue:work command but instead of starting it right away without any options or arguments we're going to configure this worker to process payments from the payments queue and the default queue as well. If we start the command without providing any options to it it's going to process jobs from the default queue only. So to do that we're going to provide an option called --queue and we're going to provide the payments queue and the default queue. Putting the payments queue first meaning that we're giving it a higher priority. A worker is going to process jobs from the payments queue first before it looks at the
Putting the payments queue first meaning that we're giving it a higher priority. A worker is going to process jobs from the payments queue first before it looks at the default queue. So let's start this worker and then vetted the route to dispatch the jobs. And as you can see here the worker picked the processPayment job first because it's in the higher priority queue before looking at the jobs in the default queue. And that's how you configure queue prioritization. Perfect. In the next episode we're going to look at how we can handle or deal with job failure.
In the next episode we're going to look at how we can handle or deal with job failure.
