While the cool thing about Laravel is that everything just works, it's important that we understand how the different moving parts work under the hood to make things work. The better we understand the internals of the framework, the better we can utilize its full power. And in this episode we are going to explore how the queue system works under the hood. The queue system does two things, dispatches jobs to the queue and processes them. The dispatching part starts with the dispatch helper. And this helper is available in every job that is created using the php artisan make:job command. Let's take a look at this helper and it's located under the Dispatchable trait.

command. Let's take a look at this helper and it's located under the Dispatchable trait. This trait is again used in every job created with the php artisan make:job command. And as we can see the helper just creates a new instance of BendingDispatch and it passes an instance of the job class and passes any arguments that the dispatch helper receives it passes it to that instance of the job. So if we call TestJob::dispatch() that means that the helper is going to create a new instance of the TestJob class, pass the arguments that are passed to dispatch to this instance and then pass the job instance to the BendingDispatch instance. Now let's take a look at the BendingDispatch class and as we can see this class accepts

and then pass the job instance to the BendingDispatch instance. Now let's take a look at the BendingDispatch class and as we can see this class accepts the job instance and it has a few methods that you can use to set the connection, the job should be dispatched to, the queue that it should be dispatched to and other configurations as well. The most interesting thing about this class though is the __destruct method here. The __destruct method is what actually sends the job to the queue component. The method is a magic method by php and it's called automatically whenever the BendingDispatch instance is cleared from the stack. That means if we are dispatching the job from inside a route action the dispatching will

dispatch instance is cleared from the stack. That means if we are dispatching the job from inside a route action the dispatching will happen after the view is returned to the browser. And if we continue to take a look at this method the destruct method we can see that it first checks if this job shouldn't be dispatched, if so it will just return and does nothing. If the job should be dispatched however there are two options, if the job should be dispatched or should run after the response is returned, if you are dispatching a job from inside an HTTP request, you may configure the job to be run after the response is sent to the browser. Or you can configure the job to be dispatched to the queue to be processed later by a queue

HTTP request, you may configure the job to be run after the response is sent to the browser. Or you can configure the job to be dispatched to the queue to be processed later by a queue worker. In this series we are only interested in dispatching jobs to the queue. So the job is either not dispatched, it's ignored or it's dispatched to the queue. Let's take a quick look at the shouldDispatch method here, and this method simply checks if this job should be unique and if so it's going to make sure that there are no instances of the same job currently in the queue so we can dispatch it there. The method does that by checking the cache lock and make sure that there is no lock acquired already giving the unique ID of that job.

The method does that by checking the cache lock and make sure that there is no lock acquired already giving the unique ID of that job. Now let's go back to the __destruct method and take a look at this line here. This line is what actually dispatches the job to the queue and it resolves an instance of the Dispatcher and calls the dispatch method to it and passes the job instance to the dispatch method. Let's take a look at this Dispatcher interface and it's located under Illuminate\Contracts\Bus\Dispatcher. The implementation of this interface is done inside Illuminate\Bus\Dispatcher. And if we check the dispatch method on the Dispatcher we can see that it receives the command as an argument which is the job that we want to dispatch to the queue and it's

And if we check the dispatch method on the dispatcher we can see that it receives the command as an argument which is the job that we want to dispatch to the queue and it's going to call another method called dispatchToQueue. Let's take a look at this method, this instance is going to resolve an instance of the queue driver that we want to push jobs to. So for example if we are pushing jobs to the database queue driver this variable here is going to hold an instance of the database queue. And finally at the end of this method it's going to call another method pushCommandToQueue so let's take a look at it. As we say the queue variable here holds an instance of Illuminate\Contracts\Queue\Queue.

so let's take a look at it. As we say the queue variable here holds an instance of Illuminate\Contracts\Queue\Queue. If we are dispatching to the database queue then it holds an instance of the DatabaseQueue and we are going to take a look at it in a bit. So depending on how you configure the job if you have a delay configured the push command to queue method is going to call the later method on the queue driver, if you don't have a delay configured it's going to call the push method. And there are other methods when you have a specific queue that you are pushing jobs to in that case it uses the pushOn method and if you have a queue configured as well as a delay configured it's going to call the laterOn method.

to in that case it uses the pushOn method and if you have a queue configured as well as a delay configured it's going to call the laterOn method. Now let's take a look at the database queue driver, it's located under Illuminate\Queue\DatabaseQueue. Let's go check the push method, the push method calls another method called enqueueUsing and creates a payload that it passes it to that method as well as instruct the method on how to actually dispatch this job to the queue driver that the database queue is configured to push to. So let's split the screen for a moment and take a look at the pushToDatabase method and this method just returns a database query that stores the job payload inside the database.

So let's split the screen for a moment and take a look at the pushToDatabase method and this method just returns a database query that stores the job payload inside the database table that holds our jobs. And if we take a look at the createPayload method, here it converts the payload array to a JSON, let's see how it creates the payload array. And since our jobs that we are dispatching to the queue are an object we are going to go to createObjectPayload and here we go, that's how the payload of the job is created. Each job has a unique ID, a display name and several other configurations. One of these interesting configurations is the job attribute here. It holds an instance of the job handler, in that case it's Illuminate\Queue\CallQueuedHandler

One of these interesting configurations is the job attribute here. It holds an instance of the job handler, in that case it's Illuminate\Queue\CallQueuedHandler and the method that's called on that handler is the call method. That handler is a wrapper around the job that we are going to take a look at in a bit. At this point you only need to know that when a worker picks the job up it's going to create an instance of CallQueuedHandler and call the call method on it and pass the job data to it. Another thing to inspect here is how the command is created or how the job is stored inside the data attribute of the payload. Here we check if the job should be encrypted and if we can resolve an instance of the encryptor.

the data attribute of the payload. Here we check if the job should be encrypted and if we can resolve an instance of the encryptor from the container, in that case we are going to use that encryptor instance to encrypt a serialized version of the job instance. If not the command attribute is just going to hold a serialized version of the job object. When a worker picks the job up it's going to unserialize the command here and pass it to the handler to run the job. So now we know how the jobs are pushed to the queue driver and we know how the job payload is created. The enqueue using method just invokes the callback that stores the job in the queue

is created. The enqueueUsing method just invokes the callback that stores the job in the queue driver and fires some events. But before that, it checks if the job should be dispatched after a database transaction is committed. In that case, instead of calling the callback right away or invoking the callback to dispatch the job to the queue driver, it's going to create an instance of the DatabaseTransactionsManager from the Laravel container and store a callback that invokes this callback method here. So that's how Laravel delays pushing jobs to the queue until database transactions commit.

here. So that's how Laravel delays pushing jobs to the queue until database transactions commit. Instead of invoking it right away it's going to store the callback inside the database transactions manager and the database transactions manager is going to invoke the callback whenever all transactions are committed. And that's basically how jobs are dispatched to the queue. The other part we're going to look at is how jobs are processed. For that let's take a look at the work command. Here is the command, the command accepts an instance of the worker in the constructor and inside the handle method it calls the runWorker method that uses the worker to start.

Here is the command, the command accepts an instance of the worker in the constructor and inside the handle method it calls the runWorker method that uses the worker to start processing jobs. So the worker instance here is going to either run the next job only and then exit, that's if the once option is used and we explained the once option in a previous episode. If the once option is not used it's going to call the daemon method on the worker. Let's take a look at the worker and it's located under Illuminate\Queue\Worker. Let's inspect the daemon method. And that method holds everything that a worker needs to do to process jobs from the queue. Let's take a look at this line here and we can see that the worker stores the last restart.

And that method holds everything that a worker needs to do to process jobs from the queue. Let's take a look at this line here and we can see that the worker stores the last restart timestamp that's stored in the cache. It uses this value later to see if there is any change that happened during the lifetime of the worker. A change in the restart timestamp means that the worker should exit to be restarted later. After that it counts the time that this worker started and it starts a counter for jobs processed by this worker. This is used to determine if the worker should exit when using the maximum time or the maximum --max-drops command options of the php artisan queue:work command.

This is used to determine if the worker should exit when using the maximum time or the maximum drops command options of the queue:work artisan command. After that we can see a while loop starting and at the beginning of the loop we can see this line here determines if the worker should pause. A worker pauses if the application is in maintenance mode. Internally the worker just calls the sleep php function to pause for a few seconds before it continues pulling new jobs. Now let's move to this line here, and this is the part where the worker queries the queue store for new job. Depends on the storage driver we use this line will inspect this store and find the

store for new job. Depends on the storage driver we use this line will inspect this store and find the next job in the queue. The return value of this getNextJob method is an instance of Illuminate\Contracts\Queue\Job which is a wrapper around values coming from the queue store. Let's take a quick look at this getNextJob method. The connection variable here holds an instance of Illuminate\Contracts\Queue\Queue which is the queue storage driver. The way the getNextJob method picks a job from the queue is by calling the pop method on the queue driver instance.

The way the getNextJob method picks a job from the queue is by calling the pop method on the queue driver instance. And as we said it returns an instance of Illuminate\Contracts\Queue\Job. Now back to the daemon method. We now have an instance of a job stored inside the job variable; it's the next job that needs to be processed. But before we look at processing jobs, let's take a look at the registerTimeoutHandler method here. Inside this method, the worker installs an alarm along with a signal handler for that alarm.

Inside this method the worker installs an alarm along with a signal handler for that alarm. The alarm will go off after the configured timeout we set for the job are on the worker level. Let's take a look at this method registerTimeoutHandler and we can see the alarm set here and the signal handler for the alarm is configured here. The handler will terminate the worker so it doesn't get stuck processing a job that exceeds its configured timeout. It does that by calling the kill method which will basically exit the process with an exit error of value one.

It does that by calling the kill method which will basically exit the process with an exit error of value one. Now let's go back to the daemon method and look at this line here. Now that we have a job and we have the alarm registered we are going to call the runJob method. So let's take a look at it. This method calls the process method and the process method calls the fire method on the job instance. And again this job instance holds a wrapper around our actual job. It holds an instance of Illuminate\Contracts\Queue\Job.

And again this job instance holds a wrapper around our actual job. It holds an instance of Illuminate\Contracts\Queue\Job. Let's take a look at this fire method. So let's open the contract and find the implementation so it's under database/jobs. Let's find the fire method. And here we can see that the fire method extracts the job command from the payload resolves it from the container and then executes the handler method. So this line here will resolve an instance of CallQueuedHandler that we discussed earlier in this episode. Once we have this instance ready we are going to call the method the handler method and in

in this episode. Once we have this instance ready we are going to call the method the handler method and in that case that method is called call. While calling the method we are going to pass the connection the queue connection and the data part of the payload of the job. Now let's take a look at this CallQueuedHandler instance. Let's close the worker here. The CallQueuedHandler class is under Illuminate\Queue\CallQueuedHandler. And we know now that the worker is going to call the call method on this class. This method does a lot of things.

And we know now that the worker is going to call the call method on this class. This method does a lot of things. First it unserializes the job instance from the payload. And while doing that, if the job instance accepts an Eloquent model in the constructor and this model wasn't found, a ModelNotFoundException was thrown, the call method is going to call the handleModelNotFound method. Which basically deletes the job from the queue and stored it in the failed job staple if that's how we configure our worker to handle nonexisting model exceptions. Now let's move to the next thing that this method does, which is releasing unique locks when the job starts processing or when it's done processing.

Now let's move to the next thing that this method does, which is releasing unique locks when the job starts processing or when it's done processing. If the job implements the ShouldBeUniqueUntilProcessing interface, the lock will be released before processing the job. Otherwise the lock will be released after processing the job. The job processing is done inside this dispatchThrough middleware method. And when the job is processed and this job didn't fail and it wasn't released back to the queue, we are going to ensure that the next job in the chain, if there is a chain, is dispatched. Or if this job is part of a batch, we ensure that a successful Batch job is recorded.