In this episode, however, we are going to explore ways we can scale workers running on the same machine. We will do that with the help of supervisor and the cron scheduler. Here I am connected to an Ubuntu 20.04 server that was provisioned using Laravel Forge. We are going to add a new program for supervisor to monitor for us. And to do that we will go to /etc/supervisor/conf.d. So let's go to this directory /etc/supervisor. And if we checked the files in this directory, we will find it empty for now. Let's create an extra_workers file. So I'm going to use the nano editor, you can use them if you want extra_workers.conf.

Let's create an extra workers file. So I'm going to use the nano editor, you can use them if you want extra workers.conf. And inside this file, I'm going to paste a program or a program configuration that I have prepared. And we're going to start by discussing a few things about these configurations. The first line here gives a name to our program. We'll call it extra workers. And we'll use this name later to start and stop this program when needed. Next in the command attribute, we configure supervisor to run the queue:work artisan command when this program starts.

Next in the command attribute, we configure supervisor to run the queue:work command when this program starts. Next we will set auto_start here to false. So supervisor doesn't start the program automatically when we restart the server. And finally, we have the number of processes set to three. And this basically tells supervisor that we are going to start three processes to run this command, which is the queue:work command. Now let's save this file. And to make sure that supervisor reads this new configuration, we are going to call supervisorctl reread, oh a typo, so supervisorctl reread.

And to make sure that supervisor reads this new configuration, we are going to call supervisorctl reread, oh a typo, so supervisorctl reread. And here we go. And then we are going to call supervisorctl update. Perfect. Now supervisor knows about our program. And now whenever we need to add more workers to the queue workforce, we can simply run the supervisorctl start extra workers, which will start three workers for us to help empty our busy queues. And when we no longer need to have these workers running, we are going to call supervisorctl.

empty our busy queues. And when we no longer need to have these workers running, we are going to call supervisorctl stop command and give it the program name, which is extra_workers. But we don't have to do this by hand. If we know our application receives a lot of jobs at a specific time of the day, we can use cron to start and stop the program based on the schedule. Let's open the /etc/crontab file to add a couple of cron entries. We can see here that Forge adds a few entries by default. And at the end of the file, we are going to add this entry. And again, I have it prepared in my clipboard.

And at the end of the file, we are going to add this entry. And again, I have it prepared in my clipboard. So this command is going to start the extra workers supervisor program at 7 a.m. every day. And then we are going to add another command to the cron tab. And this command is going to end or is going to stop the extra workers program at 11 a.m. every day. So now when we save this file between 7 a.m. and 11 a.m. every day supervisor will start three worker processes and keep them running for us. All we have to do now is save this file, the /etc/crontab file and hit enter.

three worker processes and keep them running for us. All we have to do now is save this file, the /etc/crontab file and hit enter. And now our program will be running on time. So let's clear the terminal and discuss another scenario. If we don't have a specific schedule in which our queues are going to be busy and we want to always make sure that there are enough workers to empty the queue throughout the day, Laravel has got our back. The queue:work command has a couple of handy options that we can use to achieve that. Basically, we are going to start a number of workers every 10 minutes and let them exit.

that. Basically, we are going to start a number of workers every 10 minutes and let them exit if the queue is already empty, which means those workers are not needed. To do that, we'll go to that /etc/crontab file again. So we're going to use the nano editor to edit the /etc/crontab file. And we're going to add a new entry. Let's remove the old entries from here to add a new one. And here we go. This cron entry will run every 10 minutes and it's going to call the queue:work command which will start a worker.

This cron entry will run every 10 minutes and it's going to call the queue:work command which will start a worker. We can just save now and consider the job is done. Because remember, this queue:work command is going to run every 10 minutes. So cron will start a new worker for us every 10 minutes. We don't want that, we want to just start enough workers to ensure our queue is empty. Starting a worker every 10 minutes is dangerous because eventually the workers are going to eat all the server resources and the server is going to crash. As we said, we want that worker to stop if the queue is empty. For that we are going to use the --stop-when-empty option.

As we said, we want that worker to stop if the queue is empty. For that we are going to use the --stop-when-empty option. So let's continue adding to the program and we're going to add the --stop-when-empty option to the queue:work artisan command. Now whenever cron starts this worker and the worker finds that the queue is already empty it's going to exit right away. And because we are not using supervisor to monitor this worker, supervisor is not going to start an instance of this worker again. It will just exit and vanish. But before we hit save, if our queue is really busy, the cron command will start another

It will just exit and vanish. But before we hit save, if our queue is really busy, the cron command will start another worker after 10 minutes as we said, while a previous worker is still running. This may be too overwhelming for your server if the resources are limited. For that, we are going to use another command option that will automatically stop the worker after 9 minutes, and that option is --timeout. So let's add to the command here, --timeout=540 seconds. Now this worker will stop automatically after 9 minutes, even if the queue is still busy. cron will start a new worker every 10 minutes, so when the new worker starts, this worker will be already ended.

Cron will start a new worker every 10 minutes, so when the new worker starts, this worker will be already ended. So now we hit save to save the /etc/crontab file. And that's it. So far so good. We have been scaling workers using cron by starting more workers when needed. However, in more complex scenarios, you may need to scale workers handling specific queues. If you are using the Redis queue driver, you can achieve that by using Laravel Horizon. We are not going to discuss installing Horizon in this episode. You can check the docs for that.

We are not going to discuss installing Horizon in this episode. You can check the docs for that. However we are going to take a look at the horizon.php configuration file. And under this environment, the production environment, we can see that this supervised pool of workers has a minimum of one worker and a maximum of 10 workers. This worker is also handling jobs from the deployments queue and the notifications queue. Notice also that we use the auto balancing strategy. Make sure you set it to auto in your application as well because it's not set like that by default. So what will happen here is that if the deployments queue is really busy while the notifications

than the deployments queue, horizon will still allocate more workers to the deployments queue since it has the higher time to clear. And that's a very smart way to scale your workers. And horizon makes it very simple. Also one more thing you need to learn is that horizon adds or removes a single worker every 10 seconds, based of course on the time to clear as we explained earlier. But that means it will take 9 seconds to scale the pool from 5 to 8 workers. If you think that's slow for your use case, you can control the rate of scaling by configuring the balanceMaxShift and the balanceCooldown. With balanceMaxShift equals to 3 and balanceCooldown equals to 1, horizon will add or

the balance max shift and the balance cooldown. With balanceMaxShift equals to 3 and balanceCooldown equals to 1, Horizon will add or remove 3 workers every second, which will give you much faster scaling.