Rate Limits Overview0:00
Shopify API, just like many other APIs, have rate limits and API versioning. Let's review both of these topics, because I think as an app developer, it's important to know how the versioning of APIs are done and what sort of rate limits we have to deal with. REST Admin API, for example, which we care about, because that's what we're building our app with, supports two requests per second. That's basically the standard limit, but if you're on a Shopify Plus, then your limit is 20 requests per second. Shopify implements rate limiting using the leaky bucket algorithm. Picture it as a bucket with max capacity and a small hole that allows water to leak at a constant rate. Each API request can then be visualized as a drop of water entering the bucket. And just like a leaky bucket, the requests are processed at a constant rate. So in the context of Shopify, the bucket size or the capacity is 40 requests per app, per store, per minute, with a leak rate of two requests per second. There are a couple of headers included with API responses.
So in the context of Shopify, the bucket size or the capacity is 40 requests per app, per store, per minute, with a leak rate of two requests per second. There are a couple of headers included with API responses. One is called xShopifyShopAPICallLimit, which shows the number of requests that have been made for that particular store. And another one is called retryAfter, which contains the number of seconds to wait before retrying again. There are some strategies and tips on how to avoid rate limit errors. And one of them is to basically try and average two requests per second with retry functionality. Thankfully, basic Shopify API package that provides the API functionality which Laravel Shopify uses has the built in basic rate limiter with automatic retry functionality. So technically, you don't need to worry about handling the rate limit retry functionality on your own since it's taken care of for you. But if you still encounter rate limit related issues, you could adjust the limiter or even build your own. As far as the API versioning goes, Shopify releases a new API version quarterly, so every three months.
API Versioning Basics1:51
But if you still encounter rate limit related issues, you could adjust the limiter or even build your own. As far as the API versioning goes, Shopify releases a new API version quarterly, so every three months. The current stable version as of recording this video is April of 2023, and each stable version is supported for a year. You should try and update the versions when possible and not fall too behind because they do remove versions after some time. And whenever the version is removed, if your app is using that version, it will serve the oldest supported stable version. If we scroll down here, we see the basic anatomy of the API URLs and where the API version fits in. So for REST Admin API, it's admin/api/version/endpoint.json. If we open the REST Admin API documentation for products, for example, we see the endpoints here contain the versions in them. Now, you may have noticed that we did not specify the API version in our code when making API calls to create and delete products. That's because the basic Shopify API package handles the versioning automatically and injects the proper version into the URL.
Updating API Version2:49
Now, you may have noticed that we did not specify the API version in our code when making API calls to create and delete products. That's because the basic Shopify API package handles the versioning automatically and injects the proper version into the URL. We could, however, still include it if we wanted to override the version. The API version that the package automatically injects in the URL comes from the Shopify configuration file. So if we open the shopify.app.php and scroll down to API_VERSION, this is the version that gets injected into the URL. Let's actually upgrade this to the latest stable version. So let's open the .env file and we have it set to January of 2023. Let's change that to April of 2023. Alright, as you might remember in the last video, I mentioned that we should introduce some queued jobs to basically queue these API operations to avoid having user wait on the page. That way we can give the user feedback right away.
Queueing Product Creation3:38
Alright, as you might remember in the last video, I mentioned that we should introduce some queued jobs to basically queue these API operations to avoid having user wait on the page. That way we can give the user feedback right away. So let's implement that. So first we need to create a new job. So we'll do vendor/bin/sail artisan make:job CreateProducts. Then let's copy this code from here into that job. So let's open CreateProducts Job and paste it in here. We can accept the account and the user or the shop model into the constructor. So we'll do private readonly int count and private readonly User shop. Then we can replace this with this->count.
So we'll do private readonly int count and private readonly User shop. Then we can replace this with this->count. And this will be this->shop. And same here, this->shop. Let's define this array before the loop like this. Let's go back to the controller. We can get rid of this and we can dispatch the job in here. So we can do something like createProducts, dispatch and pass the count as well as the shop. We can actually get rid of these variables. So let's just do it directly this way and take this and put it here.
We can actually get rid of these variables. So let's just do it directly this way and take this and put it here. That way we can get rid of this and get rid of that. Next, we should adjust our queue connection. We're going to be using Redis for this. So let's change the connection to Redis. We need to open our .env file. Let's scroll up, change queue connection to Redis. Let's open the browser. Click on create products.
Let's open the browser. Click on create products. We got the feedback right away. It didn't take a few seconds. Let's switch to the telescope. We have the request. As you can see, it took 58 milliseconds. Let's switch to jobs and we see that the job has been queued. We can process it by running the php artisan queue:work command. So let's go back to code.
We can process it by running the php artisan queue:work command. So let's go back to code. Open the terminal. We'll run vendor/bin/sail artisan queue:work. And as you can see, it started running createProducts and it has finished running creatingProducts. It took about three seconds to run. Let's go back to the browser to make sure that it worked. We see that the status is processed. Let's switch to our store. Click on products and we see the products have been created.
Let's switch to our store. Click on products and we see the products have been created. This is great because now the user doesn't have to wait all that time to create the products. If they select 100 products, that could take some time because, remember, we may hit some rate limits. And then if we do, the API package automatically does the retries. So it can take some time to finish executing all of those API requests. And we don't want the user to stay on page and wait for all that to finish. Instead, we give the user instant feedback and queue the job behind the scenes. We should also add some locking to prevent multiple jobs from being dispatched if the user clicks on the button multiple times. Now, there are multiple ways we can implement this.
Preventing Duplicate Jobs6:29
We should also add some locking to prevent multiple jobs from being dispatched if the user clicks on the button multiple times. Now, there are multiple ways we can implement this. We can use Laravel's withoutOverlapping middleware. We could use shouldBeUnique interface or we could create a custom locking mechanism. For our case, I think shouldBeUnique contract will work. So let's go back to our code. Let's scroll up in our job. And as you can see, it already has that shouldBeUnique contract right here. So we can simply implement it right here. Now, in addition to implementing this, we should also define the ID or the key of the lock because we want to lock this job per shop or the user and not across all shops.
So we can simply implement it right here. Now, in addition to implementing this, we should also define the ID or the key of the lock because we want to lock this job per shop or the user and not across all shops. We can do that by defining the unique ID method. So we'll do public function uniqueID. This will return a string and will simply return this shopID, which is the userID. And that's pretty unique. Now, I'll quickly show you how this is used so that it makes more sense. If we open this dispatchableTrade in here and go into the dispatch method, which uses the PendingDispatch class. If we open that in here and scroll down on the destruct method, it checks if it should dispatch this job or not. And if it should dispatch the job, then it just does the dispatch right here, depending whether it has to do after response or not.
If we open that in here and scroll down on the __destruct method, it checks if it should dispatch this job or not. And if it should dispatch the job, then it just does the dispatch right here, depending whether it has to do after response or not. And if it should not dispatch, then it just returns. Now, if we inspect this method right here, we see that if the job implements the ShouldBeUnique, then it executes this thing in here. And it does this by acquiring the uniqueLock. If we go into this method in here, we see how the key is generated for that cache. If we click on this getKey method, we see that it gets the unique ID from the return value of the uniqueId method if it exists on the job. Otherwise, it uses the uniqueId property on the job. If it exists, if not, it's blank. And this uniqueId then is appended to this key in here.
If it exists, if not, it's blank. And this unique ID then is appended to this key in here. And this key is just laravel unique job, the class name, and then whatever the unique ID is. So if we didn't define that method, this would essentially be the unique ID. And therefore, you can see how this would conflict across multiple stores. That's why setting this to userId makes this unique enough that it's not going to conflict across multiple stores. By the way, if this all sounds new to you and you're new to jobs and queues, I highly recommend you check out the laravel documentation on queues. And also, there are very good resources here at Laracasts to get you up to speed. So let's test this out to make sure that this actually works. So let's open the browser, click on the create products multiple times.
So let's test this out to make sure that this actually works. So let's open the browser, click on the create products multiple times. Let's switch to the Telescope, load the new entries on the job. And as you can see, only one job was queued. So even though multiple requests are coming in, only single job is being queued because of that contract. The one thing that I do want to adjust is that if you notice the toast message, it says that it has generated the fake data. Now, that no longer applies because it is not generating the data instantly. Instead, it's queuing it up. So a proper message would be something like we've started generating fake data or something like that. So let's go back to the code, open ProductCreator and change this to started generating fake data.
Queueing Product Deletion9:37
So a proper message would be something like we've started generating fake data or something like that. So let's go back to the code, open ProductCreator and change this to started generating fake data. Let's now do the same thing for the delete part, because in the controller, we still have this executing whenever the user tries to delete products. So let's take all of this and we'll put it in a new job. So let's create that job in here. We'll do vendor/bin/sail artisan make:job DeleteProducts. Let's open the DeleteProducts job and let's paste in the code. We're going to accept the shop in the constructor. So we'll do private readonly User $shop. And let's change this to $this->shop.
So we'll do private readOnly User shop. And let's change this to this->shop. And we also don't need this in here. Instead, we'll do this->shop. We should make this job unique as well. So let's implement shouldBeUnique contract. And let's dispatch it from the controller. So we'll do deleteProducts, dispatch, request, User. And let's get rid of that. And let's also adjust the toast message.
And let's get rid of that. And let's also adjust the toast message. So we'll open this component and change this to deleting fake data. All right. I think we should be able to test this out. I did forget to add the uniqueId method on the deleteProducts. So let's add that in here the same way. And we should be good to go to test this out. So let's open the browser. Let's click create five products.
So let's open the browser. Let's click create five products. We see that no new jobs are being queued in here. Let's try to delete fake products. And let's click it multiple times. Switch to the Telescope, load the new entries. And we see that we only have a single delete products job. For better user experience, we could probably check if the lock exists in cache. And if it does, show users some sort of warning that there already is a job in queue. Or maybe just simply disable these buttons if the job is already in queue.
And if it does, show users some sort of warning that there already is a job in queue. Or maybe just simply disable these buttons if the job is already in queue. We could also add email or other type of notification to the User whenever the products have been created or deleted. But we're not going to go that far because this is a simple demo app.
