Introducing Laravel Scout0:00
Laravel Scout is the solution to all of our search problems. It's essentially search on steroids. It wraps different services that you can pick and choose from to perform these fuzzy searches, these advanced searches for you, and you get to utilize that in your Laravel application in a very similar style and syntax to just working with an Eloquent database query. So it feels very natural. It's incredibly powerful. Let's get it integrated into our framerate application. There are several different drivers that you can use with Laravel Scout for search. Algolia is a SaaS software as a service with a very generous free tier.
There are several different drivers that you can use with Laravel Scout for search. Algolia is a SaaS software as a service with a very generous free tier. It's perhaps the simplest to get started with in one fashion because you're literally just creating an account, copying some keys into your .env file, and you're good to go. But in my case, I'm going to reach for MileySearch. MileySearch? MahuliSearch? I don't know. Well, however you pronounce it, I'm going to go with MileySearch because I have Laravel Herd Pro, and it has that as a service baked right in.
Installing and Configuring MeiliSearch1:02
Well, however you pronounce it, I'm going to go with MileySearch because I have Laravel Herd Pro, and it has that as a service baked right in. You could also set it up yourself manually if you'd like to, but feel free to use Algolia or Typesense or whatever driver is easiest for you to get going with. Okay, so let's follow the documentation and get our app configured to use MileySearch. We'll first need to require the Laravel Scout Composer package. And whilst that's installing, let's take a look at the next steps. We can publish the ScoutServiceProvider. Once that's installed, we should be able to add the searchable trait to our Post. So let's find Post.
Once that's installed, we should be able to add the searchable trait to our Post. So let's find Post. Here we are. And we can use searchable, which comes with that Scout Composer Package. MeiliSearch is going to require installing a couple of other composer dependencies. So we'll copy that from the documentation and install. And then we'll need to copy the Scout Driver environment variable along with the master key and the host. However, this is where I'm going to switch over to Laravel Herd because that will tell us the environment keys to use. So in Laravel Herd's Services tab, and again, you'll need Herd Pro for this,
because that will tell us the environment keys to use. So in Laravel Herd's Services tab, and again, you'll need Herd Pro for this, I'm going to add a search service. We can leave all the defaults. I'll leave auto start on as well, and I'll hit save. It's been initialized. Here we go. I can copy the environment variables. Then we'll jump into our IDE, open our .env file. And right at the bottom, I'm going to drop this in.
Then we'll jump into our IDE, open our .env file. And right at the bottom, I'm going to drop this in. So our Scout Driver is set to MileySearch. We have our host set up at 7700. And then Laravel Herd is our MileySearch key. I appreciate not all of you are on Herd Pro. So the MileySearch manual installation is not very difficult. We have to essentially grab this little curl command, and we're going to execute that locally. Here we go.
and we're going to execute that locally. Here we go. It's pulling it, and it's going to get it installed. While that's taking place, let's take a look. You'll see that once it's installed, we can run the MileySearch command, and we can provide a master key, which essentially correlates to, well, this key here that we're going to enter into our .env file. So let's try it out. We can run the MileySearch command, and we can provide a master key. Let's say test, and you'll see that that starts the MileySearch server.
We can run the MileySearch command, and we can provide a master key. Let's say test, and you'll see that that starts the MileySearch server. We'll have to update our .env file to use the test key that we defined, and everything else should be set. The default port is 7700, which we already have configured in the .env file here. You can check everything's configured correctly by copying the IP address and port listed in your .env file, pasting it into the browser. You'll have to copy the MileySearch key and paste it in here, and you should see your local instance of MileySearch up and running. There's nothing yet available because we've not asked it to index anything,
Indexing Posts for Search3:49
and you should see your local instance of MileySearch up and running. There's nothing yet available because we've not asked it to index anything, but this is everything up and running. MileySearch is correctly configured. Let's see if we can start seeing models in MileySearch. I'm going to use the php artisan scout:import command for this. So php artisan scout:import, and then it's the fully qualified class name of the model you want to actually use, and it has to be a model that has that little searchable trait that we added to the Post.
and it has to be a model that has that little searchable trait that we added to the Post. So in this case, scout:import app/models/Post. That will take just a moment, and then it will tell us, well, all the records have been imported. So let's see if that's the case. We'll go back to MileySearch, and sure enough, if we refresh, we have 246 posts that have been imported into MileySearch. Now already, we should be able to perform some searches in the MileySearch interface.
Now already, we should be able to perform some searches in the MileySearch interface. So let's say singing for singing in the rain. Remember that previously that returned nothing, but now in MileySearch, that actually returns the results, singing in the rain. If I say singing and rain, note that it picks up on both of those things, and again, it's returning these as results. Of course, we don't want our users to have to use MileySearch's dashboard in order to see posts in our application.
Implementing Scout Search5:07
Of course, we don't want our users to have to use MileySearch's dashboard in order to see posts in our application. So this is where we use Laravel Scout in order to be able to pull results from MileySearch back into our application. Let's jump into our PostController's index method, and just for a moment, I'm going to comment out what we have here for grabbing posts from the database. Instead, we'll grab posts using Laravel Scout. So first of all, we can use the postSearch method. That search method is made available by the searchable trait.
So first of all, we can use the search method. That search method is made available by the Searchable trait that we added to the Post, and we want to search for, well, whatever the User has passed in, which would be request, and we'll use the query method to grab the query parameter. We'll try and keep things as simple as possible, and then we'll expand on it over the episode. So the first thing I'm going to do is call paginate in order to duplicate the pagination that we already have in place.
So the first thing I'm going to do is call paginate in order to duplicate the pagination that we already have in place, and I'll also duplicate the withQueryString chained method in order to make sure that our query parameters are persisted. Okay, so with this commented out and with our Scout integration in place, let's go to our application and see if search works. And the answer is no, we get a blank page. Taking a look in the console, it says that it cannot read properties of undefined, reading slug, and it's struggling to load the slug property on the Post's topic.
it says that it cannot read properties of undefined, reading slug, and it's struggling to load the slug property on the Post's topic. The reason for this is that, well, when we were loading the posts in Eloquent, we used the eager load to actually also load the User and topic relationships, but we're not doing that in Scout. Now, something important to keep in mind when it comes to Scout, you're not dealing with Eloquent. Again, you are passing off to a third-party service to do the search. That search will return essentially the IDs of any relevant models, and then Laravel Scout is going to convert those IDs into Eloquent models.
That search will return essentially the IDs of any relevant models, and then Laravel Scout is going to convert those IDs into Eloquent models that you can paginate or grab from the database, whatever you need to do. So what I'm trying to essentially say is there's no with method available in Scout, but there is a hook we can use to eager load those relationships before we pull them from the database. Before we call paginate on Scout, we're going to use a hook called query that is given a closure that receives an Eloquent builder instance. We'll name the variable query, and here you can perform any last-minute adjustments.
We'll name the variable query, and here you can perform any last-minute adjustments to the query that will be executed to grab these models from the database. You shouldn't use this for filtering. That is what Laravel Scout is for, but you can use it for things like, well, adding eager loading. So in this case, I can chain on the with method, and I can load the User and the Topic for each Post. Let's see if that works. Well, that's already a good sign.
Let's see if that works. Well, that's already a good sign. The UI is now loading. Let's search for singing. Here we go. Well, that returns singing in the range, so search is working as before, but now check this out. If I search for singing, which before was returning no results, it now returns all of the singing in the rain results. It uses that fuzzy search provided by MileySearch.
it now returns all of the singing in the rain results. It uses that fuzzy search provided by MileySearch to actually return results to us. How cool is that? Let's say singing rain, and it still works, even though the actual search query is singing in the rain. How about if we search for Spielberg, and I'm essentially going to spell that incorrectly. It's still going to return all of the reviews of Jaws, reason being, if we actually jump in here,
It's still going to return all of the reviews of Jaws, reason being, if we actually jump in here, it's using the body of the text, and it's using a fuzzy search on Steven Spielberg's name spelled with an E instead of a U, but because of the power of MileySearch, it's able to perform that and return the correct results to us. So already, this is incredibly powerful, and it's so much more versatile than the solution we had before with a simple where like query.
and it's so much more versatile than the solution we had before with a simple where like query. Let's quickly check that our pagination still works. I'll move to page two, and yes, it does. The query string is respecting the query and the page, and you'll see that we're actually looking at Raiders of the Lost Ark now because the body of that must also contain Steven Spielberg. One thing that I know is broken is our topics. So if I click on, say, general,
Adding Topic Filtering9:26
One thing that I know is broken is our topics. So if I click on, say, general, you'll see that it doesn't actually filter down to general, and that makes complete sense. We're not doing any form of filtering for a topic inside the actual Laravel Scout search. I already said that we can't use the query hook to do filtering or we shouldn't use the query hook to do filtering. So how do we respect the topic ID when it comes to using Laravel Scout?
So how do we respect the topic ID when it comes to using Laravel Scout? Well, the Scout documentation tells us that where clauses are supported. So you'll see here, for example, that you can filter down to a certain user ID. I'll make this clear again. When you perform a WHERE query using Scout, you are not doing a database query. Instead, it's going to send that WHERE query
you are not doing a database query. Instead, it's going to send that WHERE query off to MileySearch, to Algolia, to Typesense, whatever it is you're using, and it will allow that service to perform the WHERE lookup. And now for MileySearch, that requires a little bit of additional configuration, but we'll get to that in a moment. For now, we'll just add the WHERE query. So perhaps after our query hook, we could chain on our WHERE,
For now, we'll just add the WHERE query. So perhaps after our query hook, we could chain on our WHERE, and I want to say WHERE the topic_id is equal to the given topic_id. But keep in mind that topic can be null, so we need to factor that in. We can use the conditional when method for this. So when there is a topic in place, we want to perform the following closure. That closure is going to receive a ScoutBuilder.
we want to perform the following closure. That closure is going to receive a ScoutBuilder. So that's a Laravel ScoutBuilder instance, not an EloquentBuilder instance. Again, as I say, they are separate things. We can call that query, and then we'll say query where topic_id is equal to the given topic_id. Then we call paginate and with query_string as before. Now, if you're following along
Then we call paginate and with query string as before. Now, if you're following along and you go ahead and click a topic after adding that WHERE query, you'll get a 500 error. Don't worry, you've not done anything wrong. It's just that MileySearch requires that we define ahead of time any fields that we actually want to use a WHERE query on, in this case, the topic ID.
any fields that we actually want to use a WHERE query on, in this case, the topic ID. We can do that in the scout.php config file. So from the IDE, search for a file called scout.php. This is the config file that was published when we published the vendor files earlier. And in the MileSearch configuration, line 133 for me, there is an index settings array. This is where we define the different fields that we want to be able to filter on.
This is where we define the different fields that we want to be able to filter on. So in this case, let's add an entry called posts. Posts is going to be an array, and we'll copy this filterable attributes node. And we want to set this to an array of any of the fields to filter by. In our case, it's going to be topicId. You might think that this solves the issue, but you'd be wrong.
You might think that this solves the issue, but you'd be wrong. If you go back and refresh and try again, it's still going to throw a wobbly. And that's because there's a command that we have to run that will actually sync that filterable attribute list with MileySearch. That command is php artisan scout:sync-index-settings. And it's important that you execute this anytime you change those filterable attributes.
And it's important that you execute this anytime you change those filterable attributes. In fact, you likely want to add it to your deployment script once you deploy your application so that that's always kept up to date before your application goes live. Now that that's in place, let's come back and we'll attempt to filter down to general posts with Spielberg. You'll see it works exactly as expected.
to general posts with Spielberg. You'll see it works exactly as expected. Let's go for conspiracies. Yep, works perfect. Questions, nice. Reviews, wonderful. Well, isn't that so much more powerful and versatile than the implementation we had before? One thing to keep in mind, by default, this is taking place synchronously.
Queueing Search Sync13:12
One thing to keep in mind, by default, this is taking place synchronously. That is to say that as we save a model, it is sending the information about that Post to a third party service, which can slow things down for your users. Instead, it would be better to tell Scout to use a queue to do that asynchronously in the background. Let's take a look at setting that up. Back in scout.php, we have this little queue parameter here.
Let's take a look at setting that up. Back in scout.php, we have this little queue parameter here. I'm going to set it to true, which essentially says, look, make sure this is queued up. We also want to go into our .env file and find the queue connection, which is currently set to sync. I'm going to change this to Redis as I already have a Redis server up and running with Laravel Herd.
as I already have a Redis server up and running with Laravel Herd. But you might want to use database instead. It's perhaps the simplest to set up. If so, just change to database, then go to your terminal, type php artisan make:queue-table. That will publish a migration, and then obviously you'll want to run php artisan migrate to make sure you have the latest changes.
and then obviously you'll want to run php artisan migrate to make sure you have the latest changes. That table now exists, and you're good to go. So now from the front end, if we create a Post, we'll use our little autofill, and let's say finding Nemo instead. We'll come down, hit create post, and then go to the post index and search for finding Nemo, hit enter. We get zero results.
and search for finding Nemo, hit enter. We get zero results. That's because obviously now it's been queued, and we haven't got our queue running, so that data isn't synced with MileySearch, and MileySearch cannot return it to us. So you'll have to remember when you set queue to true, you'll need a queue worker running in the background. Let's do that with php artisan queue:work, and you'll see it processes that make searchable task.
Let's do that with php artisan queue:work, and you'll see it processes that make searchable task. It's done in 26.53 milliseconds, so it doesn't take long at all, and if we come back and we refresh the page, we now have finding Nemo available. However, because it can take a little bit of time to sync, we should probably factor that into our controller. Let's imagine that our queue is busy, so it takes 30 seconds for our post.
Let's imagine that our queue is busy, so it takes 30 seconds for our Post to sync with the front end. That means that when a User creates a Post and they go to the index, they're not going to see the Post they just created. That's not a great user experience. So why don't we wrap this in an if statement? We'll say, look, if the request actually has a query parameter called query,
We'll say, look, if the request actually has a query parameter called query, then yeah, we'll use Scout to narrow down the search and find it, but if it doesn't, we're going to fall back to this older style of searching. So let's cut that out and we'll paste it in here and then I'll uncomment it. We don't need this section because this is to do with our old style of search, so I can remove that, and seen as paginate and with query string.
with our old style of search, so I can remove that, and seen as paginate and with query string are the same in both, why don't we remove that and then we'll call that once a little further down in the script. So perhaps here we could say posts paginate with query string. There we go. So that should give us a nice fallback. Let's test it out.
So that should give us a nice fallback. Let's test it out. We'll stop our queue to simulate the queue being slow. We'll head back to the front end. Let's create a post, autofill. Think of another movie we could use here, one that we've not used yet. Let's fill this out with the jungle book and we'll come down, hit create post. So that's not been synced with MileySearch
and we'll come down, hit createPost. So that's not been synced with MileySearch because we haven't executed the queue, but if we come back to the post index, yes, it is still there. If I search for jungle book, I'm not going to find anything. At least I'm not going to find the result I'm actually looking for, but at least I see it in the index.
I'm actually looking for, but at least I see it in the index. And then when I get around to running the queue or when the queue actually picks the job up and I come back and I search for jungle book again, now it will appear at the top of the search list. So I think that's a nice compromise, being able to run the two side by side depending on whether the user is actually searching or not. And this asynchronous manner.
depending on whether the user is actually searching or not. And this asynchronous manner in which Scout actually executes is just something you'll want to keep in mind because it will affect when things are available to search and find in your application. And with Scout and search implemented, folks, that's the end of the series. I know I'm pretty sad as well. I would have loved to just carry this on forever
I know I'm pretty sad as well. I would have loved to just carry this on forever because to be honest, there is no end to the features that you can add to something like.
