Finding the CFP API0:00
So while Symposium has this baseline need to have a list of conferences I can potentially apply to, I as the creator of Symposium have some particular requests, which is, I don't want to have to keep track of all the conferences that are interesting and add them all manually. So we want to go look for a source of data for that sort of information. And lo and behold, we've got one. It's called callingallpapers.com. So the first thing I want to do is go take a look and see, do they have an API that we can work with? Now for those who aren't familiar, a call for papers is a request from a conference asking people to propose a talk or a speech that they want to give at this conference. So if you are a conference speaker looking to speak at conferences, you want to look for CFPs. And that's exactly what callingallpapers.com is for. As far as I remember, it's run by the people who ran join.in, which was also some old school PHP people. So I think this is run by some old school PHP people. I can't say for sure. I do remember that I found this API. When I went here to build this course, I actually couldn't find any details about the API anywhere. I must have asked somebody a favor. But the good news is I came here. It does say the API is powered by DigitalOcean, which does tell us that there's an API of some sort. So there's two things I would do first. I would either just try typing api. to see what I can have or a view source to see if this is powered by JavaScript. So let's just get started by doing api.callingallpapers.com. And there you go. It's telling us that this is how they do everything. They start with a version and then they say CFP. So it looks like the endpoint is for CFP. And it looks like it's showing all of the CFPs.
Fetching API Data in Laravel1:21
So let's just get started by doing api.callingallpapers.com. And there you go. It's telling us that this is how they do everything. They start with a version and then they say CFP. So it looks like the endpoint is for CFP. And it looks like it's showing all of the CFPs. I'm guessing it's one of these where if you request it and your browser is looking for HTML, it says it basically requests and accepts HTML. You're going to get this HTML version. And if we were to request it with a system asking for application/json, then we would get a JSON version of it. So let's take a look. So the good thing is we can use php artisan tinker to play around with any Laravel calls we might want to try out. So if we were to use the HTTP client built into Laravel, you can just say HTTP::get and then put a URL in here. So let's try that. Let's see what we get. Is it body? Right. So we say, OK, well, we're already getting JSON, which is what we weren't getting in the browser. But let's use the json helper instead.
Right. So we say, OK, well, we're already getting JSON, which is what we weren't getting in the browser. But let's use the JSON helper instead. And now we have a destructured data, we can see it's very helpful structured data that we can work with CFPs and underneath CFPs, it's an array. Every item in that array has very helpful structured data with the dates and latitude and longitude is all great. If it was an API that didn't sense that we were looking for JSON, you can explicitly say HTTP Accept JSON. And then it would communicate, you know, because different APIs can look for are you looking for JSON different ways? It would be communicated even more strongly. That's we're looking for. But we don't need to. We've already got that right there. So we can see that in an HTTP client, we're able to get a very nice structured data from here that we can work with in our application. So let's talk for a second about what kind of an API client we actually want to build. Normally with API clients, we call them an SDK software development kit.
Choosing an API Client3:08
So let's talk for a second about what kind of an API client we actually want to build. Normally with API clients, we call them an SDK (software development kit). And that just means a piece of code that makes it easy for you to interact with this particular API in a structured way. That is predetermined often by the people who created the API, but not always. Usually when we're going to deal with an external API like this, we're going to build some kind of a structure around it. And the structure could start very simply, we could just look for file_get_contents. But it's pretty dumb. It's just a function that reads the contents into a string and echoes it back out for you. The moment you need any kind of complex structure, you're going to be disappointed you're using file_get_contents, let alone if you want to do any kind of testing and mocking.
The moment you need any kind of complex structure, you're going to be disappointed you're using file_get_contents, let alone if you want to do any kind of testing and mocking. There's Guzzle, which is the oldest API tooling you can have in the php world. Very powerful, very flexible. Also, massive overkill for a large number of projects and not exactly written the way that Laravel developers tend to want to write their code. You can use Laravel's HTTP client, which is based off of a previous open source project called ZTTP that Adam Radlon wrote. But it was pulled into the core as the HTTP client, and that's what we use just there in the test. And it makes it really easy for you to make simple GET and POST calls and pull various things out of them in a very kind of Laravel friendly way. Or you can use Saloon, which is a structured package for how to build API integrations and SDKs around bigger, more structured, more complicated, well-defined APIs. And if you are building anything with any more than a few calls, especially if it's going to be passed around for a team,
Or you can use Saloon, which is a structured package for how to build API integrations and SDKs around bigger, more structured, more complicated, well-defined APIs. And if you are building anything with any more than a few calls, especially if it's going to be passed around for a team, Saloon is a really powerful way to set up a pretty complicated set of interactions and tests and mocking and everything like that around an external API. The only downside to Saloon in my personal experience, which is kind of limited with Saloon, because it requires you to build a pretty decent number of classes to get spun up, which is going to be valuable in the vast majority of settings. You're basically going to define a connector for the API and every single request you might make, or whether it's, you know, get a list of sites or whatever, is each going to have its own PHP class. And that's fine. Makes a ton of sense in a lot of contexts, but for something super simple, which it turns out the symposium thing is, I don't always want to go the whole way out for Saloon because it's just another dependency with a couple classes that don't always necessarily make sense. So no criticism of Saloon at all. I think it's great.
I don't always want to go the whole way out for Saloon because it's just another dependency with a couple classes that don't always necessarily make sense. So no criticism of Saloon at all. I think it's great. But personally, I tend to write code where I say, if it can be really simple code using the dependencies we already have in the app, I'm going to start there before I reach for a more structured external dependency. Because then that's just something you have to keep upgraded and keep looking good and everything. So, but these are your options. I would say more complicated SDKs probably should reach for Saloon. If you got something a little bit simpler, and that's a lot of things, it's not a bad thing. It's not pejorative. It's just, if it's simple enough, I'd use the HTTP client built into Laravel. I used to use Guzzle more. I would use it less today because anything I can do with Guzzle, I can do with Laravel's HTTP client, which is based on Guzzle, just has a more enjoyable syntax.
Creating a Wrapper Client6:06
I used to use Guzzle more. I would use it less today because anything I can do with Guzzle, I can do with Laravel's HTTP client, which is based on Guzzle, just has a more enjoyable syntax. And I would say just never use this. Don't ever use file_get_contents. There's no reason to use it today. One of the tools we've relied on often in the Laravel community is the idea of creating a wrapper PHP class around your API calls. And this is because testing is significantly easier when you do that. If you could imagine a test where you'd say, given this set of data from the API, our app should do these things. Well, it's very hard to do that when getting a set of data is just a file_get_contents call. Are you going to make a fake API to return that? But if you've got a wrapper class, that wrapper class can be just swapped out in your test using something called mocking. And you can say, instead of giving the real wrapper class that's making calls to this API, you can say, give me a fake version of that wrapper class.
But if you've got a wrapper class, that wrapper class can be just swapped out in your test using something called mocking. And you can say, instead of giving the real wrapper class that's making calls to this API, you can say, give me a fake version of that wrapper class. And then that fake version, you can instruct it to give test data that's specific to this test you're working on right now. So let's just take a quick look at what one of those might look like. Everyone does a little bit different, but let's just call it services right now. You can call it integrations or whatever else. And in there, we're going to create a class called CallingAllPapers.php. OK, so in here, you usually have a constructor. If you're working with Guzzle, you would inject a version of Guzzle in here. If you're working with the Laravel HTTP client, you usually just call the facade directly.
If you're working with Guzzle, you would inject a version of Guzzle in here. If you're working with the Laravel HTTP client, you usually just call the facade directly. But if you didn't want to, just a quick tip, if you ever want to look at the class that backs a facade, you can just go to Laravel documentation and you can look up facades. And down at the bottom, there's a class reference. And so if you want to know what class the HTTP facade is backed by, it's the Illuminate\Http\Client\Factory class. If you wanted to inject it, you would just do this and you'd call this HTTP, right? You can also just call the facade directly if you're going to be swapping out in the way that we're going to be swapping it out shortly. It's really up to you. Usually, you also set either in a constructor or some kind of high-level protected property a base URL.
It's really up to you. Usually, you also set either in a constructor or some kind of high-level protected property a base URL. And there's some other complexity we deal with sometimes, but usually it's something like this, API.CallingAllPapers.com.v1.php. So for now, we actually don't need a constructor. What I often will then do is come up with a method that's basically call. And call usually has method and path and data. And then sometimes you have options or headers or something like that. So your data can be empty. Your path can't be empty.
So your data can be empty. Your path can't be empty. If you like type hinting, you can do this. And what call is going to do is it's going to basically do the HTTP call to this particular thing. And then it's going to return whatever comes from that. So we've already kind of noted that the main thing we're going to want to get from here is the CFPs. We're going to call them conferences internally, though. So we're going to say something like this. call.get.
Call. Get. And then whatever the endpoint is, which actually I think is just the root, right? Because it's just /cfp. And we don't need to pass it any data. Great. So in here, then we kind of build using whatever tool that we're working with. How do you build a call using method get out to the path, you know, base URL https:// in this case, optionally passing that data in. And the thing is, the moment this gets significantly complex, that's when you need Saloon.
slash in this case, optionally passing that data in. And the thing is, the moment this gets significantly complex, that's when you need Saloon. But if you're like, I can do everything I need in here for this particular app, I can work with this. And the cool thing is you can start with something really simple using the HTTP client and add Saloon later if you want. That's fine. So let's say return HTTP. Oops. And actually, in this case, we don't even need to worry about the method because we literally only have a get. You know what, we're not going to worry about this again. I'll often build something like this when we need it.
You know what, we're not going to worry about this again. I'll often build something like this when we need it. But I'm pretty sure we have a single call. So we're actually just going to say return response()->json(). That's it. I guess we can say JSON and say this is going to return an array if we want, right? But we don't need Saloon. We don't need this hyper-architected thing where we're injecting everything like that.
But we don't need Saloon. We don't need this hyper-architected thing where we're injecting everything like that. Because, for example, this HTTP client makes it super easy in a test to swap it out. So if we say, hey, anytime you make this particular, a call to this particular endpoint, we need different data. You can just do that. We'll look at that in just a bit. So we don't even need the level of complexity I was thinking we were going to need coming into this because it's just not that complex of a set of calls. And if it were, then we could make this more complex. But we don't need to add all of that. We don't need it for this particular API.
But we don't need to add all of that. We don't need it for this particular API. And whoops, I just realized we need to append or prepend the base URL. Now that we're making it so simple. And we actually don't even need the slash after it because this is, I actually think this is what their base is. It's V1 slash and then this is going to be CFP. That's my gut. So I think what I'd like to do is to make a command called importConferences. And we can give it a signature, which is CFP's import.
Import Command and Upsert11:10
So I think what I'd like to do is to make a command called importConferences. And we can give it a signature, which is cfp:import. And here, let's just start by injecting callingAllPapers. And then just see we're getting. All right, looks like we're getting everything. And I think it's going to be cfp. The sub item is going to be cfp's underneath it. Oops, I accidentally grabbed something. Let's just try it. Is that an object?
Let's just try it. Is that an object? There you go. So we can basically loop over those and for each of them, we can process them. So we'll say for each CFP's conferences, CFP's as conference. And then this importConference was really import or updateConference. And then we build that out. You know, it's an array. And you see the shape right here. You see the shape right here.
And you see the shape right here. You see the shape right here. And then we basically build out a class that says for a conference with something unique. See if we have that one internally. And if we do import it, and if we don't, don't import it. So you want to look for a unique ID if possible. Looks like they don't have a unique ID from their system. Look a little bit more closely. You do have this right here. So this is probably going to be unique on everyone,
You do have this right here. So this is probably going to be unique on everyone, because this is the URL for this specific conference. So even though we're not getting this ID right here, anywhere else up here, we can pull it out of the realm. So we can say conference, rel, CFP, URI. And now we have a unique ID for every single one. So we basically can add that to the URL. So we basically can add to our migration a calling all papers. So calling all papers_ID, set it to this.
So we basically can add to our migration a calling all papers. So calling all papers_ID, set it to this. And then every time we run an import, we'll just say, hey, does this exist? If so, update it. And if not, then create a new one. This is a great use case for Laravel's updateOrCreate. So the great thing about updateOrCreate is we pass in here what is going to be the search. So this would be basically calling all papers ID equals whatever. What you pass in here is what you're either updating or what you're passing as additional parameters to the one you're creating.
What you pass in here is what you're either updating or what you're passing as additional parameters to the one you're creating. So we can say something like, and we won't build the whole thing out here because it's not worth you watching me build the whole thing. Something like conference, update or create. Remember, you had two sets of parameters here, right? So this first set of parameters is something like calling all papers ID. And then it's going to be this. And then the rest of these are going to be all the other parameters.
And then it's going to be this. And then the rest of these are going to be all the other parameters that we want to pull in from the import. So for example, if on our side, it's named name, then you say conference. Well, do they call it name? Do they call it title? They called it name, right? And basically bit by bit, we use these things to create a local version of this event in our system. And now that we have this as a command, we can go schedule it. And if you have not worked with Laravel 11 before, you might be looking for ConsoleKernel, which doesn't exist. But instead, we're going to go to routes/console.php.
you might be looking for ConsoleKernel, which doesn't exist. But instead, we're going to go to routes/console.php. And here, this is where we're going to schedule things. So we can just say schedule, command, CFPSImport, and then we can do whatever we want daily or whatever else. And so now if we have our Laravel scheduler set up, which you can take a look at the Laravel box for instructions for how to do that. It's super easy. It's either a cron command or if using Laravel Forge, there's just a button for it. Then once that's set up, this is just going to define that this ConferenceImporter should be run once a day.
It's super easy. It's either a cron command or if using Laravel Forge, there's just a button for it. Then once that's set up, this is just going to define that this ConferenceImporter should be run once a day. So once a day, it's going to go make this call out to this external API. It's going to use our CallingAllPapersClient, which turned out to be so simple. We didn't even need something like Saloon or Guzzle, whatever else. And then for each of those, it's going to either import or update the Conference. It's going to look that Conference up in the database by its CallingAllPapersID, which we would have to add the migration as just a simple string. And then either going to update all of these fields, or it's going to create a new one with those fields depending on how it's set up.
And then either going to update all of these fields, or it's going to create a new one with those fields depending on how it's set up. Again, I could walk you through all of that. That's the least interesting part about it. The most interesting part about it for me is just our relationship with APIs. How do we find the API? How do we figure out how it works? What tools do we build around the API? The structure of using an import conferences command, that command calling a little internal class SDK that we have. Those are the things that are most interesting.
Testing and Mocking Imports16:18
that command calling a little internal class SDK that we have. Those are the things that are most interesting. And so as a result of that, what I do also want is I want to make sure that we have some tests and show what it looks like to write tests around something like this. Now, often it's kosher in the PHP community to make any method like this that's not going to be called by the public as a protected method. But one of the fun things about just keeping it public is it makes it super easy to test. So let's say what we wanted to do was just grab some data from the API and write a test that mocks that if you send that data in, it does a certain thing. Well, it's super easy to do because we can just create an instance of ImportConferences.
and write a test that mocks that if you send that data in, it does a certain thing. Well, it's super easy to do because we can just create an instance of ImportConferences and we can write tests against this public method import or updateConference. Or we could do it against this one right here. But then we would actually have to mock the calling AllPapers API, which again, super easy to do because it's a class here. But because this right here doesn't even require the calling AllPapers API at all, our SDK at all, we can not even worry about that and literally just pass it in an array. Or we can do both. First one can test just this method. And then the second one can actually mock it if we need to do it, however we want to handle it.
Or we can do both. First one can test just this method. And then the second one can actually mock it if we need to do it, however we want to handle it. So let's go build a test real quick. All right, about five minutes have passed. What I did was I generated a addCallingAllPapersIdToConferencesTable migration. And now we are ready to actually build a test around this. Let's make a test. We're going to call it ImportConferencesTest. All right, we're going to say it imports a conference. So we're going to create a new instance of that doesn't have any dependencies.
All right, we're going to say it imports a conference. So we're going to create a new instance of that doesn't have any dependencies. So $cfp equals new Import. Actually call it $command equals new ImportConferences. And then what you want to do is assert that given a certain set of data, you get a certain outcome in the database. So let's do. So what was our data? It was $rel equals. And then under there, there was $cfpUri.
It was rel equals. And then under there, there was CFP_URI. And that was our ID, right? And I think that the ID technically was really a string, but it had their ID at the end of it. So there we go. We have that. But it also had other data that we find interesting. And right now we're just going to do fake data working with the name. This is the new name from the API.
And right now we're just going to do fake data working with the name. This is the new name from the API. I guess not. This is the name from the API. Great. So now we want to test it. So we want to say command. And then we're testing this particular command, right? importer update conference. And then data.
Importer update conference. And then data. And then after this, we want to test that our database looks a certain way. So you can say something like $first equals conference first and run your assertions against that. So let's just try that. All right. So fails. And you must have a title. We're already off to a bad start.
And you must have a title. We're already off to a bad start. conferences table. Because look at all these non-nullable things. Woof. So for the sake of time, we're going to make all these nullable. And we're going to go from calling a name to calling a title. So let's migrate that out and run our test. And in PHPUnit, you could pass a filter to only this particular name. If you want to run just a single test in tests, you just do only.
And in PHPUnit, you could pass a filter to only this particular name. If you want to run just a single test in test, you just do only. And now that's the only test that's going to run. So it says undefined array key name. And the data we're passing in. Oops, this is not where I was supposed to change it. This is name because that's what's coming from the API. And then this is title because that's what we're saving in the database. All right. There we go.
All right. There we go. So we are getting a single entry out of the database. And that single entry is what we just created. So we could say something like this assertEquals firstName and then dataName. Nope, not firstName, firstTitle. And what we're trying to assert here is just basically saying we want to make sure. You could say also assert that there's only one entry in the database or whatever else you want. So that could be your first test. And your second test could be something like.
So that could be your first test. And your second test could be something like. It updates the Conference. So let's say we already had this one. So we can say something like conference::create. And then we'll say the title will be originalDBTitle. And the callingAllPapersID will be. The string right here. So then we create one that looks like this. And then we update the API.
So then we create one that looks like this. And then we update the API. And so, first of all, we can pull it and we make sure that the title from the database equals this new one, not the old one. Then we can also say something like this $cert equals 1. Conference::count(). We run a test on that. And we've now got two assertions passing. Now, if you did want to do this with mocking, you could do it a few ways. You can do mocking the specific class.
Now, if you did want to do this with mocking, you could do it a few ways. You can do mocking the specific class. So you can say this instance and you're going to pass in our calling all papers SDK class. And then your second parameter will say mockery::mock, calling all papers. And in here, I would say mock should receive conferences once and return. And then it's basically your data here. And so you're capable of mocking out that higher level class. You can also, however, mock the HTTP client. If you had a testing, you can see faking responses and you do HTTP::fake. And then if you ask for a specific URL, then under that specific URL, you can give a certain response.
If you had a testing, you can see faking responses and you do HTTP fake. And then if you ask for a specific URL, then under that specific URL, you can give a certain response. You know, we could do you could just say calling allpapers.com/star. And that'll be HTTP response with our data. And that's it. So we now have three ways to test this. We can test it at the wrapping service layer. We can test it at the HTTP client layer. Or as we've done here, we can just test a single method in here and make sure that it gets what we want. And it's all kind of down to where the complexity lives in your application and what you're really testing for.
Or as we've done here, we can just test a single method in here and make sure that it gets what we want. And it's all kind of down to where the complexity lives in your application and what you're really testing for. This is a 4H loop. It's not super important that we test this before it gets to this. And this is where the majority of the complexity is happening in this particular version. So that's why we're testing that here. So we did it. We build a pretty rudimentary but still extremely powerful php wrapper class. We can call it an SDK if we want. A client that wraps in this particular instance, the HTTP client for Laravel.
We can call it an SDK if we want. A client that wraps in this particular instance, the HTTP client for Laravel. We integrated it together with the command that works together with our Eloquent object to consistently either create or update these conferences every single time we get new ones in. It's scheduled to run once a day. And we built tests using three different methods to make sure that it creates and updates conferences or CFPs when we get them. It's pretty powerful. You get a lot more options that you can build on from here. But I think that's got you off to a really good start for integrating with external APIs.
