در حال بارگذاری ...

Introducing Mux AI dubbing0:00

Here's what I'm working on today. At Laracast, we recently switched video providers, and we're now using a service called Mux. So if I click play, now we're using the Mux player. Very, very cool. But I still have a bit more work to do. So have a look. If I switch over to the documentation, they offer support for automatic translation and audio dubbing using AI. Now, this is something we had before,

automatic translation and audio dubbing using AI. Now, this is something we had before, but now I need to implement it a little differently. We're going to use AI to do this, but then we're also going to pass off that implementation to codecs for a code reveal. Okay. So here's what we're going to do. I'm going to boot up Cloud, and I am going to call dangerously skip permissions because that's how I roll.

and I am going to call dangerously skip permissions because that's how I roll. Okay. So let's do this. I wanted to consume that file and learn how it's done, and then I want to ask some questions, and then we're going to get started. All right. So now, yeah, it could probably use context 7 if that's available, which I imagine it is, but also you'll find that on documentation websites,

Feeding docs to AI0:59

which I imagine it is, but also you'll find that on documentation websites, these days you will find a copy page link, which is entirely for passing off to AI. So I'm going to do that now. Consume this documentation for mucks, audio, translations. I'm going to paste that whole thing in. So I'm going to build up. I'm going to warm up the context. Okay. Now I'm going to use voice to speech.

I'm going to warm up the context. Okay. Now I'm going to use voice to speech. Let's see. We are already using mucks for our video embeds, but now I want to add audio translations and dubbing to these assets on mucks. So we need a way to review this documentation, follow the steps, create the translation for a set number of languages, and then I want to update the existing assets that we have on mucks to include this new track. So I want you to review this,

this new track. So I want you to review this, make a plan, ask me any questions you need to, and then we can get started. All right. I'm going to hit shift tag. So I'm going to use plan mode for now. And let's get going. All right. Let's see. Which language do you want to dub into? Yes. So why don't we chat about this?

Choosing target languages2:24

Which language do you want to dub into? Yes. So why don't we chat about this? I knew at some point I'd basically checked the traffic for Larracas to figure out what are most popular languages. And I was actually surprised. Things like Portuguese popped up in Denise and stuff like that. Which three languages do you think based on Larracas traffic would be the best choices for us? OK, so based on Larracas system global footprint, Spanish, Portuguese, and Japanese.

OK, so based on Larracas system global footprint, Spanish, Portuguese, and Japanese. Now, this is interesting because our Japanese traffic isn't that high. But again, maybe it's just because we don't have Japanese translations. OK, I'm going to accept this. And then always later, I can tweak it. Some of you watching right now are like, Jeffrey, you got to include this. And that's the problem with translations is there's always one more language you need to convert to.

And that's the problem with translations is there's always one more language you need to convert to. And it's not free. Anyways, all right. Let's go with those three for now. OK, so now it wants to know, are we doing a one time batch to update all of our existing assets? Or do we want this going forward as part of our pipeline? It's going to be both. OK, so we have our plan.

Implementing feature draft3:30

It's going to be both. OK, so we have our plan. And this time, very intentionally, we're going to go super fast because later we're going to have a code review from Codex. So to quickly scan it, it's going to set up an audio translations table to track the video and the language it was converted into, as well as the path to that audio dub. We can track our languages so that we can append to them later by simply updating one file. I already have an 11 labs service class.

by simply updating one file. I already have an 11 labs service class. So it's going to add a couple methods, which we might want to tweak that, but that's OK. And then ultimately, there will be a job that will get dispatched for each asset we have. And then ultimately, once it has this new translation, it will attach that to the MUX asset, which again is the video file. Attach the French translation to the video file if we offer French, which maybe we should.

Attach the French translation to the video file if we offer French, which maybe we should. Finally, we have a dubbing process command. We implement it into my upload pipeline for when we have new courses. And it looks like we're using a unique supervisor for our queue just so it doesn't overlap. Probably not needed, but it's OK. OK, so we're going to clear context, bypass permissions, and implement the first draft of this feature. All right, so all 14 tests pass.

and implement the first draft of this feature. All right, so all 14 tests pass. Good, pint is clean, and the only PHP stand error is pre-existing. All right, so AI is doing the old. It wasn't me approach, and that's OK. So let's see. On my 11 labs class, it added create dubbing, get dubbing, download dub. See, again, I feel like this should be its own 11 labs audio dub class or something like that. I find when you see suffixes like this repeated over and over,

or something like that. I find when you see suffixes like this repeated over and over, sometimes that's a hint that a class wants to break out. Anyways, Mux, same thing here. It updated my video model with a has many relationship. We have an observer here that auto dispatches dubbing jobs when a video is created and includes that OK. And then it added my supervisor to horizon. OK, so let's see. What are the next steps?

OK, so let's see. What are the next steps? All right, good question. Run the migration, verify the environment bars, restart horizon, and then test with a single video to make sure it works. Yeah, so they're recommending I find a video. I create a translation record that is set to pending, and then we dispatch a job to process that translation on the given queue. Even this, though, like this should be its own method.

on the given queue. Even this, though, like this should be its own method. I would want that to be like if I have a video instance, maybe there's a method called translate, and then I would provide the language I want to translate it to, or maybe even that's an enum. So I don't have to remember what these codes are. Should I have it work on that right now? Yes, but then after we're doing that, watch it progress in horizon, batch process everything, OK?

Yes, but then after we're doing that, watch it progress in horizon, batch process everything, OK? And then verify on the Mux player. Once the translation is complete, the Mux player should automatically show a language selector for that asset. OK, so that's the end goal here. All right, so before we get started, I just want to say, so for your test with a single video example code, I think that should probably be a method on the video model

so for your test with a single video example code, I think that should probably be a method on the video model instead. Why don't we call the method translate? It will accept the language code, or we could use an enum if you think that's appropriate. And that method would be responsible for delegating to the audio translation relationship and creating that record, and then it would also dispatch the job as well. That way we can do all of that code with a single method

that record, and then it would also dispatch the job as well. That way we can do all of that code with a single method call on the video model. All right, maybe a little verbose, but that should do it. OK, so now it's passing. And yeah, I think this looks better. We call translate. We have an enum that is now our source of truth. I don't have to remember what the relationship table is, or what the status should be, or what the job to dispatch is.

I don't have to remember what the relationship table is, or what the status should be, or what the job to dispatch is. I don't care. I just want to know that I can translate a video into a language. OK, so we're going to test this out, and hopefully it'll work. And then we're going to send it off to codecs for a code review. OK, new tab. Let's run my migrations. Cool.

Testing translation pipeline7:46

Let's run my migrations. Cool. Let's boot up Tinker. And we're going to find a video. So I can say app models video. We're going to get the latest one, but make sure it is a published one, not something that's scheduled. Give me the first one. All right, so silent money bug from Mars.

Give me the first one. All right, so silent money bug from Mars. This is from Jeremy's nightmare debugging course, which is cool. You should check it out. And now we should be able to say translate into app. And what was it called? Dubbing language. And we want Spanish. All right, so with any luck, this is going to dispatch a job.

And we want Spanish. All right, so with any luck, this is going to dispatch a job. It creates the relationship, of course. And then it dispatches a job to process this. All right, so now I'm going to boot up Horizon so we can get going. Or, of course, you can say pitch be arts and Q work if you're not using something like Horizon. It doesn't matter in this case. Yes, it started running.

It doesn't matter in this case. Yes, it started running. Oh, but it looks like it failed. OK, so we do have some issues that we need to fix. Good. So in table plus for that audio translations table we created, yeah, we can see that the status failed. And the reason is because call to undefined method gets static renditions. OK, that's the problem.

gets static renditions. OK, that's the problem. AI screwed up on this. OK, let's get back to work. So let's say you screwed up, son. And now I'm going to paste in that error. But of course, I could also say take a look at the logs or review the table. OK, and it looks like it fixed the bug. So we're going to try one more time.

OK, and it looks like it fixed the bug. So we're going to try one more time. I think I still have my video instance. Very quickly, let's do this. Let's clear out the existing try so we can start from scratch. So one more time, translate. That will dispatch a job. I will boot up our QWorker to process that job. All right, running and done. OK, so it didn't fail this time.

All right, running and done. OK, so it didn't fail this time. Let's see the status. OK, so now I fast forwarded about a minute just to let it finish, and it has. OK, so now we can see for this video, the status for Spanish translation is complete. Here's the S3 audio file, and here's the track it was attached to. So now with any luck, if I load this on the Laracast website, I should now have Spanish translations for this video.

So now with any luck, if I load this on the Laracast website, I should now have Spanish translations for this video. Let's have a look. All right, here we go. So let's disable audio, hit Play, and sure enough, we have our audio translation for Spanish. So let's give this a listen. [NON-ENGLISH SPEECH] I don't speak Spanish. How do you do?

Codex review skill workflow10:27

I don't speak Spanish. How do you do? Pretty neat, right? OK, so now we're going to assume this is feature complete, and the next step is to perform a code review. And we have a couple choices here. Now, one option we have here is we could perform the code review directly within the current context. And you and I probably do this all the time. Another option would be start from scratch.

And you and I probably do this all the time. Another option would be start from scratch. Create a brand new session with brand new context. Review all of your changed files and perform the code review that way. But actually, even better than that, I like to pass it off to an entirely different model like Codex. And that way, I get two unique perspectives with their own pros and cons.

And that way, I get two unique perspectives with their own pros and cons. And here's the way I do it, at least. I let Claude remain in charge, and Codex is simply a reviewer. It's going to say, hey, here's some things I picked up on. What do you think? But then ultimately, it's Claude's job to receive that information and determine whether or not it is relevant. So let's see if we can make this work.

whether or not it is relevant. So let's see if we can make this work. Here's what I want to do. Ideally, I just want to say codecs review, and that's it. I want it to be its own skill. So here's what we can do. We can either have Claude make the skill on its own, or you can do it by hand, whatever you want. Why don't we have AI do it for us? I want you to make a skill for me.

Why don't we have AI do it for us? I want you to make a skill for me. It's going to be called codecs-review. And its job is to request a peer review from the codecs model. And then it will implement the feedback and suggestions only if you feel they are worthy and valid. So which consists of a handful of steps. Step one would be send a request to codecs to review the uncommitted files. Step two would be present the feedback

to review the uncommitted files. Step two would be present the feedback in a clear and formatted way. And then step three would be for you to evaluate and implement the requested changes and updates only if you feel they are warranted and necessary. All right, let's give that a run. All right, let's create the skill. Notice it goes within a skills directory. You name the skill directory after what the skill is,

Notice it goes within a skills directory. You name the skill directory after what the skill is, and then the actual markdown file is capital skill.md. Now we can run it like this. Let's have a look real quick, though, at the file. OK, so let's see. Request a peer review from OpenAI's codecs model. Present the feedback, evaluate and implement only the suggested changes. Yeah, this looks reasonably good.

the suggested changes. Yeah, this looks reasonably good. So we request the review. It's going to take a look at uncommitted files. Inform the user if there is an issue. Present feedback, evaluate each piece of feedback. You're the final arbiter. I really like this. So it's letting Claude remain in charge. It receives feedback, and it decides whether it's warranted,

So it's letting Claude remain in charge. It receives feedback, and it decides whether it's warranted, because often it'll have more context built up. Do not blindly apply all suggestions. Yeah, I think this is really good. The only thing I might tweak, though, is I want to give a little more of a prompt to codecs. So I'm going to swap that out with this. All right, here we go. So we're still going to do codecs review.

All right, here we go. So we're still going to do codecs review. It's going to check the uncommitted files. But we're going to just give it more feedback. This is what you specialize in. Here's what I want you to provide for feedback. Be specific, reference exact lines. Don't comment on formatting. Blah, blah, blah. You get the idea.

Blah, blah, blah. You get the idea. Now, I think this is reasonably good to get started. But one thing I want you to be aware of is if you're using codecs, make sure that it has access to the same guidelines that you're using for Claude. Remember, at the time of this recording, it can be tricky. Claude will use its own guidelines file. And a different agent or model will use agents.md and not

Claude will use its own guidelines file. And a different agent or model will use agents.md and not Claude.md, and it's kind of a big pain, to be honest. Just make sure that you have those represented. You can do that by simply copying your guidelines file. You could set up a sim link. Or if you're using a layer of all boost, it'll take care of this for you. Let me show you. If you run PGP Artisan Boost Install,

Let me show you. If you run PGP Artisan Boost Install, you'll see that it will prompt you for which editors and models you want to use. So here I'm using Claude and codecs and PGP Storm. And that way it'll set up the duplicate guidelines for us. So you just run through this whole thing. It'll detect what you're using. It'll build up that guidelines file. And now codecs will also have access to those guidelines.

It'll build up that guidelines file. And now codecs will also have access to those guidelines. OK, so I'm ready to give this a shot. First, before we run it, let's say, commit. And this will be smart enough to commit only the new skill that was created. And that's done. OK, great. So now let's boot up Claude one more time. And we're going to run our codecs review.

So now let's boot up Claude one more time. And we're going to run our codecs review. Let's take a look. OK, and now that's done. Yeah, so this is the general workflow, right? So to repeat, stage one, have your agent do the work. Amazing. Step two, pass off that work to a second model, something like codecs or grok. If you want to use grok, don't use grok.

something like codecs or grok. If you want to use grok, don't use grok. For a code review. And then stage three, perform a human review. Go over all of the files. Look for alarm bells. Look for a squint your eye. See if anything pops out at you. That just doesn't seem right. And when you're done, at this point, you can finalize.

That just doesn't seem right. And when you're done, at this point, you can finalize. So in my case, I have a skill. I call it finalize. And here's what it does. And yeah, this is just a simple command. Very, very simple. It starts by running composer run format dirty, which basically runs rector in PHP stand and pint only on my uncommitted files.

runs rector in PHP stand and pint only on my uncommitted files. It runs our full test suite. If that fails, it's going to let me know. I need to know that. Otherwise, if the test pass, it can go ahead and run the git commit for me while providing the message. And there we go, formatted, tested, and committed. That's the workflow.

That's the workflow.

دوست دارید گاهی خبرهای Laracasts را ایمیل کنیم؟