Full Transcript
https://www.youtube.com/watch?v=XcuvwueNGzo
[00:00] AI has just changed the way we write.
[00:02] AI has just changed the way we write Playwright test scripts.
[00:04] Playwright test scripts.
[00:04] And it is not going back.
[00:07] I don't see it going back.
[00:08] In the next 12 minutes, I'm going to explain you how Playwright agents plan itself.
[00:15] And then generate the test scripts on its own and heal the test scripts whenever there is a failure.
[00:20] So, we're going to understand how this orchestration happens autonomously using Playwright MCP servers and the existing LLMs.
[00:28] Follow along with me.
[00:31] I highly recommend you to implement every step that I say in this video so that within like 12 minutes, you will have a clear hands-on and a demo to present it during your interview or to add it to your portfolio or to present it to your team.
[00:45] So, let's jump into the demo.
[00:47] Right, we should understand that AI isn't replacing quality assurance engineering.
[00:52] It is replacing quality engineers who don't use AI.
[00:54] You might have heard this phrase many times by now in 2026.
[00:57] Uh it
[01:01] phrase many times by now in 2026. Uh it applies to everybody in our industry.
[01:04] applies to everybody in our industry or everybody actually most of the industries.
[01:05] So, this is true actually.
[01:07] Imagine if you don't know what is the difference between MCP servers and how to use AI agents and what are all included that makes the agent in itself.
[01:17] You will definitely be left behind.
[01:20] You cannot work the way it operates and the way it will operate using the AI.
[01:26] That's exactly what you're going to understand in this video.
[01:27] Uh and to to set this first, as I say it's going to be a hands-on video.
[01:34] I wanted to open a VS Code.
[01:37] I'm using VS Code Insider.
[01:40] You can use VS Code as well.
[01:42] And create a new window.
[01:45] And in the new window, you can now open a new folder.
[01:49] Uh click the click open and I'm going to launch a new folder.
[01:51] Here I will call it this as um maybe uh Playwright AI demo 3 because I had previous two
[02:04] AI demo 3 because I had previous two hands-ons.
[02:06] So, I'm going to go with three and click on open.
[02:08] You can give three and click on open.
[02:08] You can give whatever the name that you want.
[02:11] Let me maximize this window a little bit.
[02:14] Right, now this is a completely a new and empty um window that you can see there.
[02:18] What I'm going to do, I have a chat.
[02:22] If you have VS Code, you will have the chat and Microsoft gives by default few tokens to play around.
[02:29] You can use that.
[02:31] In my case, I'm using a Claude.
[02:33] So, I have I went to the extension.
[02:36] If you want to get Claude for your test automation, you can go to extensions, click on Claude and then set it up and you will get this.
[02:41] You just have to log in or if you wanted to go with the default one, you can you can the way you have to do is you can click on the profile.
[02:47] Um just a sec.
[02:49] So, you can click on the profile here and then you can log in with your GitHub and you will get the GitHub Copilot here.
[02:59] You can do any one
[03:05] GitHub Copilot here.
[03:05] You can do any one setup.
[03:07] In my case, in this example, I'm setup.
[03:07] In my case, in this example, I'm going to use a Claude.
[03:10] So, let me go going to use a Claude.
[03:10] So, let me go back to the folder.
[03:11] back to the folder.
[03:11] It is completely empty.
[03:13] It is completely empty.
[03:13] Um so, you don't have to worry about any any folder structures.
[03:16] Um so, you don't have to worry about any any folder structures.
[03:17] We're going to build everything from the start.
[03:19] build everything from the start.
[03:19] There are three prompts that I'm going to use.
[03:21] are three prompts that I'm going to use.
[03:21] I'll tell you before I jump into that, I hope you are ready with this.
[03:23] I'll tell you before I jump into that, I hope you are ready with this.
[03:25] hope you are ready with this.
[03:25] Um you're you're you're ready with this empty folder.
[03:28] Um you're you're you're ready with this empty folder.
[03:28] If not, pause the video, set up the Claude or Copilot and then resume the video.
[03:30] If not, pause the video, set up the Claude or Copilot and then resume the video.
[03:32] video, set up the Claude or Copilot and then resume the video.
[03:32] Now, I'm going to go back to the PowerPoint and we'll see what we're going to do uh next.
[03:35] then resume the video.
[03:35] Now, I'm going to go back to the PowerPoint and we'll see what we're going to do uh next.
[03:38] go back to the PowerPoint and we'll see what we're going to do uh next.
[03:38] what we're going to do uh next.
[03:40] uh next.
[03:40] If you look at the prompt to text models, the way it works is that the LLM reads the prompt.
[03:42] If you look at the prompt to text models, the way it works is that the LLM reads the prompt.
[03:44] models, the way it works is that the LLM reads the prompt.
[03:44] I'll show you exactly how it does.
[03:47] the LLM reads the prompt.
[03:47] I'll show you exactly how it does.
[03:50] exactly how it does.
[03:50] We can say anything.
[03:51] We can say anything.
[03:51] Log in as admin, generate mock user data, asset dashboard, anything.
[03:53] Log in as admin, generate mock user data, asset dashboard, anything.
[03:55] generate mock user data, asset dashboard, anything.
[03:55] It will understand our context and it will use um MCP server.
[03:58] dashboard, anything.
[03:58] It will understand our context and it will use um MCP server.
[04:00] our context and it will use um MCP server.
[04:00] Here we are going to use MCP server.
[04:02] um MCP server.
[04:02] Here we are going to use MCP server.
[04:02] MCP is a is a connectivity
[04:07] MCP server.
[04:07] MCP is a is a connectivity portal for um let's say you have you portal for um let's say you have you have built an app and you want the AI to talk to your app and work on it.
[04:12] So, you build this MCP server.
[04:14] So, it's a portal through that the AI the LLMs talk to your application and get the work done and gets the result and put it back to the customer's chat the front screen.
[04:27] That's what it does.
[04:27] And if I go to further as I said in this video, we are going to use Playwright's test agents.
[04:36] So, it is kind of a trio.
[04:36] It has a play planner and it has a generator and it has a healer.
[04:43] This is exactly what we're going to see in this video.
[04:44] So, planner analyzes the DOM and the goal of the test and generator writes the Playwright actions and assertions and healer executes the test, catches the failure and loops back to fix the code if there is anything that fails.
[05:04] And all by running this command.
[05:05] We're going to run this command to set up the Playwright agent.
[05:06] So, now I'm
[05:08] up the Playwright agent.
[05:08] So, now I'm having an empty folder here.
[05:12] I mean having an empty folder here.
[05:14] I mean not this one, this one.
[05:14] I don't have anything written.
[05:17] We're going to build from the start, but if you have an existing framework like this, you can you can also install the agents on your own.
[05:27] I mean from in the middle of your framework or you can add it as an add-on and an additional feature.
[05:31] So, what we are going to do in the VS Code is that we're going to run this command npx playwright init hyphen agents hyphen hyphen loop equal to VS Code.
[05:41] The way you do that is you can go to your VS Code.
[05:46] You can launch a terminal.
[05:48] Okay, you can open a new terminal and you can paste it here.
[05:50] There are prerequisites that are required for this.
[05:51] So, what I would suggest, you need to have a node and other things installed.
[05:58] So, the easy way to do that is that you go to your Claude.
[06:03] Let me maximize.
[06:04] Let me zoom a bit.
[06:07] You go to your Claude or your Copilot and then paste this command.
[06:09] and then paste this command.
[06:12] npx playwright init hyphen agent or you can also use
[06:14] can also use your terminal.
[06:16] your terminal. If you have the prerequisites done, I'm going to press enter.
[06:19] prerequisites done, I'm going to press enter.
[06:22] So, what it will do is that it will install all the agents, the
[06:24] will install all the agents, the Playwright AI agents.
[06:26] Playwright AI agents.
[06:28] AI agents. As you can see, we got the folders here.
[06:30] folders here. And if I can expand, you can see the agents and within the agents, you got a
[06:32] And if I can expand, you can see the agents and within the agents, you got a generator agent, you have a healer agent
[06:35] generator agent, you have a healer agent and you have a planner agent.
[06:38] and you have a planner agent. The way to differentiate agent and the skills are
[06:40] differentiate agent and the skills are like agent will have a keyword.agent.md.
[06:42] like agent will have a keyword.agent.md. And if I click the agent, it will also have tools.
[06:47] And if I click the agent, it will also have tools. Okay, what tool to use for what?
[06:49] have tools. Okay, what tool to use for what? Okay, so let's see here. You need to use
[06:52] what? Okay, so let's see here. You need to use the Claude for you can go you can update this based on your um the the AI LLMs that you have.
[06:55] to use the Claude for you can go you can update this based on your um the the AI LLMs that you have.
[06:56] this based on your um the the AI LLMs that you have. You can make it as 4.6 or GPT or whatever it is.
[06:59] that you have. You can make it as 4.6 or GPT or whatever it is.
[07:02] You can make it as 4.6 or GPT or whatever it is. Opus 4.6 or whatever.
[07:05] Opus 4.6 or whatever.
[07:08] whatever. And if you look at it as the tools,
[07:11] And if you look at it as the tools, right?
[07:14] So, that means it is an agent.
[07:18] Okay, skills normally instructs the agents what to do.
[07:20] Like how to do it.
[07:22] Basically, that's the skill.
[07:24] Not actually what to do.
[07:27] It's like how to do it.
[07:30] So, now you got all the agents and all we did is like we ran this command and it took care of all the installations for us.
[07:33] And the first prompt that I'm going to paste, as I said we're going to implement this by like three prompts and this is the first prompt I'm going to paste.
[07:39] This prompt is like initialize a new Playwright project with MCP agents because your VS Code may not have MCP servers activated.
[07:50] If it's activated, that's well and good.
[07:52] It's going to identify and start the MCP server.
[07:55] So, initialize a new Playwright project with MCP agents for VS Code integrator, install the necessary dependencies including Playwright because if you look at this, we don't have Playwright installed yet.
[08:07] Okay?
[08:09] So, we need the browsers and Playwright to be installed.
[08:09] Run the seed test to validate the setup
[08:12] Run the seed test to validate the setup and start the MCP server for local testing tools.
[08:14] So, I'm going to hit enter.
[08:16] So, this will make sure we get all the Playwright here.
[08:18] While it's installing it, I'm going to reiterate on what is MCP server.
[08:20] MCP server, it makes this LLM that you see here like Claude to talk to Playwright instead of us writing all the Playwright scripts.
[08:21] It's going to understand the whole Playwright library and it does the setups.
[08:23] It writes the test cases.
[08:24] It plans and it heals.
[08:26] It does everything.
[08:29] So, similarly, we have MCP servers for many apps in today's world.
[08:32] So, that's the advantage of the MCP.
[08:34] It's called model context protocol.
[08:36] Okay?
[08:38] It's a rule that set that has set for this model context.
[08:40] So, what you need to do um by following the user instructions and connecting to the app.
[08:43] So, that's the MCP that connects.
[08:45] Like an USB, you plug your iPhone charger, so you get your
[09:12] your iPhone charger, so you get your iPhone charged.
[09:15] So, you you plug your TV charger, your TV gets charged.
[09:17] So, that's that's what it is.
[09:20] I've explained MCP servers clearly on my other video.
[09:23] So, please have a look at it if you have questions.
[09:26] But yes, we were we were waiting for this to be done.
[09:30] So, now we can see that Playwright has set up the server and it is also running.
[09:34] Playwright MCP server started in the background via npx playwright run test hyphen Um sorry, run test hyphen MCP hyphen server.
[09:47] Now, this is done.
[09:49] Now, the second prompt I'm going to try is the main one where we are going to plan, generate and heal all three whenever there is a failure.
[09:57] And lastly, we're going to fail it and then we're going to use healer to correct on its own.
[10:03] So, here's the prompt.
[10:06] And it is about OrangeHRM dashboard.
[10:08] So, it's the globally used open-source website for like test automations to practice things.
[10:12] So, the
[10:14] automations to practice things.
[10:15] So, the AI will let AI will let itself understand what is this OrangeHRM dashboard is.
[10:21] If not, get the URL from the web page.
[10:23] So, here what I'm saying is I need to write a new test for the OrangeHRM dashboard.
[10:26] Please act as the orchestrator and follow the strict lifestyle.
[10:29] First, read the rules in playwright.testplanner.agent.md, which is this one, okay?
[10:37] So, I go here.
[10:39] I'm asking it to read the rules.
[10:42] There is a set of rules defined by the playwright by themselves.
[10:43] And analyze the target page and output step-by-step strategy for the test and stop and wait for my approval.
[10:46] Once I approve, switch to the healer, sorry, the plan the generator.
[10:51] Okay?
[10:54] So, the first one was the planner actually.
[10:57] So, I was clicking the wrong file.
[11:01] So, the first one was planner.
[11:03] So, plan the test cases, read my DOM of OrangeHRM, understand what needs to be automated there, and then wait for my approval.
[11:05] Once I approve,
[11:18] wait for my approval.
[11:20] Once I approve, use the generator.agent,
[11:22] use the generator.agent, the one I mentioned here,
[11:25] the one I mentioned here, and write the actual playwright
[11:28] TypeScript code based on the plan after
[11:30] I approve. If I wanted to modify tweak
[11:33] something, it will do that. And execute
[11:36] the test cases. If the test fails,
[11:40] then use the healer.agent to analyze the
[11:44] failure and fix it. Okay? Let's hit
[11:46] enter. And yeah.
[11:48] So, now you can see that
[11:50] it is going through
[11:53] the web page. It went through the MCP
[11:56] server. Okay? Planner one HRM dashboard.
[11:58] So, this is this is the website http
[12:00] open source hyphen demo if you just in
[12:02] case if you wanted to launch it on your
[12:04] browser instead of updating it as
[12:07] OrangeHRM. So, it has generated me the
[12:08] test cases. You see test case number
[12:14] one, two, three, four, five, and six. We
[12:15] are not going to dive deep into what
[12:17] each of these are. But I'm going to just
[12:18] Say please review this plan.
[12:18] Once I approve, please review this plan.
[12:18] Once I approve, okay, I'm going to say approved.
[12:23] Okay, okay, I'm going to say approved.
[12:23] Okay, so it will go and generate the test case.
[12:26] The plan is ready.
[12:26] If you if you have noticed, it did not launch any website.
[12:29] It is using the MCP server in a headless mode.
[12:32] It went through the website and it is so quick that it has generated so many test cases for us in a fraction of second.
[12:39] Now, I'm going to click approved.
[12:39] What this approved will do now, it will create the test cases for us over here.
[12:45] And it will also execute the test cases and fix the test cases using the healer agent that we see here if there is any issues.
[12:52] Okay?
[12:52] Now, you can see that it has created a test folder.
[12:55] If I expand it, it is it is it is creating the files for each of the test cases that it has written.
[13:05] So, there's one for dashboard, there's one for login.
[13:08] Let it carry on.
[13:08] So, while it generates the test cases, I hope you understand how we orchestrated it.
[13:14] Okay?
[13:14] An AI agent here is nothing but it's not just this file, okay?
[13:18] It's
[13:20] but it's not just this file, okay?
[13:20] It's nothing it's not an agent.
[13:23] It's just a nothing it's not an agent.
[13:23] It's just a nothing but an empty text file.
[13:25] We are nothing but an empty text file.
[13:25] We are giving a life to this and making it as agent by giving an access to an MCP server and also to this LLM.
[13:27] giving a life to this and making it as agent by giving an access to an MCP server and also to this LLM.
[13:29] agent by giving an access to an MCP server and also to this LLM.
[13:32] server and also to this LLM.
[13:32] In this case, it's Claude that is writing the test cases.
[13:34] case, it's Claude that is writing the test cases.
[13:34] Okay?
[13:37] So, an agent should plan, should take an action, and it should give you a result.
[13:40] plan, should take an action, and it should give you a result.
[13:40] So, that's what an agent is.
[13:42] should give you a result.
[13:42] So, that's what an agent is.
[13:44] So, we are making this agent by utilizing these tools, right?
[13:46] agent by by utilizing these tools, right?
[13:46] The MCP and the LLMs.
[13:49] by utilizing these tools, right?
[13:49] The MCP and the LLMs.
[13:52] and the LLMs.
[13:52] Okay, now we got all the test cases generated.
[13:55] test cases generated.
[13:55] And what it says is that all six test cases are generated.
[13:57] is that all six test cases are generated.
[13:57] Now, let me also save the test plan.
[13:59] generated.
[13:59] Now, let me also save the test plan.
[14:02] So, test plan is also saved for the traceability.
[14:02] And what I wanted to run, you can run the test.
[14:05] for the traceability.
[14:05] And what I wanted to run, you can run the test.
[14:08] to run, you can run the test.
[14:08] Okay, so what I'll do, I'll copy this command and I paste the command here so that it runs the test cases.
[14:10] I paste the command here so that it runs the test cases.
[14:10] Or I can also simply say please run it for me.
[14:12] the test cases.
[14:12] Or I can also simply say please run it for me.
[14:14] please run it for me.
[14:14] Um Come on, hit.
[14:17] Um Come on, hit.
[14:17] So, now the the test case is being executed.
[14:19] Come on, hit.
[14:19] So, now the the test case is being executed.
[14:21] the test case is being executed.
[14:21] So, we'll see if everything works fine.
[14:22] we'll see if everything works fine, whatever that it has written.
[14:24] And if there's any failure, it will heal on its own.
[14:27] And by now, I hope you are also working along with me and you have written it the test cases.
[14:30] So, you can see three tests passed and three failed because it has just generated the test cases and didn't execute at all.
[14:35] So, what it's going to do that um three failed, switching to healer role.
[14:44] As you can see, switching to healer role to fix all three failures.
[14:48] So, it's going to switch to the healer role, which is this one, the healer.
[14:52] And as per the instructions given here, it's going to do the fix for us.
[14:55] Fix it.
[14:57] Scroll down.
[15:00] It fixed all three and it's going to rerun.
[15:03] After this, what we are going to do, the third prompt the third thing I said, right?
[15:05] Basically is that we are going to fail one of the test cases manually and then we are going to see if it fixes again.
[15:07] Okay?
[15:12] And that So, all three fixes applied, rerunning the test to verify.
[15:14] Okay, five
[15:22] rerunning the test to verify.
[15:22] Okay, five passed and one remaining.
[15:24] The close button locator is about dialog is a matching the more more than likely uses X icon.
[15:30] X icon.
[15:30] Let me do a final full run to confirm all six test cases passed.
[15:35] Okay.
[15:35] All right.
[15:35] Okay, all six passed.
[15:38] Right?
[15:38] And it's going to generate the results as well.
[15:41] well.
[15:41] So, face plan face planner is done.
[15:44] Six test scenarios designed, approved by me, and six pick files plus share help helper written.
[15:52] You can see the shared helper written here.
[15:52] And fixed three using the healer.
[15:57] Is a healer role played here and the healer role played here as well.
[16:00] And then it fixed couple of errors and finally we got all six out of six passed.
[16:04] That's amazing.
[16:04] Now, what I'm going to do,
[16:08] I'm going to fail one of the test case placeholder.
[16:15] Um instead of maybe this login, what I will say is that
[16:24] this login, what I will say is that update it as sign in, okay?
[16:26] update it as sign in, okay?
[16:28] Let me update it as sign up or sign in maybe.
[16:30] Instead of login, I'll make it a sign in, right?
[16:32] login, I'll make it a sign in, right?
[16:37] Now, I want let me save it as a file.
[16:39] Yeah. Auto save is enabled. Please check if auto save is enabled.
[16:41] Otherwise, if you make any change, you need to add that.
[16:43] So, what I'm going to say, I'm going to paste another prompt.
[16:44] Okay?
[16:47] Or you can simply say please run again and it will use it, but this one is like more specific.
[16:49] We don't want to So, this is why prompting is important.
[16:50] If you're on a free tire, we don't want you to waste the tokens by going back and forth conversation with the AI.
[16:53] Let's see.
[16:55] I'm going to hit enter. And this is the test file selected.
[16:58] Um I can use plus to add multiple files if you want to add multiple files.
[17:00] Um let's see if it's going to identify my the failure.
[17:02] So, this is the one we did.
[17:04] We updated a login to sign in.
[17:25] Did. We updated a login to sign in.
[17:27] So, and it identified it's like you can see that switching to healer role.
[17:30] And it has identified the error and it is updating that to login.
[17:32] And you can see it is updated here as well.
[17:36] And the test case is now passed.
[17:38] So, that's the healer.
[17:40] So,
[17:41] Right.
[17:42] Finally, we are at the end of the video.
[17:44] So, this is amazing.
[17:46] If AI does this, what I'm going to do in when AI does all of that.
[17:50] So, that's exactly why we need this human human engineering to place.
[17:53] Because AI can hallucinate, right?
[17:54] It can hallucinate and it doesn't know the the gaps in the business.
[17:56] We don't know what it just generated is right or not.
[18:00] That's why we need human engineering to provide the context.
[18:02] So, without you signing off, this cannot be trusted.
[18:05] Okay?
[18:06] Though it does all the things, you need to decide what test cases to be there.
[18:08] You need to decide what business logics will be covered.
[18:27] What business logics will be covered in that.
[18:31] So, I hope you have learned how this the Claude and other thing works.
[18:35] And I hope you have the demonstration ready on your laptop.
[18:40] That's the purpose of this video.
[18:42] If you have set it up, please comment below that you have set it up and you have executed it.
[18:47] I'll be happy to know. Or if you have any questions, please drop that in the comment as well.
[18:50] And all these three prompts are in my comments area.
[18:54] And also the Git repo I will attach for this.
[18:57] Please cross-check that. Thank you so much for watching this video.
[19:01] We'll catch up in the next