QA testing with Codex and Playwright

What Codex does for QA testing

• Writes end-to-end tests for the flows that matter: sign-up, checkout, the main screens.
• Runs the suite, reads the failures and fixes the test or the code, saying which.
• Drives a real browser to check a change by hand, and files bugs with steps and screenshots.
• Finds flaky tests and quarantines them with an issue, instead of retrying until they pass.

Codex features for QA testing

How to set up Codex for QA testing

1. Connect the tools

   Add the Playwright MCP server with one command, or the same table in config.toml. If the tests need to reach a staging site, turn on network access for the sandbox.

   # A real browser for exploratory checks
   codex mcp add playwright npx "@playwright/mcp@latest"
   
   # Or in ~/.codex/config.toml
   [mcp_servers.playwright]
   command = "npx"
   args = ["@playwright/mcp@latest"]

2. Write the rules in AGENTS.md

   Codex reads AGENTS.md before any work: ~/.codex/AGENTS.md, then the files from the repository root down to the folder it's working in. Put the job's rules there, or in a skill under .agents/skills/ if they're long. See the AGENTS.md guide.

   ## QA rules
   - Test against the local or staging app, never production.
   - Use the test accounts in .env.example, never real ones.
   - A failing test is a finding: say whether the test or the code
     is wrong before you change either.
   - Every bug report: steps, expected, actual, a screenshot and
     the commit you tested.
   - Don't delete or skip a test to make the suite pass.

3. Run it without the interactive interface

   codex exec runs one task and exits. Progress goes to stderr and the final message to stdout; -o also writes it to a file and --json streams events. The sandbox is read-only unless you pass --sandbox workspace-write.

   # Hand the failing test output to Codex
   npm test 2>&1 | codex exec "summarize the failing tests and propose the smallest likely fix"

4. Put it on a schedule

   Create a scheduled task in the ChatGPT desktop app or on the web (the CLI doesn't manage them), or run codex exec from cron. See how to make Codex proactive.

Source: Codex documentation

Limits and safety

• It tests what it can reach: a local or staging app with test data, not production.
• A passing suite covers only the flows someone wrote tests for. Ask it to list what isn't covered.
• Browser runs are slow and use more of your plan. Run the full suite on a schedule and a smoke test on each change.
• Codex sandboxes commands by default: writes stay in the workspace and the network is off. Keep danger-full-access for an isolated runner or container.

Claude Code or Codex for QA testing?

The rules carry over. Write them once in AGENTS.md and put @AGENTS.md in CLAUDE.md, and both agents read the same file.

Step by step for the other agent: QA testing with Claude Code.

Make QA testing someone's job

Commands run when someone remembers to run them. Crewmark turns your own Codex into an AI employee with a role, a goal in numbers, working hours and a heartbeat, so the work happens on schedule and shows up on a board.

1. Start from the role

   The AI software engineer, with these rules in its brief.

2. Put the work on its board

   Add the job as a recurring issue. The employee wakes on its heartbeat, does it and reports in a comment.

3. Related

   GitHub issues covers what happens to the bugs it files.

FAQ