Testing applications built on ChatGPT presents a problem traditional testing frameworks weren't designed for. Your code doesn't run in a browser you control: it runs inside ChatGPT's Apps SDK runtime, which means multiple display modes (inline, picture-in-picture, fullscreen), theme variations, and tool invocations initiated by ChatGPT itself.
Manual testing across these combinations isn't practical. The combinatorics are brutal.
What's Different
ChatGPT Apps operate with constraints unfamiliar to most enterprise developers. Your React components render inside a specialized runtime with ChatGPT frontend state, tool invocations with specific inputs, backend session state, and persistent widget state. Testing each scenario manually would be inefficient at best, impossible at worst.
Sunpeak's framework addresses this by pre-configuring two testing layers: Vitest with jsdom for unit tests (pnpm test) and Playwright for end-to-end tests against the ChatGPT simulator (pnpm test:e2e).
The key mechanism is simulation files. These JSON configurations in tests/simulations/ define deterministic states: user messages, tool definitions with input schemas, call parameters accessible via useToolInput(), and result data passed to components via useWidgetProps(). This lets you test specific states without manual setup.
Testing Across Display Modes
The createSimulatorUrl utility generates test URLs with configurable parameters: simulation file name, display mode (inline/pip/fullscreen), theme (light/dark), device type, and touch/hover capabilities. You can test a counter component starting at 5 in fullscreen dark mode, then immediately test the same component inline with light theme.
The Broader Pattern
This reflects a challenge beyond ChatGPT Apps. As organizations adopt AI-native applications running on proprietary runtimes, testing methodologies must account for environments where you have limited visibility into how code behaves across different runtime states.
Traditional testing assumes direct control over application environments. That assumption breaks when your application is one component in a larger AI system.
The trade-off: frameworks like Sunpeak reduce setup complexity but tie you to specific testing patterns. Whether that's acceptable depends on how much control your organization needs over its testing infrastructure.
What to Watch
Testing tools for AI-native applications remain relatively nascent. Production effectiveness data is limited. The question isn't whether these tools work in demos but whether they catch the bugs that matter in production.
We'll see.