Preetest 以 AI 測試智能體檢查自家應用程式，首輪得 36/100、發現 79 項問題

Disclosure first: I work on Preetest (Yamama Software, Saudi Arabia). This post is about running our own product on our own apps. Every number below comes from a real report, and the report links are public. This write-up was drafted by the AI assistant that runs our testing and fixes, from those reports, at the founder's direction.

What Preetest is, in one paragraph

You give it a URL (and a login if the app needs one). It runs a set of AI testing agents, each with a specialty (accessibility, performance, security headers, mobile layout, SEO, forms, caching and so on; 17 in all), and writes one report: a score out of 100, every issue with severity and the element it found it on, and a fix prompt per issue that you can paste into Cursor, Claude Code or whatever you build with.

Run 1: our internal office suite

A 10-page signed-in scan of the staff suite we use ourselves (documents, mail, calendar, chat).

Score: 36/100. 79 issues: 3 high, 31 medium, 45 low.
Report: https://app.preetest.com/r/z1LctGGKyBvPlZIcTkeh4w

The findings that mattered:

Every static file was served with Cache-Control: max-age=0. Every visit re-downloaded every script. High severity, and correct.
The cross-app sign-in put the user's email and name in a GET ?t= parameter. Medium. Also correct; that lands in server logs and browser history.
The "export all my data" action was a plain GET link with no confirmation. Medium.
Bottom navigation at 390 px wide cut off the "Mail" label. Low, but it is the first thing a phone user sees.

Run 2: the same app, after two small fixes. The score went down.

We fixed the export confirmation and one accessible-name issue, re-ran with the same settings, and got 29/100 with 101 issues.
Report: https://app.preetest.com/r/AIykp03BiPhDdXf2jn-Lww

Why worse? The second run reached more screens (the calendar, the staff guide, the privacy and download pages), so it found more. Coverage is budget-based: each run stops when its token reservation runs out, and the two runs stopped at different pages. So this is not a clean before/after, and I am not going to present it as one. What the second run added was the most useful finding of the day:

The calendar had 49 column-header cells with no role="row" parent, and 42 day buttons using aria-pressed, which their role does not support. Plus text at 4.12:1 contrast where 4.5:1 is required.
LCP of 4.7 s on the documents page. The assistant greeting waited on about 1.5 MB of scripts.

Run 3: a small Expo web app

Score: 53/100. 27 issues: 5 high, 11 medium, 10 low. 100 % coverage, 143 tokens used.
Report: https://app.preetest.com/r/BTlW0oUUg-ybJZQxCpPyKA

Unknown URLs returned 200 instead of 404 (soft 404). High.
No header / main / footer / nav landmarks. High.
The onboarding "Next" control was not exposed as a button to screen readers. High.
sitemap.xml served as HTML; a JSON data file served with no caching.

What we fixed (and what we did not)

Fixed and live:

Calendar ARIA: headers and weeks now sit inside role="row", day cells use aria-selected, arrow keys move between days. axe on that page went from 92 ARIA violations plus 33 contrast failures to 0.
Contrast: the faint purple went from #7c6fd6 (4.12:1) to #5a4cc0 (6.5:1).
Mobile bar: five apps plus a "More" button; nothing cut off at 360, 390 or 412 px.
Caching: versioned JS/CSS now cached for a year. A repeat visit to the documents page renders in under a second.
Expo app: real 404s, landmarks, a real button role on "Next", a real XML sitemap, long cache on versioned files.
A bug the 404 fix exposed: 19 icon font files had been missing from the deploy the whole time. The old catch-all returned an HTML page with status 200 for them, so nothing ever errored. Soft 404s hide real breakage; that one finding paid for the run.

Not fixed, or not proven:

The documents page first load is still about 4 s. Only repeat visits got faster.
We moved the sign-in token out of the URL, but have not yet tested a real signed-in hop between apps with a staff account, so I am not calling it fixed.
We have not re-scanned since the bigger fixes, so there is no honest "after" score yet. When there is, I will add it here, whichever way it goes.

Where the agents were wrong

It flagged /api/config on the Expo app as "returns private data with no auth". It returns public settings (app name, support links, public client ids). False positive.
It flagged a missing aria-current on the docs sidebar. The source already renders it on the active button.
It flagged noindex on the internal editor as high. That is intended; it is a staff tool.

So: a sharp junior QA that works at 3 a.m., not an oracle. Read the report, do not paste every fix prompt blindly.

Pricing, honestly

The free plan is 50 tokens. Our smallest real run above used 143, so the free tokens get you a project set up and a look at the flow, not a full inspection. A standard inspection is roughly 1,000 tokens, which is the $29/month plan. It also refuses to scan its own domain, so you cannot run it on preetest.com.

If you try it on your own app, I would genuinely like to hear where it was wrong. That list above is the one we work from.