Prompt monitoring is the operational half of AI visibility monitoring: deciding which questions to ask, how often, with which model, and at what cost. The prompt set is the measurement instrument, and almost every failure of this discipline is an instrument failure — prompts that nobody would actually type, wording edited mid-series, branded questions inflating a headline rate, or a set so large that most of it is near-duplicates.
SiteRank AI treats the prompt set as versioned, approved, categorised configuration, and every run as an observation with recorded conditions.
Prompts you approve, never prompts we invent
Nothing is monitored until you save it. SiteRank AI can suggest candidates, but it does so from the site's own content — page titles, headings already phrased as questions, taxonomy terms and topic-cluster labels — with no external model call at all. Each suggestion carries the page or term it came from, so you can see why it was proposed.
Suggestions are capped at 25 per pass and deliberately spread across intents. The cap is the design, not a limitation: highly similar prompts are not independent observations, and averaging over a thousand near-identical variants manufactures precision that does not exist while multiplying the bill. A small set you recognise as your real questions beats a large generated one.
A suggestion writes nothing and measures nothing. It becomes an instrument only when you approve and save it.
Six intent categories
Every prompt is filed under one of six categories, because they answer different business questions and must not be averaged together.
| Category | Question shape | What it tells you |
|---|---|---|
informational |
"How does X work?" | Whether your material is used to explain the subject |
comparison |
"X vs Y for Z" | How you are characterised against a named alternative |
navigational |
"Where do I find X's documentation?" | Whether the model can route someone to you correctly |
recommendation |
"What should I use for Z?" | Whether you are in the consideration set at all |
brand |
"Is [your brand] any good for Z?" | What is said when the asker already knows you |
competitor |
"Is [rival] any good for Z?" | What is said about the alternatives you track |
Brand prompts nearly always produce a mention, because the brand is in the question. Reporting a headline mention rate driven by brand prompts is self-deception, so categories stay separate in reporting and every figure states which prompts drove it. The recommendation and informational categories are where the honest test lives. Monitor the questions AI users ask about your brand covers how to source the questions in the first place.
Each prompt also carries the target aliases to match against (your brand's real spellings and legal name) and the tracked competitor list used for share of voice. Choosing that list well is its own decision — choosing competitors for LLM visibility monitoring covers it.
Prompt text is immutable within a series
Editing the wording of a prompt does not update it. It creates version 2.
Changing the words changes the question, and comparing answers across a wording change compares two different experiments. So an edit to prompt text bumps the prompt version and recomputes its hash, every observation records the version and hash it was taken under, and charts group by prompt and version and break the line at a version change rather than joining across it.
Metadata that does not change the stimulus — category, tags, schedule, the enabled flag, the competitor list — is edited in place with no version bump, because it does not alter what was asked.
Schedules and samples
Each prompt has its own schedule: manual, daily, weekly or monthly. Manual means it never runs unattended. Scheduled prompts become due once their interval has elapsed since the last run, and the queue picks up whatever is due — a design that catches up after a missed window rather than assuming a cron fired punctually, because on a low-traffic WordPress site it often does not.
Each prompt also sets its own samples per run, from 1 to 10. Samples are what turn a single answer into a rate: one response is an anecdote, and the same prompt an hour later can name a different set of companies. More samples narrow the Wilson interval and make volatility measurable at all — below two samples, volatility reports as not measurable rather than as zero.
Provider, model and the web-search flag are set per prompt and recorded per observation, so changing them shows up as a marked discontinuity in the series rather than as a silent step change in the numbers.
The budget guard runs before the call
Monitoring spends your money on your own provider key, so the limit is enforced before the request, not after. A budget checked once the run has billed you is a receipt, not a control.
Before each call the guard evaluates two limits:
- Monthly budget in USD, compared against month-to-date estimated spend plus the estimated cost of the call about to be made.
- Maximum runs per day, compared against runs already made today.
A budget of zero blocks monitoring rather than meaning "unlimited", because an unlimited default on a metered API is a trap. Spend figures are estimates derived from token counts and a per-model price map, and they are labelled as estimates everywhere they appear — they are not your provider invoice, and grounded web-search surcharges are not modelled, so the estimate is a floor. Because estimates can undercount, a separate pause-on-budget setting stops scheduled runs outright rather than merely warning, and a warning threshold tells you when you are approaching the limit while there is still room to react.
Running monitoring requires its own capability, siterank_ai_manage_visibility, deliberately separate from general settings access, because it spends money. Configuring provider credentials requires another, for the same reason.
Every observation keeps its provenance
A stored response without its conditions is unusable, so each observation records the prompt id and version, the prompt hash, the provider, the model requested and the model returned, whether web search was enabled, the locale, the timestamp, the call status, token usage, the estimated cost, the raw response and the extracted citations.
That is what makes any reported rate expandable into the individual responses behind it. It is also what makes a chart honest: when the returned model changes mid-series, the instrument changed, and the chart marks a break.
Failed calls are stored too, as failures. They are excluded from both the numerator and denominator of every rate, and the excluded count is reported, because a timeout is not evidence that a model omitted you.
What monitoring does not do
- It does not send your content anywhere. Only prompt text goes to the provider. Your posts, corpus, clusters and audit findings never leave the install.
- It does not use a key we supply. Bring your own. Credentials resolve from an environment variable, then a
wp-config.phpconstant, then an encrypted stored value, and never reach the browser in any form. - It does not drive consumer chat products. Observations come from provider APIs under recorded conditions, which is a different thing from what a person sees in a chat app, and is labelled as such on every chart.
- It does not report a position. Rates, shares, coverage and volatility — never a rank. See LLM rank tracking.
- It does not run without you. No prompt is measured until you approve it, and no run happens outside the budget you set.
Who it is for
Teams that want a repeatable measurement rather than a screenshot. If you can write ten questions your buyers genuinely ask, this turns them into a series you can defend a year from now — same wording, same conditions, sample sizes visible, breaks marked where the instrument changed.
Official sources & further reading
- Web search tool — OpenAI
Related reading
Frequently asked questions
How many prompts should I start with?
Fewer than you expect — ten questions you recognise as the ones your buyers actually ask is a working instrument. Near-identical prompts are not independent observations, so a set of two hundred variants costs proportionally more while adding almost nothing you can defend. Start small, run it long enough to see the intervals narrow, and add a prompt only when there is a question the current set genuinely does not cover.
If I reword a prompt, do I lose the history I have already collected?
No — the earlier observations stay exactly where they are, attached to version 1 of that prompt. The new wording becomes version 2 and starts its own series, and the chart marks a break between them rather than joining two different questions into one line. You keep both records; what you do not get is a continuous rate spanning the change, because there is no honest way to produce one.
Can I compare my mention rate with a number from another tool or another site?
No. Every rate here is a proportion within your prompt set, your samples, your model and your web-search setting, so a different prompt set is a different instrument measuring a different thing. The numbers are built to be compared against your own earlier runs under identical recorded conditions, and that is the only comparison they support.
What will monitoring actually cost me?
That depends on your provider's pricing, the model you choose, how many samples each prompt takes and how often it runs — SiteRank AI cannot tell you a figure in advance, and does not pretend to. What it does is estimate the cost of each call from a per-model price map, check that estimate against your monthly budget and daily run limit before the request goes out, and label the running total as an estimate everywhere it appears. Treat it as a floor rather than an invoice: grounded web-search surcharges are not modelled.
Should brand prompts be in the set at all?
Yes, but read them separately from everything else. A question containing your brand name nearly always produces a mention, so it tells you how you are characterised once someone already knows you — not whether you are discoverable. Keeping the six categories apart in reporting is what stops a healthy-looking headline figure that is really just your own name being read back to you. The measurement methodology sets out each metric's numerator and denominator in full.