AI search, SEO & online discovery
Practical coverage from Dubai

Saved
Measurement

How to Report Visibility When Some Questions Receive No Answer

Summarize with AI
Analysing business data; illustrative photograph.
Photo by nappy / PexelsIllustrative photography
In this article 6 sections

Two AI search surfaces are tested on the same twelve questions. The first names the business in four of nine scorable answers. The second names it in five of eight. Based on those rates, the second looks stronger. Restrict the comparison to the six questions that produced scorable answers on both surfaces and the order reverses. The first names the business four times and the second names it three times.

Both comparisons can be useful. The first describes the answers a reviewer could score on each surface. The second asks what happened on questions both surfaces answered in a usable way. Publish coverage alongside both figures so a reader can see how much of the original test each conclusion represents. This guide shows how to build that comparison and keep unanswered questions visible in the report.

Start with a fixed list of planned slots

Give each planned question and condition a slot ID before collecting results. A slot might be one English question about a Dubai branch on one surface during a particular reporting period. A planned repeat has its own slot when the method calls for multiple independent observations. Assign a separate attempt ID each time an operator submits the question. If an attempt fails and your method allows a retry, attach the retry to the same slot and retain the earlier result.

Set the unit of analysis before reviewing the answers. A monthly business dashboard often benefits from one selected outcome per planned slot. An operations dashboard might instead need every attempt to measure errors or timeouts. These views can share a capture log. They answer different questions and need separate counts. Five retries on one difficult question should have the weight of one planned slot in the visibility comparison.

Record the final disposition of every slot. Include questions your team did not attempt, questions that received a refusal, and responses for which evidence was lost. That reconciliation makes the denominator reviewable. Survey reporting offers a useful analogy. AAPOR publishes standard definitions for classifying the final outcomes of sampled cases and calculating outcome rates. The slot labels here apply to AI answer audits and have their own eligibility rules. AAPOR Standard Definitions

Decide whether the answer supports the metric

A business mention can be scored when enough of the answer is preserved to determine whether the business appears, under a written rule. A citation may require a separate source panel capture. If the full answer text survives but the panel was missed, the run can qualify for the mention metric and remain unknown for the citation metric. Mark those eligibility decisions in separate columns.

Some replies combine a refusal with useful commercial information. For example, a surface may decline to guarantee that a Dubai shop is open and still list shops the reader can contact. Under a mention rule covering the answer's suggestions, the names in the list can still qualify. A generic refusal with no business content falls into a different group if the metric asks which businesses appeared in substantive answers. Publish the eligibility rule so readers can tell how mixed replies were treated.

An omitted observation and an observed absence have different meanings. If the full qualifying answer lists competitors and leaves out the business being tracked, score the business as absent. If the response stopped before the business list loaded, the outcome may be indeterminate. A visible mention in a fragment can be preserved as evidence. For a symmetric primary metric, use the same evidence requirement for positive and negative classifications or specify an explicit partial-evidence policy. Otherwise, a report can quietly include partial positives while dropping partial negatives.

A public feature announcement can help readers identify the product under discussion. The Google Maps Instagram announcement below illustrates a product-level description. Whether a particular answer was delivered, and which parts of it were available to score, belongs in the test log for that specific surface and slot. A Maps announcement does not establish anything about completion on a separate Search interface.

Reconcile coverage before comparing visibility

Give each surface or month a row with the count planned, count attempted, count scorable for the metric, and count positive. Also show the excluded dispositions with enough detail to tell an execution gap from a product response. Keep the observation window and retry policy beside that row. A surface that usually requires a second attempt may look similar to another after retries even if customers have a different experience on their first attempt.

In this constructed fixture, both surfaces are scheduled for twelve identical slots. Surface A has twelve attempts at the slot level, with one refusal and two incomplete captures. Surface B has eleven attempted slots, with one refusal and two incomplete captures. The remaining Surface B slot was never attempted. Each fixture slot has at most one attempt. Business mention is the only visibility outcome measured.

Report line Surface A Surface B
Planned slots 12 12
Attempted slots 12 11
Refusal with no scorable answer 1 1
Incomplete capture 2 2
Not attempted 0 1
Scorable answers 9 8
Business mentioned in scorable answers 4 5
Scorable coverage of planned slots 9/12 (75%) 8/12 (67%)
Mention rate among scorable answers 4/9 (44%) 5/8 (63%)

A separate measure, confirmed mentions per planned slot, would be four of twelve for A and five of twelve for B. It describes how many positive observations the audit secured relative to its plan. Label it carefully. An unattempted slot reduces that observed yield even though the platform had no chance to answer. If you want a first-attempt customer experience measure, start from actual first submissions and report what the team observed for those submissions, with capture failures identified separately.

Choose an explicit convention when nothing is scorable. With zero eligible answers, display the conditional mention rate as unavailable and show the coverage count. A rate of zero is appropriate when at least one answer met the eligibility rule and none of those answers named the business. That distinction also matters when exporting data to charts, where blank or unavailable values can be rendered as an apparent drop if the chart treats them as zero.

Recompute the comparison on common scorable questions

Match slots across surfaces using the original question ID and any conditions you intended to hold fixed. For comparisons across months, also check whether wording and eligibility rules stayed the same. Use an explicit flag for questions scorable on both sides. A pairing based on row order can silently compare different questions after an analyst filters one export.

In the fixture, six slots are scorable on both surfaces. On those six, Surface A has four mentions and Surface B has three. On the three slots scorable only on A, the business is absent every time. On the two scorable only on B, it appears twice. One slot is unscorable on both. Those four groups account for all twelve original slots and explain the reversal. B's two extra positive observations contribute to its own conditional rate, while the three extra A observations bring A's conditional rate down.

Give the common-question comparison as four of six versus three of six and show the six-of-twelve common coverage. If a decision depends on the particular difference, list the paired outcomes as well. The fixture has two slots positive on both, two positive only on A, one positive only on B, and one negative on both. The observed difference on the common set comes from three discordant slots. A chart of the two percentages alone would hide how little data drives it.

The common set can also be selective. Suppose both surfaces answer broad questions but only one can handle questions with a district or a tight service constraint. The shared subset then tells you more about broad queries than it does about the full plan. Show coverage by question family or language when those distinctions affect the decision. Keep counts next to rates for small groups.

For a recurring report, freeze the definition of a comparable pair before looking at the month’s scores. If an answer becomes scorable after a permitted retry, the pair can qualify for a selected-outcome comparison under that retry policy. A first-attempt comparison still needs its own common set. Preserve both flags when teams use the same file to assess visibility and the reliability of collecting evidence.

Show how unknown slots could affect the conclusion

A sensitivity calculation can show why the unscored observations matter. If the question is hypothetical visibility under full scorable coverage, assign every unknown a possible binary outcome and report the extremes. For A, four mentions are confirmed and three slots are unknown under that hypothetical, giving four to seven out of twelve. For B, five are confirmed and four are unknown, giving five to nine out of twelve. These are logical ranges under stated assumptions, with no sampling confidence level.

That calculation requires a meaningful hypothetical. A missing screenshot of an otherwise completed answer could conceal a positive or negative result. A complete refusal without any commercial content is a known failure to show the business if you define your metric as exposure on observed attempts. Treat the refusal as a known non-exposure for that metric and keep it separate from the conditional mention rate. An unattempted question still has an unknown customer outcome. State which dispositions you treat as unknown for each measure rather than reusing one interval for every purpose.

When missing slots cluster, decide whether the current report can support the intended comparison. A language outage could remove almost all evidence for Arabic questions. A stricter source-capture workflow might reduce eligibility for one month even if the answer experience stayed similar. Publish the row counts, describe the gap and separate the affected comparison from results that still have comparable evidence. If you collect replacements after the window, keep their dates and show whether they are a new observation or a permitted retry.

Keep the monthly note short enough to read

The existing AI Visibility monthly reporting guide emphasizes publishing the question set and run conditions, and shows that source lists can move even between repeats. Keep that context beside your coverage table. A shared subset can remove differences in which questions were scored, while variation in the answers themselves can still affect the results. Keep run-to-run variation and missingness as separate items in the report. AI Visibility monthly reporting guide

A short reader-facing note could say that B had more observed mentions in its scorable answers, while A had more mentions on the six questions both could be scored on. Follow that sentence with the question coverage and the dispositions that account for the gap. For a real report, specify the surface, month and business. When reporting two different outcomes, such as a name appearing and a website citation, provide each with its own scorable count.

Visibility coverage record

Retain the plan version, slot IDs and conditions, attempt IDs and retry rule, final dispositions, eligibility by metric, and the selected answer evidence. In the monthly report show the planned, attempted, scorable and positive counts, the excluded dispositions, and the size and result of the paired comparison. Record the method used for any hypothetical bounds and the decision the available evidence can support.

AI Visibility is published by Lunasol, which offers GEO and AEO services. The two-surface numbers in this guide are constructed to demonstrate the reporting method. About AI Visibility

You’ve reached the end of this article. Read as plain text

Search the blog

Summarize with AI

Choose your assistant. Copy the article and summary request, then paste it into a new chat.

The full article is included, so your assistant can read it without opening this website. Nothing is submitted automatically.

View or copy the summary requestOpen assistant

Summaries are generated by your chosen service and may miss details. Its account requirements and terms apply.