AI search, SEO & online discovery
Practical coverage from Dubai

Saved
Measurement

How to Separate Positive Neutral and Negative Business Mentions

Summarize with AI
Analysing business data; illustrative photograph.
Photo by Jakub Zerdzicki / PexelsIllustrative photography
In this article 5 sections

An answer says a business has a branch in the district, describes its staff as helpful and criticises the booking process. A single sentiment label for the whole answer would hide which fact was neutral and which parts were praise or criticism. A reader interested in booking friction needs to see that distinction, even if the answer’s overall impression sounds favourable.

Code the opinion toward the specific business and aspect before combining results. Keep the words that support the label and document what the reviewers agreed on. AI Visibility’s monthly reporting guide explains how to put observations into a recurring report. This walkthrough focuses on sentiment coding, including how to separate an evaluative statement from a fact or a conditional suggestion. AI Visibility monthly reporting guide

Identify the target and aspect before choosing polarity

Start with a specific passage and identify which business it describes. A sentence may name two companies and praise only one of them. The speaker may also be evaluating the category or a neighbourhood rather than the business. If the target remains uncertain after reading the surrounding text, flag the passage. A label is only useful when another reviewer can see what it refers to.

Record the aspect. Price, staff, location, availability and booking can each carry a different evaluation. The original SemEval work on aspect-based sentiment analysis illustrates the distinction between sentiment for a particular aspect and sentiment applied to an entire sentence or review. The codebook in this guide uses that idea for sampled business mentions. It does not adopt the original task’s dataset or claim comparable scores. SemEval aspect-based sentiment task paper

Label What counts Example from fictional text
Positive A favourable judgment directed at the target The team was responsive
Neutral A factual or descriptive statement with no clear evaluation The business has a branch in the district
Negative An unfavourable judgment or clearly adverse outcome under the rule The booking process was confusing
Mixed Positive and negative evaluations in the same defined unit The work was careful, but the turnaround was poor
Unresolved Target, meaning or context is insufficient to label A pronoun could refer to either of two businesses

Decide whether the unit is an aspect passage or an answer-and-business summary. In an aspect-level report, careful work and poor turnaround can be two rows. In an answer-level report that combines them, the same passage can be mixed. Choose the unit before counting and record whether a single answer can contribute more than one row. Otherwise a long answer can dominate the report simply because reviewers split it into more pieces.

Separate facts from favourable or unfavourable wording

A number on its own is usually descriptive. “The price is AED 90” does not state whether it is fair, cheap or expensive. The codebook should tell reviewers to avoid using their own expectations about what a service ought to cost. If the answer says the price is higher than alternatives and presents that as a disadvantage, retain the comparison and label the evaluation supported by the text.

Apply the same rule to practical limitations. “The branch closes at six” reports a time. “It closes too early for your requested appointment” evaluates the fit against a stated need. Record both the qualifier and the aspect. If the exact wording makes an unfavourable implication clear, write down why. If it requires the reviewer to invent an unstated expectation, use neutral or unresolved according to the codebook.

An official Home Box UAE Instagram post illustrates the need to identify who is speaking. Its regional retail account and exact post were previously verified. Promotional wording on a company’s account represents the publisher’s own presentation. If an answer repeats a claim from a promotion, preserve the attribution when coding the answer. A positive statement attributed to a brand does not establish an independent customer assessment. The social post is an illustration of source scope, outside the tally below. Official Home Box UAE Instagram post

Recommendation and sentiment need separate fields. An answer can compliment a business and still prefer a different one for a specific task. It can also suggest considering a business because of its location without expressing an opinion about service quality. A citation or first position in a list does not itself establish positive sentiment. Record the wording rather than reading intent into the layout.

Compare reviewer decisions before reconciling them

Give both reviewers the same text, target and codebook. Ask them to label the overlap independently and save their first decisions. If they see each other’s answers before coding, a final agreement rate cannot tell you how similarly they understood the rules on their own. Use a sample with neutral facts, obvious judgments and difficult cases so a common label does not account for most of the agreement by itself.

Start with the paired count and exact agreement. Count one match when both reviewers assign the same label to the same unit and divide by the number of paired labels. Show a small cross-tabulation when it helps. A table with one reviewer’s labels on the rows and the other reviewer’s labels on the columns shows which categories caused disagreement. Review those cases before recalculating the sentiment distribution.

For a sufficiently informative overlap, Cohen’s kappa is one way to describe agreement for nominal labels relative to agreement expected from the reviewers’ marginal label distributions. Keep the paired counts and category distribution alongside it. A small or highly skewed sample can be difficult to interpret, and the coefficient does not tell you whether the final labels were factually correct. This walkthrough uses exact agreement for its small teaching example. Cohen’s original agreement paper

Reconcile by reading the passages, not by averaging labels. If one reviewer called a bare price negative and the other called it neutral, inspect the written rule and decide whether the passage actually evaluates the price. Save the two first labels, the reconciled label and the reason. When the disagreement exposes a gap in the rubric, update the rule and recheck relevant earlier rows. Report agreement from the original paired labels and the final distribution from the reconciled ones.

Before measuring agreement, settle what the reviewers are asked to label. If one labels a whole answer and the other labels each clause, their rows cannot be compared directly. A practical first pass is to agree the units and target identities, then let reviewers assign polarity without seeing each other’s labels. If finding the units is part of the research question, measure agreement on that step separately. Otherwise a disagreement about boundaries can be mistaken for a disagreement about tone.

Keep unpaired observations out of the paired denominator and say how many there were. A missing reviewer label is different from a reviewer choosing unresolved. If the codebook treats unresolved as a valid category, include it in the comparison when both reviewers submitted a decision for the same row. If some rows were dropped after disagreement, show that change before presenting the final distribution.

Follow a worked example with two disagreements

A fictional eight-row panel demonstrates the calculation. Two labelled columns represent example reviewer decisions. They are constructed records, rather than the results of a new human agreement study. Six rows had the same label in both columns. The two disagreements concerned a bare price and a suggestion that a company might suit people who need a local branch.

In the first disagreement, one example reviewer marked the price neutral and the other marked it negative. The reconciliation retained neutral because the text stated a price without an evaluation. In the second, the labels were neutral and positive. The reconciliation retained neutral under the example rubric because a tentative location fit did not include a favourable judgment about the service or branch. A different rubric could code that language differently, but should explain its rule and apply it consistently.

The tally produced six matches out of eight paired labels, or 75% exact agreement. After reconciliation, the eight rows comprised two positive, three neutral, two negative and one mixed. These figures describe the fictional rows only. The agreement percentage and the sentiment distribution have different meanings even though both calculations used eight rows here.

Lunasol’s historical twelve-run Google AI Mode study reports that one answer gave each listed business a labelled line of praise. Its study counted names and citations rather than scoring sentiment. The observation shows why it can help to keep the exact aspect and wording. This article did not recode the answers or infer how much of the historical sample was positive. Lunasol’s published 12-run study

Report counts without overreading them

Use a Sentiment coding record with the answer ID, business, aspect, exact passage, attribution, reviewer labels and final decision. Record the planned runs, captured answers and qualifying mentions separately. For sentiment share, state whether the denominator is all qualifying mentions, only resolved mentions or one summary per answer and business. If you exclude unresolved rows from a percentage, show their count beside the denominator. An unobserved answer does not supply a neutral mention.

A balanced summary can be more useful than a single net score. Show the category counts and note which aspects drew criticism. Two negative comments about booking can point to a different question from two about service quality. If the panel covers English and Arabic answers, keep the languages identifiable and have an appropriately fluent reviewer interpret the full passage. Tone, negation and idiomatic praise can be lost if the reviewer sees only isolated translated words.

For a repeated report, compare the question set and the coding rules before interpreting changes. A panel that adds more prompts about complaints may produce more negative passages even when the earlier prompts remain stable. A codebook that newly treats conditional suitability as positive can also move the numbers. Version both the prompt panel and the rule, and rerun the same sample when you want to separate a definition change from a change in answer language.

Keep sentiment, factual accuracy and commercial outcomes in separate records. A favourable claim can be wrong, and a critical claim can describe a genuine limitation. Verify the facts where they matter and avoid using sentiment as a substitute for checking service quality. An authorised analytics or CRM export could describe visits or enquiries under its own definitions. This guide used no private exports, and its labels carry no inference about what customers saw or did.

When a report quotes an answer, keep enough context to preserve the evaluation. A sentence saying that a branch is convenient can be followed by a restriction that matters to the same user. If the publication only has room for a short excerpt, keep the fuller passage in the evidence record and explain which aspect the quote supports. A second reviewer should be able to trace the label without guessing at the omitted context.

AI Visibility is published by Lunasol, which offers GEO and AEO services. The historical example comes from its published study and the agreement demonstration uses fictional labels. About AI Visibility

You’ve reached the end of this article. Read as plain text

Search the blog

Summarize with AI

Choose your assistant. Copy the article and summary request, then paste it into a new chat.

The full article is included, so your assistant can read it without opening this website. Nothing is submitted automatically.

View or copy the summary requestOpen assistant

Summaries are generated by your chosen service and may miss details. Its account requirements and terms apply.