Understanding how Scope works helps you interpret your scores accurately and make better optimization decisions. This article explains our data collection methodology, scoring formula, and refresh cadence.
How We Collect AI Responses
Prompt Execution
Scope runs a curated library of prompts across four AI platforms: ChatGPT (OpenAI), Claude (Anthropic), Gemini (Google), and Perplexity. For each business we monitor, we run 25-100+ prompts depending on your plan.
How we select prompts:
- Category-specific prompts based on your business type (e.g., "best [your category] in [your city]")
- Location-specific prompts using your business's city and region
- Service-specific prompts based on your primary services
- Competitive prompts that include competitor names to understand relative positioning
How we run prompts:
- Each prompt is executed independently without personalization (no account context that would skew results)
- We use the default web-accessible version of each AI platform
- Prompts are run from servers in your business's geographic region to simulate local user queries
- We do not provide any identifying information that would cause AI to favor your business
Response Analysis
After running each prompt, Scope analyzes the AI response to determine:
- Was your business mentioned? (binary: yes/no)
- If mentioned, what position in the response? (first mention vs. subsequent)
- What sentiment was the mention? (positive, neutral, or with caveats)
- Were specific attributes mentioned? (location, reviews, specialties)
The AI Visibility Score Formula
Your AI Visibility Score (0-100) is a composite metric calculated from:
| Component | Weight | Description | |---|---|---| | Mention Rate | 50% | Percentage of prompts where you are mentioned | | Position Score | 25% | Whether you are mentioned first vs. later in the response | | Sentiment Score | 15% | Quality of the mention (positive vs. neutral vs. qualified) | | Platform Coverage | 10% | Whether visibility is consistent across all four platforms |
Example calculation: A business mentioned in 60% of prompts, usually in the first position, with positive sentiment on 3 of 4 platforms would score approximately:
- Mention Rate: 60 × 0.50 = 30
- Position Score: High position score × 0.25 ≈ 20
- Sentiment: Positive × 0.15 ≈ 12
- Platform Coverage: 3/4 × 0.10 ≈ 7.5
- Total: ~70
Score Refresh Cadence
Starter plan: Weekly full scans (all prompts re-run once per week) Growth plan: Daily monitoring with rolling scan execution
Because running 100+ prompts across four AI platforms is computationally intensive, we distribute scan execution across the week rather than running all prompts at midnight. This means your score represents a rolling window of recent data, not a single point-in-time snapshot.
Handling AI Response Variation
AI responses are inherently non-deterministic — the same prompt asked twice may produce different answers. We account for this through:
Repetition: We run high-importance prompts multiple times and aggregate results to reduce noise.
Rolling averages: Your score is calculated from a rolling window of recent responses, smoothing out one-time variations.
Variance tracking: In your detailed prompt view, you can see individual prompt results including variations across runs.
Platform-Specific Methodology
Each AI platform requires slightly different monitoring approaches:
| Platform | Response type | Key factors | |---|---|---| | ChatGPT (with Browse) | Retrieval + generation | Bing index strength, website quality, review platforms | | Claude | Training-based + optional web | Training data presence, authoritative sources | | Gemini | Google-indexed retrieval | GBP data, Google's index, E-E-A-T signals | | Perplexity | Real-time web retrieval | Bing + web index, content freshness, direct citations |
Industry Benchmarks
Scope compares your AI Visibility Score to other businesses in your industry category. Benchmarks are calculated from:
- Aggregated, anonymized scores of all Scope-monitored businesses in your category
- Weighted by market similarity (same-city businesses weighted higher)
- Updated monthly as our dataset grows
A score at the 50th percentile means you perform equal to the median business in your category. A score at the 75th percentile means you outperform 75% of monitored businesses in your category.
Data Retention and Privacy
- Monitoring data: Retained for 12 months on all plans
- Prompt results: Individual prompt responses stored for 90 days; aggregated scores stored indefinitely
- Business information: We store only the information you provide (business name, URL, category, location)
- We do not store: Customer data, PII from AI responses, or personal information about your customers
For full details, see our Privacy Policy.
Q: Why does my score vary between scans? A: AI responses are probabilistic, not deterministic. The same prompt can produce different results on different executions. Your score represents a statistical aggregate of many prompt runs to reduce this variance, but some variation is inherent and expected. Focus on the monthly trend rather than day-to-day fluctuations.
Q: How do I know Scope is accurately representing real AI responses? A: Our team manually spot-checks AI responses weekly to verify that our automated analysis matches what users would see. We also maintain test cases where we know the expected output to validate our pipeline. If you ever see a discrepancy between what Scope shows and what you see manually, please contact support@scope.online.
Q: Does Scope's monitoring itself affect my AI Visibility Score? A: No. Our monitoring queries are designed to be non-identifying and are run without any context that would cause AI platforms to treat them differently from a real user query. We don't log in to AI platforms with accounts that have history related to your business.