In this Article
Sentiment analysis is usually discussed as a modelling problem. In practice, the model is the commodity part: usable classifiers are available from every cloud provider. What separates a useful sentiment programme from a misleading one is which text goes in.
This guide covers the four sources sentiment data comes from, what each platform’s rules allow, why sampling bias dominates accuracy, and how to read a score without over-claiming.
Key Facts
- Collection decides accuracy more than the model does. A strong classifier on a biased sample produces a confident wrong answer.
- Platform terms, not technical difficulty, set the boundary. The major social platforms restrict automated collection and offer paid APIs instead.
- Review sites are the most usable source for most commercial questions: public, structured, and directly about products.
- Sarcasm, negation and domain vocabulary are where models fail, and they fail silently, producing a score rather than an error.
- A sentiment score is a relative instrument. Trends against your own history are meaningful; absolute percentages compared across vendors are not.
Where does sentiment data come from?
Four sources, each with different access rules and a different bias. We call it the 4-part sentiment source model.
| Source | Access | Built-in bias |
|---|---|---|
| 1. Review platforms | Public pages, some with official APIs | Bimodal: people write when delighted or angry |
| 2. Social platforms | Official APIs, mostly paid; automated collection restricted | Skews to the demographics of each platform |
| 3. Forums and communities | Public, some with documented APIs | Expert-heavy, unrepresentative of casual users |
| 4. Your own channels | Support tickets, surveys, sales notes | Only people who contacted you; the most actionable source anyway |
The fourth row is consistently under-used. Support tickets are already yours, already about your product, and carry none of the access or licensing questions attached to everything above them.
What do platform rules actually allow?
This is where sentiment projects hit their real constraint, and it is contractual rather than technical.
Social platforms restrict automated collection in their terms and offer paid API tiers as the sanctioned route. Those tiers set volume, history depth and field availability, so what your project can measure is decided at purchase time. Building around the API rather than through it puts the account and the project at risk.
Review platforms vary. Some publish official APIs or partner feeds; others permit reading public pages but prohibit systematic extraction. Read the terms for each one rather than assuming a category-wide answer.
Personal data travels with the text. Usernames, profile links and free-text content are personal data under the GDPR and comparable regimes. Aggregate early, store only what your purpose needs, and keep a deletion path. General information, not legal advice.
Where ordinary collection infrastructure fits is the public, permitted end: review pages, forums and news, which frequently render differently by region. DataImpulse residential covers that at $1 per GB across 195 countries.
Why does sampling bias beat model accuracy?
Because every source over-represents somebody, and a classifier has no way to know.
Review platforms attract the delighted and the furious, so the middle of your customer base is invisible. Social platforms differ in demographics and in which topics get discussed at all. Forums skew to technical users. Each source produces an internally consistent picture of a different population, and averaging them does not produce the truth, it produces a blend with unknown weights.
Two practices keep this honest. Report by source rather than merged, so a shift in one is visible instead of diluted. And track volume alongside sentiment: a sentiment score that improves while mention volume collapses is usually a collection failure rather than good news, and that failure mode is common when a platform changes its access rules.
How should a sentiment score be read?
| Question | Sentiment data answers it when | It does not when |
|---|---|---|
| Is perception moving? | Compared against your own history, same sources | Compared against a competitor measured differently |
| What are people complaining about? | Themes are extracted and read by a person | Only a polarity score is produced |
| Did a launch land badly? | Volume and sentiment are read together | Volume is ignored, hiding the size of the reaction |
| Should we change the roadmap? | Combined with support and churn data | Used alone; loud is not the same as common |
The models themselves fail in known ways: sarcasm, negation across clauses, and domain vocabulary where a word that is negative in general English is neutral in your category. These produce a confident score rather than an error, so spot-checking a sample by hand each month is not optional overhead, it is the only detection method available.
Frequently Asked Questions
What data does a sentiment analysis API need?
Text, and the quality of that text decides everything. Sources are review platforms, social platforms, forums and your own support channels, each with different access rules and a different built-in bias toward some part of your audience.
Can I scrape social media for sentiment analysis?
The major platforms restrict automated collection in their terms and sell API access instead, so the sanctioned route is the paid tier, whose limits then define what your project can measure. Review sites, forums and news are generally more accessible for commercial questions.
Why do two sentiment tools give different numbers?
Mostly because they sample different sources, and each source over-represents a different population. Model differences matter less than that. Compare a tool against its own history rather than against another tool’s absolute percentage.
Where do sentiment models fail?
Sarcasm, negation spread across clauses, and domain-specific vocabulary where an ordinarily negative word is neutral in context. These failures produce a confident score rather than an error, so a monthly hand-check of a sample is the only practical detection.
What is the most under-used sentiment source?
Your own support tickets and sales notes. They are already yours, already about your product, carry no third-party access questions, and are the most directly actionable text in the whole category.
Public sources, collected the same way every run
Review pages, forums and news render differently by region, and a sentiment trend built on a shifting sample is not a trend. DataImpulse residential proxies pin collection to a country at $1 per GB across 195 countries. Create an account and fix your sampling context.
Related: data enrichment · is web scraping legal · proxies for web scraping.
Last updated: September 17, 2026.
State/City/Zip/ASN Targeting 
