Sentiment Analysis Engine.

Project Overview

Mines opinion and emotion from reviews and social posts at scale — turning mountains of free text into a readable pulse of what people feel.

Every day people write vast amounts about products, brands, services and events — reviews, tweets, comments, support tickets — and buried in that text is something organisations badly want: how people feel. Reading it all is impossible, so this project builds a sentiment analysis engine that reads free text and classifies its opinion and emotion — positive/negative/neutral, and often finer emotions — automatically, at scale, turning a firehose of unstructured text into a measurable pulse of sentiment.

At its core it is a text classification problem: map a piece of text to a sentiment label. Approaches range from classic (lexicon methods that score sentiment-bearing words; bag-of-words models like TF-IDF plus a classifier) to modern transformer models fine-tuned for sentiment, which understand context far better. On top of the basic label, useful engines add aspect-based sentiment (a review can be positive about battery life but negative about price — sentiment per aspect, not just overall) and aggregation (trends over time, by product, by source), which is where the business value lives.

The value is scale and structure: continuous, quantified sentiment across thousands of texts, revealing what customers love and hate and how it shifts. It is honest about the genuine hard parts of language that trip up naïve models: sarcasm and irony ("great, another update that breaks everything"), negation ("not bad" is positive), context and domain-specific meaning, mixed sentiment, and bias in training data; accuracy is good but never perfect, so it informs rather than replaces judgement. Built honestly — a well-chosen model, aspect awareness where it matters, sensible aggregation, and clear-eyed about sarcasm/negation limits — it is both a genuinely useful text-mining tool and a definitive lesson in NLP text classification.

What this project does

• Classifies text sentiment (positive/negative/neutral, emotions)
• Reads reviews, social posts, comments, tickets at scale
• Supports aspect-based sentiment (per feature/topic)
• Aggregates sentiment over time, product and source
• Turns unstructured text into measurable signal
• Reveals what people love, hate and how it shifts
• Informs decisions with a quantified pulse of opinion

Real-World Applications

Deployment contexts where a build of this kind earns its keep.

Features & Capabilities

• Text-classification sentiment (classic → transformer)
• Emotion/opinion labelling
• Aspect-based sentiment
• Aggregation and trends
• Scale over large text volumes
• Confidence-scored outputs
• Honest about sarcasm, negation and bias

Difficulty, Time & Required Skills

Skills you should have (or will pick up)

• Text classification (classic and transformer)
• Aspect-based sentiment
• Aggregation and trend analysis
• Handling sarcasm/negation/context limits
• Evaluation and bias awareness

Bill of Materials

Every part below is commonly available from Indian and international hobby-electronics suppliers. Prices are indicative 2026 retail figures in Indian rupees and will drift — treat them as a budgeting guide, not a quotation.

Estimated total: ₹0, excluding tools, shipping and consumables.

Tools and consumables

• Soldering iron (temperature controlled, 350 °C) with 0.8 mm 60/40 or lead-free solder
• Digital multimeter — continuity, DC volts and current ranges
• Wire strippers, flush cutters and a small set of precision screwdrivers
• Heat-shrink tubing and a heat gun (or a lighter, carefully)
• A laptop with a USB port and the toolchain listed above

Hardware Specifications

This is a pure-software NLP system with no electronic hardware to specify. The "platform" is a computer: a CPU handles lexicon and TF-IDF models and moderate transformer inference, while a GPU accelerates transformer fine-tuning and high-volume classification.

Memory and storage scale with the volume of text ingested and the aggregation history retained; a production deployment adds ingestion connectors (reviews/social/tickets) and a dashboard store for the aggregated pulse. The rest of the system is the software stack, libraries and models below.

Software Requirements & Development Environment

Reference toolchain: Python 3.11 + NLP (transformers/classical). Anything newer normally works; anything older may lack the board definitions used here.

• Install the Arduino IDE 2.3.x (or PlatformIO if you prefer a real editor and dependency locking).
• Add https://espressif.github.io/arduino-esp32/package_esp32_index.json under File → Preferences → Additional Board Manager URLs, then install esp32 from the Boards Manager.
• Set the correct port under Tools → Port. On Linux add yourself to the dialout group: sudo usermod -aG dialout $USER and log out and back in.
• Open the Serial Monitor at 115200 baud — every sketch here logs its state there.
• Keep File → Preferences → Show verbose output during: compilation switched on while you are debugging build errors.

Required libraries

Block Diagram

The block diagram shows the functional decomposition of the system — what senses, what decides, what acts, and where the data ends up.

Circuit Diagram & Wiring

The "wiring" is the analysis data flow — text is preprocessed and classified into sentiment (overall and per aspect), then aggregated into trends and reports.

Wire one row at a time and tick it off — most "it does not work" reports trace back to a single swapped pair.

Wiring explanation

• Ingest text from reviews, social posts, comments or tickets.
• Preprocess (clean, tokenise) for the model.
• Classify sentiment overall and, where useful, per aspect.
• Aggregate over time, product and source into a readable pulse.
• Mind sarcasm/negation/context — accuracy is good, not perfect.

System Architecture

Read the stack from the bottom up: physical hardware, the firmware that drives it, the transport that moves data off the device, and the software a human actually looks at.

Working Principle

Sentiment analysis exists to solve a scale problem: the opinions organisations want are scattered across more free text than any human could ever read, so the goal is to read it automatically and turn feeling into a measurable signal. Framed technically, it is a text classification task — map a piece of text to a sentiment label (positive/negative/neutral, or finer emotions) — and everything else is refinement of that core. The payoff is converting an unstructured firehose into structured, quantified data you can chart, segment and act on.

The approaches sit on a clear sophistication spectrum, and understanding it is half the lesson. Lexicon methods score text by looking up sentiment-bearing words in a dictionary (fast, transparent, but blind to context). Bag-of-words models (TF-IDF features into a classifier) learn from labelled data which word patterns signal which sentiment — better, but still ignore word order and context. Transformer models fine-tuned for sentiment read text in context, understanding how surrounding words change meaning, and are markedly more accurate on hard cases. The right choice depends on the accuracy needed, the compute available, and how much labelled domain data you have.

The refinement that unlocks real value is aspect-based sentiment. Overall sentiment is coarse: a product review saying "amazing camera, but the battery is terrible and it's overpriced" is not simply "positive" or "negative" — it is positive about the camera, negative about battery and price. Aspect-based sentiment attributes feeling to specific aspects (camera, battery, price, service), which is what a product team actually needs — not "reviews are 60% positive" but "praise for the camera, complaints about battery life". Pairing this with aggregation — sentiment trends over time, broken down by product, feature and source — is where sentiment analysis stops being a classifier and becomes business intelligence.

The honesty this project requires is about the ways language defeats naïve models, and they are not edge cases — they are everywhere in real text. Sarcasm and irony invert surface sentiment ("oh great, another crash" is negative despite "great"), and are genuinely hard even for strong models. Negation flips polarity ("not bad" is positive; "I can't say I love it" is negative) and trips up lexicon and bag-of-words methods that miss the "not". Context and domain shift meaning ("this vacuum really sucks" is praise). Real text has mixed sentiment, informal spelling, emoji and code-switching, and models inherit bias from their training data. So accuracy, while good, is never perfect, and the engine should inform judgement, not replace it — aggregate signals are trustworthy at scale even when individual classifications occasionally err. Built with a model matched to the need, aspect awareness where it matters, sensible aggregation, and clear eyes about sarcasm, negation and bias, the engine delivers exactly what its users want — a scalable, quantified pulse of public opinion — while teaching the central skills and the central humility of NLP text classification.

The maths behind it

Sentiment classification

model(text) → sentiment label (+ confidence)

  {positive, negative, neutral}  (or finer emotions)

Lexicon → bag-of-words (TF-IDF) → transformer
(rising context-awareness and accuracy).

Aspect-based sentiment

For a review mentioning aspects {camera, battery, price}:

  sentiment(aspect) for each aspect
  → "positive camera, negative battery/price"

Far more actionable than one overall label.

Aggregation (the value)

pulse = aggregate(labels) over time / product / source

  trend(t), share_positive, top complaints/praises

Individual errors wash out at scale; the AGGREGATE signal
is trustworthy. Inform judgement, do not replace it.

Program Flowchart

The firmware is a single cooperative loop. Nothing blocks for long, so networking, sensing and the user interface all stay responsive.

Assembly Instructions

Build on a breadboard first and only commit to solder once the whole system has run for an hour without a fault.

1. Choose and set up a sentiment model

   Pick a model for the need (lexicon/TF-IDF for simple/transparent, transformer for accuracy) and classify text into sentiment.

2. Add aspect-based sentiment

   Where it matters, attribute sentiment to specific aspects (camera, battery, price) rather than one overall label.

3. Aggregate into a pulse

   Aggregate sentiment over time, product and source into trends and reports — where the value lives.

Step-by-Step Implementation Guide

Work through these in order. Each step ends in something you can observe, so a failure is always localised to the step you just finished.

1. Classify sentiment and per aspect

   Classify overall sentiment, and where useful attribute sentiment to specific aspects of the text.

   pythonanalyse.py Copy Download

   def classify(text, model):
       out = model(text)                    # contextual sentiment (transformer)
       return {"label": out.label, "confidence": float(out.score)}
   
   def aspect_sentiment(text, model, aspects):
       result = {}
       for a in aspects:                    # per-feature opinion, not just overall
           span = sentence_about(text, a)   # the clause discussing this aspect
           if span:
               result[a] = classify(span, model)["label"]
       return result                        # {"camera": "pos", "battery": "neg"}

   out = model(text) # contextual sentiment (transformer)A context-aware model reads the text in context, handling much of what lexicon/bag-of-words methods miss.

   for a in aspects: # per-feature opinion, not just overallAspect-based sentiment attributes feeling to specific aspects — the actionable form a product team needs.

   result[a] = classify(span, model)["label"]Each aspect gets its own sentiment from the clause about it, so "great camera, bad battery" is captured faithfully.

2. Aggregate into trends

   Aggregate individual labels over time, product and source into a readable pulse, where individual errors wash out at scale.

Complete Source Code

The listing below is complete and compiles as written — there are no elided sections. Read the annotations under each block before you upload it.

#!/usr/bin/env python3
"""
Sentiment Analysis Engine

Classifies text sentiment (and per-aspect), at scale, from classic to
transformer models, and AGGREGATES into a quantified pulse over time,
product and source. Sarcasm/negation are hard — accuracy is good, not
perfect; trust aggregates at scale and inform judgement, don't replace it.
"""
from collections import Counter, defaultdict

class SentimentEngine:
    def __init__(self, model, aspects=None):
        self.model = model               # lexicon / TF-IDF / transformer
        self.aspects = aspects or []

    def classify(self, text):
        out = self.model(text)           # sentiment + confidence
        return {"label": out.label, "confidence": float(out.score)}

    def aspects_of(self, text):
        # per-aspect sentiment (the actionable form)
        res = {}
        for a in self.aspects:
            span = sentence_about(text, a)
            if span:
                res[a] = self.classify(span)["label"]
        return res

    def analyse(self, item):
        return {"overall": self.classify(item.text),
                "aspects": self.aspects_of(item.text),
                "source": item.source, "ts": item.ts}

    def pulse(self, items):
        # THE VALUE: quantified sentiment across many texts
        overall = Counter()
        by_aspect = defaultdict(Counter)
        for it in items:
            a = self.analyse(it)
            overall[a["overall"]["label"]] += 1
            for asp, lab in a["aspects"].items():
                by_aspect[asp][lab] += 1
        return {"overall": overall, "by_aspect": by_aspect}
        # Individual errors (sarcasm/negation) wash out; the trend is reliable.

if __name__ == "__main__":
    engine = SentimentEngine(sentiment_model, aspects=["camera","battery","price"])
    report = engine.pulse(REVIEWS)
    # A quantified pulse of opinion — informs decisions, doesn't replace judgement.

Configuration & Calibration

Configuration steps

• Configure the sentiment model (classic/transformer) and classes/emotions.
• Configure aspects for aspect-based sentiment.
• Configure aggregation windows and segments (product/source/time).
• Configure confidence gating and hard-case handling.

Calibration procedure

An uncalibrated sensor produces confident, precise, wrong numbers. Do this once per physical unit and record the constants.

1. Model choice

   Pick the model matching accuracy needs and available labelled data/compute.

2. Hard cases

   Evaluate on sarcasm/negation/mixed examples; set expectations honestly.

3. Aggregation

   Verify aggregate trends are stable and meaningful over your volumes.

Dataset, Model & Training

Dataset

Sentiment-labelled text; domain-matched data matters (product reviews differ from tweets). Transformers can be fine-tuned; classic models train on TF-IDF features.

Coverage of sarcasm, negation, mixed sentiment and informal text improves robustness.

Data preprocessing

• Clean and tokenise text; handle emoji, informal spelling, negation cues.
• For transformers, use the model tokeniser; for classic, build TF-IDF features.
• Segment aspects/sentences for aspect-based sentiment.

Hyperparameters

Training process

• Fine-tune a transformer (or train a TF-IDF classifier) on domain-matched labelled data.
• Include sarcasm/negation/mixed examples; validate on a hard-case set.
• Check for bias; evaluate on held-out real text, not just clean benchmarks.

Evaluation, Metrics & Deployment

Accuracy/F1 per class matters, but so does honest evaluation on hard cases (sarcasm, negation) and the reliability of aggregate trends.

Figures from the reference training run described above — reproduce them before trusting your own changes.

Inference example

from collections import Counter

def classify(text, model):
    out = model(text)                    # transformer/classic sentiment
    return {"label": out.label, "confidence": float(out.score)}

def aspect_sentiment(text, model, aspects):
    # sentiment per aspect (camera/battery/price), not just overall
    result = {}
    for a in aspects:
        span = sentence_about(text, a)   # the part discussing this aspect
        if span:
            result[a] = classify(span, model)["label"]
    return result                        # e.g. {camera: pos, battery: neg}

def aggregate(items, model):
    # the value: a quantified pulse over many texts
    labels = [classify(t, model)["label"] for t in items]
    return Counter(labels)               # individual errors wash out at scale
    # Sarcasm/negation are hard; accuracy is good, not perfect — inform, don't replace.

Testing Procedure & Expected Output

Test from the bottom up. Confirm power, then each sensor in isolation, then the integrated loop — the first failing step tells you exactly where to look.

Bench-test checklist. If a row fails, stop and fix it before moving on.

Expected output

Per-text sentiment (overall and per aspect) and an aggregated pulse over time, product and source.

{
  "overall": { "positive": 612, "neutral": 190, "negative": 198 },
  "by_aspect": {
    "camera": { "positive": 410, "negative": 60 },
    "battery": { "positive": 90, "negative": 280 },
    "price": { "positive": 70, "negative": 210 }
  },
  "note": "trust the aggregate; sarcasm/negation err individually"
}

Across a thousand reviews: broadly positive overall, but aspect-based sentiment reveals the real story — the camera is loved while battery and price draw complaints, exactly the actionable insight aggregation provides.

Troubleshooting: Common Errors & Fixes

Performance Optimisation

• Match the model to accuracy needs, data and compute.
• Add aspect-based sentiment for actionable insight.
• Aggregate at scale — trends are reliable even when items err.
• Handle negation/context with a context-aware model.
• Replace every delay() with a millis() comparison — blocking delays are the single most common cause of dropped readings.
• Sample sensors on a fixed cadence and publish on a slower one; you almost never need to transmit at the sampling rate.
• Move networking into its own FreeRTOS task so a slow DNS lookup cannot stall the control loop.
• Use uint8_t / uint16_t where the range allows; on an 8-bit AVR a 32-bit add costs four times as much.
• Profile before optimising — print micros() deltas around each stage and fix the slowest one first.

Safety Precautions

• Accuracy is never perfect (sarcasm/negation) — inform judgement, do not replace it.
• Trust aggregate trends over individual classifications for decisions.
• Watch for and mitigate bias inherited from training data.
• Analysing user text raises privacy obligations — handle lawfully.
• Wear eye protection when soldering or cutting, and solder in a ventilated space — rosin flux fumes are a respiratory irritant.
• Power the circuit through a bench supply with a current limit while you are testing. A 300 mA limit turns a wiring mistake into a beep instead of a dead board.
• Disconnect power before changing any wiring. Hot-plugging a sensor onto a live bus is the fastest way to lose a controller.

Maintenance

• Refresh models/data as language and domains evolve.
• Re-test on hard cases (sarcasm/negation) and for bias.
• Update aspects and aggregation as products change.
• Monitor drift in sources and vocabulary.
• Re-check every screw terminal and header after the first week — thermal cycling loosens connections that felt tight on day one.
• Recalibrate at the interval given in the calibration section, and keep the constants in a text file next to the firmware — not only in flash.
• Keep a short logbook of firmware versions and what changed. Six months later you will not remember why that constant is 1.083.

Future Improvements & Upgrades

A working v1 is a platform, not a finish line. These are the upgrades that add the most capability for the least rework.

• Add emotion granularity (joy/anger/fear, etc.).
• Add multilingual and code-switching support.
• Add sarcasm-aware modelling and stance detection.
• Add real-time streaming sentiment and alerting.
• Design a proper PCB. Once the breadboard version has run for a month, moving to a two-layer board removes the intermittent-contact failures that dominate prototype faults.
• Add connectivity — an ESP32 and an MQTT publish turn a local gadget into something you can graph, alert on and analyse over months.
• Add persistent local storage (microSD or the on-chip flash) so a network outage does not create a hole in your data.
• Move configuration out of the source: a captive-portal setup page or a JSON config file makes the build reusable without a recompile.
• Add a battery and solar option so the unit survives a power cut and can be sited away from a socket.
• Write a small test harness that feeds synthetic sensor values through the decision logic, so you can validate thresholds without physically triggering the event.

Frequently Asked Questions

References & Learning Resources

These are the primary sources worth reading in full. Manufacturer datasheets always outrank forum posts when the two disagree.

1. Sentiment analysisReference
2. Text classificationReference
3. Aspect-based sentiment analysisReference
4. Transformers (NLP)Reference
5. TF-IDFReference