Music Genre Classifier.

Project Overview

Classifies songs into genres from their audio — turning sound into features (or spectrograms) a model can learn to tell rock from jazz.

Sorting music into genres — rock, jazz, classical, hip-hop, electronic — is a classic and approachable machine-learning problem that teaches the whole craft of audio classification. This project builds a model that listens to a song (or a clip) and predicts its genre from the audio itself, learning the sonic signatures that distinguish styles: rhythm and tempo, instrumentation and timbre, harmonic content and energy.

The core lesson is that you cannot feed raw audio waveforms straight to a classifier effectively — you must first turn sound into meaningful features. There are two classic routes. The first extracts hand-crafted audio features — MFCCs (capturing timbre), tempo, spectral properties, chroma (harmony), zero-crossing rate — and feeds them to a classifier. The second turns the audio into a spectrogram (a picture of frequency content over time) and treats genre classification as an image classification problem with a CNN, since a spectrogram looks different for different genres. Both work; the feature-based route is transparent and light, the spectrogram-CNN route is powerful and reuses image-model machinery.

The value is a hands-on introduction to audio ML — feature extraction, spectrograms, and classification — with an intuitive, fun task and famous datasets (like GTZAN). It is honest that genre is fuzzy: genres overlap, blend and are partly subjective and cultural, so there is no perfect ground truth and even humans disagree on boundary cases; benchmark datasets (GTZAN especially) have known flaws and biases that inflate scores; and models can latch onto production artefacts rather than musical content. So accuracy is meaningful but not absolute, and the interesting cases are the ambiguous ones. Built honestly — feature or spectrogram-based, with realistic expectations about fuzzy labels — it is both a satisfying project and the clearest lesson in turning audio into something a model can classify.

What this project does

• Classifies songs/clips into music genres from audio
• Extracts audio features (MFCC, tempo, spectral, chroma)
• Or uses spectrograms + a CNN (audio as images)
• Learns sonic signatures of genres
• Outputs genre with confidence
• Teaches audio feature extraction and classification
• Handles fuzzy, overlapping genre labels honestly

Real-World Applications

Deployment contexts where a build of this kind earns its keep.

Features & Capabilities

• Hand-crafted audio features + classifier
• Spectrogram + CNN (image-style) route
• Genre prediction with confidence
• Feature-vs-spectrogram comparison
• Works on clips or full tracks
• Uses classic datasets (e.g. GTZAN)
• Honest about fuzzy labels and dataset flaws

Difficulty, Time & Required Skills

Skills you should have (or will pick up)

• Audio feature extraction (MFCC, tempo, spectral, chroma)
• Spectrograms and audio-as-image CNNs
• Classification and evaluation (accuracy, confusion)
• Handling fuzzy/overlapping labels
• Dataset awareness (GTZAN flaws)

Bill of Materials

Every part below is commonly available from Indian and international hobby-electronics suppliers. Prices are indicative 2026 retail figures in Indian rupees and will drift — treat them as a budgeting guide, not a quotation.

Estimated total: ₹0, excluding tools, shipping and consumables.

Tools and consumables

• Soldering iron (temperature controlled, 350 °C) with 0.8 mm 60/40 or lead-free solder
• Digital multimeter — continuity, DC volts and current ranges
• Wire strippers, flush cutters and a small set of precision screwdrivers
• Heat-shrink tubing and a heat gun (or a lighter, carefully)
• A laptop with a USB port and the toolchain listed above

Hardware Specifications

This is a pure-software audio-ML system — no electronic hardware to specify. The "platform" is a computer: a CPU handles feature extraction and classical models; a GPU accelerates spectrogram-CNN training.

Memory scales with audio length and batch size; storage holds the audio dataset and features/spectrograms. A deployment adds an upload/classify UI. Everything else is the software stack, models and libraries below.

Software Requirements & Development Environment

Reference toolchain: Python 3.11 + audio ML (librosa/PyTorch). Anything newer normally works; anything older may lack the board definitions used here.

• Install the Arduino IDE 2.3.x (or PlatformIO if you prefer a real editor and dependency locking).
• Add https://espressif.github.io/arduino-esp32/package_esp32_index.json under File → Preferences → Additional Board Manager URLs, then install esp32 from the Boards Manager.
• Set the correct port under Tools → Port. On Linux add yourself to the dialout group: sudo usermod -aG dialout $USER and log out and back in.
• Open the Serial Monitor at 115200 baud — every sketch here logs its state there.
• Keep File → Preferences → Show verbose output during: compilation switched on while you are debugging build errors.

Required libraries

Block Diagram

The block diagram shows the functional decomposition of the system — what senses, what decides, what acts, and where the data ends up.

Circuit Diagram & Wiring

The "wiring" is the classification data flow — audio is converted into features (or a spectrogram) and passed to a classifier (or CNN) that predicts the genre.

Wire one row at a time and tick it off — most "it does not work" reports trace back to a single swapped pair.

Wiring explanation

• Take an audio clip or track as input.
• Extract features (MFCC/tempo/spectral/chroma) OR compute a spectrogram.
• Feed features to a classifier, or the spectrogram to a CNN.
• Output the predicted genre with confidence.
• Remember genre labels are fuzzy — ambiguous tracks are expected.

System Architecture

Read the stack from the bottom up: physical hardware, the firmware that drives it, the transport that moves data off the device, and the software a human actually looks at.

Working Principle

Music-genre classification is the ideal first audio machine-learning project because it makes the field's central lesson unavoidable: you cannot learn effectively from raw audio waveforms. A waveform is a long, high-rate sequence of amplitude samples with the musically meaningful information (rhythm, timbre, harmony) buried in its frequency structure over time, not visible in the raw samples. The whole art of audio ML is transforming sound into a representation that exposes what matters, and this project teaches exactly that transformation, twice.

The first route extracts hand-crafted features that summarise musically relevant properties. MFCCs (mel-frequency cepstral coefficients) capture timbre — the "colour" of the sound that distinguishes a distorted guitar from a piano — and are the workhorse audio feature. Tempo and rhythmic features capture the beat; spectral features (centroid, rolloff) capture brightness and energy distribution; chroma captures harmonic/pitch content. Feed a vector of these to a standard classifier and it can learn that, say, high tempo plus certain spectral energy plus particular timbres tends to mean one genre. This route is transparent and lightweight, and it teaches which acoustic properties define genres.

The second route reframes the problem as image classification. Compute a spectrogram — a 2-D image with time on one axis, frequency on the other, and intensity as colour — and a genre's characteristic patterns (a four-on-the-floor kick, dense orchestral harmonics, a hip-hop beat) literally look different in the image. So you can feed the spectrogram to a CNN, exactly as in image classification (project A07), letting the network learn the discriminative visual-audio patterns itself. This route is more powerful and elegantly reuses image-model machinery, which is why the "audio as spectrogram images" trick is so widely used across audio AI.

The honesty this project needs is about the fuzziness of the label itself, which is more fundamental than the usual accuracy caveats. Genre is not a clean, objective category: genres overlap and blend (where exactly does rock become metal, or pop become electronic?), they are partly subjective and cultural, and songs deliberately cross boundaries — so there is no perfect ground truth, and even human experts disagree on boundary cases. This means a genre classifier can never be "perfectly accurate", because the target is itself blurry, and the interesting cases are precisely the ambiguous ones. Compounding this, the famous benchmark datasets — GTZAN above all — have well-documented flaws (duplicate/mislabelled tracks, artist repetition across splits) that inflate reported accuracy, and models can cheat by latching onto production or recording artefacts rather than musical content. So results should be read with realism: accuracy is meaningful but not absolute, and clean-looking benchmark numbers may not reflect real generalisation. Built with either representation route and clear eyes about fuzzy labels and dataset flaws, the classifier is a genuinely satisfying project and the clearest possible lesson in the foundational move of audio ML — turning sound into features (or spectrograms) a model can learn from.

The maths behind it

Represent audio (the key move)

Route 1 (features):  audio → [MFCCs, tempo, spectral, chroma]
Route 2 (spectrogram): audio → time×frequency image

You CANNOT feed raw waveforms effectively — represent first.

Classify

Route 1: classifier(features) → genre probabilities
Route 2: CNN(spectrogram) → genre probabilities   (audio as images)

genre = argmax(probs); confidence = max(probs).

Genre is fuzzy (be honest)

genres overlap/blend; partly subjective/cultural
  → no perfect ground truth; humans disagree on edges
GTZAN etc. have flaws (dupes, artist leakage) → inflated scores

Accuracy is meaningful, not absolute; ambiguous cases are the point.

Program Flowchart

The firmware is a single cooperative loop. Nothing blocks for long, so networking, sensing and the user interface all stay responsive.

Assembly Instructions

Build on a breadboard first and only commit to solder once the whole system has run for an hour without a fault.

1. Represent audio as features or spectrograms

   Choose a route: extract hand-crafted features (MFCC/tempo/spectral/chroma), or compute spectrograms for a CNN.

2. Train and evaluate honestly

   Train a classifier (features) or CNN (spectrograms) on clean, artist-disjoint splits, and inspect the confusion matrix.

3. Classify and interpret

   Predict genre with confidence, and treat ambiguous/blended tracks as the interesting cases, not failures.

Step-by-Step Implementation Guide

Work through these in order. Each step ends in something you can observe, so a failure is always localised to the step you just finished.

1. Extract features and classify

   Turn audio into features (or a spectrogram) and predict the genre with confidence.

   pythongenre.py Copy Download

   import numpy as np, librosa
   
   def features(path):                        # represent audio (the key move)
       y, sr = librosa.load(path, duration=30)
       mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=20).mean(axis=1)   # timbre
       tempo = librosa.beat.tempo(y=y, sr=sr)                             # rhythm
       chroma = librosa.feature.chroma_stft(y=y, sr=sr).mean(axis=1)     # harmony
       return np.concatenate([mfcc, tempo, chroma])
   
   def classify(path, model, labels):
       probs = model.predict_proba([features(path)])[0]
       i = int(np.argmax(probs))
       return {"genre": labels[i], "confidence": round(float(probs[i]), 2)}

   def features(path): # represent audio (the key move)The essential step: raw audio is turned into meaningful features, because a classifier cannot learn from waveforms directly.

   mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=20).mean(axis=1) # timbreMFCCs capture timbre — the sonic colour that distinguishes instruments and, in aggregate, genres.

   tempo = librosa.beat.tempo(y=y, sr=sr) # rhythmTempo and rhythm features capture the beat, a strong genre signal.

   return {"genre": labels[i], "confidence": round(float(probs[i]), 2)}The prediction includes confidence, which is especially meaningful given how fuzzy genre boundaries are.

2. Compare routes and read results realistically

   Optionally compare the feature route with a spectrogram CNN, and interpret accuracy knowing genre is fuzzy and datasets flawed.

Complete Source Code

The listing below is complete and compiles as written — there are no elided sections. Read the annotations under each block before you upload it.

#!/usr/bin/env python3
"""
Music Genre Classifier

Classifies audio into genres by REPRESENTING sound as features
(MFCC/tempo/spectral/chroma) for a classifier, OR as spectrograms for a
CNN (audio as images). You cannot learn from raw waveforms directly.
Genre is FUZZY (no perfect ground truth); benchmark datasets (GTZAN)
have flaws that inflate scores — read accuracy with realism.
"""
import numpy as np, librosa

def extract_features(path):                 # Route 1: hand-crafted features
    y, sr = librosa.load(path, duration=30)
    mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=20).mean(axis=1)   # timbre
    tempo = librosa.beat.tempo(y=y, sr=sr)                             # rhythm
    centroid = librosa.feature.spectral_centroid(y=y, sr=sr).mean()   # brightness
    chroma = librosa.feature.chroma_stft(y=y, sr=sr).mean(axis=1)     # harmony
    return np.concatenate([mfcc, tempo, [centroid], chroma])

def to_spectrogram(path):                   # Route 2: audio as image (for a CNN)
    y, sr = librosa.load(path, duration=30)
    return librosa.power_to_db(librosa.feature.melspectrogram(y=y, sr=sr))

class GenreClassifier:
    def __init__(self, model, labels, route="features"):
        self.model = model; self.labels = labels; self.route = route

    def classify(self, path):
        x = extract_features(path) if self.route == "features" else to_spectrogram(path)
        probs = self.model.predict_proba([x])[0] if self.route == "features" \
                else self.model.predict(x[None])[0]
        i = int(np.argmax(probs))
        return {"genre": self.labels[i], "confidence": round(float(probs[i]), 2)}

if __name__ == "__main__":
    clf = GenreClassifier(trained_model, GENRES, route="features")
    print(clf.classify("track.mp3"))
    # Use artist-disjoint splits; be sceptical of inflated benchmark scores.

Configuration & Calibration

Configuration steps

• Configure the route (features vs spectrogram) and clip length.
• Configure feature set (MFCC count, tempo, spectral, chroma) or spectrogram params.
• Configure the classifier/CNN and genre labels.
• Configure clean, artist-disjoint dataset splits.

Calibration procedure

An uncalibrated sensor produces confident, precise, wrong numbers. Do this once per physical unit and record the constants.

1. Representation

   Verify features/spectrograms capture genre-relevant content; compare routes.

2. Evaluation

   Use artist-disjoint splits; inspect the confusion matrix; be sceptical of inflated scores.

3. Confidence

   Calibrate confidence; expect low confidence on ambiguous/blended tracks.

Dataset, Model & Training

Dataset

Genre-labelled audio (e.g. GTZAN, FMA). GTZAN is classic but has documented flaws (duplicates, mislabels, artist leakage) that inflate scores.

Balanced, clean, artist-disjoint splits matter for honest evaluation.

Data preprocessing

• Segment audio into clips; resample; normalise.
• Route 1: extract MFCC/tempo/spectral/chroma features.
• Route 2: compute (mel) spectrograms as model inputs.

Hyperparameters

Training process

• Route 1: train a classifier on extracted features. Route 2: train/fine-tune a CNN on spectrograms.
• Use clean, artist-disjoint splits; do not trust GTZAN numbers naively.
• Inspect the confusion matrix — confusions are often between genuinely similar genres.

Evaluation, Metrics & Deployment

Accuracy and the confusion matrix, read with realism: genre is fuzzy, so confusions between similar genres are expected, and benchmark scores can be inflated by dataset flaws.

Figures from the reference training run described above — reproduce them before trusting your own changes.

Inference example

import numpy as np, librosa

def features(path):                       # Route 1: hand-crafted features
    y, sr = librosa.load(path, duration=30)
    mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=20).mean(axis=1)  # timbre
    tempo = librosa.beat.tempo(y=y, sr=sr)                            # rhythm
    centroid = librosa.feature.spectral_centroid(y=y, sr=sr).mean()  # brightness
    chroma = librosa.feature.chroma_stft(y=y, sr=sr).mean(axis=1)    # harmony
    return np.concatenate([mfcc, tempo, [centroid], chroma])

def classify(path, model, labels):
    probs = model.predict_proba([features(path)])[0]
    i = int(np.argmax(probs))
    return {"genre": labels[i], "confidence": float(probs[i])}
    # Genre is fuzzy — ambiguous tracks are expected; GTZAN scores can inflate.

Testing Procedure & Expected Output

Test from the bottom up. Confirm power, then each sensor in isolation, then the integrated loop — the first failing step tells you exactly where to look.

Bench-test checklist. If a row fails, stop and fix it before moving on.

Expected output

A predicted genre with confidence per track, read with realism about fuzzy labels.

{
  "track": "track.mp3",
  "genre": "jazz",
  "confidence": 0.71,
  "runner_up": "blues (0.22)",
  "note": "genre is fuzzy; jazz/blues genuinely overlap"
}

A track classified as jazz with blues a close runner-up — the model correctly reflecting that these genres genuinely overlap, exactly the kind of fuzzy boundary that makes genre classification interesting.

Troubleshooting: Common Errors & Fixes

Performance Optimisation

• Represent audio well (features or spectrograms) — the key move.
• Evaluate on clean, artist-disjoint splits; distrust inflated benchmarks.
• Inspect confusions; expect similar-genre overlap.
• Use GPU for spectrogram CNNs; feature route is lighter.
• Replace every delay() with a millis() comparison — blocking delays are the single most common cause of dropped readings.
• Sample sensors on a fixed cadence and publish on a slower one; you almost never need to transmit at the sampling rate.
• Move networking into its own FreeRTOS task so a slow DNS lookup cannot stall the control loop.
• Use uint8_t / uint16_t where the range allows; on an 8-bit AVR a 32-bit add costs four times as much.
• Profile before optimising — print micros() deltas around each stage and fix the slowest one first.

Safety Precautions

• Genre labels are fuzzy and subjective — do not present predictions as objective truth.
• Respect music copyright and licensing of any audio used.
• Benchmark scores can be inflated — report honest, leakage-free evaluation.
• Be aware models may key on artefacts rather than musical content.
• Wear eye protection when soldering or cutting, and solder in a ventilated space — rosin flux fumes are a respiratory irritant.
• Power the circuit through a bench supply with a current limit while you are testing. A 300 mA limit turns a wiring mistake into a beep instead of a dead board.
• Disconnect power before changing any wiring. Hot-plugging a sensor onto a live bus is the fastest way to lose a controller.

Maintenance

• Refresh datasets and genres as needed; keep splits clean.
• Re-evaluate honestly as models change.
• Compare feature vs spectrogram routes over time.
• Watch for artefact-driven predictions.
• Re-check every screw terminal and header after the first week — thermal cycling loosens connections that felt tight on day one.
• Recalibrate at the interval given in the calibration section, and keep the constants in a text file next to the firmware — not only in flash.
• Keep a short logbook of firmware versions and what changed. Six months later you will not remember why that constant is 1.083.

Future Improvements & Upgrades

A working v1 is a platform, not a finish line. These are the upgrades that add the most capability for the least rework.

• Add multi-label / sub-genre and mood classification.
• Add larger, cleaner datasets (FMA) and audio transformers.
• Add explainability (what audio drove the prediction).
• Add streaming/real-time genre tagging.
• Design a proper PCB. Once the breadboard version has run for a month, moving to a two-layer board removes the intermittent-contact failures that dominate prototype faults.
• Add connectivity — an ESP32 and an MQTT publish turn a local gadget into something you can graph, alert on and analyse over months.
• Add persistent local storage (microSD or the on-chip flash) so a network outage does not create a hole in your data.
• Move configuration out of the source: a captive-portal setup page or a JSON config file makes the build reusable without a recompile.
• Add a battery and solar option so the unit survives a power cut and can be sited away from a socket.
• Write a small test harness that feeds synthetic sensor values through the decision logic, so you can validate thresholds without physically triggering the event.

Frequently Asked Questions

References & Learning Resources

These are the primary sources worth reading in full. Manufacturer datasheets always outrank forum posts when the two disagree.

1. Music information retrievalReference
2. MFCCReference
3. SpectrogramReference
4. GTZAN dataset (and criticism)Reference
5. librosa audio libraryDocs