Pose-Estimation Fitness Coach.

Project Overview

Watches you exercise, counts your reps, and calls out your form in real time by tracking your body's keypoints — a coach in a camera.

A good exercise coach does two things a mirror cannot: they count your reps so you don't have to, and they watch your form and correct it before a sloppy squat becomes a hurt knee. This project builds a camera-based coach that does both automatically, in real time, by tracking the positions of your body's joints as you move. It turns any camera into a rep-counter and form-checker — useful for home workouts, physiotherapy, and anyone training without a trainer.

The foundation is human pose estimation: a model detects the body's keypoints — shoulders, elbows, hips, knees, ankles and so on — in every frame, giving a live stick-figure of the body. From those keypoints the coach computes joint angles (knee bend, elbow bend, hip hinge), and everything follows from tracking how those angles change. A rep is a characteristic up-and-down cycle of the relevant angle (a squat is the knee angle going down past a threshold and back up), so counting reps is detecting those cycles. Form feedback compares the angles against the correct pattern for the exercise (knees not collapsing inward, back angle maintained, full range of motion) and flags deviations live.

The value is objective, tireless feedback — accurate counts and instant form cues without a human trainer. It is honest about the limits of a single camera: pose estimation struggles with occlusion and unusual angles, a 2-D view can miss depth-dependent errors, and camera placement matters; and — importantly — this is a fitness aid, not medical or professional coaching, so form rules are heuristic and it should never be relied on for injury-sensitive rehab without professional oversight. Within those bounds, as a real-time pose-driven rep-counter and form-checker, it is both a genuinely helpful workout companion and a clear lesson in turning pose keypoints into meaningful, actionable analysis.

What this project does

• Counts exercise reps automatically from body motion
• Gives live form feedback (angles, range, alignment)
• Tracks body keypoints with pose estimation
• Computes joint angles and detects rep cycles
• Compares form against the correct pattern per exercise
• Works for home workouts and guided practice
• Provides objective, tireless feedback

Real-World Applications

Deployment contexts where a build of this kind earns its keep.

Features & Capabilities

• Real-time pose (keypoint) estimation
• Joint-angle computation
• Rep counting via angle-cycle detection
• Form checking against exercise patterns
• Live cues (range, alignment, tempo)
• Works on edge/phone/GPU
• Honest about single-camera limits and non-medical scope

Difficulty, Time & Required Skills

Skills you should have (or will pick up)

• Human pose estimation (keypoints)
• Joint-angle geometry from keypoints
• Rep detection via signal (angle) cycles
• Rule-based form checking
• Real-time pose pipelines and their limits

Bill of Materials

Every part below is commonly available from Indian and international hobby-electronics suppliers. Prices are indicative 2026 retail figures in Indian rupees and will drift — treat them as a budgeting guide, not a quotation.

Estimated total: ₹10,400, excluding tools, shipping and consumables.

Tools and consumables

• Soldering iron (temperature controlled, 350 °C) with 0.8 mm 60/40 or lead-free solder
• Digital multimeter — continuity, DC volts and current ranges
• Wire strippers, flush cutters and a small set of precision screwdrivers
• Heat-shrink tubing and a heat gun (or a lighter, carefully)
• A laptop with a USB port and the toolchain listed above

Hardware Specifications

Consolidated electrical and interface specifications for every active part in the build.

Power Budget & Supply Sizing

Add up the typical active current of every part, then size the supply with at least 50 % headroom so transmit bursts and motor inrush never brown out the controller.

Summed typical draw is 1450 mA. With a 1.5× design margin the supply should deliver at least 2200 mA continuously at the stated rail voltage.

Software Requirements & Development Environment

Reference toolchain: Python 3.11 + MediaPipe / PyTorch. Anything newer normally works; anything older may lack the board definitions used here.

• Flash Raspberry Pi OS (64-bit) with Raspberry Pi Imager; pre-configure Wi-Fi, hostname and SSH in the Imager settings so the board comes up headless.
• Update first: sudo apt update && sudo apt full-upgrade -y, then reboot.
• Work inside a virtual environment — python3 -m venv ~/venv && source ~/venv/bin/activate. Bookworm blocks system-wide pip install by design.
• Enable the buses you need with sudo raspi-config → Interface Options (I²C, SPI, Serial, Camera).
• Develop over VS Code Remote-SSH so you edit on your laptop but run on the Pi.

Required libraries

Block Diagram

The block diagram shows the functional decomposition of the system — what senses, what decides, what acts, and where the data ends up.

Circuit Diagram & Wiring

The "wiring" is the analysis data flow — a camera feeds frames to pose estimation; joint angles drive rep counting and form checks, which produce live counts and cues.

Wire one row at a time and tick it off — most "it does not work" reports trace back to a single swapped pair.

Wiring explanation

• Position the camera to see the whole body for the exercise, minimising occlusion.
• A pose model extracts keypoints per frame.
• Compute joint angles from keypoints; detect rep cycles and check form.
• Give live counts and form cues via screen/audio.
• Placement and view angle matter — a 2-D view can miss depth-dependent errors.

System Architecture

Read the stack from the bottom up: physical hardware, the firmware that drives it, the transport that moves data off the device, and the software a human actually looks at.

Working Principle

The whole system rests on reducing the messy visual problem of "watching someone exercise" to a clean, numeric one: track the body's keypoints and reason about angles. Human pose estimation — a well-developed vision capability — detects the coordinates of the body's joints (shoulders, elbows, wrists, hips, knees, ankles) in each frame, producing a live skeletal representation. That skeleton is the abstraction that makes everything else tractable: instead of analysing pixels, the coach analyses a handful of joint positions, exactly the data a human coach implicitly reads when they watch your body move.

From keypoints, the coach computes joint angles, and angles are the language of both counting and form. A joint angle is simple geometry — the angle at the knee, for instance, is the angle between the thigh (hip→knee) and shin (knee→ankle) vectors. As you perform an exercise, the relevant angle traces a characteristic waveform over time: in a squat the knee angle falls as you descend and rises as you stand. This turns exercise analysis into signal analysis of a joint-angle time series, which is a huge simplification.

Rep counting then becomes cycle detection on that waveform. A rep is one full down-and-up excursion of the driving angle: the angle crosses below a "down" threshold (you reached the bottom) and back above an "up" threshold (you returned to the top), completing a cycle. Counting reps is counting those cycles, with hysteresis (two thresholds, not one) so a wobble at the bottom doesn't double-count. This same idea generalises across exercises — you just pick the driving joint and thresholds per movement.

Form feedback is where the coach earns its name, and it works by comparing the observed angles against the correct pattern for the exercise. Good form has geometric signatures: adequate range of motion (did the knee actually reach depth, or was it a half-squat?), alignment (do the knees track over the toes rather than collapsing inward — a knee-valgus check from the hip/knee/ankle geometry?), posture (is the back angle maintained?), and tempo. When an angle or relationship strays outside the acceptable band for that exercise, the coach flags it live, so you can correct mid-set rather than reinforce a bad habit. The honest caveats are essential: a single camera gives a 2-D view, so pose estimation suffers from occlusion (a limb hidden behind the torso) and cannot always see depth-dependent errors, and camera placement strongly affects what can be measured (a side view sees squat depth; a front view sees knee alignment). And the form rules are heuristics, not clinical judgement: this is a fitness aid, not medical or professional coaching, and it must not be leaned on for injury-sensitive rehabilitation without professional oversight. Within those bounds, though, turning keypoints into angles, angles into rep cycles, and angle-deviations into live cues gives a genuinely useful, tireless, objective coach — and a textbook example of extracting meaningful analysis from pose data.

The maths behind it

Joint angle from keypoints

For a joint B with neighbours A and C:

  v1 = A − B,  v2 = C − B
  angle = acos( (v1 · v2) / (|v1| |v2|) )

e.g. knee angle from hip(A), knee(B), ankle(C).

Rep counting (cycle + hysteresis)

Track the driving angle θ over time:

  state DOWN when θ < θ_low   (reached the bottom)
  state UP   when θ > θ_high  (returned to top)
  count a rep on a DOWN→UP transition

Two thresholds (hysteresis) stop wobble double-counting.

Form checks (vs pattern)

range_ok  : θ reached the target depth (θ_low low enough)
align_ok  : knee tracks over foot (valgus angle within band)
posture_ok: back/hip angle within band
tempo_ok  : rep duration within band

Deviation → live cue. Rules are HEURISTIC (fitness aid).

Program Flowchart

The firmware is a single cooperative loop. Nothing blocks for long, so networking, sensing and the user interface all stay responsive.

Assembly Instructions

Build on a breadboard first and only commit to solder once the whole system has run for an hour without a fault.

1. Set up pose estimation

   Stream camera frames and extract body keypoints with a pretrained pose model; smooth to reduce jitter.

   Place the camera to see the whole body for the exercise.

2. Compute angles and count reps

   Compute joint angles from keypoints and detect rep cycles with two-threshold hysteresis.

3. Add form checks and cues

   Compare angles/alignment against the exercise's correct pattern and give live cues; tune thresholds per exercise and view.

Step-by-Step Implementation Guide

Work through these in order. Each step ends in something you can observe, so a failure is always localised to the step you just finished.

1. Turn keypoints into angles, reps and cues

   Compute the driving joint angle, detect rep cycles with hysteresis, and flag form deviations live.

   pythonrep_form.py Copy Download

   import numpy as np
   
   def angle(a, b, c):                        # joint angle at b
       v1, v2 = np.subtract(a, b), np.subtract(c, b)
       cos = np.dot(v1, v2) / (np.linalg.norm(v1)*np.linalg.norm(v2) + 1e-9)
       return np.degrees(np.arccos(np.clip(cos, -1, 1)))
   
   def update_squat(kp, st, low=90, high=160):
       knee = angle(kp["hip"], kp["knee"], kp["ankle"])   # driving angle
       if knee < low: st["phase"] = "down"                # reached the bottom
       if knee > high and st["phase"] == "down":          # returned to top
           st["phase"] = "up"; st["reps"] += 1            # one rep = one cycle
       cue = None
       if st["phase"] == "down" and knee > low + 15:
           cue = "go deeper"                              # range-of-motion form
       return st["reps"], cue

   knee = angle(kp["hip"], kp["knee"], kp["ankle"]) # driving angleThe knee angle is computed from three keypoints — the geometry that turns pose into a measurable exercise signal.

   if knee < low: st["phase"] = "down" # reached the bottomThe low threshold marks the bottom of the movement; two thresholds (hysteresis) prevent wobble from double-counting.

   if knee > high and st["phase"] == "down": # returned to topA rep is counted on the down-to-up transition — cycle detection on the angle waveform.

   cue = "go deeper" # range-of-motion formForm feedback compares the angle against the correct pattern and cues live — here, insufficient depth.

2. Deliver counts and feedback live

   Show the rep count and speak/display form cues in real time so the user corrects mid-set, and tune per exercise/view.

Complete Source Code

The listing below is complete and compiles as written — there are no elided sections. Read the annotations under each block before you upload it.

#!/usr/bin/env python3
"""
Pose-Estimation Fitness Coach

Tracks body KEYPOINTS, computes JOINT ANGLES, counts REPS via angle-cycle
detection (with hysteresis), and gives live heuristic FORM feedback per
exercise. A fitness aid, not medical coaching. Single-camera limits apply.
"""
import numpy as np

def angle(a, b, c):
    v1, v2 = np.subtract(a, b), np.subtract(c, b)
    cos = np.dot(v1, v2) / (np.linalg.norm(v1)*np.linalg.norm(v2) + 1e-9)
    return np.degrees(np.arccos(np.clip(cos, -1, 1)))

class Coach:
    def __init__(self, exercise="squat"):
        self.ex = EXERCISES[exercise]     # driving joint, thresholds, form bands
        self.phase = "up"; self.reps = 0

    def update(self, kp):
        if not confident(kp, self.ex["joints"]):     # skip unreliable poses
            return self.reps, ["can't see you clearly"]
        theta = angle(*[kp[j] for j in self.ex["joints"]])  # driving angle

        # rep = one DOWN->UP cycle (hysteresis)
        if theta < self.ex["low"]:  self.phase = "down"
        if theta > self.ex["high"] and self.phase == "down":
            self.phase = "up"; self.reps += 1

        # live form cues vs the correct pattern
        cues = []
        if self.phase == "down" and theta > self.ex["low"] + 15:
            cues.append("increase range of motion")
        if self.ex.get("valgus") and knee_valgus(kp) > self.ex["valgus"]:
            cues.append("keep knees over toes")
        if not posture_ok(kp, self.ex):
            cues.append("keep your back straight")
        return self.reps, cues

if __name__ == "__main__":
    coach = Coach("squat")
    for frame in camera():
        kp = pose_keypoints(frame)                   # pose estimation
        reps, cues = coach.update(kp)
        show(reps, cues)                             # live count + feedback
    # Heuristic fitness aid — not a substitute for professional coaching.

Configuration & Calibration

Configuration steps

• Configure the pose model, per-exercise driving joint and thresholds.
• Configure form bands (range/alignment/posture/tempo) per exercise.
• Configure smoothing and keypoint-confidence gating.
• Configure count/cue output (screen/audio) and camera view.

Calibration procedure

An uncalibrated sensor produces confident, precise, wrong numbers. Do this once per physical unit and record the constants.

1. Rep thresholds

   Set θ_low/θ_high per exercise so full reps count and partials/wobbles do not.

2. Form bands

   Tune range/alignment/posture tolerances to flag real errors without nagging.

3. View/robustness

   Validate across body types and camera placements; pick views that see the target errors.

Dataset, Model & Training

Dataset

Pose estimation uses a pretrained model (trained on large keypoint datasets); the coaching logic is largely geometric/rule-based on the resulting angles.

Optional: labelled good/bad-form clips to tune or learn form thresholds per exercise.

Data preprocessing

• Extract per-frame keypoints; smooth to reduce jitter.
• Compute joint angles; normalise for body proportions where needed.
• Handle low-confidence/occluded keypoints gracefully.

Hyperparameters

Training process

• Mostly configuration/geometry: set per-exercise thresholds and form bands.
• Optionally learn form thresholds from labelled good/bad clips.
• Validate counts and cues across body types and camera placements.

Evaluation, Metrics & Deployment

Success is accurate rep counts and useful, correct form cues across users and views — not a single accuracy number.

Figures from the reference training run described above — reproduce them before trusting your own changes.

Inference example

import numpy as np

def joint_angle(a, b, c):                 # angle at b (A-B-C)
    v1, v2 = np.array(a) - np.array(b), np.array(c) - np.array(b)
    cos = np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2) + 1e-9)
    return np.degrees(np.arccos(np.clip(cos, -1, 1)))

class SquatCoach:
    def __init__(self, low=90, high=160):
        self.low, self.high = low, high      # rep thresholds (hysteresis)
        self.state = "up"; self.reps = 0

    def update(self, kp):                     # kp: keypoints
        knee = joint_angle(kp["hip"], kp["knee"], kp["ankle"])
        # rep = a DOWN->UP cycle of the knee angle
        if knee < self.low:  self.state = "down"
        if knee > self.high and self.state == "down":
            self.state = "up"; self.reps += 1     # counted a rep

        cues = []
        if self.state == "down" and knee > self.low + 15:
            cues.append("go deeper (range of motion)")     # form: depth
        if knee_valgus(kp) > VALGUS_BAND:
            cues.append("knees out (alignment)")           # form: alignment
        return self.reps, cues
        # NOTE: heuristic form rules — a fitness aid, not medical coaching.

Testing Procedure & Expected Output

Test from the bottom up. Confirm power, then each sensor in isolation, then the integrated loop — the first failing step tells you exactly where to look.

Bench-test checklist. If a row fails, stop and fix it before moving on.

Expected output

Live rep counts and form cues, driven by joint angles from pose keypoints.

{
  "exercise": "squat",
  "reps": 8,
  "knee_angle": 84,
  "phase": "down",
  "cues": ["keep knees over toes"],
  "note": "fitness aid, heuristic form rules"
}

Mid-squat at 84° knee angle on rep 8, with a live alignment cue — objective counting and instant feedback, honestly scoped as a fitness aid.

Troubleshooting: Common Errors & Fixes

Performance Optimisation

• Reduce to angles early — reps and form both come from angles.
• Use hysteresis and smoothing for robust counting.
• Match camera view to the errors you want to catch.
• Gate on keypoint confidence to avoid false counts/cues.
• Pin the hot loop to one core with taskset and leave the others free for the OS.
• Prefer MJPEG over raw YUY2 when capturing from USB cameras — the decode cost is far lower than the USB bandwidth cost.
• Log to a tmpfs RAM disk and flush to the SD card once a minute; per-sample SD writes are what kills cards.
• Run the service under systemd with Restart=always so a crash never means a dead deployment.
• Profile before optimising — print micros() deltas around each stage and fix the slowest one first.

Safety Precautions

• This is a fitness aid, not medical or professional coaching — do not rely on it for injury-sensitive rehab without professional oversight.
• Form rules are heuristic and a single 2-D view has real blind spots.
• Cameras raise privacy obligations — notice/consent and data minimisation.
• Encourage users to stop if something hurts, regardless of the app.
• Wear eye protection when soldering or cutting, and solder in a ventilated space — rosin flux fumes are a respiratory irritant.
• Power the circuit through a bench supply with a current limit while you are testing. A 300 mA limit turns a wiring mistake into a beep instead of a dead board.
• Disconnect power before changing any wiring. Hot-plugging a sensor onto a live bus is the fastest way to lose a controller.

Maintenance

• Add/tune exercises and their thresholds/bands over time.
• Re-validate across body types, views and lighting.
• Update the pose model as better ones appear.
• Keep the non-medical scope clearly communicated.
• Re-check every screw terminal and header after the first week — thermal cycling loosens connections that felt tight on day one.
• Recalibrate at the interval given in the calibration section, and keep the constants in a text file next to the firmware — not only in flash.
• Keep a short logbook of firmware versions and what changed. Six months later you will not remember why that constant is 1.083.

Future Improvements & Upgrades

A working v1 is a platform, not a finish line. These are the upgrades that add the most capability for the least rework.

• Add multi-view or depth cameras for 3-D form.
• Add more exercises and personalised baselines.
• Add tempo/eccentric-timing and fatigue cues.
• Add progress tracking and workout summaries.
• Design a proper PCB. Once the breadboard version has run for a month, moving to a two-layer board removes the intermittent-contact failures that dominate prototype faults.
• Add connectivity — an ESP32 and an MQTT publish turn a local gadget into something you can graph, alert on and analyse over months.
• Add persistent local storage (microSD or the on-chip flash) so a network outage does not create a hole in your data.
• Move configuration out of the source: a captive-portal setup page or a JSON config file makes the build reusable without a recompile.
• Add a battery and solar option so the unit survives a power cut and can be sited away from a socket.
• Write a small test harness that feeds synthetic sensor values through the decision logic, so you can validate thresholds without physically triggering the event.

Frequently Asked Questions

References & Learning Resources

These are the primary sources worth reading in full. Manufacturer datasheets always outrank forum posts when the two disagree.

1. Pose estimationReference
2. Human keypoint detectionReference
3. Range of motionReference
4. MediaPipe Pose / MoveNetDocs
5. Signal cycle detectionReference