Telepresence Robot.

Project Overview

A mobile video-call on wheels — you drive it around a remote place and see, hear and speak through it, putting your presence somewhere your body is not.

A video call lets you talk to a remote place; a telepresence robot lets you be there — a screen, camera, microphone and speaker on a mobile base that you drive from anywhere, so you can move through a remote space, look around, and hold conversations as if you were physically present. It is used for remote work and site visits, for doctors to round on patients, for people to attend school or events they cannot travel to. This project builds one: a wheeled robot that streams two-way audio and video and that you steer remotely over the network, turning a static video call into embodied remote presence.

Two subsystems make it work. The first is real-time two-way audio/video: a camera and microphone on the robot stream to the remote pilot, and the pilot's voice (and optionally face) stream back to a speaker and screen on the robot — low-latency enough for natural conversation and for driving by what you see. The second is remote teleoperation: the pilot's drive commands travel over the network to the robot's differential-drive base, so they can move it around, with the video as their eyes. The engineering challenge that ties them together is latency — laggy video makes driving into a frustrating guessing game — and safe remote driving, since a pilot seeing a delayed, narrow camera view can easily hit things.

Because you drive by a delayed video feed through a limited field of view, the robot must protect against the pilot's blind spots: an obstacle stop that overrides commands before a collision, sensible speed limits, and a safe stop if the connection drops (a robot that keeps driving when it loses its pilot is dangerous). It is honest that good telepresence is fundamentally about connectivity quality and latency, that a small robot is a prototype of the interaction rather than a commercial product, and that driving a robot in a real space where people move demands genuine care. But as a mobile, network-driven, two-way audio/video robot with obstacle protection and safe-stop, it delivers the real thing — your presence, moving, in a place your body is not.

What this project does

• Streams two-way audio and video between robot and remote pilot
• Lets the pilot drive the robot remotely over the network
• Moves through a remote space as the pilot's embodied presence
• Keeps latency low enough for natural conversation and driving
• Overrides driving to stop before obstacles (pilot blind spots)
• Safe-stops if the connection drops
• Puts your presence where your body is not

Real-World Applications

Deployment contexts where a build of this kind earns its keep.

Features & Capabilities

• Real-time two-way A/V streaming
• Network teleoperation of a mobile base
• Low-latency, drive-by-video control
• Obstacle-stop override (blind-spot protection)
• Connection-loss safe stop + speed limits
• Camera pan / look-around (optional)
• Honest about connectivity/latency and safe remote driving

Difficulty, Time & Required Skills

Skills you should have (or will pick up)

• Real-time two-way audio/video streaming (low latency)
• Network teleoperation and command latency handling
• Differential-drive control from remote commands
• Safety overrides: obstacle stop, connection-loss stop
• Drive-by-video UX considerations

Bill of Materials

Every part below is commonly available from Indian and international hobby-electronics suppliers. Prices are indicative 2026 retail figures in Indian rupees and will drift — treat them as a budgeting guide, not a quotation.

Estimated total: ₹12,470, excluding tools, shipping and consumables.

Tools and consumables

• Soldering iron (temperature controlled, 350 °C) with 0.8 mm 60/40 or lead-free solder
• Digital multimeter — continuity, DC volts and current ranges
• Wire strippers, flush cutters and a small set of precision screwdrivers
• Heat-shrink tubing and a heat gun (or a lighter, carefully)
• A laptop with a USB port and the toolchain listed above

Hardware Specifications

Consolidated electrical and interface specifications for every active part in the build.

Power Budget & Supply Sizing

Add up the typical active current of every part, then size the supply with at least 50 % headroom so transmit bursts and motor inrush never brown out the controller.

Summed typical draw is 1701 mA. With a 1.5× design margin the supply should deliver at least 2600 mA continuously at the stated rail voltage.

Software Requirements & Development Environment

Reference toolchain: Raspberry Pi OS Bookworm (64-bit) + Python 3.11 + VS Code Remote-SSH. Anything newer normally works; anything older may lack the board definitions used here.

• Flash Raspberry Pi OS (64-bit) with Raspberry Pi Imager; pre-configure Wi-Fi, hostname and SSH in the Imager settings so the board comes up headless.
• Update first: sudo apt update && sudo apt full-upgrade -y, then reboot.
• Work inside a virtual environment — python3 -m venv ~/venv && source ~/venv/bin/activate. Bookworm blocks system-wide pip install by design.
• Enable the buses you need with sudo raspi-config → Interface Options (I²C, SPI, Serial, Camera).
• Develop over VS Code Remote-SSH so you edit on your laptop but run on the Pi.

Required libraries

Block Diagram

The block diagram shows the functional decomposition of the system — what senses, what decides, what acts, and where the data ends up.

Circuit Diagram & Wiring

Every signal line in the build is shown below, followed by a pin-by-pin connection table you can work through with a multimeter in hand.

Wire one row at a time and tick it off — most "it does not work" reports trace back to a single swapped pair.

Wiring explanation

• Mount the camera and screen high (head height) for natural presence; keep the mic/speaker clear for good audio.
• Drive the base through an H-bridge; the Raspberry Pi handles A/V streaming and command reception.
• Fit an obstacle sensor ring to cover the pilot's blind spots around the base.
• Ensure a good network link (Wi-Fi) — connectivity quality dominates the experience.
• Provide a watchdog so motors stop if commands stop arriving (lost pilot).

System Architecture

Read the stack from the bottom up: physical hardware, the firmware that drives it, the transport that moves data off the device, and the software a human actually looks at.

Working Principle

Telepresence turns communication into presence. A phone or video call sends your voice and image to a fixed point; a telepresence robot gives that call a body you control — it can move through the remote space, turn to look, approach a person, follow along a corridor — so instead of being a face on a screen in one room you are a mobile participant who can go where the conversation goes. That mobility is the whole difference: presence you can direct. Achieving it means marrying two subsystems that each have to work well and, harder, work well together over a network.

The first subsystem is real-time two-way audio and video. The robot captures video and audio and streams them to the remote pilot; the pilot's voice — and usually their face on the robot's screen — streams back. Both directions must be low-latency, because this feed is doing double duty: it carries the conversation (and lag makes people talk over each other) and it is the pilot's eyes for driving. Video compression, network transport and buffering all trade latency against quality, and telepresence lives or dies on getting that trade right — which is why good telepresence is, at heart, a connectivity and latency problem more than a mechanical one.

The second subsystem is remote teleoperation. The pilot's steering commands travel over the network to the robot's differential-drive base, and the pilot drives by what the camera shows them. This closes a control loop that runs through a human and a network: see (delayed video) → decide → command (delayed) → robot moves → see the result (delayed again). Every delay in that loop makes driving harder, turning a laggy connection into a frustrating guessing game where the robot is always a moment behind the pilot's intent. Designing for this means keeping latency low, keeping speeds modest, and giving the pilot a view wide enough to drive by.

The consequence that dominates the design is safe remote driving, because the pilot is fundamentally handicapped: they see a delayed image through a narrow field of view, missing things a physically-present person would notice — the table leg just out of frame, the person stepping in from the side, the delay between deciding to stop and the robot stopping. So the robot cannot simply obey; it must protect against the pilot's blind spots. An obstacle stop overrides drive commands to halt before a collision the pilot did not see; speed is limited so the delayed loop stays controllable; and — critically — a connection-loss safe stop ensures a robot that loses its pilot stops rather than continuing blindly (a driverless robot rolling through a space is a real hazard). The design is honest that this is a prototype of the interaction: a small robot demonstrates the presence and the driving, but a commercial unit invests heavily in connectivity, latency, camera coverage and safety, and driving any robot through a space where people move demands genuine care. Within that honest frame, it delivers the real experience — see, hear, speak and move through a place your body is not — which is what makes telepresence more than a video call.

The maths behind it

Teleoperation loop latency

Total control latency the pilot feels:

  T = T_capture + T_encode + T_uplink
    + T_network + T_downlink + T_decode + T_display
    + T_command_back

Every term adds lag between intent and seeing the result.
Low T → natural driving; high T → a guessing game.

Safe speed vs latency

Distance travelled before the pilot can react to a hazard:

  d_react = v × (T_video + T_human + T_command)

Keep d_react small: limit speed v as latency rises, and let
the robot's own obstacle stop cover what the pilot misses.

Safety overrides

Robot obeys the pilot ONLY if safe:

  if obstacle within d_stop:      override → STOP
  if no command for t_timeout:    lost pilot → SAFE STOP
  v ≤ v_max                        (bounded speed)

A robot that keeps driving without its pilot is dangerous.

Program Flowchart

The firmware is a single cooperative loop. Nothing blocks for long, so networking, sensing and the user interface all stay responsive.

Assembly Instructions

Build on a breadboard first and only commit to solder once the whole system has run for an hour without a fault.

1. Build the presence platform

   Assemble the differential-drive base with a tall mast, mount the camera/mic/speaker (and optional screen) at head height, and fit an obstacle-sensor ring around the base.

2. Set up low-latency two-way A/V

   Stream camera+mic to the pilot and the pilot's voice (and face) back, tuned for low latency so conversation and driving feel natural.

3. Add safe teleoperation

   Receive drive commands over the network and apply them through safety filters: obstacle-stop override, speed limit, and safe-stop on lost connection.

Step-by-Step Implementation Guide

Work through these in order. Each step ends in something you can observe, so a failure is always localised to the step you just finished.

1. Apply remote drive commands safely

   Take the pilot's drive command and execute it only if safe — override to stop for obstacles the pilot cannot see, and safe-stop if the connection is lost.

   pythonteleop.py Copy Download

   import time
   
   V_MAX = 0.35                     # bounded speed (m/s)
   D_STOP = 0.30                    # obstacle stop (m)
   LINK_TIMEOUT = 0.5              # s without a command = lost pilot
   
   class SafeDrive:
       def __init__(self, base, obstacles):
           self.base = base; self.obs = obstacles
           self.last_cmd_t = 0
   
       def on_command(self, v, omega):          # from the remote pilot
           self.last_cmd_t = time.time()
           v = max(-V_MAX, min(V_MAX, v))        # bound speed for laggy loop
           if self.obs.min_distance() < D_STOP:  # pilot's blind spot
               self.base.stop()                  # OVERRIDE: don't hit it
               return
           self.base.drive(v, omega)             # execute the pilot's intent
   
       def watchdog(self):                       # runs continuously
           if time.time() - self.last_cmd_t > LINK_TIMEOUT:
               self.base.stop()                  # lost pilot -> SAFE STOP

   v = max(-V_MAX, min(V_MAX, v)) # bound speed for laggy loopSpeed is capped so the delayed see-decide-command loop stays controllable — the faster it drives, the further it travels before the pilot can react.

   if self.obs.min_distance() < D_STOP: # pilot's blind spotThe robot overrides the pilot to stop for an obstacle they cannot see through a narrow, delayed view — protecting against the pilot's blind spots.

   self.base.drive(v, omega) # execute the pilot's intentOnly when it is safe does the robot carry out the pilot's command — the robot obeys, but never blindly.

   if time.time() - self.last_cmd_t > LINK_TIMEOUT:If commands stop arriving the pilot is lost, so the robot safe-stops rather than continuing to roll driverless through the space.

2. Stream low-latency two-way A/V

   Stream the robot's camera and microphone to the pilot and the pilot's audio (and face) back, minimising latency so conversation is natural and the pilot can drive by the video.

Complete Source Code

The listing below is complete and compiles as written — there are no elided sections. Read the annotations under each block before you upload it.

#!/usr/bin/env python3
"""
Telepresence Robot — mobile two-way A/V presence

Streams low-latency two-way audio/video and lets a remote pilot drive
a differential-drive base by the video. Every drive command is gated by
safety: obstacle-stop override (pilot blind spots), bounded speed, and
a connection-loss SAFE STOP. A prototype of embodied remote presence.
"""
import time, threading

V_MAX, D_STOP, LINK_TIMEOUT = 0.35, 0.30, 0.5

class TelepresenceRobot:
    def __init__(self, base, camera, mic, speaker, obstacles, link):
        self.base, self.cam, self.mic = base, camera, mic
        self.spk, self.obs, self.link = speaker, obstacles, link
        self.last_cmd = 0
        threading.Thread(target=self.av_loop, daemon=True).start()
        threading.Thread(target=self.watchdog, daemon=True).start()

    def av_loop(self):                         # two-way, low latency
        while True:
            self.link.send_video(self.cam.frame())     # robot -> pilot
            self.link.send_audio(self.mic.chunk())     # robot -> pilot
            a = self.link.recv_audio()                 # pilot -> robot
            if a: self.spk.play(a)                      # pilot's voice
            # (optional) show pilot's face on the robot screen

    def on_drive(self, v, omega):              # remote pilot command
        self.last_cmd = time.time()
        v = max(-V_MAX, min(V_MAX, v))          # bound speed
        if self.obs.min_distance() < D_STOP:    # blind-spot obstacle
            self.base.stop(); return            # OVERRIDE
        self.base.drive(v, omega)               # safe -> obey

    def watchdog(self):                        # lost-pilot safe stop
        while True:
            if time.time() - self.last_cmd > LINK_TIMEOUT:
                self.base.stop()                # no pilot -> SAFE STOP
            time.sleep(0.1)

    def run(self):
        for cmd in self.link.commands():        # driven remotely
            self.on_drive(cmd["v"], cmd["omega"])

if __name__ == "__main__":
    robot = TelepresenceRobot(Base(), Camera(), Mic(), Speaker(),
                              ObstacleRing(), NetworkLink())
    robot.run()

Configuration & Calibration

Configuration steps

• Configure the camera/mic/speaker and the two-way streaming (prioritising latency).
• Configure the network link and drive-command reception.
• Configure the obstacle-stop distance, speed limit and connection-loss timeout.
• Configure optional camera pan and pilot-face display.

Calibration procedure

An uncalibrated sensor produces confident, precise, wrong numbers. Do this once per physical unit and record the constants.

1. Latency

   Measure and minimise end-to-end A/V and command latency so driving and conversation feel natural.

2. Safety overrides

   Verify the obstacle stop triggers before contact and the safe-stop fires promptly on connection loss.

3. Drive feel

   Tune speed limits and command smoothing so the delayed loop is controllable and comfortable.

Mechanical Assembly & Motion

Mechanical assembly

• Differential-drive base with a tall mast placing the camera and screen at head height for natural, eye-level presence.
• Camera, microphone and speaker positioned for a clear two-way view and good audio; optional screen shows the pilot's face.
• An obstacle-sensor ring around the base covering the pilot's blind spots.
• Stable, not top-heavy despite the mast — it must not tip when starting/stopping.

Motion logic

Motion is differential drive commanded remotely: the pilot sends forward/turn intents and the base executes them, but every command passes through a safety filter first.

The robot moves deliberately and at limited speed, because the pilot drives through a delayed, narrow view; the robot's own obstacle stop overrides motion the pilot cannot see is unsafe.

Sensor integration

• Camera + microphone: the pilot's eyes and ears (and the video they drive by).
• Obstacle sensor ring (ultrasonic/ToF): covers blind spots for the override stop.
• Connection/heartbeat monitoring: detects a lost pilot for the safe stop.

Actuator explanation

Two geared DC motors via an H-bridge for differential drive, executing filtered remote commands.

Speaker (and optional screen) render the pilot's presence; an optional camera pan lets the pilot look around.

Kinematics

The base uses standard differential-drive kinematics; the distinctive element is that the control input arrives from a remote human over a laggy network and is gated by safety overrides.

v     = (v_R+v_L)/2,   omega = (v_R−v_L)/L     # differential drive

Remote command c(t−T) arrives delayed by latency T.
Apply only if safe:
  if obstacle or link_lost:  v = omega = 0          # override
  else:                      v = min(c.v, v_max)

Testing Procedure & Expected Output

Test from the bottom up. Confirm power, then each sensor in isolation, then the integrated loop — the first failing step tells you exactly where to look.

Bench-test checklist. If a row fails, stop and fix it before moving on.

Expected output

A robot you drive remotely while seeing, hearing and speaking through it — with obstacle and connection-loss protection.

{
  "session": "active",
  "video_latency_ms": 180,
  "pilot_command": { "v": 0.2, "omega": 0.0 },
  "obstacle_m": 0.9,
  "applied": "driving",
  "link": "ok"
}

An active session driving forward gently with 180 ms video latency and a clear path — presence on the move; had an obstacle or a dropped link appeared, the robot would have stopped itself.

Troubleshooting: Common Errors & Fixes

Performance Optimisation

• Prioritise low latency over resolution for driving and conversation.
• Bound speed so the delayed teleoperation loop stays controllable.
• Override the pilot with an obstacle stop for blind spots.
• Safe-stop on connection loss — never drive without a pilot.
• Pin the hot loop to one core with taskset and leave the others free for the OS.
• Prefer MJPEG over raw YUY2 when capturing from USB cameras — the decode cost is far lower than the USB bandwidth cost.
• Log to a tmpfs RAM disk and flush to the SD card once a minute; per-sample SD writes are what kills cards.
• Run the service under systemd with Restart=always so a crash never means a dead deployment.
• Profile before optimising — print micros() deltas around each stage and fix the slowest one first.

Safety Precautions

• A robot that keeps driving when it loses its pilot is dangerous — always safe-stop on connection loss.
• Override drive commands with an obstacle stop; the pilot cannot see blind spots through a narrow, delayed view.
• Limit speed for the delayed loop, especially where people move.
• Respect privacy — a mobile camera/mic in a shared space needs consent and clear indication it is active.
• Lithium cells vent and burn when abused. Only use protected cells or a proper BMS, never charge below 0 °C, and never leave a charging pack unattended on a wooden desk.
• Moving parts pinch. Keep fingers, cables and hair out of gear trains and wheels, and always test motion with the drivetrain unloaded and the robot on blocks first.
• Motors are inductive — always fit a flyback diode across a DC coil, or use a driver that already has one, or the back-EMF spike will destroy your GPIO.
• Wear eye protection when soldering or cutting, and solder in a ventilated space — rosin flux fumes are a respiratory irritant.
• Power the circuit through a bench supply with a current limit while you are testing. A 300 mA limit turns a wiring mistake into a beep instead of a dead board.
• Disconnect power before changing any wiring. Hot-plugging a sensor onto a live bus is the fastest way to lose a controller.

Maintenance

• Monitor and keep latency low; check network quality where it operates.
• Test obstacle-stop and safe-stop overrides regularly.
• Keep camera/mic/speaker clean and correctly positioned.
• Check base stability, drive wear and battery health.
• Re-check every screw terminal and header after the first week — thermal cycling loosens connections that felt tight on day one.
• Log pack voltage. When resting voltage after a full charge drops below about 4.0 V, the cell is near end of life — replace it.
• Listen for gear chatter. A servo that buzzes at rest is either stripped or fighting a mechanical bind — fix it before it burns out.
• Recalibrate at the interval given in the calibration section, and keep the constants in a text file next to the firmware — not only in flash.
• Keep a short logbook of firmware versions and what changed. Six months later you will not remember why that constant is 1.083.

Future Improvements & Upgrades

A working v1 is a platform, not a finish line. These are the upgrades that add the most capability for the least rework.

• Add WebRTC for robust low-latency two-way A/V.
• Add assisted/semi-autonomous navigation (waypoints, follow-me).
• Add a wider/depth camera and better blind-spot coverage.
• Add auto-docking/charging and multi-robot session management.
• Design a proper PCB. Once the breadboard version has run for a month, moving to a two-layer board removes the intermittent-contact failures that dominate prototype faults.
• Add connectivity — an ESP32 and an MQTT publish turn a local gadget into something you can graph, alert on and analyse over months.
• Add persistent local storage (microSD or the on-chip flash) so a network outage does not create a hole in your data.
• Move configuration out of the source: a captive-portal setup page or a JSON config file makes the build reusable without a recompile.
• Add a battery and solar option so the unit survives a power cut and can be sited away from a socket.
• Write a small test harness that feeds synthetic sensor values through the decision logic, so you can validate thresholds without physically triggering the event.

Frequently Asked Questions

References & Learning Resources

These are the primary sources worth reading in full. Manufacturer datasheets always outrank forum posts when the two disagree.

1. TelepresenceReference
2. Telerobotics / teleoperationReference
3. WebRTC (real-time A/V)Reference
4. Latency (teleoperation)Reference
5. Raspberry Pi cameraRaspberry Pi