Sound Detection

Demonstration of BEATRIX Reacting to Sound

BEATRIX responding to voice commands:

BEATRIX responding to music:

Task 1 (difficulty: low)

Objective

Make the robot react to external loud sounds with a relatively high amplitude compared to the background noise. The robot should dance by rotating one of the motors left and right starting from a known home position without exceeding the allowable rotation threshold of the mechanical frame.

Tasks

  1. Adjusting the Robot’s Home Position: The MATLAB code must first prompt the user to adjust the robot’s pose to match the home position, the user then should answer MATLAB’s inquiry by typing true and pressing ENTER.
  2. Audio Analog Read: The MATLAB code must then start a loop in which it received fresh analog inputs from the microphone.
  3. Signal Processing: The MATLAB code will group a number of successive samples to form processing windows. The Mean Absolute Value (MAV) of each window should be computed after removing the background noise offset.
  4. Dancing Action: If the MAV of a window exceeds a specific threshold, the robot should start to dance by rotating one of the motors left and right by a specific numbe of steps starting from the home position using MATLAB’s playtone() function. Students are expected to adjust the frequency and duration of the tone accordingly.

Challenges

  • MATLAB has no straightforward homing mechanism and there are no feedback sensors. You should implement an odometer or a step counter yourself using frequency and time calculations.
  • The MAV window parameters like sample size and background noise level need careful tuning.
  • Stepper motor control is not straightforward in MATLAB and you need some tricks to control the motor behaviour (hint: use MATLAB’s playtone() function)

Task 2 (difficulty: medium)

Objective

Make the robot follow a sound source by rotating in the direction of the audio source starting from a home position.

Tasks

  1. Adjusting the Robot’s Home Position: The MATLAB code must first prompt the user to adjust the robot’s pose to match the home position, the user then should answer MATLAB’s inquiry by typing true and pressing ENTER.
  2. Audio Analog Read: The MATLAB code must receive fresh inputs from both microphones and compute the MAV for sample windows. The code then should compute the differential signal by subtracting the MAV’s of both microphones.
  3. Step Magnitude and Direction: The code should assign a proper number of steps to reach the sound source destination. The direction should be a function of the sign of the differential signal. If the differential signal is almost equal to zero, the robot should stop.

Challenges

  • It is hard to calibrate both microphones simultaneously, the parameters of each one might differ
  • The differential signal needs a specific threshold to allow the robot to move, since both microphones have their own noise and calibration parameters. You need to adjust the threshold accordingly so that the robot won’t execute random motions.

Task 3 (difficulty: high)

Objective

Make the robot execute a set of three distinct commands (right-left-stop) starting from the home position. This task relies on MATLAB’s speech recognition toolbox and requires connecting a headset to the PC.

Tasks

  1. Adjusting the Robot’s Home Position: The MATLAB code must first prompt the user to adjust the robot’s pose to match the home position, the user then should answer MATLAB’s inquiry by typing true and pressing ENTER.
  2. Speech Recognition: The students should use and understand the example code provided by Mathworks. They should display the output prediction of the neural network in the command window to validate the accuracy of the neural network.
  3. Finite State Machine Design: The robot should be programmed according to a finite state machine whose output depends on the current state of the robot (left, right or stop/midway) and the fresh input from the deep learning module. For example, if the robot is already headed left and the user says ‘left’, the robot won’t move. If the robot is at the stop position, and the user says ‘right’, the robot should rotate so that its head points to the right, and so on.

Challenges

  • The speech recognition code needs to be carefully read and tweaked before attempting the finite state machine task. Tweaking might be a bit time consuming.

Citations