Final Documentation

Project Overview - MIDI Gloves
What I have created are a set of gloves that allow a user to compose, produce, and record music with future age gloves. With the gloves you can map any type of instrument sound to your fingers and record and loop grooves and melodies in real time. You can be the drummer, the pianist, the guitarist and the bassist with these gloves. You can record backgrounds and melodies and piece together sections of music to create a full song just through the use of the gloves. You even have the capability of adding and altering audio effects like frequency, volume, pitch, panning, by just bending your fingers. They are truly the musicians best friend.

How do they work?
They are composed of 15 sensors, 8 buttons, 2 circuit boards, and an LCD screen. On each fingertip there is a force sensitive resistor (FSR) also known as a pressure sensor. These pressure sensors register touches and taps and in response triggers notes. Each pressure sensor also sends a certain pitch as well as a volume at which the note should be played. This volume is based off the speed at which the finger is tapped and results in a much more organic and natural sounding music.

On the back of the index and middle fingers of each hand there are two flex sensors that, when bent, send certain values from 0-127 to control variable effects like volume, frequency, and panning. Each sensor is open to be used for any purpose and can be mapped to any trigger in Ableton, even triggering sound clips and audio samples.

On the side of each index finger there are four buttons that each correspond to different controls within the program. There is navigation control within the program by sending up and down signals allowing the user to toggle between different sounds. There is also an undo button, a stop/start button, and up octave and down octave buttons allowing the user to change the finger note values while playing.

Attached to the board is an LCD screen that acts as an interface between the gloves, the user, and the program, insuring the user is aware of what is being sent through the gloves to the program. The LCD allows you to program the gloves to any scale in any key. (Major, Minor, Blues, Pentatonic). In the future the user will also be able to program any finger to any note.

To power it all I have 4 coin cell batteries and a 3.3V power source from the computer. I used two coin cell batteries to power the LCD screen and a single coin cell for each hand to send enough power to each sensor. For each hand battery, I designed and 3D printed a battery holder that sandwiches the battery with a positive and negative lead. You can see a picture of these holders below.

Finally there is really one circuit board that controls it all and processes the information, the Teensy 3.2. Before this project I knew nothing about the Teensy but after working on this, the Teensy has proved to be an amazing piece of hardware capable of amazing things. Through the Teensy I am able to send Serial, Keyboard, and MIDI signals all at the same time. It has proved to be more than capable of handling dozens of analog inputs and digital outputs and processing all extremely fast.
Attached to the Teensy I have a MUX Breakout Board that expands the Teensys analog and digital pins so that I can receive the triggers from the left hand.
All this works together to create a fully function electronic musical instrument that is versatile, portable, and easy to use for musicians and non-musicians alike.

Physical Appeal:
One of the hardest and most daunting challenges of this project has been making the gloves look nice while keeping functionality. I tried many different solutions to this such as sewing fabric over the wires, covering the wires with a fabric sheet, covering the fingers with cut off glove fingers, and even fitting a second glove over the main glove. In the end all of these methods seemed to make the glove much more restrictive and much less comfortable to wear. So I concluded to not cover the wires and deal with the physical appeal after testing and further iterations. However, to make the gloves more eye-catching I decided to add a laser engraved triangle of acrylic with an LED to light up the design. BMP of design below.

Components Used:
1x Teensy 3.2
1x Analog MUX Breakout Board
11x Force Sensitive Resistors
4x Flex Resistors
8x Switches/Buttons
1x LCD Screen 16x2
4x Coin Cell Batteries
2x 3D Printed Single Coin Cell Holder
1x Double Coin Cell Holder
1x 3D Printed LCD Case
1x Pair of Gloves
1x Wrist Brace
1x Laser Engraved Acrylic
1x 3D Printed Acrylic holder
1x Blue LED
Arduino IDE
USB_Keyboard Library
and much more....

Circuit Diagram

 
Usage Graphic

Code:
https://drive.google.com/open?id=0B8O3izVcHJBzdHh2Zm9VaTViX0U

Below are images of the final, fully functional prototype

More process photos below...

Further Functionality

Now that the glove's basic functionality is working well I wanted to add some more buttons and switches to add more minor functionality during the music making process.

I have added 4 buttons on each index finger to allow for navigation in Ableton, undo, stop, start, and octave change

Left Hand Glove

One of the largest challenges so far has been duplicating the wiring and functionality of the glove over to the left hand to give me a whole 10 note range.

There are three methods that I found to accomplish this task.

1. Duplicate same wiring on second glove and attach second USB to computer, essentially making the left hand a second, separate, midi controller.

2. Duplicate the wiring on the second glove and use some sort of communication (Serial, I2C, etc) to make the gloves communicate with each other.

3. Duplicate the wiring on the second glove and use a MUX Breakout Board to expand the analog pins to the other hand.

I decided not to pursue option 1 on the basis that I wanted it to function as a single midi controller and I wanted to be able to control both gloves with the use of just 1 board.

I originally tried to pursue option 2 because I had a second Teensy board and thought it might be the best way of making the boards communicate. 

After hours of fidling and trying both I2C and Serial communication I decided this method wouldn't work for what I was trying to do. I needed extreme precision in value recall from the slave glove so that I could assign different values to different controls. This method didn't work because I found there to be significant error in the serial communication between the boards. Also, using a second Teensy board significantly raises the production cost of the glove sets.

I finally decided to go with Option 3 and use a MUX Breakout Board. After getting my board and spending a few hours to test it, it turned out to be extremely accurate and the perfect tool for my purposes. 

 

https://www.sparkfun.com/products/9056

I assigned each pin to different FSRs and Flex resistors and used the sent values to trigger midi through the single Teensy board on the right hand.

This method seems to work extremely well and has been quite successful.

Below is an image of the left hand with the MUX Breakout and full wiring.

Updated Glove Pictures

Pictures of Updated Glove

Wiring has been transferred over to a new glove with a more modern design and has been redone to create a more robust, cleaner, and reliable circuit.

LCD has been attached to a wrist brace and I have modeled and 3D printed an LCD case to improve
the look and overall feel of the product.

Curcuit Diagram

Wiring Diagram for the glove

Project Timeline

Timeline
04/05/16	Attach sensors and RGB LED on second glove
04/06/16	Print Battery Holder and wire to battery and glue sensors on glove
04/07/16	Write code to get serial communication fully working
04/08/16	Test to make sure everything is working
04/09/16	Clean up wires
04/10/16	Make cover over top to hide wiring on glove and wrist
04/11/16	Create Logo Design
04/12/16	Laser Cut and attach 2 acrylic pieces with Logo
04/13/16	Take good pictures and videos
04/14/16	Work on song/film performance
04/15/16	Work on song/film performance
04/16/16	Work on song/film performance
04/17/16	Work on song/film performance
04/18/16	Work on song/film performance
04/19/16	Work on song/film performance
04/20/16	Work on song/film performance
04/21/16	create poster
04/22/16	Touchups
04/23/16	Touchups
04/24/16	Create Media
04/25/16	Create Media
04/26/16	Create Media
04/27/16	EXPO

Bill of Materials

MIDI Glove Bill of Materials
Part	Number	Price Per Part	Total Price	Link
Teensy 3.2	2	$20.00	$40.00	https://www.sparkfun.com/products/13736
Small FSR	10	$6.00	$60.00	https://www.sparkfun.com/products/9673
Large FSR	1	$7.00	$7.00	https://www.sparkfun.com/products/9375
Short Flex Resistor	4	$8.00	$32.00	https://www.sparkfun.com/products/10264
Buttons	8	$0.50	$4.00	https://www.sparkfun.com/products/9190
LCD Screen	1	$16.00	$16.00	https://www.sparkfun.com/products/709
Gloves Pair	1	$5.00	$5.00
10k Resistor	10	$0.05	$0.50	https://www.sparkfun.com/products/10969
27k Resistor	4	$0.05	$0.20	https://www.sparkfun.com/products/10969
330 Resistor	6	$0.05	$0.30	https://www.sparkfun.com/products/10969
Wrist Strap	1	$3.00	$3.00	McGuckins
Door Stoppers Pack	1	$3.00	$3.00	McGuckins
RGB LED	2	$2.00	$4.00	https://www.sparkfun.com/products/10820
Coin Cells	4	$1.00	$4.00
Wire,heat shrink, etc.	1	$10.00	$10.00
Total	$189.00

Andy Chuong,  Brady Risendal, and Kristof Klipfel

ATLS 3519 – Object

Project 2: Artificial Human Companion

Rockstar

Description and Summary of Project:

For our project, we decided to create a “stuffed rock” that plays rock music. The goal of this is to have a teddy bear-like companion that plays music. This would be an object you could lay in bed with, place on your desk, throw across the room, or punch. The initial plan was to be able to throw the figure and have it played music based off of the acceleration of the figure but had trouble with . We also wanted to have the figure have a squeezable feature similar to many stuffed animals.

The process of creating this object started with the enclosure. We cut up a pair of jeans we found in the BTU Lab. To create the sphere like shape we used the template below:

After cutting out the piece into the shape of our template, we used hot glue to “stitch” the figure together. We left one seam open to allow us to remove/add stuffing to the inside. We also left a hole on the left and right sides to add speakers. To give our sphere/rock a companion feel, we added googley eyes to it.

The next step was to figure out the electronics for this. We wanted to be able to enclose the figure and not have external wires. To do this, we tried a couple different audio shields and break out boards.

The first one was the SparkFun Audio-Sound Breakout Board. (https://www.sparkfun.com/products/11125)

It turned out to be more of a hassle than we had planned. The first problem we ran into with this board was that it needed a 1GB micro SD card which is surprisingly hard to find nowadays. After finding one online and waiting for it to arrive, we tried triggering audio again. Even with the correct micro SD card, we still weren’t able to get an audio output. We checked everything with a multimeter and our circuit seemed to be good. The only strange occurrence was a constant change in input voltage to the board. After a while we decided to change boards and go for the Sparkfun MP3 Player Shield.

The second one (https://www.sparkfun.com/products/12660)proved to be much better. We were able to stack the shield right on top of the Redboard and successfully feed power to it. We then followed the Sparkfun MP3 Shield Tutorial. ( https://learn.sparkfun.com/tutorials/mp3-player-shield-hookup )

After getting the MP3 Shield to play the audio file we needed to increase the volume of the speaker. We originally couldn’t get the speaker to play loud enough. We decided to add the Sparkfun Mono Audio Amp Breakout and used it to boost the volume of the speaker. (https://www.sparkfun.com/products/11044)

Personal Impression

Working on this project in a group was a very good experience. I think we all did a pretty equal amount of work and when we ran into errors it was nice having other people to help come up with trouble shooting ideas. I feel like we all learned quite a lot and could reproduce this project individually if we were to do it ourselves. I think doing this project as part of this group was very helpful to me and was also very fun.

Parts:

Project Specific:

Sparkfun RedBoard: https://www.sparkfun.com/products/12757

Mp3 Shield: https://www.sparkfun.com/products/12660

Speaker: https://www.sparkfun.com/products/9151

Amplifier: https://www.sparkfun.com/products/11044

SD Card: https://www.sparkfun.com/products/13833

General:

    Button

    Wire

    Male and female crimps

    Cloth material (jeans used in this project)

    Hot glue

    Googley Eyes (Optional)

    9v battery with barrel adapter for the Redboard (or another similar power supply)

Wiring Diagram

Made using Fritzing

The Fritzing file can be downloaded from: https://creative.colorado.edu/~anch3636/object/proj2.fzz

Code:

In order to actually get the code working we had to install a separate library called the SFEMP3Shield Library to the Arduino IDE. The library repository can be found here:

(https://github.com/madsci1016/Sparkfun-MP3-Player-Shield-Arduino-Library )

We first uploaded one of the simple sketches found in the SFEMP3Shiled library examples called FilePlayer. Once we had the code working through the Serial Monitor we extracted some code from further down and added an if statement to increase the volume to max, trigger, and play the MP3 file with a button click.

if (xValue == 1023){

    //Serial.write("+++++++++++++++++++++++");

    //Serial.write(1);

    MP3player.setVolume(0, 0);

    MP3player.playTrack(1);

 }

To prevent clutter, the code can be downloaded/viewed from: http://creative.colorado.edu/~anch3636/object/project2_file_player.ino

Images of Final Product

Videos of Final Project

https://drive.google.com/a/colorado.edu/file/d/0B_3j9nUbOORyaEw5UXNGT2xJTWs/view?usp=sharing

https://drive.google.com/a/colorado.edu/file/d/0B_3j9nUbOORyR3FyRnEtRzNvbWc/view?usp=sharing

Interaction Diagram

Midi Glove Updates and Pictures

Here are a few pictures of the updated glove so far:

All the pressure sensors and flex sensors are working and there are also 2 buttons by the thumb allowing navigation within the program.

I struggled for many hours to configure the Arduino IDE to be able to send and receive MIDI, Serial, and Keyboard commands.

I had to configure the usb_conf.h file to allow keyboard commands to be sent.

I have also attached an LCD screen to program the 5 fingers to different scales.

I will upload a video of me using it soon.

Object Project 2 Brief and Pictures

For our project 2 we have decided to build a pet rock that is interpreted in a different way than the well known "Pet Rock". We are going to build a stuffed "rock" that has a microcontroller inside and that plays rock music when triggered.

We hope to have it look like a rock while at the same time giving it some personality and living characteristics like eyes and possibly a mouth.

Below are some pictures of some steps along the building process.

 We have been using the Sparkfun audio breakout board that you can see in the middle of the screen. The purpose of the board is to play audio files with the click of a button.

Unfortunately for us it didn't work out quite that well. We tried every thing we could to troubleshoot the problem but we couldn't figure out how to get it working. We followed tutorials exactly and did every kind of testing we could from amperage, voltage, conductivity, etc and had no luck. The only odd thing we came across was that we were only getting 1.1 volts to the board and that value kept on gradually increasing and decreasing.

So next thing we did was purchase the Sparkfun MP3 shield to fit on the redboard. After some fiddling with drivers and libraries, we got the board to flash and also got it to play music. Our problem right now is getting enough power to the board to make the speaker loud enough.

Next step is to hook up a button and a 9v battery to the board and have the rock music played on button press. Once we have that, we will just put it inside our stuffed pet rock and we will be ready to rock out.

Documentation

MIDI GLOVE

Interaction with music is something that has driven me to do a variety of projects over the past few years with this being the most recent and involved project to date.

The idea behind my MIDI Glove is to be able to compose, control, and manipulate music just using the dexterous properties of your hands.

The glove is meant to act as a midi controller for any DAW (Digital Audio Workstation) to record melodies, loops, etc. to create music in a non-conventional, more interactive way.

The Glove is composed of many different sensors, each with a different purpose and completely customizable. Each sensor can trigger different parameters within the program to allow the user to control the music in whatever fashion fits their style best.

The glove itself is composed of 9 sensors and 2 buttons. There is a pressure sensor on each of the five fingers and there is a flex sensor on the back of each finger excluding the thumb.

The 5 pressure sensors are FSRs (Force Sensitive Resistors) that register different resistances based on the pressure on the sensor. This allows for velocity sensitive control meaning the harder the sensor is pressed, the louder the volume value will be sent in the MIDI signal.

On the back of 4 fingers there are bend resistors that change resistance the further they are bent. These are used to change variable values within the program such as frequency control, volume control, effect presence within a sound, etc.

The glove allows for many different combinations of controls and movement to expressively interact with the music.

Below you can see pictures of the current build stage of the glove.

Ableton (DAW)

Sewing

Next step was to figure out how to secure down the sensors so that I could test all the components together. I thought I might try to sew each individual one down.
I quickly learned two things:
First, that I am terrible at sewing
Second,  that sewing each individual component was not the most efficient way to go about it.

First Test

Pics After Wiring

Just a few pics to show after wiring it all up and strapping it to my hand.

Wiring

Wiring has been a little bit of an issue and will be an ongoing process.
I originally started by hooking up all the sensors in parallel and connected them directly to the Teensy 3.0.

I kept on running into issues of the board shutting off and/or not turning on again while there were multiple sensors connected.

I discovered that the sensors were exceeding the boards amp rating and needed an external power source to function properly.

So, after learning that I decided to hook up a coin cell battery to power the resistors. So far it has worked out pretty well. I havent really run across any problems but only time will tell.

Code Challenges

Next challenge was learning how to optimize code for each individual sensor.

One challenge that I had faced during testing the flex sensors was that there was a lot of inconsistent data being sent.
The data was very sporadic and jumped up and down very quickly causing very rough control within the audio program.

I tried doing some math to get things working but it didn't work out too well.
I kept doing research and eventually came across this arduino page about data Smoothing.

https://www.arduino.cc/en/Tutorial/Smoothing

This is exactly what I needed. Now the data is much smoother and the sensor is much easier to control.

The other problem, as far as code goes, was getting the pressure sensors to register the peak value so that I could use the FSRs as velocity sensitive devices.
I have yet to really improve this but so far I just have a delay after the sensor registers a trigger value simulating when the peak of the tap should be.


 

Heres the code for smoothing:

/*

  Smoothing

  Reads repeatedly from an analog input, calculating a running average
  and printing it to the computer.  Keeps ten readings in an array and
  continually averages them.

  The circuit:
    * Analog sensor (potentiometer will do) attached to analog input 0

  Created 22 April 2007
  By David A. Mellis  <dam@mellis.org>
  modified 9 Apr 2012
  by Tom Igoe
  http://www.arduino.cc/en/Tutorial/Smoothing

  This example code is in the public domain.


*/


// Define the number of samples to keep track of.  The higher the number,
// the more the readings will be smoothed, but the slower the output will
// respond to the input.  Using a constant rather than a normal variable lets
// use this value to determine the size of the readings array.
const int numReadings = 10;

int readings[numReadings];      // the readings from the analog input
int readIndex = 0;              // the index of the current reading
int total = 0;                  // the running total
int average = 0;                // the average

int inputPin = A0;

void setup() {
  // initialize serial communication with computer:
  Serial.begin(9600);
  // initialize all the readings to 0:
  for (int thisReading = 0; thisReading < numReadings; thisReading++) {
    readings[thisReading] = 0;
  }
}

void loop() {
  // subtract the last reading:
  total = total - readings[readIndex];
  // read from the sensor:
  readings[readIndex] = analogRead(inputPin);
  // add the reading to the total:
  total = total + readings[readIndex];
  // advance to the next position in the array:
  readIndex = readIndex + 1;

  // if we're at the end of the array...
  if (readIndex >= numReadings) {
    // ...wrap around to the beginning:
    readIndex = 0;
  }

  // calculate the average:
  average = total / numReadings;
  // send it to the computer as ASCII digits
  Serial.println(average);
  delay(1);        // delay in between reads for stability
}

 

Tutorials

First step for me was learning how to use each different type of sensor using various tutorials.
Many of the tutorials were from Sparkfun, Buildr, or Instructables.

Here are a few examples of some of the sensors and tutorials I followed to get started:

FSR Tutorial - http://bildr.org/2012/11/force-sensitive-resistor-arduino/

Flex Resistor Tutorial - http://bildr.org/2012/11/flex-sensor-arduino/

Midi Instructable - http://www.instructables.com/id/Easy-3-Pot-Potentiometer-Arduino-Uno-Effects-Midi-/

Board Choice

I originally started with the Sparkfun RedBoard as my primary control surface but quickly realized that I couldnt control all my sensors due to the lack of analog pins.
I searched around to see if I could convert digital pins to analog and didn't find anything that would really work too well.

I inquired about different boards and someone mentioned the teensy board.

After doing some research I decided that the teensy would be my best option.
It comes with 20 analog pins and enough digital pins for my project.

As a bonus, it can send midi messages and doesn't require external software to convert serial to MIDI.

Final conclusion:
Teensy 3.0 is definitely the best way to go.

MIDI Glove Vision Statement

I will be creating a device I am referring to as a MIDI Glove. Its purpose will be to send MIDI signals to various programs to trigger certain controls.
The main purpose and intention for this is to control and create music solely with the use of hand gestures and finger taps. It will be able to trigger audio clips, musical notes, songs, audio effects, software controls, etc.
It will be able to be used for song composition, live dj control, electronic music production, live looping, light control, and more.

This glove will be composed of a variety of sensors on different parts of the hands to pick up different changing outside factors.
I plan to have pressure sensors on each finger, flex sensors on the top of each finger, tilt sensors, accelerometers, and buttons.
Each will be able to be programmed to different controls within the audio program to allow for complete customization.

I also hope to include LEDs and visual aspects on the top of the globe to make the performance and production process more engaging and entertaining for both the user and the audience.