Wireless gesture control of Arduino using LeapMotion and ESP8266

After having played around with Wirelessly Controlling Arduino with LeapMotion through Bluetooth 2 years back, I didn’t get an opportunity to tinker more with LeapMotion and Arduino. I installed a couple of Apps from the App Store and played around.

When the call for project submissions for India’s first Mini Maker Faire opened, I decided to re-do this project, albeit use the popular ESP8266 WiFi module instead of a Bluetooth one. Continue reading “Wireless gesture control of Arduino using LeapMotion and ESP8266”

Making LeapMotion and Arduino talk using Bluetooth

After having received the Dev Kit from LeapMotion team a while back, we enjoyed playing with the many demos, examples and libraries posted by fellow Leap Developers. Many asked for linking with H/W including Android, Raspberry Pi and Arduino. A few mustered the courage to get Hardware rolling using the Leap. Some examples included the control of a Labyrinth maze using the Leap (http://www.youtube.com/watch?v=eHpD3Wuj2Co).

In another recent development, Xavier (http://xseignard.github.io/2013/06/25/interfacing-leap-motion-with-arduino-thanks-to-nodejs/) successfully Interfaced the Leap and Arduino using the Node.js library. This was pretty cool, except for the fact that the Arduino had to be connected to the USB of the PC  from where it was receiving data using the Firmata protocol. A rather unfortunate(or fortunate) thing is that I burned up 2 of my USB ports leaving only one port working on my Laptop to which I had a choice of connecting either the Leap or the Arduino. So, in essence, we couldn’t try the above demo.

That set me thinking wouldn’t it be a lot cooler if we could use our Laptops’s inbuilt Bluetooth module to send Leap data wirelessly to a ‘Bluetooth enabled Arduino’ and have some fun? That idea, coupled with some intense coding, a boring Saturday, some electric shocks resulted in this:

For those already familiar with Java, Eclipse, LeapMotion, Bluetooth RXTX using Java, here’s the Source code with the references:

1. http://rxtx.qbang.org/wiki/index.php/Two_way_communcation_with_the_serial_port

2. http://mfizz.com/oss/rxtx-for-java (DLLs for 64 Bit Machine)

3. http://playground.arduino.cc//Interfacing/Java

4. Leap Java Example from SDK

Java Code:


package com.leaparduino.hello;

/******************************************************************************\
* Copyright (C) 2012-2013 Leap Motion, Inc. All rights reserved. *
* Leap Motion proprietary and confidential. Not for distribution. *
* Use subject to the terms of the Leap Motion SDK Agreement available at *
* https://developer.leapmotion.com/sdk_agreement, or another agreement *
* between Leap Motion and you, your company or other organization. *
\******************************************************************************/

import gnu.io.CommPort;
import gnu.io.CommPortIdentifier;
import gnu.io.SerialPort;

import java.io.IOException;
import java.io.OutputStream;

import com.leapmotion.leap.Controller;
import com.leapmotion.leap.Frame;
import com.leapmotion.leap.Listener;
//import java.lang.Math;
class SampleListenerMain extends Listener {

public void onInit(Controller controller) {
System.out.println("Initialized");
}

public void onConnect(Controller controller) {
System.out.println("Connected");
}

public void onDisconnect(Controller controller) {
System.out.println("Disconnected");
}

public void onExit(Controller controller) {
System.out.println("Exited");
}
public void onFrame(Controller controller) {
Frame frame = controller.frame();
System.out.println(frame.fingers().count());
LeapIntoArduino.writeToArduino(frame.fingers().count());
try {
Thread.sleep(500);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

class LeapIntoArduino {
static OutputStream out = null;

public static void main(String[] args) {
//Connect to Arduino BT
try
{
(new LeapIntoArduino()).connect("COM12");
}
catch ( Exception e )
{
e.printStackTrace();
System.exit(0);
}

// Create a sample listener and controller
SampleListenerMain listener = new SampleListenerMain();
Controller controller = new Controller();

// Have the sample listener receive events from the controller
controller.addListener(listener);
// Keep this process running until Enter is pressed
System.out.println("Press Enter to quit...");
try {
System.in.read();
} catch (IOException e) {
e.printStackTrace();
}

// Remove the sample listener when done
controller.removeListener(listener);
}

void connect ( String portName ) throws Exception {

CommPortIdentifier portIdentifier = CommPortIdentifier.getPortIdentifier(portName);
if ( portIdentifier.isCurrentlyOwned() )
{
System.out.println("Error: Port is currently in use");
}
else
{
CommPort commPort = portIdentifier.open(this.getClass().getName(),2000);

if ( commPort instanceof SerialPort )
{
SerialPort serialPort = (SerialPort) commPort;
serialPort.setSerialPortParams(9600,SerialPort.DATABITS_8,SerialPort.STOPBITS_1,SerialPort.PARITY_NONE);
out = serialPort.getOutputStream();
}
else
{
System.out.println("Error: Only serial ports are handled by this example.");
}
}
}

public static void writeToArduino(int noOfFingers)
{
String tmpStr = Integer.toString(noOfFingers);;
byte bytes[] = tmpStr.getBytes();
try {
out.write(bytes);
} catch (IOException e) { }
}
}

Arduino Code:


#include <LiquidCrystal.h>
LiquidCrystal lcd(8,9,10,11,12,13);
char ch;
void setup()
{
Serial.begin(9600);
lcd.begin(16,2);
pinMode(2,OUTPUT);
pinMode(3,OUTPUT);
pinMode(4,OUTPUT);
digitalWrite(2,LOW);
digitalWrite(3,LOW);
digitalWrite(4,LOW);
}
void loop()
{
lcd.setCursor(0,0);
lcd.print("Remote Value:");

if (Serial.available()) {
lcd.setCursor(0,1);
ch = Serial.read();
lcd.print(ch);
if(ch == '1')
digitalWrite(2,HIGH);
if(ch == '2')
digitalWrite(2,LOW);
if(ch == '3')
digitalWrite(3,HIGH);
if(ch == '4')
digitalWrite(3,LOW);
if(ch == '5')
digitalWrite(4,HIGH);
if(ch == '6')
digitalWrite(4,LOW);
delay(50);
}
delay(10);
}

Quick Notes:

  1. The Bluetooth serial port in my Laptop was COM13, hence used that in the code. Please replace this with your COM PORT NO.
  2. The Arduino sketch includes code for LCD also. Since I didn’t have an extra serial port on Laptop to debug, I used this. This can be removed.
  3. The if conditions in Arduino can be replaced with well mannered if-else checks
  4. Please use the right version of rxtxSerial.dll (32 / 64 bit) in your Java JRE’s bin.
  5. Make sure you have added the LeapJava.jar (and also the DLLs) and RXTXcomm.jar to the Eclipse project

I’ll update a detailed step-by-step version soon. Meanwhile, if you’ve any questions, please ask.

Thanks!