"Welcome to the Virtual World...may I take your username?"

Quick note: I wrote this a long time back, but never published - thought it wasn't good enough. This just occurred to me --> it's just a post - doesnt need to be perfect right?!? So posting this now - Please be sure to leave a comment with what you think!

Over the last few years, there have been some amazing advancements in gadgets - specifically the new imaging devices have emerged and a new horizon for virtual reality has been set. These devices have opened up a door for limitless future development possibilities. "Occulus Rift", the virtual reality head-mounted device that was received with overwhelming Kickstarter response, has been the pioneer in developing the key tech that can realise the virtual reality. The reason behind this huge success is an amazing idea which transfers all your senses into a digitally created world. Although this concept has been recently recognised, the idea has been around for decades.

Saying hello to the Internet of Things!

A while back I signed up for Microsoft Developer Program for Internet of Things ( #iot for more info ). As much as I love exploring new things this was extremely exciting thing for me.

I have always had the curiosity to know more and try to hack things my own way. Even as a kid I had an investigative mind which always tried to discover more about how everything works. You can imagine this curiosity by the fact that I got severe electric shock as a kid, when I tried to cut a live wire from "Clothes Iron". This curiosity grew more and more in me, to a point that I did an engineering degree (Yes! I was born with an engineer's mind). I have always been interested in hacking different devices to make something more useful out of it.

One Image hiding over eight thousand different stories...

Working with large datasets has its own pros and cons. Whatever the implementation or field might be, there is always a need for training a machine learning algorithm to recognize the pattern in that data. We often discuss this "Pattern" in many different instants and a big chunk of literature addresses this recognition problem. However it is often not considered important to get to know how this pattern looks like? why is it even called "Pattern" in the first place??

Interestingly the answer lies in the above image which shows a collection of 8000 different samples, arranged in columns. Here the first thing to notice is that there actually is a repeating pattern in the data. This is the exact pattern which we are trying to learn. It may not make sense when looking at it, however with correct label representation, each sample can be used to build a model which is able to identify each class with high accuracy.

Future of Music == Gesture + AI + Singing

While doing some research, I found this talk in which a musician talks about how she was able to use different gestures to compose a song. The actual talk can be seen below:

What is the first thing that comes to your mind after watching this? yes, it is amazing to see such a performance for a music fanatic. But for me it is even more interesting to see the different aspects of data fusion involved. By looking at her performance, there are a number of things that comes to my mind:

1. Hand gestures recognition using data gloves.
2. Body posture recognition using Kinect Sensor.
3. Localization of the person on stage using Kinect Sensor.

You might have noticed, that there are different hand gestures which are used to start the editing or instrument playing sequence. While the hand gestures are used to play specific notes as well, body posture specifies the different after effects/post processing. Similarly the location of the singer is used to relate it to different music effects.

This really shows the potential of natural interaction technology, and what might be achieved if new ideas are integrated into these natural interaction methods.

Reference:
http://www.kinecthacks.com/imogen-heap-talks-ableton-controlling-gloves/

How to train your dragon?

toothless

No! this post is not about the dragon from the animated movie (although it's one of my favourite). However this picture explains almost everything there is about the topic. For those of you who have seen this movie, computer vision machines can be thought to be like the dragon which can not fly. You have to train it about every single incident and how it should react to each one of them in order to fly.

Yup! that's right, this post is about training. Training a computer!

Just a tip of the Iceberg..

I have been working for a while on my dissertation. The main aim of this project is to build a gesture recognition system which is both reliable and robust; to the extent that it can replace the already existing human computer interaction (HCI) techniques. Despite the idea being clear, I have been asked a lot of questions about the project.

Right now I am still in the final stages to finish the approach I have planned to use. However to answer some of these questions, I have uploaded a video which shows an implementation of similar gesture recognition technique I did in the start of this project.

Here is a short demo video:

This was not very reliable or robust technique. However I learned a lot of vital information while implementing this. The idea behind this is still at the base of a very complex dynamic hand gesture recognition system I am currently working on.

Will be uploading more information soon..