Smartphone based Guidance System for Visually Impaired Person

This post is about an android application based on my publication in the following conference:

International Conference on Image Processing Theory, Tools and Applications, Istanbul, Turkey

The publication is online now and can be found on the following link:
Smartphone based Guidance System for Visually Impaired Person
( Notice: You may require IEEE memebership in order to access this paper)

For demonstration the application can be downloaded from the following link:
https://www.dropbox.com/s/ar6g91u0hbvc6xb/GuidanceSystemBlind.apk

This application uses OpenCV 2.4.2 libraries for android. Therefore it requires installation of the following two packages from Google Play Store.

1. OpenCV Manager
2. OpenCV 2.4 pack armeabi-v7a neon

After installation of the above packages you can install and run the demo software.

Recognizing Dynamic Hand Gestures

I have finished my final dissertation project, and now I am working on the thesis. The final outcome of this project is:

Currently I am working towards my thesis, after which I plan to write a journal paper, which I will share here  as well.

How to train your dragon?

toothless

No! this post is not about the dragon from the animated movie (although it's one of my favourite). However this picture explains almost everything there is about the topic. For those of you who have seen this movie, computer vision machines can be thought to be like the dragon which can not fly. You have to train it about every single incident and how it should react to each one of them in order to fly.

Yup! that's right, this post is about training. Training a computer!

Just a tip of the Iceberg..

I have been working for a while on my dissertation. The main aim of this project is to build a gesture recognition system which is both reliable and robust; to the extent that it can replace the already existing human computer interaction (HCI) techniques. Despite the idea being clear, I have been asked a lot of questions about the project.

Right now I am still in the final stages to finish the approach I have planned to use. However to answer some of these questions, I have uploaded a video which shows an implementation of similar gesture recognition technique I did in the start of this project.

Here is a short demo video:

This was not very reliable or robust technique. However I learned a lot of vital information while implementing this. The idea behind this is still at the base of a very complex dynamic hand gesture recognition system I am currently working on.

Will be uploading more information soon..

AndroidWifi is live!

Following my post few months back about how to set up a simple script to share your connection with Android devices in particular, I found that there was no application which provided full capabilities as my batch scripts did. I started work on a very simple application to convert these scripts into an easy to use application. Today I am releasing first version of this application.

Why AndroidWifi?
While reading this post, some of you might be wondering, why use AndroidWifi? why go through all these steps?

The answer is simple. This is the only free software available to date which enables you to share all types of internet connections over wifi.

These include:

• 3G dongle internet
• Wired internet
• Wifi internet from another router
• internet accessed through USB adapters

Update 13/08/2013: I am seeing people copying this blogpost as it is on their blogs. For this reason I have put a password on the file.

The software can be downloaded at: AndroidWifi_0.9  AndroidWifi_0.91 AndroidWifi_0.95
Download password: seevisionc.blogspot.co.uk

Congestion? Not a problem!

This post is about another course project which I did for my Master's degree in Computer Vision Engineering. It's as simple and innovative as it could get. The idea is to estimate the flow of traffic in real-time using the CCTV video. Four categories of congestion/flow are first categorized, which are Low, Medium, Medium-High and High Congestion.

Here is a short demo video for this project:

While implementing this, the most difficult part was to distinguish between Medium-High and High Congestion, as both types of congestion have data which overlaps and is difficult to model with a machine learning algorithm such as K-means Clustering.

Compiling OpenCV with OpenNI for Microsoft Kinect Sensor

I have implemented code for using OpenCV 2.4.x with latest OpenNI 2.x SDK. Detailed tutorial and code on this newer method can be found here

After installing OpenNI, as I am more familiar with OpenCV, therefore I wanted to work with the data from Kinect sensor using OpenCV. I searched online and a lot of people had been able to compile and run OpenCV with OpenNI. However I could not find any guide on how to do this. I managed to compile OpenCV successfully with OpenNI after some research, and now I am writing this tutorial for those of you who are in the same situation as me.

If you already know how to compile OpenCV with Visual C++ then this tutorial will be straight forward for you. However for completeness sake I will write everything in detail, so even if you are new to OpenCV you wont have any problems compiling from the source code.

If you want to use OpenCV with Microsoft Kinect using OpenNI SDK then, obviously, the first pre requisite is that you should have already installed and set up Kinect sensor using OpenNI and NITE libaries. If you have not already done that then please head over to OpenNI installation Tutorial and first install everything needed to set up. If you have successfully set up everything correctly then precompiled samples inside OpenNI and NITE folders should be working with your Kinect Sensor., which means you are good to go with this tutorial.

Installation of OpenNI with NITE middleware for Microsoft Kinect

Update (31/03/2013): Since writing this blog post, there have been a number of updates to the OpenNI SDK. Now the framework is totally changed, and the links to the OpenNI and NITE binaries used in this tutorial have been moved to another location. You will need to download the specific version of binaries from the new links, rest of the installation process is still the same.

Okay, so this post is again a guide for installation of another library. This time it is OpenNI library for Microsoft Kinect. I am currently working on a computer vision project which utilizes the power of Microsoft Kinect's depth sensor. While researching for my project, I found out that OpenNI library is better than Kinect SDK in many ways. It is opensource, cross-platform, has skeletal as well as hand tracking (which I needed for my project), gesture recognition and ability to use OpenCV were many reasons for choosing OpenNI over Kinect SDK.

OpenNI contains the SDK to access RGB and Depth data from depth sensors containing primesense hardware which includes Microsoft Kinect. However to use skeletal tracking, hand tracking, gesture recognition or any other natural interaction functions in an application a natural interaction library called NITE is required. To use these libraries with Microsoft Kinect, drivers for using Kinect sensor with OpenNI and NITE are required. To install these libraries and drivers is the easiest part, however to make the sensor working with these libraries is really difficult. I followed a number of guides online, however I could not find any guide which could completly resolve all the problems which come after installation and make the sample programs work. I am writing this guide to make sure that most people get the sensor working with these open source libraries.

Select And Save - An application

This post is just to give an intro to a small and simple application I wrote which I named after what it does "Select and Save". The idea is very simple, you want to save an image or part of an image from your webcam, and at the same time sort them into a series of files with distinct but well defined names.

When you start the application, it shows a window with the webcam output and a console window for instructions. There are basically two functions which can be performed, while the capture window is in focus. These functions are explained below:

Capturing the whole frame

This is a very simple function, it captures the whole frame and saves it, as a 640x480 jpg, into a file named in a format which I will discuss towards the end of this post.

Capturing a region within the frame

This function might replace most commonly used 'cropping-the-image' by some photo tool methods. The main aim of this function is to capture only the information selected by the user. The implementation is simple and natural. You just have to point, click down, drag to required region and viola!!

Now some word about the file name format. The files are stored in the folder where the application is running, with the following format:

Image-YEAR-MM-DD_HR_MIN_SEC.jpg

This name takes the current local date and time, which is really distinct for two images taken, considering they are not taken within a second of each other.

Here are some screenshots of how it actually works:

Selecting a region using mouse

Selected Region output

Finally here is the link for binary for this project: mouseSelectAndSave
It uses OpenCV libraries, which I have copied inside the application folder, just in case you dont have OpenCV installed or have different version than mine.

AndroidWifi

To read the update messages on this guide click here

 

A few months back, I bought an android phone. I found android was way better than iOS, except for one little issue, which I was fortunate enough to get around. I have always been sharing my internet connection with my phone, be it an iPhone, iPad or any other device. They all worked pretty well with my windows Ad-hoc connection sharing. Except for this phone which is really a major disadvantage.

A lot of people use connectify to get around this on windows 7. For me, as it has always been, I wanted something simple and fast. I did some search, and found out an easy way of making a virtual router. This involved using command line to give commands for enabling virtual wifi, so it was easy for me to write a batch script which did all the work for me. Its pretty much same as connectify, but way faster and simpler, plus instead of using that 'advertising' SSID in connectify, you can choose your own SSID name with this method. I have divided the setup into few simple steps, so everyone can understand easily.

Settings on your phone

While I was trying to make a virtual router on my laptop I could connect successfully but after a while the phone had problem browsing, connecting or even discovering the virtual router. I started troubleshooting, and found out that the problem was not with the virtual router, but it was in the phone settings. My android phone used network based location services which, whenever the phone connected, tried finding the location using the network. It was this service which made the wifi connection impossible. I disabled this and it started working as a charm. To disable this service on Nexus S (Ice Cream Sandwich), go to Settings->Location Services and disable all services as shown below.

Disabling these options will be similar in other android phones as well. Just look for Location options in settings, and you will get there.

Settings on your pc

 

UPDATE (21/07/2012): I have written an application for doing this step. 

Here is the link: AndroidWifi is live!

If you use this application then this step can be skipped. 

I have written this guide using Windows 7, however similar method can be used on Windows Vista. To start with settings on your pc, first thing you have to do is make a batch file. To do this open a new notepad window and copy the following lines.

netsh wlan set hostednetwork mode=allow ssid=MySsid key=MyPassword keyUsage=persistent 

netsh wlan start hostednetwork

Replace MySsid with the SSID name you want, and MyPassword with your own password. Save this file as TurnWirelessOn.bat, and make sure the extension is .bat.

As you might have already guessed this batch file will be used to turn on the wireless connection. We need one more batch file to turn off the virtual router. To do this follow the same procedure as above replacing the commands with the following commands.

netsh wlan stop hostednetwork

 Name this file as TurnWirelessOff.bat for simplicity.

Sharing your connection
Now comes the tricky part. You have to run the above batch file, which will first create a virtual wireless connection. After it has been created, you can use it to share your LAN connection to. To do this, right click  on TurnWirelessOn.bat file and select run as administrator.

 This will enable an internet connection with the SSID and password you provided. Now go to Network and Sharing Center->Change Adapter Settings. You will notice two wireless connections, as shown below:

Note down the name of the wireless connection which uses Microsoft Virtual Wifi Miniport. In this case it is called Wireless Network Connection 2.

Now right click and open properties of Local Area Connection. Go to Sharing and enable  connection sharing and choose Wireless Network Connection 2.

Once you apply the settings, you might get a notification that the connection is currently active. This can be fixed by TurnWirelessOff.bat. Opening this file as administrator will turn the connection off, enabling you to set the sharing options.

Once everything is set up, just use the two batch scripts to share your internet connection to your android, and enjoy!

Update (07/07/2012):  As I can not try this method on different version of windows, with different models and android version, therefore I request everyone to please leave a comment with windows version, mobile model, type of internet shared and android version. 

If it is working great for you, why not share it with others. It will just cost you a simple LIKE at alternativeto page for this software (I don't really think it is, but lets just call it a software)

Cheers!

 

This work is produced under the following creative commons license: 

 

AndroidWifi by Muhammad Asad is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Based on a work at http://seevisionc.blogspot.co.uk/2012/06/androidwifi-alternative-to-connectify.html.

_______________________________________________________________________________________
Update(13/08/2013): I am seeing a lot of bloggers copying this blogpost as it is on their blogs without mentioning the source where they copied it from. For this reason I have created a creative commons license for this, which means that you can copy, modify and use the content as long as you share your software free of charge and mention the source.
_______________________________________________________________________________________
Update (11/03/2013): I have been really really busy with work, and therefore have limited time to answer everyone's questions. If you get stuck somewhere, then please read all comments as solution to almost every problem you can come across can be found in the comments. If you succeed please post the following details in comments below, as these might help anyone trying to share internet with similar setup as yours:

1. Windows Version
2. Mobile Model
3. Type of Internet Shared 
4. Android Version.

If you find out any additional detail that is missing, or anything which solved the problem you are having and is not listed below, PLEASE DO WRITE IN COMMENTS BELOW. THANK YOU!!

_______________________________________________________________________________________
Update (21/04/2013): I am seeing a number of interesting links coming in to this post. I thought it might be interesting for you people to find out about what other devices/areas this script is able to work.

1. This guide has been used by a user of roku streaming player to share his existing internet connection with the roku player. The original forum post can be found here, and might be able to solve a very common issue of not being able to access internet, despite being connected to the virtual router.

2. A question from a reddit user, which can be found here.

3. Other bloggers, translating and writing the same guide again with some modifications, so people can understand in their native language. This post can be accessed here

4. A senior forum member at xda developers forums recommended this method to other users. The full thread including the post can be found here

5. Users have recommended this method on pcmweb.nl forums. This post in Dutch. Thanks to google translate, the users are recommending this method over buying a wireless repeater hardware.

Click here to go to back to the start of this post
_______________________________________________________________________________________

Light bulb!

"An Idea" Yes! I did this Illustration as well

While I was tangled in a web of courseworks for the past few months, I had some really amazing ideas about a lot of different computer vision applications. Fortunately for me, I managed to write few of these  down on a piece of paper whenever they struck me. I have always had the habit of being lazy and not writing things down, when I have them in mind. Because of this, I somehow always ended up forgetting some of the awesome ideas. Knowing the ideas were awesome and not knowing what they actually were always made me feel like hitting myself into a wall.

GSVI v2.0 real-time

After having finished the prototype guidance system earlier, an idea struck into my mind to make it real-time. Since everything was implemented in matlab (for the prototype) and because matlab is not good for real-time systems, I had to implement everything again (well sort of) in C++. I love working with OpenCV so there I began, doing all the stuff again in C++ using OpenCV library.

After much work, and tunning it with different algo's, i finally had it up and running. However, now came the main problem of testing this system in a real envoirnment, and since I have implmented everything on my desktop, it was impossible to take the desktop in a real path. So I took my camera and shot a small clip simulating a blind person walking, and deviading in both left and right directions slightly and then coming back to straight path.

I opened up this video using OpenCV and simply ran my code, the output was really impressive. This was the first time my system was working in a real world scenario, without controlled conditions.

Here is the output from the actual code: