Project description
Purpose
The purpose of this robot is to tell children stories. The course of the stories can be interactive influenced by the kids.Super-Yano interacts with synthesized voice output, speech recognition, eye-contact and simulates human behavior.
Starting basis
The mechanical basis is an electronic doll called "Yano" which I modified and expanded.The idea to this project came as I found the "Robotic F.A.C.E.", a MIT student project.
They modified a Yano doll to control its facial expression with a computer.
Motivated by this project my intention was also to build and program an robot based on a Yano doll. The ambition wasn't to create an intelligent system, but to maintain the primary purpose of the Yano as an interactive story teller with expanded abilities.
Inside the original head of the Yano doll there are only three motors controlling its facial functions. And therefore most movements are coupled (see Robotic F.A.C.E.).
I wasn't satisfied with this limitation and so the first step was to replace the original mechanics inside the Yano head with an own servo system.
Camera system
I assembled 11 servo motors in this robot.And to let him see, I installed two rotable cameras in his eyes.
Here's a video of my first tests with the eye movements:
Servo control
To control all the servos I used a Parallax servo controller card. It can control up to 16 servos independently with different speed rates and is driven via a serial connection.Furthermore there is a home-made controller card to control a LCD which is used to display spoken text.
Power is provided by a standard computer power supply.
Software
The software was developed with Visual Studio 2005 and Visual C++.It's structured into 5 sections:
- GUI & main program
- robot movement control
- face detection
- voice output
- speech recognition
The class movement control section is definetly the most important part because it controls all robot movements, makes the robot doing mouth movements during speaking, steers the eye blinking and makes him waggle his ears.
Ok, you may wonder why a robot waggles his ears - but I think that such small gimmicks make a robot more humanoid. And therefore more acceptable as a story teller.
Face detection was done with a image library from Intel called OpenCV.
OpenCV comes along with many functions and filters. Among other things it includes a trained face detection.
To integrate this in my project I had to make a detour with a wrapper DLL because OpenCV is unmanaged C++, but my project is written in managed C++.
But although it wasn't actually so hard to implement, the ability to focus on faces and to keep eye-contact makes a great lasting impression on people...
Voice ouput was integrated with Microsoft's Speech API.
The advantage is, that the Speech API is expandable with many voices in all languages and available for free.
You can find a simple code fragment in Delphi at my code snippets section here.
Also speech recognition can be done with the Speech API - but currently only in English and Japanese.
For my daughter I had to find something in German and found Naturally Speaking from DragonSoft, a dictation software which I included "through the back-door" to my project by accessing its ActiveX components.
And finally, there is a main program used as a central control unit for all modules and provides a graphical user interface (GUI).
Now see the whole thing in action:
Conclusion
Well, this one was a very complex and large project. And although I see it more unemotional from my point of view, again and again Super-Yano achieves incredible enthusiasm with people.This description was just the tip of the iceberg.
If you are interested in more details you may read the
project documentation (PDF, 6.3MB). But sorry - it's in German only since I'm just to lazy to translate 80 pages into my clumsy English...
Anyway, you also may have a look at the image gallery and the demonstration videos on YouTube.