solid and dynamic team
Multidisciplinary team of graduates, masters and doctors, with complementary knowledge from distinct areas such as computer science, engineering and psychology.
VIRTUALSIGN:
Our main objective is to simplify the learning and communication process between all users of Sign language, through the hardware and software we designed and developed. He hope that the implementation of this project will help those who have communication limitations, providing more equitable access to education and opportunities.
Multidisciplinary team of graduates, masters and doctors, with complementary knowledge from distinct areas such as computer science, engineering and psychology.
Working daily with dedication to eliminate inequalities and to break down barriers to the communication for full integration into society.
Sharing our knowledge with you, and taking the project to the Sign Language users and everyone else who wants to know a little more about the VirtualSign.
The Virtual Sign application consists of a bidirectional translator for the Portuguese Sign Language. In other words, the application translates the performed Sign Language gestures to written text and translates written text to their respective gestures in Sign Language. If you want to know more technical details about the functioning of the translator, you can consult the article Real Time Bidirectional Translator of Portuguese Sign Language.
Gesture to TextUser performs gestures in Sign Language that are captured by the kinetic camera (Kinect) and the sensor gloves. The Kinect provides information about the movement and position of the hands and body. The gloves provides a precise information about the configuration of the hands/fingers.
The data collected is sent to the application, which using advanced data mining and classification algorithms processes the data in real time translating the gestures into the respective text.
Text to Gesture
User writes the text that want to translate on his personal computer. The text is then processed by the application, giving rise to a mapping between the words/phrases submitted and their representation in Sign Language.
Each gesture has a unique identifier that is passed to an Avatar which interprets the received identifier and performs the corresponding animation. The animation includes body motion and facial expression.
Learn while playing
The game has three main levels taking place in a desert, a jungle and a futuristic scenario respectively. The difficulty increases with the advance in the game as as it advances from the learning of letters to the learning of sentences.
The learning is inherent with the gameplay itself. As the player progresses in the game, he will learn new gestures in sign language starting from the alphabet, then through the words until be able to perform complete sentences.
Several challenges for the player, such as obtaining items to gain access to others zones, checkpoints to improve the knowledge of Sign Language and fun mini-games.
The player interact with his avatar and the objects in the scenes. When playing with Kinect the player will be asked to preform the gesture himself using the VS Translator.
The game is prepared to run on personal computers, but soon it will be exported to the mobile devices, concretely for Android, iOS and Windows Phone.
If you liked this project and want to know more details about the game, you can consult the Virtual Sign Translation in Serious Games paper.
use our avatar and translate the text you want to Portuguese Sign Language
visit the game webpage and have fun playing the game while learning Portuguese Sign Language
join our community and tell us your opinion and suggestions to improve the project
