Background and objectives
Whether at school, in daily life or in professional life, we need to consult maps, diagrams, graphs, etc. These maps are necessary for building skills and mental models used in many fields (geography, biology, mathematics, mechanics, etc.). For people with visual impairments, access to these different graphics is not excellent, which can have consequences on learning.
The “Augmented Space Reality” (or SAR in English) project started in November 2018 within the framework of the European VISTE project, then continued within the framework of the Accessimap project carried by “Cherçons pour Voir”. It was born out of the desire displayed by certain teachers specializing in visual impairment to create interactive audio-tactile educational content themselves. The advantage of spatial augmented reality is that it makes it possible to “augment” an already existing object with digital information. For example, you can trigger a sound when an area of the graph is touched.
Presentation of the system
PapARt is a spatial augmented reality system developed and marketed by the company RealityTech. The system consists of a color and depth camera and a tower-mounted projector, a microphone and a speaker. The camera makes it possible to locate objects and follow the movement of users’ hands.
After placing an object (thermoformed paper, object, board game board, etc.) on the work area, it is easy to create audio-tactile zones by adding already recorded sounds or by recording your own voice on the system request (“creation” mode). These interactive objects and graphics can then be used (“use” mode) very intuitively. The device makes it possible to be completely free of software since the transition from creation mode to use mode is carried out using raised maps that the user places on the work area. The drawing of the shapes is done with the hands (no mouse, no keyboards!).
Demonstration
A demonstration video is available here.
Project actors
The project was initiated by Anke Brock (researcher at ENAC Toulouse), Christophe Jouffrais (director of “Cherchons pour Voir”), Martin Hachet (INRIA Bordeaux) and Lauren Thévin (post-doctoral fellow at INRIA Bordeaux then at ” Cherchons pour voir “). This team was reinforced by the arrival of Nicolas Palard, engineer “Cherchons pour Voir” and Nicolas Rodier, trainee “Cherchons pour Voir” from the ISIS engineering school of Castres from mid-February to August 2019.
Many visually impaired professionals participated in different stages of this project (needs analysis, design and assessments) including: specialized teachers in Romania and Greece (within the framework of the European VISTE project), IRSA professionals from Bordeaux and CTEB from Toulouse, and the members of the AccessiJeux association from Paris, as well as six people with visual impairments. We thank them all.
Project status and follow-up
This project was an exploratory project to demonstrate the concept of using augmented reality to create audio-tactile content. The results show that it is very easy to use by professionals with visual impairment and that the device responds well to the need.
For the moment, no transfer to a company has been envisaged but if you know a company likely to be interested, do not hesitate to send them to the “Cherchons pour voir” laboratory!