Context and objectives
Visually impaired people cannot use maps, diagrams, or charts since most of them rely on visual information. In addition, Braille clogs a document very quickly and does not allow capturing all of the information for reading. However, these maps are essential for the construction of mental representations of spatial configurations and for the mastery of knowledge (geography, biology, mathematics, etc.). These difficulties correspond to a real obstacle to access to education.
The “SAR-Creation” project started in November 2018 as part of the European VISTE project. It was born out of the impossibility of teachers specializing in visual impairment to create their own interactive audio-tactile educational content themselves without going through computer scientists or researchers. It is therefore advisable to democratize this practice by developing free tools to make available to them. The advantage of spatial augmented reality is that it makes it possible to augment an already existing tactile object with digital information and to update it very easily. In the context of visual impairment, this is audio information.
Presentation of the system
PapARt is a spatial augmented reality system developed and marketed by the company RealityTech (http://rea.lity.tech). In the framework of the VISTE and Accessimap projects, several laboratories have collaborated (“Cherchons pour voir”, Toulouse; ENAC, Toulouse; INRIA, Bordeaux) to work on the uses of the device, in particular for people with visual impairments.
The system consists of a color and depth camera and a tower-mounted projector, a microphone and a speaker. The camera tracks the movement of users’ hands. By depositing an embossed content (thermoformed paper, object, board game board, etc.) on the work area, we will be able to create audio-tactile areas (“creation” mode) and to interact not only with them but also objects (“exploration” mode).
The creation mode allows you to draw concave or convex shapes and straight lines on a relief support. It is then possible to associate a sound with each of its forms.
The device makes it possible to completely free oneself from the computer since the passage from one mode to the other is carried out using wooden cards to present to the camera. In addition, the drawing is made using his finger.
To find out more, a demonstration video is available on the laboratory’s YouTube channel at the following address: https://youtu.be/z6h6mKT0h1M
Consortium and actors involved
The project was initiated by Anke Brock (ENAC Toulouse), Christophe Jouffrais (IRIT), Martin Hachet (INRIA Talence) and Lauren Thévin (INRIA Talence). This team was reinforced by the arrival of Nicolas Pallard, engineer trainee from the University of Bordeaux, from November 2018 to the end of February 2019 and Nicolas Rodier, engineer trainee from the ISIS school of engineers in Castres in mid- February to August 2019.