How it Works

A Behind the Scenes Look

Software

Audio Interface

The audio interface implemented in Louis allows users to communicate with applications using voice commands. It converts voice input to text using Google’s speech recognition API to listen to what users have to say. It also generates speech from text based on Google’s Text- to-speech API (gTTS) so that visually impaired users can listen to audio instructions from Louis.

Learn App

Learn basic braille alphabets on the Learn app. When you tell Louis which category of braille characters (choose one from letter, number, indicator and punctuation), your learning step begins. Louis prints out each character with an audio instruction such as "This is letter a.", allowing you to read it with your finger while listening to what Louis says.

Tutor App

Test your braille knowledge on the Tutor app. After telling Louis how long do you want your test to be(choose one from short, medium and full). It shows you a number of randomly selected braille characters (they could be any of the following: letter, number, indicator or punctuation), and you are given three chances to suggest your guess of what each character would be. The characters that you answered wrong will be remembered so that you could re-learn them after the test.

Headlines App

Test your braille reading skills if you believe you have learned enough to read news articles. You can read through BBC news articles from different categories of your choice. It retrieves a number of today's articles of the chosen category from BBC website and let you read them by printing characters on the cell(s) you have. Each cell recalibrates right after being read in order to allow users with small number of cells can benefit from this app.

Riddles App

Stimulate learning, by solving interesting and comical riddles with answers that they can only find out by reading braille! This motivational approach has been previously adopted by Royal Blind, Scotland’s largest vision impairment organisation. Easily browse through the riddle library by using voice commands. Keeps track of which riddle has been viewed most recently, so you can conveniently pick up where you left off.

Memory App

Play the traditional Memory card game, for one or two players, where the cards are individual cells corresponding to a braille alphabet character. Developed to stimulate learning, by practising the memorisation of the braille alphabet characters in a recreational way.

Players first determine if they are playing on their own or not. In solo mode, the number of turns is recorded, so that the player can attempt to find all pairs in as little turns as possible. When playing together, the app will keep track of the score. A player can ‘flip a card’ by pressing the button on a cell. That cell will go from a blank output to rendering the associated character. A second cell can be selected in the same way. The player gets time to inspect the characters, and can move on to the next turn by saying ‘next’. The output will then return to blank again if it was not a pair. If the two cells were showing corresponding characters, they will stay ‘facing upwards’. The character will be read out to reinforce learning and the player earns a point. In multiplayer mode, the same player gets another turn.

Hardware

The hardware design was inspired by combination locks and Gray Code. Each cell uses a single motor, which rotates an axel holding two disks. The top disk is smaller than the bottom one and it rests directly on top of it. This enabled us to represent half a Braille character with the larger disk and the other half with the smaller disk. The smaller disk is fixed on the axel, while the big disk is loose and rotates only when a catch, attached to it from below, is pushed by the axel. This enables us to rotate the axle in one direction to position the larger disk then rotate the axle in the other direction to position the smaller disk.

Initially, apart from the difference of the size, the two disks were identical and had whole segments raised, which themselves were pushing pins up, forming the braille characters. At the end of the third iteration though, in an effort to decrease the size, we decided to raise small bumps on the surface of the disks which would represent the braille characters and thus avoid the difficulties of working with pins. Furthermore, we altered the big and small disks so they would have triple and double encoding respectively, which enabled for smaller and faster transitions between the different combinations.

Interactive Disks Render

Our cell walls were made from lego bricks during the testing phase and were intended to be replaced by a combination of 3D printed and transparent CNCd walls before the start of the fourth iteration.

The cells are controlled by the main controller which contains a speaker, a microphone, the Raspberry Pi single-board computer and the rechargeable batteries. On a cell there are the pins that display a braille character as well as a button, which can be used as an input in particular apps. The cells are connected end-to-end with the main controller and each of them is operated using our own custom-made PCB board.

Firmware

Communications Protocol

The main controller and cells communicate using a four-byte command protocol. The first byte contains the destination of the command. This is required because the main controller and cells are chained together, meaning that a message from the main controller to the last connected cell would need to pass through all the connected cells in order to reach the last cell.

The second byte in the protocol identifies the specific command to be performed. This includes rotations of either the large or small disk, button presses, calibration, etc..

The third and fourth bytes in the command protocol hold the data that is required to process the specified command. This includes the angle for a rotation of the disks, the cell number when a button is pressed, and a status message when initialising louis.

Firmware

The firmware running on the microcontroller in each cell implements a Finite State Machine (FSM). The purpose of this FSM is to receive incoming commands, read the commands, then act on them accordingly. Each of the steps in the FSM is outlined below.

Wait for command: in the starting state, the firmware idles, continuously polling the serial ports until it receives a command.
Check command: when a command is received through one of the serial ports, the firmware first checks the header of the received command to check the destination.
Process command: if the destination of the command is the current cell, the command will be processed by the cell. For example, this could involve displaying a certain braille character by performing a specific rotation.
Forward command: if the destination of the command is not the current cell, the firmware forwards the command in the direction that will lead to the current cell.

Interactive Examples

The following three interactive widgets demonstrate three core features of the firmware: sending output from the main controller to connected cells, sending input from the connected cells to the main controller, and determining the number of cells that are connected.

Sending Output to Cells

Step 1 of 4

The Main Controller wishes to send a message to Cell 3. (For example, to tell it to output a specific Braille character). Since there is no direct connection between the Main Controller and Cell 3, the Main Controller must first send the message to Cell 1.

Receiving Input from Cells

Step 1 of 3

Assume that the button on Cell 2 has been pressed. Cell 2 now needs to alert the Main Controller that this event has occurred. Since there is no direct connection between the Cell 2 and the Main Controller, Cell 2 first sends the message to Cell 1.

Discovering All Connected Cells

Step 1 of 9

The Main Controller sends a message to Cell 1. The payload of the message tells Cell 1 to identify itself as cell number 1, and to continue the discovery process.

Custom Printed Circuit Board (PCB)

A major goal of louis is to keep the unit cost of individual cells to a minimum. Although the firmward had been developed using the Arduino hardware and software, it was decided that a custom PCB would allow for reduced costs in the long run, particularly if produced at scale. This PCB was designed to include only the absolute minimum number of components in order to reduce the Bill of Materials.