Computer Vision News - February 2017

6 Computer Vision News Application Application dramatically improved by using the Sesame phone. The application is a background service and to turn it on, you say “ Open sesame ”. Then the phone opens up. The front-facing camera automatically opens to search for the user’s face. Once it locks on the face, that screen goes away. Then a mouse cursor on screen follows his or her head movements. If the head moves to the right, the mouse goes to the right. Then left, up, and down. Then it’s just a matter of focusing on the interaction the users wants to have with the smartphone. To dial someone, user moves the head so that the cursor moves to the phone icon, dwelling there for a second. Then for simplicity, it taps the phone. Then the phone dialer opens it. The head brings then the cursor to the contact which needs to be called. It pauses there for another second, and it dials that person. The beauty is that they’ve integrated it with almost the entire Android operating system, not just that dialer. It’s not just a messaging app: any app you download from the store works out of the box with this technology. Ben Dov relates of seeing children play Angry Birds for the first time in their lives. Selfies are a number one user request , but it’s still problematic because the camera is already used to track the face. They’re working on that and hope to come up with a feature soon. They actually started with a prototype, which was their MVP (Minimal Viable Product), and decided to progress in gradual steps. At first, they used Viola- Jones , the face detecting algorithm, connecting that to a cursor on screen as an overlay. Then they gradually improved different elements of the system that were far from optimal in Viola-Jones, like speed, accuracy and steadiness. Tests were run with test users, including of course Giora, their first prime tester. Then they went on building and after that started adding more and more beta users. Ben Dov shared with us the computer vision techniques they tried and the challenges they encountered. It comes out that Viola-Jones is not sufficient for a turning face. Viola-Jones can detect the front facing picture of a person’s face. But the Sesame phone needs a solution for users who look at the screen and turn their head. At some point, they reduced optical flow to track where the face was moving. That too had its problems, mainly with tracking points being lost or some errors in the tracking. They went to a beta user, and suddenly, he was using it perfectly. At that point they realized that the reason was just Giora having very limited head movements. It wasn’t good enough for him. For someone who could move the head just like everybody, the app was working perfectly. That finding was good in two ways: on one hand, that meant the product was already suitable for some people; on the other, it pointed the developers to a direction where improvements were to be done. When they went back to the optical flow , they improved it even more and it became accurate enough for Giora to use, even with his limited head movements. “I’ve seen children play Angry Birds for the first time in their lives”