Team Pupil

It’s 2018, and access to the internet is crucial for research, logistics management, online commerce, data storage, etc. However, the web is still only partially accessible to many users with vision difficulties. Screen readers partially mitigate the challenge of using the web by converting web page content into actionable elements that are read aloud for users. However, the functionality of screen readers (e.g. Google ChromeVox) is limited to only narrating actions and reading webpage text, relying on developers to add alternate descriptions to images and other non-textual elements, leaving users with vision impairments unable to view embedded images. What it does Our group aimed to solve the web accessibility problem with Pupil, a chrome extension that utilizes visual recognition to automatically caption web images with alternate descriptions in real-time. These descriptions are injected in the source code of the webpage to be read off to the user by whatever screen reader they already have.

The three of us became friends while studying computer science at Brown. We decided to participate to Hack@Brown together, where we formulated and developed Pupil. We were later contacted by Christine, a project manager for the Imagine Cup, through our project submission on devpost because our project utilized Microsoft Cognitive Services, specifically the computer vision API. Sean Rayment, Brown University, Computer Science Sc.B., 2020 - Sean is a computer science concentrator at Brown University who became in web accessibility after hearing a talk given by Larry Goldberg (Oath) about closed captioning standards in Brown’s User Interfaces and User Experience course. Otherwise, Sean is generally passionate about building applications for the web. In his free time, Sean competes in Brazilian Jiu Jitsu and plays Dungeons and Dragons. Sathya Anisetti, Brown University, Applied Mathematics-Computer Science Sc.B., 2020 - Sathya is a student at Brown University an incoming summer research intern at MIT Lincoln Laboratories and. After spending the summer after freshman year assisting in computer vision research, Sathya grew very passionate about the applications of automatic image recognition in areas ranging from recognizing handwritten digits to creating self-driving cars. Sathya hopes that after college he can continue working in the areas of machine learning and artificial intelligence. Pranav Nagalamadaka, Brown University, Applied Mathematics-Computer Science Sc.B. 2020 Pranav is a student at Brown University, interested in the areas of Applied Mathematics, Computer Science, Data Analytics. In his free time, he loves to play piano, run track, do meditation, and read books. He seeks an active life, playing basketball, soccer, table tennis and badminton. His future goals are to become a Data Scientist, join a non-profit organization fighting poverty alleviation or lack of education and spread the philosophy and practice of meditation.