During the events we hosted, many seniors described everyday problems they faced, including misreading ingredients and buying the wrong products. Realizing the issue was widespread, we created a mobile application that analyzed ingredient labels via machine learning. The app provides a multimodal interface—visual, audio, and touch elements, verbally informing the user if the product contains ingredients outside of the user’s diet, and empowering seniors to be more independent when shopping.
Science Research Award
Synopsys Science Fair Award
Jonathan Park presented his research on a machine learning model to automatically analyze allergens from food/drug ingredient labels for seniors in Synopsis Science Fair. He was awarded as “Honorable Mention” and received a special award: $50 and letter of recognition from Northern California Institute of Food Technologists (NCIFT). Here is his poster.
Junior Science & Humanities Symposium (JSHS)
Jonathan also was the Semi-finalist at California Junior Science & Humanities Symposium. He submitted a 9 page research paper and presented his work to the JSHS committee members. Here are the abstract and the presentation slide.
With the increase in the senior population, senior-related health issues have gained significant attention. One of the most common problems is vision disorder or loss, which makes it difficult for seniors to identify allergens from ingredient labels. In order to overcome this issue, I have developed a Convolutional Neural Network (CNN) based smart app, ReSight, that reads ingredient labels to automatically detect drug and food allergens for seniors. ReSight provides a novel hybrid detection method to improve analysis even on cylindrical products. ReSight’s model was compared with two baselines: Tesseract, one of the most accurate open-source optical character recognition (OCR) systems and Google Vision API, an industry-leading OCR system. The experimental results on the five most common allergens (wheat, peanut, soy, milk and egg) demonstrate that ReSight performed significantly better than Tesseract and performed as well as Google Vision API. After detecting allergens, ReSight uses a multimodal interface (visual, audio, and touch mode) to highlight the allergens on the screen and verbally inform the user of their presence. ReSight, with its multiple functions, offers a promising solutions to seniors who have health and dietary restrictions.
To see the whole slide, click the slide below.