Khosla-backed HealthifyMe introduces AI-powered image recognition for Indian food

Indian health and wellness startup HealthifyMe has introduced an AI-powered feature that automatically recognizes Indian food from images for calorie intake logging, allowing users to track their meal intake more efficiently.

Tracking Indian foods through images is challenging, given there is a vast variety of cuisines. Also, a number of people eat on a plate called Thaali in Hindi, which contains different portions of various food items. A model has to recognize all that along with portion sizes.

HealthifyMe’s latest feature, called Snap, lets users take pictures of their meals and try and recognize food items captured within the frame. Alternatively, users can also give access to their gallery, and the app scans food pictures automatically. Users can review those pictures and food items within them later. This allows users to take pictures from their cameras and deal with the calorie logging later.

In terms of privacy, the company says the model on the device detects food pictures and sends them to servers for specific dish recognition. The company also mentions its gallery-based model works better as it has more time to recognize food items than the option to take pictures of your meal for recognition.

If the model detects there are multiple items in an image, Snap will ask you to tap on an item and add it to calorie tracking. Users will see an adjustable rectangular box to focus on different items.

HealthifyMe co-founder & CEO Tushar Vashisht told TechCrunch that the company started as a food-tracking app almost 11 years ago and that remains the primary use case for its free users.

“It’s hard to engage with food tracking at times because you have to physically type and remember to log food. So we wanted to increase retention and engagement with a feature like Snap,” he said.

Vashisht said that the startup had tried implementing image-based food recognition over the years, but the availability of better generative AI models made it easier to create Snap. The company said the feature currently is trained to recognize 150,000 Indian food items.

HealthifyMe claims to offer 60-70% accuracy in terms of automatically recognizing food. Even if the model does not recognize the food item properly, users still get suggestions about what the item could possibly be, the company said. The company has human reviewers who look at false recognitions and correct them. Additionally, users can manually tag these falsely recognized photos to improve the model. Vashisht asserted that he is confident that in the next month, the accuracy of the model will be more than 80%.

Notably, HealthifyMe is not alone in the race, as other companies are also working on AI-powered food recognition. Samsung’s meal planning platform Samsung Food plans to introduce this feature next year. Former Google Lens engineer Wade Norris’ startup Snapcalorie is also working on the same problem with backing from Y Combinator, Index Ventures and Accel.

In the next few weeks, HealthifyMe aims to give more options to users for food logging. With the launch of Snap, it is allowing users to send pictures to HealthifyMe WhatsApp or tag them on X with a food image. The company is working on a voice input feature along with improving its existing AI-powered assistant Ria.

HealthifyMe, which has raised over $130 million in capital to date from Investors like LeapFrog Ventures and Khosla Ventures, has a basic paid plan starting at $4.80 (₹399) per month that gives users access to an AI-powered fitness and nutrition assistant, meal planning and healthy recipes. The company also offers pro-paid plans starting from $48 per month (₹4,000) with features like an AI-based meal planner, nutrition and fitness coaches, and a smart scale.

The higher tiers of the Pro plans have more benefits such as multiple doctor consultations, metabolic panel tests and continuous glucose monitoring (CGM) devices. The company said it has more than 200,000 users as subscribers with nearly 40% of them paying for the Pro plan.

In 2020, HealthifyMe partnered with Indian food delivery service Swiggy to curate healthy meals and restaurants. The company is already in talks with multiple food and grocery services that could leverage its technology.