NEXATMART

The problem

Online grocery stores are built around scanning pages and clicking small buttons, which shuts out blind and partially sighted customers. NEXATMART lets you shop by speaking: say what you want and the store finds it, reads back the price, adds it to your basket and takes you through checkout.

Decisions that shaped it

1. 1

   Product matching allows for misheard speech. The server tests every 1 to 4 word chunk of what was heard against each product name, scoring with Jaro-Winkler, Levenshtein distance and Soundex. A close match is added to the basket straight away. A weaker one returns the top three "did you mean" options.

2. 2

   It understands Tamil, Hindi and everyday phrasing. Before matching, a synonym dictionary maps romanised Tamil and Hindi words to product names (thakkali to tomato, vengayam to onion). Filler phrases like "can I please have some" and words like "fresh" or "organic" are removed first.

3. 3

   The browser's alternative guesses get used too. The Web Speech API is asked for up to six alternative transcripts with confidence scores, and the server tries each of them in confidence order. If the top guess falls below 25% confidence, the assistant asks the customer to repeat it, up to two times.

4. 4

   Every result is spoken, and you can follow up by voice. The assistant reads out each result, then listens again for replies like "add that", "the second one", "my cart" or "checkout". Before checkout it reads back everything in the basket and waits for a spoken yes or no.

5. 5

   Errors are spoken too. If the microphone is blocked, nothing is heard, the network drops or an item is out of stock, the assistant says what happened and either listens again or closes.

Built with

• C#
• ASP.NET Core MVC
• Entity Framework Core
• SQLite
• Razor
• Web Speech API