Computer Vision Firms Focus on Food Retail

The steady stream of Amazon Go store openings is whetting America’s appetite for checkout-free retail experiences and setting the stage for any number of competitive responses from retail and technology rivals.

“When you look at how much traction Amazon Go is getting, we regard them as a great market enabler and encourage them to be fruitful and multiply,” Andy Radlow said with a laugh. Radlow is VP of sales and marketing for Grabango, a Berkeley, Calif.-based startup developing a similarly enabled solution—only one that is intended to be deployed in large retail stores with thousands of items.

Grabango, whose founder Will Glaser also founded the streaming radio company Pandora, is testing with four clients, including what Radlow described as three top 30 grocers and one of the 10 largest convenience store chains in the U.S. He declined to identify them but said the company was hopeful it would announce a rollout with at least one of its current partners before the end of the year.

Not unlike e-commerce order fulfillment solutions—which also use newly available technology in service of making shopping easier for shoppers and more efficient for retailers—there’s a bit of a race developing in checkout. In addition to Amazon Go and Grabango, there are other companies, such as Standard Cognition, Zippin, Trigo Vision and Sensei, which are also developing some form of line-free technology using computer vision. Standard and Zippin are currently exhibiting their technologies in their own stores in California; Trigo Vision, based in Tel Aviv, recently announced an agreement to supply its technology to 272 stores in the Shufersals supermarket chain in Israel. And Sensei, based in Portugal, is looking to expand in Europe, reportedly with the financial backing of Germany’s Metro AG chain.

Proponents argue they represent a next step from checkout options using in-store scanners or consumers' phones, in part because the latter solutions tend to shift the labor of checking out to the consumer, and in part because more convenient checkout is only the head of the snake for computer vision-enhanced stores: The same system can also track inventory, spot shoplifters and provide the retailer and suppliers with rich merchandising and traffic data.

Virtual Baskets

Grabango, which in Silicon Valley terminology came out of “stealth mode” just months ago, promises a new take on the “Go” concept, enabling an enterprise-class solution to be deployed in existing stores as large as 100,000 square feet. The solution uses ceiling-mounted “g rails” with clusters of cameras, each with a proprietary computer behind them. In practice, they will keep an eye on every shopper and every product in a store, and digitally recognize all of the interactions between them, Radlow says.

Shoppers in a store using Grabango shop as they normally would while the system assigns and tracks a “virtual basket” to each shopper, which is recognized at checkout. Those who have prearranged to pay through an app can simply walk out; others are presented with an option of how they’d like to pay, but the items in their basket needn’t be scanned.

“We have embraced an inclusive design that recognizes that in large grocery, cash is important,” Radlow said.

The Grabango system “learns” where objects are by tracking their arrival and departure from shelves over time, a term known in computer vision as “object permanence,” Radlow said. This, he said, produces accurate results even when items are pulled from deep in a shelf that’s out a camera’s line of vision.

Photograph courtesy of Grabango

“In combination with object permanence, plus the fact that we can see people reaching to the back of the shelf and see as they pull it to the front of the shelf, the combination of all those gives us a high degree of accuracy of what’s occurring between said inventory and said humans,” Radlow said.

Not every last detail is worked out quite yet. Objects sold by weight are currently flagged and handled by cashiers using the same point-of-sale processes they use today, but Radlow said Grabango intends to develop automated methods for identifying and weighing objects to be added to virtual baskets. The company said it would also work with retailers “to strike the optimal blend of strategies to streamline checkout of fresh or bulk items.” That could mean changes in merchandising, depending on partner input, “but our goal is to minimize any changes to shopper or store operations,” Radlow said.

Cashiers will still have a role in verifying the accuracy of the virtual baskets, aggregating tickets for shoppers who may have multiple carts of people in a trip; they also, for example, will verify ages for alcohol purchases.

In cases where the Grabango system is not certain as to the right object, a snippet of video could be sent to a person in the loop to verify. The system is designed, however, to require fewer such verification as time goes on. “We recognize that the grocery environment is dynamic. Endcaps are always changing, and who knows what the DSD folks are going to be putting on the shelves overnight,” said Radlow. “The system will recognize when it doesn’t know what the heck is on the endcap this week or what particular item the AB InBev delivered this week, but that’s where our humans on the loop will answer the alert and make a correction.”

Return on Investment

Grabanago reps say its system—in part because it uses proprietary hardware—can cost up to 50% less per square foot than a technology such as Amazon Go, which reportedly absorbs costs of $1 million, or about $500 per square foot, for a 2,400-square-foot store.

“When you’re in the business of retrofitting existing stores that have 1% to 3% net margins, that’s an absolute deal-killer,” Radlow says. “There is no way a grocery store can afford to spend another penny on another thing that doesn’t have a clear and present and significant return on investment. If it doesn’t have a large ROI, they won’t even have a meeting with you.”

Photograph courtesy of Grabango

That’s where benefits of a computer vision-enabled store go beyond enabling shoppers with a quicker trip. Radlow said the same system enabling checkout can also be used to track inventories, potentially reducing out-of-stocks. The system can also help guard against employee or customer shoplifting and can identify hazards leading to slip-and-falls and inventory loss. Many retailers are already using computer vision-assisted robots to scan the aisles for such tasks now.

Grabango could address nearly half a store’s shoplifter and insider fraud issues in the first year, Radlow contended.

“We just increased your net profit by 50%, and that doesn’t have anything to do with the 100 basis points of out-of-stock; the 150 basis points that are in inventory management or other savings that would come from quadrupling cashier effectiveness because they no longer have to manage items dumped onto a conveyor belt, and they don’t have to scan or bag anymore,” he said.

In a presentation at the ShopTalk conference earlier this year Michael Gabay, CEO of Trigo Vision, said he co-founded the company with his brother after a stint in Israel’s elite defense force, for whom they developed technologies. They decided to turn their attention to supermarket retail in particular because it was behind other industries in technology adoption and therefore represented a big opportunity for advancement.

“It’s all about experience for the shoppers—and value for the retailer,” Gabay said.