Can A.I. usher in a new era of hyper-personalized food?

Artificial Intelligence in food and beverage, with hyper-personalized products for age, ethnicity, and gender

Alex Fine

Artificial Intelligence in food and beverage, with hyper-personalized products for age, ethnicity, and gender

Alex Fine

Artificial intelligence is helping to develop flavors tweaked precisely for your age, ethnicity, and gender. Think beer and snacks as unique as your fingerprint, and a future where your food knows more about you.

“My take is that pretty much all the food and beverage products on the market today are awful,” Jason Cohen tells me, with fierce conviction. “There are literally no products engineered for me.”

Cohen is the founder and CEO of Analytic Flavor Systems, an NYC-dobased start-up that aims to usher in a new era of hyper-personalized food. We are meeting at a swank Australian coffee shop near the company’s office in the financial district—the kind of place that offers multiple single-origin pour-over options—so he can tell me about his artificial intelligence (A.I.) platform, Gastrograph, which he says can be used to map taste preferences with unprecedented ease and precision. Cohen is lanky and self-possessed, with hair the color of damp straw. He drinks his coffee with the studied concentration of someone who takes flavor extremely seriously.

Like many start-up CEOs, Cohen interprets his own dissatisfaction as a sign of a more general problem. It’s not just that most grocery store offerings, from snack-cakes and yogurt to green tea and IPA, don’t fully thrill our senses. They’re also aimed at the lowest common denominator: There’s nothing out there truly designed for you. The world of food and beverage manufacturing, Cohen says, is still oriented around “the predominant demographic,” the flavors of things tailored to please a coarse approximation of majority appetites. The result, in his view? Endless shelves of products that most people like, but few people really love.

The dream is of a day when we’ll each have a Dorito of our own.
Yes, the processed food industry has gotten pretty good at making food to please the masses. New Yorker staff writer Helen Rosner once argued that anyone who’s tasted chicken tenders loves them, even if they no longer choose to ingest them. New York Times reporter Michael Moss famously explained how snack food companies have learned to lure us by tweaking proportions of salt, sugar, and fat to a “bliss point” ratio most human beings find irresistible. But Cohen’s argument is that existing models of flavor design only work in crude broad strokes. And he thinks his artificial intelligence (AI) tool is the doorway into a new landscape where food and beverage companies know more about us than ever before, with product offerings that respond ever more individualized hungers.

In its quest to make food that knows more about you, Analytic Flavor Systems’s main data collection tool its smartphone app, Gastrograph. The app’s central feature is a wheel with twenty-four spokes, where each sliver represents a discrete category of sensory experience—such as “meaty,” “bitter,” or “mouthfeel.” Tasters map the contours of flavor perception by tracing the spokes corresponding to the qualities they detect, designating the intensity of each on a scale from one to five. A submenu allows for a more granular record of experience: specifying that “meaty” quality, for instance, as beefy, sausage-like, or more exotic options (moose, kangaroo). Tasters are then prompted to give the product a preference rating, on a scale from one to seven.

Using the gastrograph AI system to map a beer flavor profile Analytic Flavor Systems

A flavor analysis of a craft beer, Dogfish Head’s 120-minute IPA, mapped using the Gastrograph app. Each of the wheel’s 24 spokes corresponds to a specific flavor, ranked from 1 to 5 in intensity. Most categories break down into submenus for an even finer analysis.

The Gastrograph app also gathers data about the person doing the tasting—demographic information, socioeconomic status, past experience with the product, smoking habits, and more— as well as information about the ambient environment, such as temperature, barometric pressure, and noise levels, all of which can shade our experience of how things taste. “We literally turn on every sensor that the device has,” Cohen explains, including the microphone, light meter, and GPS. “We even collect magnetic field data, which is right now not predictive of anything, but one day…” He shrugs. Who knows what the data might reveal about the influence of the magnetosphere on our ability to detect saltiness?

All this is an attempt to crack open most that most private realm: the intimate, ineffable world of flavor. The tastes that tantalize and repel us, after all, are highly individual, shaped by biology, culture, and personal history. But the power of the Gastrograph A.I. lies in its ability to model and predict the flavor preferences of increasingly narrow slices of the consuming public, giving food and beverage companies the information they need to develop products optimized for more and more specific sensibilities. Cohen dreams of a day when we’ll each have a Dorito of our own.

An algorithm has no tastebuds; a neural net never gets the munchies. So can a robot brain really tell us what we’ll want to eat? The question is whether A.I. systems will be able to excel in the sensual, creative work of tasting and developing new foods—and what we stand to gain or lose by inventing foods that really have our number.

???

How should food be made to taste? This question has vexed manufacturers since the earliest days of factory-made foods, when industrial processing created new challenges—and new possibilities—for flavor. The unprecedented ability to manipulate raw ingredients raised two connected conundrums, both still top-of-mind for the industry today. The first has to do with consistency. No grain of wheat, no cocoa bean, is identical—yet each Oreo that tumbles off the production line must be, as far as possible, indistinguishable from the next. How can nature’s variety be commoditized and rendered uniform, with sensory experience that’s guaranteed? Second, there is the problem of deliciousness. What makes one crème-filled cookie preferable to another crème-filled cookie? How can pleasure be measured?

jason-cohenAnalytic Flavor Systems

Jason Cohen, founder and CEO of Analytic Flavor Systems, a start-up that aims to usher in a new era of hyper-personalized food

By the middle of the twentieth century, food scientists, chemists, home economists, consumer researchers and experimental psychologists forged a new discipline to address these questions: sensory science. If you’re like most people, you’ve probably never heard of sensory science. But its methods shape basically everything you eat. It’s what ensures the uncanny consistency of Budweiser from can to can, calculates the ideal crunch of a Pringle, and determines the optimal cheesiness for a Ritz cracker.

The achievements of sensory science depend on the work of an exclusive group of highly specialized human beings: the trained sensory panel. Sensory panel tasters learn to taste and smell analytically. They learn to describe their experiences using standardized vocabularies of taste, aroma, texture, and mouthfeel. They are taught to set their personal preferences aside, and report only what they perceive. Over the years, electronic noses, electronic tongues, and other sensing devices have threatened to replace human tasters with automated machine equivalents, but for now the tasting panel remains the primary instrument of sensory science. Today, sensory panels operate within large food and beverage companies; in academic, government, and military research labs; and at an international network of sensory evaluation and consulting companies.

Christopher Findlay is the chairman and founder of one of the world’s leading sensory consulting companies, Compusense, with a client roster that includes many of the top food, beverage, and flavor brands. “We keep a pool of 30 panelists,” he told me — not full-time employees, but local Guelph, Ontario residents, who are chosen for specific projects (such as a recent six-week French fry study) and compensated for their time. “They get paid,” Findlay says, “but it’s not all about the money. They cheerfully make the commitment. They come here in the middle of snowstorms.”

Each individual is trained using Compusense’s trademarked Feedback Calibration Method, which, Findlay explains, allows tasters to achieve exceptional precision in describing their flavor experiences. This approach aims for a kind of objectivity, with participants expected to respond in reasonably coherent and consistent ways to the same sample. Ultimately, flavor panels are thought to produce reliable, reproducible information about subjective flavor experiences—allowing companies to figure out how their grapefruit seltzer compares with other grapefruit seltzers, for instance, or determine whether an ingredient change will cause a noticeable alteration to its taste.

an oreoAlex Fine

Launching a new food or beverage into the highly competitive US market is a perilous venture

Of course, there is the messier problem of consumer behavior—“a different animal,” says Findlay. Compared with trained tasters, ordinary consumers respond in unsystematic, inconsistent ways when asked about what they taste, and whether they like it. Understanding consumer desires, and using those insights to forecast trends, is a large part of what sensory evaluation companies do. Still, a lot of forecasting comes down to intuition—which may be why new product development in the food and beverage industry is generally recognized as a bloodbath. The claim that ninety percent of new food products crash and burn within a year gets bandied about a lot; that figure is probably overstated. But however you crunch the numbers, launching a new food or beverage into the highly competitive US market is a perilous venture, one that’s expensive and fraught with hazards. One recent calculation estimated that new product failure costs the U.S. food industry $20 billion a year. Suffice it to say, food and beverage companies are hungry for more reliable ways to foresee the future of flavor.

This is where Cohen believes sensory science has failed to deliver, and where Analytic Flavor Systems sees its opportunity. If he’s right, it could mean a fundamental realignment in the field of flavor science, a shift away from the monolithic declarations handed down by tasting panels. It would mean, instead, embracing the ostensible chaos of popular opinion, and finding new ways to map complex patterns long dismissed by the industry as noise.

eight hands, each pulling one slice of pizzaAlex Fine

The vaunted efficiencies of scale that allowed the industrial food production system to achieve its productivity and cheapness have also severed the connection people want to feel to the things they eat

Cohen pulls up a slide on his laptop to show me an image of a tasting panel at Miller Molson Coors. It shows five or so people sitting around a table with plastic sample cups of what looks like beer in front of them; they are writing their notes on sheets of paper, smiling and chatting with each other. He can barely conceal his disdain. “This is obviously not a scientific way to develop products,” he says. What’s more, “every single one of these individuals is white. They are going to develop products for national or international launch, and they literally cannot perceive what other demographics around the world perceive when they are tasting these products.”

Conventional sensory evaluation methods, according to Cohen, are both fundamentally flawed and fundamentally misguided. By treating the trained tasters on panels as neutral instruments, who are expected to set aside their personal preferences and achieve consensus on matters of taste, sensory science denies the clear differences between people. When tasters on a panel are “calibrated” to use a standardized language to describe their experiences, “you are literally throwing away data about the variations in perception in the broader underlying market,” Cohen says. (Experts disagree with this characterization of sensory science, but more on that later.)

In other words, Cohen accuses traditional sensory science of artificially straight-jacketing tasters in order to achieve a false objectivity. His beef with sensory science dates to his undergraduate days at Penn State, where he was one of the founders of the Tea Institute, a student group devoted to the appreciation and study of tea and tea ceremonies. A tea obsessive, Cohen was determined to find a way to determine whether a rare green tea was actually what it claimed to be, or calculate the changes to the taste of a pu-erh as it ages. A basic course in sensory science seemed to yield only unsatisfactory answers.

Everyone experiences flavor a little differently. Gastrograph looks for demographic patterns in the very places where consensus falls apart.
Then he discovered data science. In 2010, Cohen began graduate work with Professor John Yen at Penn State’s College of Information Sciences and Technology, where he built a basic A.I. platform that recorded tea information and tasting notes. The goal was a program that could tell you the kind of tea you were drinking when you entered in what you were tasting. “We achieved superhuman performance on some things in 2012,” he told me, when the system outstripped his and his peers’ ability to pinpoint a tea’s place of origin. This was the basis of the Gastrograph AI. Cohen left graduate school before completing his degree—in order to protect his intellectual property, he says—and devoted himself full-time to Analytic Flavor Systems beginning in 2013. Two members of the Tea Institute, Ryan Ahn and Aislynn van Clief, soon joined him, and now serve as the company’s director of research and creative director, respectively. Along with a data scientist, they form the core of Analytic Flavor Systems’ small team.

Rather than calibrate to a universal baseline, Gastrograph’s guiding principle is that everyone experiences flavor a little differently. The tool, then, looks for demographic patterns in the very places where consensus falls apart. The superior power of machine learning, Cohen explains, is that it can descry patterns in what traditional sensory methods dismissed as noise. Even more, Cohen says, it can locate unanticipated drivers of preference in the information that sensory science discards. “We think of flavor as an infinite-dimensional Hilbert space,” he says, referring to the mathematical concept of a complex algebraic system with an infinite number of variables. What he means is that the sensory possibilities for how a flavor may be experienced are, practically speaking, limitless. Sensory science constrains the data it collects, for instance, by limiting tasters to standard vocabularies—but an A.I. has no need to respect these limits. The idea is that Gastrograph achieves “superhuman performance” precisely by ruminating on the things we record but fail to notice.

Can a robot brain really tell us what we’ll want to eat?
AI can raise the specter of technology run amok: a post-singularity future with robots in command. Of course, AI-based technologies are already widespread, shaping many of our interactions with the digital world. Essentially, A.I. describes any system that utilizes machine learning — computational algorithms, including neural networks and natural language processing — to churn meaning from aggregations of data, finding patterns, making predictions, and, crucially, displaying the capacity for self-improvement. Whether filtering spam, identifying potential new drugs, or recommending the next show to binge-watch, AI-based systems decrease their error rate over time. They get better at giving us what we seem to want.

In order to do all this, however, A.I. need something from us in return: our data.

?☕?

You can download a free public version of the Gastrograph app for your phone. I try it out myself, hesitating for a moment when it asks permission to access my microphone and location. Ultimately, I cave and duly supply the kind of personal information that, in the age of Cambridge Analytica and ransomware, we’ve all grown more cautious about sharing: my race, ethnicity, and gender; my birthday; my first language; my hometown.

Data, after all, is the lifeblood of AI, or maybe more like its endocrine system—it’s how the system grows, matures, and develops, fills out and builds its muscles. If we want to help usher in a new era of personalized food, then, surrendering personal information is probably a necessary evil.

I pour myself a glass of a 2015 COS Pithos Bianco, a biodynamic orange wine from Sicily. I love drinking wine, and have very strong opinions about it, but when it comes to naming what I’m tasting and putting words to flavors, I fall into an abyss of uncertainty. I know what I like, but I can’t begin to tell you why. Aren’t I precisely the kind of puzzle the Gastrograph is meant to solve? Will it be A.I. that finally unmuddles my perceptions, locating the engines that drive my preferences?

 

Tasters map the contours of flavor perception by tracing the spokes corresponding to the qualities they detect, designating the intensity of each on a scale from one to five. Here, the “floral” quality of a tea has been marked at level 5, and “wet” at level 4

I take careful little sips as I fiddle with the app, making a messy cobweb of flavor notes — herbaceous, dry, mineral, astringent—guessing wildly at intensity levels. I add “marine,” and then scroll helplessly through the submenu. Is it more like winkles, or like snails, or any of the other two dozen options, from kelp to cuttlefish? Finally, I enter my own custom reference: beach sand. Overwhelmed by doubt, I abandon my half-finished review and drink my wine, un-analytically and with gusto.

If data is the lifeblood of AI, it is also its biggest liability.
Of course, the Gastrograph is not a referral engine, but the input terminal for a vast data collection system. It won’t help me figure myself out; instead, my data helps the A.I. refine its model of how people resembling me perceive the edible, quaffable world — which then, presumably, will help food companies design products to sell me. At the moment, Analytic Flavor Systems does not rely on hapless amateurs like me to populate its database. Only a small fraction of the company’s flavor database comes through the public version of the app—a few hundred reviews per month, Cohen estimates. (He tells me that it is particularly popular among a Reddit subgroup of cigar aficionados.) The primary users are trained sensory panels and product developers at food and beverage companies, who have supplied more than 200,000 reviews for the company’s main R&D database. They also have access to a database of approximately a million consumer reviews, which they use primarily to validate consumer preference predictions.

If data is the lifeblood of AI, it is also its biggest liability. It is expensive to collect, and it can be restricted in ways inconvenient to a growing A.I. enterprise. Food and beverage companies are notoriously protective, and I wonder if any of them have had concerns about their taste panel responses going into the Gastrograph database. “That’s a little controversial,” Cohen admits. “We are fairly mum on the topic. We would obviously never make a prediction for two competitors on each others’ products, but everyone’s data is going into improving the system.”

“There are a lot of untapped markets lots of white space everywhere.”
Taken as a whole, the data collected by Gastrograph reveal large-scale patterns and emergent preferences. For instance, Cohen tells me, US consumers’ growing affection for bitterness has recently begun arcing towards the sour. Evidence of this can be seen across the marketplace: craft beer drinkers making room for lambics alongside their IPAs; Greek-yogurt-eaters checking out Icelandic skyr; and in the increasing popularity of grapefruit-flavored things, a flavor that neatly combines bitter and sour.

The point, however, is not macrocosmic trend-tracking, but microcosmic customization: to predictively model the perceptions and preferences of increasingly narrow demographic clusters.  “You can enter in a flavor profile for that latte,” Cohen says, pointing to my half-finished drink, “and the system will spit out an optimization—what to change to make it even better.”  There is no platonic ideal of a latte, of course. There is only your ideal latte: the one optimized to be maximally delicious to your palate. The Gastrograph would not merely suggest that a seltzer be grapefruit flavored, for instance, but how to tweak its floral, fruity, bitter, and sour dimensions to captivate the fancy of Northeastern millennial guys, or German ladies over forty.

The Gastrograph system can’t yet deliver at that level of precision. Theoretically, however, the more tasting data it collects, the better its models, and predictions, become. A big question for the company is where that data will come from.

The food industry has long been moving in the direction of more specialized products. Howard Moskowitz, one of the pioneers of sensory science, emphasized this in his work for Prego in the 1980s. As his consumer research demonstrated, there is no one perfect spaghetti sauce; there are different spaghetti sauces for different people. The key to Prego’s success was not improving its classic tomato sauce, but in recognizing that there was a group whose preference for a chunkier style sauce was not served by the market. (If you think you’ve heard this story before, you probably have, in this Malcolm Gladwell TED talk, among other places.)

a latteAlex Fine

There is no platonic ideal of a latte. There is only your ideal latte: the one optimized to be maximally delicious to your palate

But where Moskowitz divided Americans into three groups—those who want their sauce plain, those who want it spicy, and those hungry for something chunky—Cohen imagines a much, much finer grind. AI-assisted analysis can reveal the variegated map of preferences. “There are a lot of untapped markets,” he says, “lots of white space everywhere.”

The vision is not to simply cater to underserved tranches of fussy gourmands. “It will mean better products for everybody,” Ryan Ahn, the company’s director of research and development, tells me. While he admits that the technology isn’t quite there yet, he assures me that “personalized products are on the horizon. And we hope to be the brain behind personalized products.”

???

When compared with self-driving cars or Amazon Alexa, AI’s incursions into the food business have been notably less heralded. Many of the most notable companies are using A.I. to transform the big picture of food production: the farms, feedlots, factories, and other large-scale systems that make and distribute what we eat and drink. Analytic Flavor Systems wants to apply the power of machine learning to the most intimate and personal aspect of food: flavor. So far, however, the Gastrograph has mainly been used for quality control in the craft beer industry, as a tool to track and monitor flavor consistency from batch to batch. More recently, the technology has helped companies tweak the flavor profiles of a dozen new and forthcoming products, including craft beers, flavored waters, chocolates, and salty snacks.

Ultimately, what Analytic Flavor Systems is selling is not a food or beverage: It is a descriptive picture of experience, a predictive image of desire, and a vision of a food system fragmented into niches of highly attached consumers. If the future is, as its founders say, in foods optimized to our most personal appetites, the company’s success will ultimately depend on the Gastrograph’s ability to tell food and beverage companies what you will love, the flavors that you won’t be able to live without—and to do this more accurately, efficiently, or cost-effectively than established companies.

For all Cohen’s claims that his A.I. operates outside of the limits of sensory science, data still must enter the system in categories.
But while Cohen claims his company is disrupting the foundations of sensory science, experts in the field take issue with his characterization of the discipline.

Ana María Ulloa, PhD, an anthropologist who studies sensory scientists at work, disagrees with Cohen that sensory scientists consider taste panels to be neutral instruments. “Precisely, their practice with taste panels reveals all the little nuisances that come with the evaluation of food,” she says, and their experimental techniques productively capture these uncertainties and the differences among tasters. John Hayes, PhD, a professor of food science at Penn State who once had Cohen as a student, laments his cursory understanding of the field. Sensory scientists have long grappled with the problems that Analytic Flavor Systems claims to solve, Hayes says, and have developed and tested a sophisticated and evolving set of methodological and statistical instruments to improve the quality, reliability, and functionality of the data they collect, including methods that combine descriptive evaluations with preference ratings.

“There’s a wisdom-of-the-crowds angle here that most people haven’t leveraged in sensory [science],” he concedes, “but the idea that you have to throw away calibration, or good sensory practices, to get the scale [of data] he’s talking about — I just don’t think that’s true.”

In other words, all the machine-learning capabilities in the world won’t deliver better results if the data used to train and grow the algorithm is flawed or faulty. And for all Cohen’s claims that his A.I. operates outside of the limits of sensory science, data still must enter the system in categories, which impose their own constraints. Critics of Cohen’s approach argue that an indiscriminate data gathering system doesn’t mean that groups of people won’t be excluded from its models, and that perhaps A.I. technology would be better applied to improving the quality of sensory data collected, rather than expanding the quantity.

Food and beverage flavors formulated specifically for your age and genderAlex Fine

Ultimately, what Analytic Flavor Systems is selling is not a food or beverage: It is a predictive image of desire.

It remains to be seen whether the Gastrograph can deliver on its promises. In the meantime, some major players in the investment world are betting on them. The company has attracted investment from Techstars and ZX Ventures, AB InBev’s global innovation fund. Analytic Flavor Systems is currently in the midst of a funding round seeking $2 million.

So it’s worth pausing to consider the ends to which this powerful technology is to be deployed. Is a utopian vision of hyper-personalized food all that it’s cracked up to be?

For the company’s financial backers, flavor personalization could be a potential gold mine. Eamonn Carey, managing director of Techstars Connection, an accelerator program that was an early supporter of Analytic Flavor Systems, calls personalization the “holy grail” for food and beverage companies. Just as targeted ads on Instagram or Facebook observe our online behavior to ever more precisely serve up what we seem to want, he predicts that food companies will one day match their products to our appetites, and use flavor to secure their place in our kitchens, and our hearts. “They can then sell to the consumer directly,” Carey muses, “disintermediating coffee shops and bodegas. They can sell me the perfect chocolate bar for me, maybe as a subscription service, four of them a month, tailored for my cholesterol levels.”

Better food for more data: Are we willing to make this exchange?
The bigness of big food is a problem. The vaunted efficiencies of scale that allowed the industrial food production system to achieve its productivity and cheapness have also severed the connection people want to feel to the things they eat, leaving many consumers alienated from the huge multinational brands and agribusiness enterprises that provide many of our calories. So there’s something undeniably alluring about a more personalized industrial food system, one that could take cultural and demographic differences into account, and that actually seems responsive to individual wishes. But there’s obvious logistical challenges here, challenges that none of the people I spoke with at Analytic Flavor Systems or its backers managed to convincingly explain away. If we’re currently trapped in a system that fails to serve us because its vastness can’t acknowledge our real hungers, are we any less trapped by a system that tracks us down with the promise of our perfect chocolate bar, but only by subscription?

Further, making this vision a reality will require something from us: our data, whether we contribute it directly or indirectly. Cohen demurs when I ask him about this, but it’s undeniable that we’re in the middle of a food information explosion. Whether it’s our smart fridge’s silent sensors, our orders to Amazon Alexa or Siri, or the Instagram photos we post of our dinners, there’s more information out there about the food we eat, and how we feel about it, than ever before. Analytic Flavor Systems stands to be a beneficiary of all of this data. Research groups such as IC FOODS at UC Davis — which counts Analytic Flavor Systems as a partner—are attempting to build an “internet of foods,” a standard, common set of data languages that can harness the maelstrom and turn it into computable, actionable knowledge. Tools such as the Gastrograph, predicts Matthew Lange, IC FOODS’ director, “will likely become part of the engine that drives this more ambient collection of food sensory experiences on a large scale.”

Better food for more data: Are we willing to make this exchange? I noticed that almost everyone who spoke to me about the virtues of personalized food made sure to also genuflect toward personalized nutrition—foods tailored to our peculiar metabolic needs or allergen sensitivities. Not only better tasting; also better for you! I also wonder about the presumption of universality implied by this promise of personalization. Who will get to enjoy the benefits of custom-tailored foods, and who will be left eating undifferentiated potato chips, non-specific TV dinners?

“I’d like to buy the world a Coke,” began one of the most popular advertising jingles of all time, offering a common cola beverage to unite a fractured globe. We’re all cynics jaded about capitalism’s false promises to unite us through commerce. In the Gastrograph-driven future that Analytic Flavor Systems envisions, we will each drink our own separate Cokes, exulting in a beverage optimized to be maximally delicious to our individual, irreconcilable palates.  So it’s a good time to ask: Is a marketplace of proliferating varieties targeted to our private hungers, to our intimate biological needs, really one that serves us all best?

Nadia Berenstein holds a PhD in History & Sociology of Science from the University of Pennsylvania. She is currently working on a book based on her dissertation, recounting the history of flavor science and the flavor industry in the United States beginning in the mid-nineteenth century. Her writing has appeared in Food & Wine, Lucky Peach, among other places. She has been a guest on Science Friday, The Nod, and Gastropod, and she was an advisor for the Museum of Food and Drink’s inaugural exhibition, "Flavor: Making It and Faking It." You can read more about her research on her blog, "Flavor Added."