The Limits of Nudges and the Role of Experiments in Applied Behavioral Economics

In a recent article in Nature, Evidence from a statewide vaccination RCT, authors found that eight different nudges previously shown to be effective for encouraging flu and COVID vaccination failed to show impact when tested on more reluctant populations. I think a knee jerk reaction is that maybe nudges aren't effective ways to increase vaccination after all. But that completely misses a very important aspect of nudges. Nudges work in most cases because humans can be sensitive to context. And applied behavioral design processes work to understand this context and test the impact of interventions to know if they are effective in a given context. At the highest level I think this paper is less about the ineffectiveness of nudges per say and more about the important role of changing context on behavior. 

I'd like to try to unpack more by focusing on the following: 

• The importance of testing. You can’t blindly chuck nudges over the fence at your customers and simply assume they will be effective just because they worked in prior published studies or in other businesses. In this paper they tested 8 different nudges. If they had just scaled any or all of these without testing we would not have learned anything about effectiveness or the other lessons that follow relating to why they may not have worked. And vice versa - just because something failed to replicate in one context doesn't invalidate prior work or imply it won't work in yours. We just know findings are not generalizable across all contexts. The only way to really know about your context is to test. That is part of the value of business experiments.

• Context matters. In the paper they discussed important differences in context between late stage COVID vaccination and vaccination earlier in the pandemic, as well as differences between flu and COVID.

• The utility of behavioral personas and behavioral mapping to guide our thinking about why a given nudge may work or not. To take context a bit further, authors discussed differences in populations (age) and different challenges to flu vs COVID vaccinations and the differential impact related to how both logistical and psychological barriers may have been addressed in different populations and different contexts with different designs. All of these are things that we can point to or think about in the framework of behavioral mapping. Other issues related to 1) different kinds of hesitancy and changing norms over time, 2) whether some participants may have already been vaccinated (and not mentioned perhaps how prior infection may have changed the sense of effectiveness or urgency). These things may relate more to the kinds of personas that any given nudge may speak to. Although the paper doesn't discuss behavioral mapping or developing personas their utility here seems palpable.

• Behavioral design frameworks. Additionally, authors discussed the impact of things like message saturation and novelty effects in addition to timing. These are things that I tend to think about in the context of Stephen Wendel’s CREATE action funnel as a design framework that speaks to issues like the importance of Cue and Timing. (Actually every aspect of CREATE speaks to almost all of the aspects of this messaging in some way).

• The importance of operationalizing applied behavioral science through repeatable iterative cycles of learning. Even if one constructed behavioral maps and personas in the design of these nudges, the findings in this paper (and in many instances where we leverage experiments to test impact) dictate that we go back and revise our maps and personas based on learnings like these.

There has also been some recent discussion about the failure of nudges because they focus too much on individual behavioral (i-frame) vs. larger systemic issues  (s-frame). It seems to me that best practices in the 'diagnosis' phase of behavioral design process would be helpful in both of these areas if the behavioral lens is widened to include deeper thinking about the broader system (s-frame). As discussed in The Consitution of Knowledge: A Defence of Truth Jonathan Rouch discusses the challenges of changing behavior when beliefs and identity become tightly braided together. Sometimes people first have to be moved to a 'persuadable place emotionally' and their 'personal opinions, political identities, and peer group norms' have to be 'nudged and cajoled simultaneously, which is a long slow process.' To quote Jim Manzi, you can't test your way out of a bad strategy. It does not mean that we should give up on leveraging applied behavioral science to make a positive change in society, but it does make understanding of the larger ecosystem in the implementation of nudges all the more critical. 

As discussed in a recent article in The Behavioral Scientist:

"Our efforts at this stage will determine whether the field matures in a systematic and stable manner, or grows wildly and erratically. Unless we take stock of the science, the practice, and the mechanisms that we can put into place to align the two, we will run the danger of the promise of behavioral science being an illusion for many—not because the science itself was faulty, but because we did not successfully develop a science for using the science." 

The authors follow with 6 guidelines echoing some of the above sentiments above that are well worth reading. 

Reference: 

Rabb, N., Swindal, M., Glick, D. et al. Evidence from a statewide vaccination RCT shows the limits of nudges. Nature 604, E1–E7 (2022). https://doi.org/10.1038/s41586-022-04526-2

Chater, Nick and Loewenstein, George F., The i-Frame and the s-Frame: How Focusing on Individual-Level Solutions Has Led Behavioral Public Policy Astray (March 1, 2022). Available at SSRN: https://ssrn.com/abstract=4046264 or http://dx.doi.org/10.2139/ssrn.4046264

Agricultural Economics in the Healthcare Space

During the pandemic, it wasn't too uncommon to hear the criticism that economists should stay in their lane when it comes to issues related to health. So I thought I would write a short piece discussing what role I have had as an applied (agricultural) economist working in the healthcare space for almost a decade now. 

Economics is the study of people's choices and how they are made compatible. At a high level, agricultural economics focuses on choices related to food, fiber, natural resources, and energy production and consumption. This makes the intersection of food, health, and the environment an interesting space in agricultural economics. 

How do choices in this space impact health? What factors lead individuals to make healthy choices? In graduate school I specifically focused on why people seem to pick and choose their science and the role of evidence in food choices and attitudes toward food technology. What is the role of information and disinformation in the formation of consumer preferences and the choices they make? How can we design better policies, products, services, interventions, or choice architectures for better outcomes? How can we communicate science and risk more effectively? And, what are the best approaches in experimental design and causal inference to measure the impact in these areas? How do we bring this all together to make better decisions as individuals, business leaders, and as a society? At an applied level, which is where I work, this is not so much about making a novel contribution to the literature or advancing the field as much as it is about implementation - applying the principals of economics to develop solutions or provide frameworks to solve or better understand questions and problems in this space.

This line of reasoning has value not just in the context of food choices but for a myriad of behaviors related to healthcare at both the patient and provider level. From a business perspective, this is about how to identify opportunities to move resources from a lower to a higher valued use, and how we monetize behavior change. Of course applying this economic lens also requires bringing an ethical perspective to the table as well, which is important when we consider all of the tradeoffs involved in human decision making. 

When we are faced with wicked problems that may have alternative solutions, we can't just jump directly form the science to a cure, better policy, or product or service.  We learned from the pandemic the difference between having a vaccine and having people get vaccinated. At the end of the day there are no solutions really, only tradeoffs, and we need a framework for understanding those tradeoffs so we can make better decisions about food and health. That is squarely in the lane of theoretical and applied economists.

Related Posts and Readings

Why Study Economics / Applied Economics 

The Convergence of AI, Life Sciences, and Healthcare

The Economics of Innovation in Biopharma

Science Communication for Business and Non-technical Audiences 

The Value of Business Experiments

Statistics is a way of thinking not a toolbox

Causal Decision Making with Non-Causal Models

Rational Irrationality and Behavioral Economic Frameworks for Combating Vaccine Hesitancy 

Consumer Perceptions, Misinformation, and Vaccine Hesitancy

Using Social Network Analysis to Understand the Influence of Social Harassment Costs and Preferences Toward Biotechnology

Fat Tails, The Precautionary Principle, and GMOs

Innovation, Disruption and Low(er) Carbon Beef

Examining Changes in Healthy Days After Health Coaching. Cole, S., Zbikowski, S. M., Renda, A., Wallace, A., Dobbins, J. M., & Bogard, M. American Journal of Health Promotion. (2018)

Intrapersonal Variation in Goal Setting and Achievement in Health Coaching: Cross-Sectional Retrospective Analysis. Wallace A.M., Bogard M.T., Zbikowski S.M. J Med Internet Res 2018;20(1):e32

The Economics of Innovation in Biopharma

This podcast touches on the lack of innovation in pharma and criticism about outsourcing innovation. Do these criticisms ignore recent technological advances in biotech (and the convergence of AI and genomics) that have reduced the minimum efficient scale in drug discovery creating new opportunities for startups, small firms, and scientist entrepreneurs? When we think of therapeutics as dispensing knowledge packed into a capsule or syringe, knowledge that has properties of both a private and public good (i.e. non-rival and partially excludable) scientist entrepreneurs are better incentivized and able to capture greater value from their discoveries in a venture capital funded startup environment than a larger institution like pharmaceutical companies or universities (even with Bayh-Dole Act). Drug discovery is risky, but by combining option value and discovery of new information with staged investment VC firms can discover positive NPV projects that would otherwise be rejected under conventional financing models. The combination of technological change, the economics of knowledge, and venture capital seems to reduce the comparative advantage of innovating 'in-house.' Maybe it is the case that large pharmaceutical firms have more of a comparative advantage navigating the valley of death that lies between a discovery and a cure by focusing on the regulatory approvals and marketing efforts necessary to deliver those products than they have in drug discovery?

The Convergence of AI, Life Sciences, and Healthcare

Several years ago I was writing about the convergence of AI and genomics in agriculture:

"The disruptions of new technology, big data and genomics (applications like FieldScripts, ACRES, MyJohnDeere or the new concept Kinze planters that switch hybrids on the go etc.) will require the market to continue to offer a range of choices in seeds and genetics to tailor to each producer's circumstances of time and place." (1)

We have also seen a similar convergence in healthcare:

"A series of breakthroughs in medical science and information technology are triggering a convergence between the healthcare industry and the life sciences industry, a convergence that will quickly lead to more intimate—and interactive—relationships among people, their doctors, and biopharmaceutical companies."  (2)

This excellent segment on WBUR just a few years later picks up on the same themes:

Nobel Laureate and MIT Institute professor Phil Sharp has an even broader vision of this convergence: It’s not just computer science and biology that are converging, but engineering, physics, material science and agriculture too, he says.

“Life science is part of all of those processes and bringing physicists and engineering and information technology together to integrate life science with the translation to solving those problems is what convergence is about,” Sharp says. “It'll be decades of exciting science and exciting technology.” (3)

There are a number of parallels I want to discuss below including outcomes and value based pricing, precision medicine and precision agriculture, venture capital and digital solutions, and how these trends are leading to products and solutions that can address some of society's biggest problems like healthcare quality and cost, social determinants of health, and climate change.

Outcomes and Value Based Pricing

Due to this convergence, better data and technology are creating new opportunities. Health insurance companies, healthcare providers, and seed companies are entering into value based contracts where payments are based on outcomes and quality. 

In healthcare:

"By leveraging appropriate software tools, big data is informing the movement toward value-based healthcare and is opening the door to remarkable advancements, even while reducing costs. " (4)

"Value-based healthcare is a healthcare delivery model in which providers, including hospitals and physicians, are paid based on patient health outcomes. Under value-based care agreements, providers are rewarded for helping patients improve their health, reduce the effects and incidence of chronic disease, and live healthier lives in an evidence-based way." (5)

(See below or  https://healthinformatics.uic.edu/blog/shift-from-volume-based-care-to-value-based-care/ for an excellent infographic explaining this promising shift in healthcare)

In food and agriculture we are seeing risk sharing and outcomes based pricing contracts as well:

"...executives are touting their new pricing model, outcome-based pricing, as the potential pricing paradigm of the future. The model involves Bayer setting an expected yield outcome for a product or seed, based on a farm's data and history stored on the company's digital ag platform, FieldView, as well as the company's own research on their products. If a farmer's final yield falls below that expected value, the company will rebate a certain portion of the original price of the product. If the yield instead surpasses the initial set value, the farmer shares a pre-agreed portion of that additional income with the company." (6)

Precision Medicine and Precision Agriculture

Instead of one size fits all best practices for seed, pest management, tillage, and nutrient management recommendations driven by research from university and industry trials, growers can get individually customized prescriptions, not just at the farm or field level, but within field and moving closer and closer to the row foot level for some decisions. The combination of advanced genomics with big data generated from precision agricultural applications (remote sensing, IoT, automated steering, GPS/GIS) makes one size fits all obsolete. 

As I quoted previously: 

"That's also why the market has driven companies to treat hybrid selection like a 'big data' problem and they are developing multivariate recommender systems as tools to assist in this (like ACRES and FieldScripts). The market's response to each individual producer's unique circumstances of time and place also ensures continued diversity of crop genetics planted. There are numerous margins that growers look at when optimizing their seed choices and it will require a number of firms and seed choices to meet these needs as the industry's focus moves from the farm and field level to the data gathered by the row foot with each pass over the field." (1)

Similarly, in healthcare, the golden age of medicine driven by the 'omics' revolution and big data will allow us to move away from one size fits all generalizations of research and medicine allowing us to "tailor medical treatment to the specific characteristics of each patient involving the ability to classify individuals into subpopulations that are uniquely susceptible to a specific treatment, sparing expense and side effects and is derived from doubts on the results of subgroup analyses and on non responders in clinical trials" (7)

"Health systems will have to go rapidly from a one-size-fits-all model of treatment to a more customized model, which still uses mass-manufactured but where treatments are selected for patients based on specific biomarkers," Joshi said. "But we can now see the next advance in personalized medicine potentially going even further, something much more personalized, like a tailor-made suit...."Big data and advances in our understanding of genomics are providing us with the footholds into establishing and understanding, for the first time ever, the causal genetic factors that help us manage that golden triangle of treatment: the right target, the right chemistry, and the right patient." (2)

Venture Capital and Digital Platforms and Solutions

Monsanto's (now Bayer Crop Science) acquisition of The Climate Corporation occurred about the same time I was penning my first post on this convergence, and was the first major move in industry that solidified these potential synergies in my mind at least. This convergence has drawn the interest and has been fueled by a number of startups and venture capital firms. Farmer's Business Network (FBN) seems to be positioning itself as a disruptor, like the Amazon of agribusiness providing a platform that includes everything from purchasing inputs, crop analytics, finance and marketing, and more direct access to genetics. In the livestock space, companies like AAD (Advanced Animal Diagnostics) and Connecterra are building tools and services analogous to a Fitbit for cows. Body Surface Translations (BST) is a company whose image processing technology has targeted both problems in animal and human health.  Tim Hammerich (the Future of Agriculture) and Sarah Nolet (AgTech So What?, AgThentic, Tenacious Ventures) have weekly discussions with innovators pioneering new solutions in this space covering a range of topics including automated irrigations systems, blockchain, regenerative agriculture, carbon sequestration and a range of companies from startups to larger players including Wal-Mart and Coca-Cola. Where Food Comes From is leveraging QR codes and mobile technology paired with their source verification processes to connect consumers to information about the people and processes behind the food they consume.  IN10T is a digitally powered data driven company helping bridge the gaps between innovations and real world application of these technologies. Venture capital firm Foresite Capital even leverages data science to drive their investment strategy in therapeutics, diagnostics, and devices. This includes digital health apps like mindstrong which is leveraging AI for better diagnosis, monitoring, and treatment of behavioral health conditions and everlywell focused on actionable healthcare diagnostics and health engagement. Evidation is a company that leverages data from digital devices and sensors capturing, quantifying, and analyzing behavior, or mapping the 'behaviorome' in the context of human health (8). This is just a tiny survey of companies and products that I have encountered in just the last few years.

Addressing Society's Bigger Problems

This convergence is allowing us to address problems in healthcare like quality, cost, access and health equity. When it comes to the food we eat, AI, technology, and genomics is providing us the tools to combat issues like climate change, water quality, nutrition, safety, equity, and access. 

It's obvious when you look at the big picture, this convergence is leading to progress that is both complimentary and synergistic across a range of industries related to food and healthcare. Better food and a healthier environment and planet  led to better health outcomes. Healthcare payers and providers are realizing the importance of these issues in healthcare. Each is separately addressing key social determinants of health in ways that were not possible before:

"During the past several decades, it has become increasingly apparent that a person’s “health” is influenced by many more factors than health care alone. These other determinants are defined by the conditions and environment in which people are born, grow, live, work, and age, reaching beyond just what the delivery of acute care services can influence. These “social determinants of health” result in billions of dollars of additional costs annually. By working to mitigate the negative impacts of these factors, significant benefits can be achieved that improve both access and outcomes for individuals and lower overall costs." (9)

As I stated several years ago:

"as big data drives more diversity into every seed planted in every acre across every field, we may possibly begin to mitigate some of the risks and concerns traditionally associated with monoculture. So it is true, when you look across row after row and see only corn, you might technically call it 'monoculture' but it's not your grandparent's monoculture." 

As a result of the convergence of AI and life sciences, it's not your grandparent's healthcare either. 

References and Related Readings:

(1) Monoculture vs. the Convergence of Big Data and Genomics. Matt Bogard. October 13, 2017. https://www.linkedin.com/pulse/monoculture-vs-convergence-big-data-genomics-matt-bogard/ (previously published as: Big Data + Genomics != Your Grandparent's Monoculture. Economic Sense. December 22, 2014. http://ageconomist.blogspot.com/2014/12/big-data-genomics-your-grandparents.html

(2) Big Data Gets Personal as Healthcare and Life Sciences Converge. By Bob Evans, Senior Vice President, Oracle.  https://www.oracle.com/industries/oracle-voice/big-data-gets-personal.html

(3) Next Chapter For Biotech? Many Say 'Convergence' With Data Science. WBUR. NPR. Bioboom June 8, 2018. https://wbur.fm/2MaaMkA

(4) Healthcare Big Data and the Promise of Value-Based Car. NEJM Catalyst. Brief Article. Jan 1, 2018

(5) What Is Value-Based Healthcare?. NEJM Catalyst. Brief Article. Jan 1, 2017

(6) Q&A With Bayer on Outcome-Based Pricing. By Emily Unglesbee. DTN Progressive Farmer. 10/2/2019 

(7) Capurso L. Evidence-based medicine vs medicina personalizzata [Evidence-based medicine vs personalized medicine.]. Recenti Prog Med. 2018 Jan;109(1):10-14. Italian. doi: 10.1701/2848.28748. PMID: 29451516. 

(8) Why Foresite Capital is Betting Big on the Convergence of AI and Biotech. August 23, 2018. https://soundcloud.com/levine-media-group/why-foresite-capital-is-betting-big-on-the-convergence-of-ai-and-biotech   Check out their current portfolio of investments: https://www.foresitecapital.com/portfolio/ 

(9) Beyond the Boundaries of Health Care: Addressing Social Issues https://www.ahip.org/beyond-the-boundaries-of-health-care-addressing-social-issues/ 

Related: 

What does the farmer say...about seed choices? (Channeling Hayek) http://ageconomist.blogspot.com/2013/12/what-does-farmer-say-about-seed-choices.html 

Big Data: Causality and Local Expertise Are Key in Agronomic Applications. http://econometricsense.blogspot.com/2014/05/big-data-think-global-act-local-when-it.html

Modern Sustainable Agriculture Annotated Bibliography. http://ageconomist.blogspot.com/2011/02/modern-sustainable-agriculture.html

University of Illinois at Chicago