I recently ran across a very interesting article in Forbes titled "Big Data vs. Hayek" that made some observations about how companies were
using big data in ways that might at first seem anti-hayekian. For
example:
“They found the centralized algorithms were outperforming decentralized local manager knowledge consistently, and profits went up….It’s not just apartment owners either. So-called revenue management software is also widely used by airlines and hotels….It’s also notable that Uber uses an algorithm to set prices rather than letting drivers set them”
“What’s interesting about such centralized, algorithmic approach to price setting is how un-Hayekian it is. In particular, I’m thinking of Hayek’s essay “The Use of Knowledge in Society” where he makes the case for the use of decentralized markets to utilized widely dispersed information to make choices.”
I think of course, yes corporations do tend to act and behave in ways that are anti-hayekian. (see this EconTalk podcast Coase, externalities, the firm, and the state of economics):
"If capitalism and markets and prices, the Hayekian system of communicating information via price signals, if it works so well, why do firms exist? Because firms are almost by definition top down rather than bottom up. They use command and control rather than purchases within the firm, although there are exceptions to that. Some firms do use price signals for their decision-making inside the firm. But many firms do not. Their decisions are made not by prices but by fiat, by decisions on the top. "
Firms exist in general because they seem to find that the 'costs' of using the price system to allocate resources internally can tend to outweigh the 'benefits.' There is a lot more to Coase's theory of the firm, but given this basic premise it is not surprising that in many cases firms will choose to use a more centralized approach than a decentralized one, and in certain cases Big Data, the internet of things (IoT), and modern computing power and analytics can toy with those tradeoffs at the margin. So in some cases we might find big data incenting more centralization and less in others. The Forbes article actually makes this point in the conclusion:
"Instead, these marketplaces provide ratings and other informational systems that help buyers and sellers overcome information asymmetries, and are helping markets function better and even more dispersed than before..."
It could be the case that when it comes to internally facing resource allocation decisions, certain practices within firms seem to have more of a decentralizing flavor, while big data is forcing the invisible hand on the customer facing decisions by allowing firms more than ever to leverage very granular bits of data related to each customer's particular knowledge of circumstances, time, and place. The internet of things is one example. As I have mentioned before, in the agriculture sector, local knowledge can be key in big data applications (See Big Data: Causality and Local Expertise are Key in Agronomic Applications).
Dan Frieberg points out some very important things to think about when
it comes to using agronomic data in a Corn and Soybean Digest article "Data Decisions: Meaningful data analysis involves agronomic common sense, local expertise."
The following quote from the article is telling:
"big data analytics is not the crystal ball that removes local
context. Rather, the power of big data analytics is handing the crystal
ball to advisors that have local context"
Also, when it comes to product line offerings in terms of seed choices and hybrid seed technology, big data and analytics is forcing companies to offer more options tied to local knowledge (see What does the farmer say...about seed choices? (Channeling Hayek))
"The choice of what crops we should grow, how they should be produced in
terms of management practices and technology, and ultimately the variety
of foods we choose to consume is an example of what economists refer to
as the knowledge problem. While it might be possible to patent a given
trait or hybrid, no one company can get too firm a grasp on this
knowledge problem, regardless of their market share in the seed industry
today. (not to mention, no government agency would have sufficient knowledge
either). Given the vast array of considerations in seed choice and
management practices, there is always going to be an incentive for some
supplier to cater to the unique needs of individual producers, as
advances in genomics and technology drive production not farm by farm or
acre by acre but inch by inch."
I agree 100% with the conclusion of the Forbes article:
"we should not get ahead of ourselves in declaring the death of decentralized knowledge and decision-making."
Related:
Farm Link and the Rise of Data Science in Agriculture
