Dead reckoning, maps, and errors

In my last post I introduced dead reckoning as used during the exploration explosion. In this post I will describe the errors these explorers (Dias, Columbus, da Gama, etc.) typically encountered in dead reckoning (DR) when sailing on the oceans, and why dead reckoning could be usefully accurate despite the fact that trying to map those dead reckoning directions onto a normal map would be very inaccurate.

To get a taste of the issue, first consider the following abstract navigation problem -- hiking in foggy hills:

1. There are only two useful landmarks, 1F (the origin or "first fix") and 2F (the destination or "second fix").
2. It’s very foggy, so you have no way to use the hills as recognizable features. But the dead reckoning directions are of sufficient accuracy to get you within sight of landmark 2F. (For simplicity assume 100% accuracy).
3. You don’t know and can’t measure hill slope angles. Indeed there are only two things the hikers can measure: (a) magnetic compass direction, and (b) distance actually walked. Observe that this is not distance as the crow flies, nor is it distance projected onto the horizontal plane. If a hill happens to be a pyramid, and you happen to be walking straight up it (and thus walking up the hypotenuse of a triangle), the distance measured is the length of the hypotenuse, not the length of the horizontal leg of that triangle.
4. The first person who discovered 2F, starting from 1F, recorded dead reckoning directions to there and back as a sequence of tuples { direction, speed, time }.

We can draw a useful head-to-tail diagram of these directions on a piece of paper. But we can’t use these directions to figure out the distance as the crow flies between 1F and 2F, because we don’t know the slopes of the hills traversed. And for the purposes of our loose analogy to long-distance ocean navigation, our hikes are short and could be in all steep terrain or all flat, so that over the course of our hike the slopes don’t converge on a knowable average.

Since we have insufficient information to determine "crow flight" distances, we don’t have enough information to accurately draw our dead reckoning itinerary on maps as we know them (i.e. Ptolemaic maps). Yet such faithfully recorded directions are sufficient to get any hiker (who can also exactly measure bearings and distances) from 1F to 2F and back.

Most maps as we know them – Ptolemaic maps -- are projections from a sphere to a Euclidean plane based on lines of latitude and longitude where lines of longitude converge at the celestial poles. Latitude is determined by measuring the altitude of a celestial object, and latitude is also ultimately defined by what navigators call the celestial sphere (although by "Ptolemaic map" I will refer to any map that shows actual earth surface distances proportionately on the map, i.e. "to scale"). There are also non-Ptolemaic maps, for example subway maps, which show the topological relationships between entities but not proportional distances. This chart of the kind Zheng He may have used, or was drawn using information from those or similar voyages, was of such a topological nature (the west coast of India is along the top and the east coast of Africa is along the bottom):

 

A set of dead reckoning directions can be diagrammed.  But although it contains more information than a subway map, it doesn’t contain enough information to plot on a Ptolemaic map. Thus like a subway map this dead reckoning "space" cannot be accurately projected, or "mapped" in mathematical terminology, onto a normal (Ptolemaic) map without further information.

A subway map is in no way "to scale": the distances on are not proportional to any measured values.  By contrast a dead reckoning map can be drawn "to scale" in its own distinct Euclidan plane.  But not only cannot this dead reckoning space without further information be accurately projected (i.e. projected with proportions intact or "to scale") onto a Ptolemaic map, but two different dead reckoning itineraries drawn on a  Euclidean plane will also generally be in error relative to each other, as I will describe below.  And now to the central point I want to get across in this article: these two kinds of errors -- from trying to Ptolemaically map a dead reckoning itinerary on the one hand and between two dead reckoning itineraries on the other hand -- are very different.  They are quite distinct in kind and usually produce errors of very different magnitudes.

The unknown values ignored in a dead reckoning itinerary, analogous to the hill slopes in the scenario above, can be any spatially variable but temporally constant distances, directions, or vectors that are unknown to the navigators writing and following the directions. The three most important spatially variable but temporally constant sets of vectors generally unknown to or ignored by dead reckoners on ocean and sea voyages from the 13^th century through the era of the exploration explosion were were magnetic variation (shown below as green arrows), current (red arrows), and curvature of the earth (ignored in this post, but the same argument applies). Since these temporally constant but spatially variable factors (analogous to the slopes of our foggy hills) were unknown or ignored, they had no way to map such pure dead reckoning directions onto a Ptolemaic map. The information they rigorously measured and recorded for the purposes of dead reckoning was insufficient for that purpose. Yet that information was sufficient to enable navigators to retrace their steps (to get back on course if blown off course) or follow a previously recorded dead reckoning itinerary (or a nearby course, as I'll show below) with usefully small error.

Temporally constant but spatially variable vectors shown on a diagram.  Only the dead reckoning (DR) vectors are shown added head-to-tail, since these are all the dead reckoning navigator  in the exploration explosion era usually measured. The vectors shown here are magnetic variation (green) and current (red). Since these vectors were unknown, dead reckoning directions could not be accurately plotted on a Ptolemaic map. Curvature of the earth, not shown here, is also temporally constant and can thus also be ignored for the purposes of dead reckoning.

However some kinds of dead reckoning errors were due to unknowns variables that changed over time. These produced small but cumulative errors in dead reckoning even for the purposes of specifying repeatable directions. Errors in measuring bearing, speed, and time were of this nature. Externally, different winds required different tacking angles, creating "leeway", where the boat moves not straight forward but at an angle. If the directions don’t account for this, or account for it imperfectly, there will necessarily be a cumulative error. It was thus important to "fix" on landmarks or soundings. The more accuracy needed (such as when approaching shorelines, much more hazardous than open-ocean sailing), the more often fixes had to be taken. I hope to say more about fixes and temporally variable errors in future posts. This post is about dead reckoning between two fixes and errors that vary spatially but can be reasonably treated as constant in time.

A dead reckoning diagram made on a chart, with "fixes" or adjustments (dashed line)s to a landmark or sounding (yellow "X") diagrammed on the chart. The start and end of points of the voyage are also landmarks, so there is also a fix for the final landmark. Note that the chart still does not have to be Ptolemaic for this purpose -- the fixes need not be shown with proportionally correct distances to each other. Indeed the Zheng He era chart above is roughly in this form, with only one crude dead reckoning vector between each fix: it labels each arc with a crude time or distance estimate along with a (much more accurate) bearing estimate, but like a subway map it doesn't care about showing distances as proportional.

When sailing over continental shelves, European mariners (and sometimes Chinese mariners) of that era took "soundings" that measured depth and sampled the bottom soil, creating a unique signature of { depth, soil type} that functioned like landmarks but on open ocean. Soundings could be taken when sailing over the relatively shallow areas of continental shelves. As you can see, most parts of the oceans are too deep for this, but most shorelines are fronted by at least a few miles of soundable shelf, and sometimes hundreds of miles. Soundings were very useful for navigating in clouds, fog, and at night far enough away from the shore to avoid the hazards of very shallow water, yet close enough for the water to be shallow enough to sound. Pilots that used soundings thus had a set of "landmarks" for fixing their dead reckoning directions that allowed them to avoid hazardous navigation too close to land.

Notice that these kinds of fixes still do not give Ptolemaic coordinates -- they simply map or "fix" a particular point in our dead reckoning "space" to a particular point on the earth's surface of unknown Ptolemaic (celestial) coordinates, and indeed of unknown distances relative to other fixes.

(Side note -- explorers between Cao and Magellan usually could not get a celestial "fix" on a rolling deck of sufficient accuracy to be useful, i.e. more accurate than their dead reckoning -- and even in the case of Magellan this was only useful because there was nothing better, dead reckoning errors having accumulated to very high levels by the time they were in the mid-Pacific.  So like them we will have to ignore this way, both more ancient and more modern, but generally unused during the exploration explosion, of correcting DR errors at sea).

It's all fine and good for dead reckoning to provide, as shown above, repeatable directions to a destination, despite being Ptolemaically unmappable, when the same itinerary is exactly repeated.  But the best itinerary over the oceans depends on the wind.  These winds vary, and the early explorers of new oceans searched for the best courses and seasons in order to catch the best winds.  So the early explorers usually did not exactly repeat dead reckonings recorded on prior voyages.  They usually took courses a few hundred miles away from the prior voyages' itinerary in order to catch more favorable winds.  So the question arises: if the navigator adjusts his course by a few hundred miles, roughly what amount of resulting error should the navigator generally expect.

(BTW, it us  important to note that dead reckoning directions, while they did not have to account for currents, magnetic variation, and the curvature of the earth, for the reasons given in this article, did have to account for variations in winds and the related leeway from tacking, since these reasons do not apply to vectors with substantial temporal variability.  So we assume, as the navigators themselves probably did in some fashion, that the velocity vectors in our dead reckoning itineraries aren't strictly those measured, but are those measurements adjusted for variations in wind).

To reiterate the most important point: this is a different question than the question of what the error is when plotted on a normal map.  Historians trying to recreate these voyages, in order to figure out where their landfalls were, or plot them on maps, or to estimate what navigational acccuracy of European navigators achieved in that era, usually haven't understood this crucial distinction. Indeed, because currents and magnetic variation don't in most places in the open ocean change in extreme or sudden ways, the resulting errors in dead reckoning navigation tended to be much smaller than the errors when plotting the dead reckoning directions on a Ptolemaic map. If you can scrutinize some more complicated diagrams I can demonstrate this by example here. First consider two dead reckoning itineraries, unadjusted for current and magnetic variation and thus plotted non-Ptolemaically:

Black = DR velocity in given time period

Red = average current velocity in given time period

Green = average magnetic variation in given time period

A, B = Two different DR itineraries as recorded (i.e. not adjusted for unknown magnetic variation and current). B has different first and third leg plus different currents on last two legs (only DR measurements added head-to-tail) – navigator would not actually plot these on a chart of geographic location, or at least would not consider such plotting accurate.

1F, 2F = first fix, next fix (same in each case, but their geographical location doesn’t need to be known)

For simplicity I am treating magnetic variation as uniform and spatially varying only the current, but the same argument I make here applies even more strongly to magnetic variation (and even more strongly to curvature of the earth, which can be treated as another set of vectors).  The second fix (2F) has a question mark in front of it to indicate that the second itinerary (B) won't actually arrive at the same spot as A arrives at -- due to the different currents it encounters, it will arrive at a different spot.  We assume, as was usually the case out of sight of shore, that our early explorer doesn't know the current.  But the explorer did want to know, as historians want to know: roughly how large can such errors in typical oceans be expected to be?  To demonstrate the mathematics of this, I've created a Ptolemaic map of the itineraries (dashed lines) by adding in the currents and magnetic variations head-to-tail.  I've also superimposed the original non-Ptolemaic diagram (just the dead reckoning vectors added up) to show the much larger error that occurs when trying to project that onto a Ptolemaic map.

A‘, B’ = A and B adjusted to show difference in geographic location (all vectors added head-to-tail). The navigator in Columbus’ day could not usally compute these, since he typically did not know the current and magnetic variation values.

NA, NB = net effect of spatially variable but temporally constant current on geographic (i.e. Ptolemaic or celestial) location. Error if unadjusted itineraries Ptolemaically mapped. Separate red arrow diagram shows the same net effect of the two separate sets of currents.

Dashed blue line next to 2F = actual navigator’s error of two DR itineraries against each other when neither set of itineraries adjusts for current or magnetic variation. The next fix lies somewhere on this line, assuming no other errors.

(BTW if you can copy and paste arrows it's easy to make your own examples).

As you can see, the errors (solid blue lines labeled NA and NB) from trying to superimpose the non-Ptolemaic dead reckoning itineraries (solid lines) on the Ptolemaic map are much larger than the actual error (dashed blue line labeled 2F) that occurs from following itinerary A instead of B or vice versa (shown on dashed lines when adjusted for current.  The magnetic variation is held constant, but the same argument applies to that, and to the curvature of the earth.

Note that the error in locating our second fix 2F is simply the same as the difference between the two separately added sets of current vectors:

It would be instructive to create a computer simulation of this which plugs in actual values (which we now know in excrutiating detail) for current, magnetic variation, and curvature of the earth.

Dead reckoning and the exploration explosion

Navigation is the art or science of combining information and reducing error to keep oneself on, or return oneself to, a route that will get you where you want to go. Note what I did not say here. Navigation is not necessarily the art or science of locating where you are. While answering the latter question – i.e. locating oneself in a Euclidean space, or a space reasonably projectable onto a Euclidean space – can usefully solve the navigation problem, figuring out such a location often requires different, and often more, information than you need to answer the questions of how to stay on or return to your desired route. And indeed this is what dead reckoning does – it gets you where you want to go with different information than what you would need to draw or find yourself on a normal map. I hope to explain more about this important incompatibility between the pilots’ and cosmographers’ systems during most of the age of exploration in a future post, but for now I will give an overview of the historical development of dead reckoning.

Between Italy of the late 13^th century and the advent of GPS, dead reckoning formed the basis of most modern navigation. Dead reckoning was in particular the primary method of navigation used during the exploration explosion of the late 15^th and early 16^th centuries – the startlingly unprecedented voyages across unknown oceans of Dias, da Gama, Columbus, Magellan, and so on.

Dead reckoning is based on a sequence of vectors. Each vector consists of two essential pieces of information: direction and distance. Distance is typically calculated from time and speed, so each vector typically consists of the tuple {direction, time, speed}. With only speed and time, we have only a scalar distance value – it could be in any direction. With time but not speed, or speed but not time, we don’t have enough information to determine the distance covered.

From the start of a voyage to the last docking at the home port, dead reckoning was a strict regimen that never stopped: day and night, in calm and in storm, its measurement, recording, and diagramming protocols were rigorously followed.

Measuring or estimating the speed of a ship was a craft mystery the nature of which is still debated today, so I’ll skip over that and focus on the two more straightforward innovations in measurement, both of which occurred in or reached Italy and were first combined there in the 13^th century: in measuring direction and in measuring time.

For measuring time mariners used the sand glass, invented in Western Europe during that same century. I have discussed this invention here. A strict regimen of turning the glasses was kept non-stop throughout a voyage.

For measuring direction, the ships of the exploration explosion typically had at least two magnetic compasses, usually built into the ship to maintain a fixed orientation with the ship. Typically one compass was used by the helmsman, in charge of steering the ship, and the other by the pilot, in charge of ordering both the sail rigging and the general direction for the helmsman to keep.

The magnetic compass was probably first invented in China, used first for feng shui and then for navigation by the early 12^th century. Somehow, without any recorded intermediaries, it appears in the writings of authors in the region of the English Channel in the late 12^th century where it was quite likely being used for navigation in that often cloudy region. Its first use in Italy was associated with the then-thriving port city of Amalfi. As both Amalfi and the English Channel were at the time controlled by the Normans, this suggests to me either a Norman innovation, or arrival via Norse trade connections to the Orient via Russia combined with now unknown Chinese trade routes. This is conjectural. Neither the Norse sagas nor writings about the Normans during earlier periods mention a magnetic compass, nor do Arab sources mention it until the late 13^th century in the Mediterranean. In any case, it is the Italians who made the magnetic compass part of a rigorous system of dead reckoning.

Green dots indicate, in the case of northern Europe, the location of authors who mention use of the magnetic compass for navigation in the late 12th and 13th centuries, and for Italy, the traditional Italian association of the invention of the compass with Amalfi in the 13th century. Red indicates areas controlled by the Normans.

A dead reckoning itinerary can be specified as a sequence of tuples { direction, speed, time }. It can be drawn as a diagram of vectors laid down head-to-tail. However, as mentioned above, this diagram by itself, for nontrivial sea and ocean voyages, contains insufficient information to map the arrows accurately onto a Ptolemaic map (i.e. maps as we commonly understand them, based on celestial latitudes and longitudes), yet sufficient at least in theory to guide a pilot following such directions to their destination.

For recording speed and direction for each sand glass time interval (e.g. half hour), pilots used some variation of the traverse board, in which these values were specified by the locations of pegs in the board.

Traverse board. Pins on the upper (circular) portion indicate compass heading and (via distance from the center) for each half hour. Pins on the lower (rectangular) portion indicate estimated speed during each hour. The board thus allows an a pilot on a wet deck unsuitable for a paper log to record an equivalent of a sequence of tuples { direction, speed, time } over four hours, after which time this information is transferred to the ship’s written log(normally kept indoors), the progress is plotted as a head-to-tail diagram on a chart (also kept indoors), and the traverse board is reset. Note that the direction is read directly off the magnetic compass: thus north (the fleur-de-lis) is magnetic north, not geographic (celestial) north.

In a future post I hope to discuss more about dead reckoning directions and explain how the errors that can accumulate in such directions over long distances were corrected. I will also explain why neither the directions nor even the corrections could be accurately drawn on a normal (Ptolemaic or celestial coordinate) map, and yet such dead reckoning directions are sufficient at least in theory for the pilot to guide his ship from the starting port to the intended destination port. In practice, pilots "fixed" errors in their dead reckoning using landmarks and sounding, which I will also describe. And I hope to describe how this resulted in two incompatible systems of “navigation” (broadly speaking) during exploration explosion -- the pilot’s dead reckoning methods versus the cosmographers’ maps and globes based on latitude and longitude.

I also hope to someday figure out just why the exploration explosion occurred when it did. The advent of rigorous dead reckoning -- combining the compass, the sand glass, and decent estimates of speed with rigorous log-keeping -- did not occur in Asia (where the Chinese, lacking the sand glass at least, made a less systematic use of the compass), nor with the Arabs (who seldom used either sand glass or compass), which along with naval superiority explains why the exploration explosion occurred from western Europe. The puzzle of why the explosion started specifically in the 1480s, and not sooner or later, however, remains a mystery to be solved.

Authority and ad hominem

Argument from authority ("I'm the expert") goes hand-in-hand with the ad hominem ("you're not"). Each may be rebutted by the other, and the average quality as evidence of arguments from authority are about the same as the average quality as evidence of ad hominem. By necessity, these two kinds of evidence are the dominant forms of evidence that lead each of us as individuals to believe what we believe, since little important of what you believe comes from your own direct observation. Authority's investment costs are one good proxy measure for evaluating the value of such evidence. But contrast the law of the dominant paradigm. Perhaps the latter is superior for judging claims about the objective world, whereas investment costs are superior for judging the intersubjective.

Proxy measures, sunk costs, and Chesterton's fence

G.K. Chesterton ponders a fence:

In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, "I don't see the use of this; let us clear it away." To which the more intelligent type of reformer will do well to answer: "If you don't see the use of it, I certainly won't let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it."

This paradox rests on the most elementary common sense. The gate or fence did not grow there. It was not set up by somnambulists who built it in their sleep. It is highly improbable that it was put there by escaped lunatics who were for some reason loose in the street. Some person had some reason for thinking it would be a good thing for somebody. And until we know what the reason was, we really cannot judge whether the reason was reasonable. It is extremely probable that we have overlooked some whole aspect of the question, if something set up by human beings like ourselves seems to be entirely meaningless and mysterious. There are reformers who get over this difficulty by assuming that all their fathers were fools; but if that be so, we can only say that folly appears to be a hereditary disease. But the truth is that nobody has any business to destroy a social institution until he has really seen it as an historical institution. If he knows how it arose, and what purposes it was supposed to serve, he may really be able to say that they were bad purposes, that they have since become bad purposes, or that they are purposes which are no longer served. But if he simply stares at the thing as a senseless monstrosity that has somehow sprung up in his path, it is he and not the traditionalist who is suffering from an illusion.

Contrast the sunk cost fallacy, according to one account:

When one makes a hopeless investment, one sometimes reasons: I can’t stop now, otherwise what I’ve invested so far will be lost. This is true, of course, but irrelevant to whether one should continue to invest in the project. Everything one has invested is lost regardless. If there is no hope for success in the future from the investment, then the fact that one has already lost a bundle should lead one to the conclusion that the rational thing to do is to withdraw from the project.

The sunk cost fallacy, according to another account:

Picture this: It's the evening of the Lady Gaga concert/Yankees game/yoga bootcamp. You bought the tickets months ago, saving up and looking forward to it. But tonight, it's blizzarding and you've had the worst week and are exhausted. Nothing would make you happier than a hot chocolate and pajamas, not even 16-inch pink hair/watching Jeter/nailing the dhanurasana.

But you should go, anyway, right? Because otherwise you'd be "wasting your money"?

Think again. Economically speaking, you shouldn't go.

Has Chesterton committed the sunk cost fallacy? Consider the concept of proxy measures:

The process of determining the value of a product from observations is necessarily incomplete and costly. For example, a shopper can see that an apple is shiny red. This has some correlation to its tastiness (the quality a typical shopper actually wants from an apple), but it's hardly perfect. The apple's appearance is not a complete indicator -- an apple sometimes has a rotten spot down inside even if the surface is perfectly shiny and red. We call an indirect measure of value -- for example the shininess, redness, or weight of the apple -- a proxy measure. In fact, all measures of value, besides prices in an ideal market, are proxy measures -- real value is subjective and largely tacit.

Cost can usually be measured far more objectively than value. As a result, the most common proxy measures are various kinds of costs. Examples include:

(a) paying for employment in terms of time worked, rather than by quantity produced (piece rates) or other possible measures. Time measures sacrifice, i.e. the cost of opportunities foregone by the employee

(b) most numbers recorded and reported by accountants for assets are costs rather than market prices expected to be recovered by the sale of assets.

(c) non-fiat money and collectibles obtain their value primarily from their scarcity, i.e. their cost of replacement.

Proxy measures are important because we usually can't measure value directly, much less forecast future value with high confidence. And often we know little of the evidence and preferences that went into an investment decision. You may have forgotten or (if the original decision maker was somebody else) never learned the reason. In which case, the original decision-maker may have had more knowledge than you do -- especially if that decision-maker was somebody else, but sometimes even if that decision-maker was you. In which case it can make a great deal of sense to use the sunk cost as a proxy measure of value.

In the first account of sunk cost, there seems to be no uncertainty: by definition we know that our investment is "hopeless." In such a case, valuing our sunk costs is clearly erroneous. But the second, real-world example, is far less clear: "you've had the worst week and are exhausted.." Does this mean you won't enjoy the concert, as you originally envisioned? Or does it mean that in your exhaustion you've forgotten why you wanted to go to the concert? If it's more likely to mean the latter, then my generalization of Chesterton's fence, using the idea of proxy measures, suggests that you should use your sunk costs as a proxy measure of value, and weigh that value against the costs of the blizzard and the benefits of hot chocolate and pajamas, to decide whether you still will be made happier by going to the concert.

If your evidence may be substantially incomplete you shouldn't just ignore sunk costs -- they contain valuable information about decisions you or others made in the past, perhaps after much greater thought or access to evidence than that of which you are currently capable. Even more generally, you should be loss averse -- you should tend to prefer avoiding losses over acquiring seemingly equivalent gains, and you should be divestiture averse (i.e. exhibit endowment effects) -- you should tend to prefer what you already have to what you might trade it for -- in both cases to the extent your ability to measure the value of the two items is incomplete. Since usually in the real world, and to an even greater degree in our ancestors' evolutionary environments, our ability to measure value is and was woefully incomplete, it should come as no surprise that people often value sunk costs, are loss averse, and exhibit endowment effects -- and indeed under such circumstances of incomplete value measurement it hardly constitutes "fallacy" or "bias" to do so.

In short, Chesterton's fence and proxy measures suggest that taking into account sunk costs, or more generally being averse to loss or divestiture, rather than always being a fallacy or irrational bias, may often lead to better decisions: indeed if it is done in just those cases where substantial evidence or shared preferences that motivated the original investment decision have been forgotten or have not been communicated, or otherwise where the quality of evidence that led to that decision may outweigh the quality of evidence that is motivating one to change one's mind.. We generally have far more information about our past than about our future. Decisions that have already been made, by ourselves and others, are an informative part of that past, especially when their original motivations have been forgotten.

References:

Chesterton's Fence

Sunk Cost Fallacy  (1), (2)

Endowment Effects/Divestiture Aversion: 

Loss Aversion:

Cost as a Proxy Measure of Value

Three philosophical essays

From Algorithmic Information Theory:

Charles Bennett has discovered an objective measurement for sophistication. An example of sophistication is the structure of an airplane. We couldn't just throw parts together into a vat, shake them up, and hope thereby to assemble a flying airplane. A flying structure is vastly improbable; it is far outnumbered by the wide variety of non-flying structures. The same would be true if we tried to design a flying plane by throwing a bunch of part templates down on a table and making a blueprint out of the resulting overlays.

On the other hand, an object can be considered superficial when it is not very difficult to recreate another object to perform its function. For example, a garbage pit can be created by a wide variety of random sequences of truckfulls of garbage; it doesn't matter much in which order the trucks come.

More examples of sophistication are provided by the highly evolved structures of living things, such as wings, eyes, brains, and so on. These could not have been thrown together by chance; they must be the result of an adaptive algorithm such as Darwin's algorithm of variation and selection. If we lost the genetic code for vertebrate eyes in a mass extinction, it would take nature a vast number of animal lifetimes to re-evolve them. A sophisticated structure has a high replacement cost.

Bennett calls the computational replacement cost of an object its logical depth. Loosely speaking, depth is the necessary number of steps in the causal path linking an object with its plausible origin. Formally, it is the time required by the universal Turing machine to compute an object from its compressed original description.


From Objective versus Intersubjective Truth:

Post-Hayek and algorithmic information theory, we recognize that information-bearing codes can be computed (and in particular, ideas evolved from the interaction of people with each other over many lifetimes), which are

(a) not feasibly rederivable from first principles,

(b) not feasibly and accurately refutable  (given the existence of the code to be refuted)

(c) not even feasibly and accurately justifiable (given the existence of the code to justify)

("Feasibility" is a measure of cost, especially the costs of computation and empircal experiment. "Not feasibly" means "cost not within the order of magnitude of being economically efficient": for example, not solvable within a single human lifetime. Usually the constraints are empirical rather than merely computational).

(a) and (b) are ubiqitous among highly evolved systems of interactions among richly encoded entities (whether that information is genetic or memetic). (c) is rarer, since many of these interpersonal games are likely no more diffult than NP-complete: solutions cannot be feasibly derived from scratch, but known solutions can be verified in feasible time. However, there are many problems, especially empirical problems requiring a "medical trial" over one or more full lifetimes, that don't even meet (c): it's infeasible to create a scientifically repeatable experiment. For the same reason a scientific experiment cannot refute _any_ tradition dealing with interpersonal problems (b), because it may not have run over enough lifetimes, and we don't know which computational or empirical class the interpersonal problem solved by the tradition falls into. One can scientifically refute traditional claims of a non-interpersonal nature, e.g. "God created the world in 4004 B.C.", but one cannot accurately refute metaphorical interpretations or imperative statements which apply to interpersonal relationships.

As Dawkins has observed, death is vastly more probable than life. Cultural parts randomly thrown together, or thrown together by some computationally shallow line of reasoning, most likely result in a big mess rather than well functioning relationships between people. The cultural beliefs which give rise to civilization are, like the genes which specify an organism, a highly improbable structure, surrounded in "meme space" primarily by structures which are far more dysfunctional. Most small deviations, and practically all "radical" deviations, result in the equivalent of death for the organism: a mass breakdown of civilization which can include genocide, mass poverty, starvation, plagues, and, perhaps most commonly and importantly, highly unsatisying, painful, or self-destructive individual life choices.


From Hermeneutics: An Introduction to the Interpretation of Tradition:

Hermeneutics derives from the Greek hermeneutika, "message analysis", or "things for interpreting": the interpretation of tradition, the messages we receive from the past... Natural law theorists are trying to do a Heideggerean deconstruction when they try to find the original meaning and intent of the documents deemed to express natural law, such as codifications of English common law, the U.S. Bill of Rights, etc. For example, the question "would the Founding Fathers have intended the 1st Amendment to cover cyberspace?" is a paradigmatic hermeneutical question...[Hans-Georg] Gadamer saw the value of his teacher [Martin] Heidegger's dynamic analysis, and put it in the service of studying living traditions, that is to say traditions with useful applications, such as the law . Gadamer discussed the classical as a broad normative concept denoting that which is the basis of a liberal eduction. He discussed his historical process of Behwahrung, cumulative preservation, that, through constantly improving itself, allows something true to come into being. In the terms of evolutionary hermeneutics, it is used and propagated because of its useful application, and its useful application constitutes its truth. Gadamer also discusses value in terms of the duration of a work's power to speak directly.

Pascal's scams (ii)

Besides the robot apocalypse, there are many other, and often more important, examples of Pascal scams.  The following may be or may have been such poorly evidenced but widely feared or hoped-for extreme consequences (these days the fears seem to predominate):

1. That we are currently headed for another financial industry disaster even worse than 2008 (overwrought expectations often take the form of "much like the surprise we most recently experienced, only even more extreme").


2. That global warming has caused or will cause disaster X (droughts, floods, hurricanes, tornadoes, ...)


3. A whole witch's brew of "much like what just happened" fears were the many terrorist disaster fears that sprouted like the plague in the years after 9/11: suitcase nukes, the "ticking-time bomb" excuse for legalizing torture, envelopes filled with mysterious white powders, and on and on.
   


4. On the positive daydream side, Eric Drexler's "molecular nanotechnology" predictions of the 1980s: self-replicating robots, assemblers that could make almost anything, etc. -- a whole new industrial revolution that would make everything cheap. (Instead, it was outsourcing and a high-tech version of t-shirt printing that made many things cheap, and "nanotechnology" became just a cool buzzword to use when talking about chemistry).


5. A big hope of some naive young engineers during the previous high oil price era of the late 1970s: solar power satellites made from lunar materials, with O'Neill space colonies to house the workers. Indeed, a whole slew of astronaut voyages and industries in space were supposed to follow after the spectacular (and spectacularly expensive) Apollo moon landings -- a "much like recently experienced, only more so" daydream.


6. The "Internet commerce will replace bricks-and-mortar and make all the money those companies were making" ideas that drove the Internet bubble in the late 1990s. Indeed, most or all of the bubbles and depressions in financial markets may be caused by optimistic and pessimistic Pascal fads respectively.

History is replete with many, many more such manias and scares, whether among small groups of otherwise smart people, or among the vast majority of a society.  Sometimes poorly evidenced consequences do happen to occur, just in way(s) very different from expected -- for example Columbus, following the advice of well respected authorities like Strabo and Toscanelli and heading west for India -- ending up instead in America.  And sometimes a lucky penny prophecy of a wonderful or terrible but very unlikely event comes true -- although hardly any of us ever seem to learn about these sage predictions until after the event. Then they only make us believe enough in prophecy that we fall for the next scam.

Pascal's scams

Beware of what I call Pascal's scams: movements or belief systems that ask you to hope for or worry about very improbable outcomes that could have very large positive or negative consequences. (The name comes of course from the infinite-reward Wager proposed by Pascal: these days the large-but-finite versions are far more pernicious).  Naive expected value reasoning implies that they are worth the effort: if the odds are 1 in 1,000 that I could win $1 billion, and I am risk and time neutral, then I should expend up to nearly $1 million dollars worth of effort to gain this boon. The problems with these beliefs tend to be at least threefold, all stemming from the general uncertainty, i.e. the poor information or lack of information, from which we abstracted the low probability estimate in the first place: because in the messy real world the low probability estimate is almost always due to low or poor evidence rather than being a lottery with well-defined odds:

(1) there is usually no feasible way to distinguish between the very improbable (say, 1 in 1,000) and the extremely improbable (e.g., one in a billion). Poor evidence leads to what James Franklin calls "low-weight probabilities", which lack robustness to new evidence. When the evidence is poor, and thus robustness of probabilities is lacking, then it is likely that "a small amount of further evidence would substantially change the probability. "  This new evidence is as likely to decrease the probability by a factor of X as increase it by a factor of X, and the poorer the original evidence, the greater X is.  (Indeed, given the nature of human imagination and bias, it is more likely to decrease it, for reasons described below).

(2) the uncertainties about the diversity and magnitudes of possible consequences, not just their probabilities, are also likely to be extremely high. Indeed, due to the overall poor information, it's easy to overlook negative consequences and recognize only positive ones, or vice-versa. The very acts you take to make it into utopia or avoid dystopia could easily send you to dystopia or make the dystopia worse.

(3) The "unknown unknown" nature of the most uncertainty leads to unfalsifiablity: proponents of the proposition can't propose a clear experiment that would greatly lower the probability or magnitude of consequences of their proposition: or at least, such an experiment would be far too expensive to actually be run, or cannot be conducted until after the time which the believers have already decided that the long-odds bet is rational. So not only is there poor information in a Pascal scam, but in the more pernicious beliefs there is little ability to improve the information.

The biggest problem with these schemes is that, the closer to infinitesimal probability, and thus usually to infinitesimal quality or quantity of evidence, one gets, the closer to infinity the possible extreme-consequence schemes one can dream up,  Once some enterprising memetic innovator dreams up a Pascal's scam, the probabilities or consequences of these possible futures can be greatly exaggerated yet still seem plausible. "Yes, but what if?" the carrier of such a mind-virus incessantly demands.  Furthermore, since more than a few disasters are indeed low probability events (e.g. 9/11), the plausibility and importance of dealing with such risks seems to grow in importance after they occur -- the occurrence of one improbable disaster leads to paranoia about a large number of others, and similarly for fortuitous windfalls and hopes. Humanity can dream up a near-infinity of Pascal's scams, or spend a near-infinity of time fruitlessly worrying about them or hoping for them. There are however far better ways to spend one's time -- for example in thinking about what has actually happened in the real world, rather than the vast number of things that might happen in the future but quite probably won't, or will likely cause consequences very differently than you expect.

So how should we approach low probability hypotheses with potential high value (negative or positive) outcomes?  Franklin et. al. suggest that "[t]he strongly quantitative style of education in statistics, valuable as it is, can lead to a neglect of the more qualitative, logical, legal and causal perspectives needed to understand data intelligently. That is especially so in extreme risk analysis, where there is a lack of large data sets to ground solidly quantitative conclusions, and correspondingly a need to supplement the data with outside information and with argument on individual data points."

On the above quoted points I agree with Franklin, and add a more blunt suggestion: stop throwing around long odds and dreaming of big consequences as if you are onto something profound.  If you can't gather the information needed to reduce the uncertainties, and if you can't suggest experiments to make the hope or worry falsifiable, stop nightmaring or daydreaming already. Also, shut up and stop trying to convince the rest of us to join you in wasting our time hoping or worrying about these fantasies.  Try spending more time learning about what has actually happened in the real world.  That study, too, has its uncertainties, but they are up to infinitely smaller.

More short takes

Perhaps I should take up Twitter, but I already have this blog, and even my short takes tend to go a bit over 140 characters. So here goes:

* The most important professions in the modern world may be the most reviled: advertiser, salesperson, lawyer, and financial trader. What these professions have in common is extending useful social interactions far beyond the tribe-sized groups we were evolved to inhabit (most often characterized by the Dunbar number). This commonly involves activities that fly in the face of our tribal moral instincts.

* On a related note, much mistaken thinking about society could be eliminated by the most straightforward application of the pigeonhole principle: you can't fit more pigeons into your pigeon coop than you have holes to put them in. Even if you were telepathic, you could not learn all of what is going on in everybody's head because there is no room to fit all that information in yours. If I could completely scan 1,000 brains and had some machine to copy the contents of those into mine, I could only learn at most about a thousandth of the information stored in those brains, and then only at the cost of forgetting all else I had known. That's a theoretical optimum; any such real-world transfer process, such as reading and writing an e-mail or a book, or tutoring, or using or influencing a market price, will pick up only a small fraction of even the theoretically acquirable knowledge or preferences in the mind(s) at the other end of said process, or if you prefer of the information stored by those brain(s). Of course, one can argue that some kinds of knowledge -- like the kinds you and I know? -- are vastly more important than others, but such a claim is usually more snobbery than fact. Furthermore, a society with more such computational and mental diversity is more productive, because specialized algorithms, mental processes, and skills are generally far more productive than generalized ones. As Friedrich Hayek pointed out, our mutual inability to understand a very high fraction of what others know has profound implications for our economic and political institutions.

* A big problem in the last few years has been the poor recording of transfers of ownership of mortgages (i.e. of the debt not the house). The issue of recording transfers of contractual rights is very interesting. I have a proposal for this, secure property titles. This should work just as well for mortgage securities and other kinds of transferable contractual rights as it does for the real estate itself or other kinds of property. Anytime you transfer rights to a contract it should be registered in such a secure and reliable public database in order to avoid the risk of not being able to prove ownership in court.

* Not only should you disagree with others, but you should disagree with yourself. Totalitarian thought asks us to consider, much less accept, only one hypothesis at a time. By contrast quantum thought, as I call it -- although it already has a traditional name less recognizable to the modern ear, scholastic thought -- demands that we simultaneoulsy consider often mutually contradictory possibilities. Thinking about and presenting only one side's arguments gives one's thought and prose a false patina of consistency: a fallacy of thought and communications similar to false precision, but much more common and imporant. Like false precision, it can be a mental mistake or a misleading rhetorical habit. In quantum reality, by contrast, I can be both for and against a proposition because I am entertaining at least two significantly possible but inconsistent hypotheses, or because I favor some parts of a set of ideas and not others. If you are unable or unwilling to think in such a quantum or scholastic manner, it is much less likely that your thoughts are worthy of others' consideration.

Short takes

During my blog absence, I've been studying, designing, and implementing a style of programming I call temporal programming. It is useful for, among other things, implementing smart contracts. Meanwhile, I encourage readers interested in programming to check out Node.js. Temporal programming starts with event-oriented programming and takes it further. Temporal programming will give us control over when our instructions get executed: the plodding do this, then that, then the other, as if machine activities are only supposed to happen in one big long sequence and merely output some big long tape, will be relegated to secondary status. More to come.

--------

Is Netflix management really stupid? To summarize Megan McCardle, no: under copyright law, DVDs are covered by the first sale rule -- once you buy a DVD, you can't make a copy, but you can sell, rent or give away the DVD itself. Streaming, on the other hand, essentially involves making a copy, and you can't do it legally without the copyright owner's permission. So to stream the most demanded content, you usually need the permission of owners, and thus have to pay what they demand.

It thus makes complete sense that Netflix has to charge high prices for streaming -- because for the kind of content they want to stream (i.e. the copyrights that are so valuable that their owners bother to prevent the content from being distributed on YouTube), content owners are demanding revenues similar to what they are accustomed to via cable.

I'd add that the first sale rule also explains why Netflix, contrary to its name, originally succeeded in out-competing a gaggle of Internet video streaming companies by the stone-age method of shipping DVDs by mail. The main thing Netflix may have done wrong was, after succeeding in pivoting from Internet streaming to mailing DVDs, going back to their original goal in a way that set false expectations about prices (i.e. that streaming prices would be closer to DVD rental prices than to cable TV). Probably what happened is that Netflix CEO Reed Hastings thought he could use his mail-order rental business as leverage to negotiate lower prices with copyright owners, but this strategy did not succeed.

It also makes sense that Netflix (and their streaming competitors) lack licensed content due to copyright owners' long-standing aversion to Internet streaming. All this was happening to Netflix's video-streaming competitors long before Netflix's much more recent emphasis on that business. Netflix apparently hasn't, after all, solved the institutional problem that their DVD-shipping model worked around.

---------

Bit gold and I make brief appearances in Wired's Bitcoin article.

---------

Water remains fun. Digital fountains show how precisely drops can be located and timed:





This one is a bit different, drops fall with uniform regularity but are used to display light:

Of wages and money: cost as a proxy measure of value

The subjective theory of value, used by the Salamanca school in Renaissance Spain and most recently in neoclassical and Austrian economics, is undoubtedly the correct one, despite the fact that figures ranging from Smith to Ricardo to Marx used labor as a fundamental measure of value. This labor theory of value is profoundly wrong. If I spend hours digging a hole in your back yard, that doesn't provide you any value unless you wanted a hole in your back yard.

Nevertheless, as we shall see, using labor, or more generally cost, as a measure of value is a common strategy of our institutions and in itself often quite valuable thing to do. How does this come about? The key to the puzzle is Yoram Barzel's idea of a proxy measure, which I have described as follows:

The process of determining the value of a product from observations is necessarily incomplete and costly. For example, a shopper can see that an apple is shiny red. This has some correlation to its tastiness (the quality a typical shopper actually wants from an apple), but it's hardly perfect. The apple's appearance is not a complete indicator -- an apple sometimes has a rotten spot down inside even if the surface is perfectly shiny and red. We call an indirect measure of value -- for example the shininess, redness, or weight of the apple -- a proxy measure. In fact, all measures of value, besides prices in an ideal market, are proxy measures -- real value is subjective and largely tacit.

Such observations also come at a cost. It may take some time to sort through apples to find the shiniest and reddest ones, and meanwhile the shopper bruises the other apples. It costs the vendor to put on a fake shiny gloss of wax, and it costs the shopper because he may be fooled by the wax, and because he has to eat wax with his apple. Sometimes these measurement costs comes about just from the imperfection of honest communication. In other cases, such as waxing the apple, the cost occurs because rationally self-interested parties play games with the observable...

Cost can usually be measured far more objectively than value. As a result, the most common proxy measures are various kinds of costs. Examples include:

(a) paying for employment in terms of time worked, rather than by quantity produced (piece rates) or other possible measures. Time measures sacrifice, i.e. the cost of opportunities foregone by the employee.

(b) most numbers recorded and reported by accountants for assets are costs rather than market prices expected to be recovered by the sale of assets.

(c) non-fiat money and collectibles obtain their value primarily from their scarcity, i.e. their cost of replacement.

(From "Measuring Value").

The proxy measure that dominates most of our lives is the time wage:

To create anything of value requires some sacrifice. To successfully contract we must measure value. Since we can’t, absent a perfect exchange market, directly measure the economic value of something, we may be able to estimate it indirectly by measuring something else. This something else anchors the performance – it gives the performer an incentive to optimize the measured value. Which measures are the most appropriate anchors of performance? Starting in Europe by the 13th century, that measure was increasingly a measure of the sacrifice needed to create the desired economic value.

This is hardly automatic – labor is not value. A bad artist can spend years doodling, or a worker can dig a hole where nobody wants a hole. Arbitrary amounts of time could be spent on activities that do not have value for anybody except, perhaps, the worker himself. To improve the productivity of the time rate contract required two breakthroughs: the first, creating the conditions under which sacrifice is a better estimate of value than piece rate or other measurement alternatives, and second, the ability to measure, with accuracy and integrity, the sacrifice.

Piece rates measure directly some attribute of a good or service that is important to its value – its quantity, weight, volume, or the like -- and then fix a price for it. Guild regulations which fixed prices often amounted to creating piece rates. Piece rates seem the ideal alternative for liberating workers, but they suffer for two reasons. First, the outputs of labor depend not only on effort, skills, etc. (things under control of the employee), but things out of control of the employee. The employee wants something like insurance against these vagaries of the work environment. The employer, who has more wealth and knowledge of market conditions, takes on these risks in exchange for profit.

In an unregulated commodity market, buyers can reject or negotiate downwards the price of poor quality goods. Sellers can negotiate upwards or decline to sell. With piece rate contracts, on the other hand, there is a fixed payment for a unit of output. Thus second main drawback to piece rates is that they motivate the worker to put out more quantity at the expense of quality. This can be devastating. The tendency of communist countries to pay piece rates, rather than hourly rates, is one reason that, while the Soviet bloc’s quantity (and thus the most straightforward measurements of economic growth) was able to keep up with the West, quality did not (thus the contrast, for example, between the notoriously ugly and unreliable Trabant of East Germany and the BMWs, Mercedes, Audi and Volkswagens of West Germany).

Thus with the time-rate wage the employee is insured against vagaries of production beyond his control, including selling price fluctuations (in the case of a market exchange), or variation in the price or availability of factors of production (in the case of both market exchange or piece rates). The employer takes on these risks, while at the same time through promotion, raises, demotions, wage cuts or firing retaining incentives for quality employee output.

Besides lacking implicit insurance for the employee, another limit to market purchase of each worker’s output is that it can be made prohibitively costly by relationship-specific investments. These investments occur when workers engage in interdependent production -- as the workers learn the equipment or adapt to each other. Relationship-specific investments can also occur between firms, for example building a cannon foundry next to an iron mine. These investments, when combine with the inability to write long-term contracts that account for all eventualities, motivate firms to integrate. Dealing with unspecified eventualities then becomes the right of the single owner. This incentive to integrate is opposed by the diseconomies of scale in a bureaucracy, caused by the distribution of knowledge, which market exchange handles much better. These economic tradeoffs produce observed distributions of firm sizes in a market, i.e. the number of workers involved in an employment relationship instead of selling their wares directly on a market.

The main alternative to market exchange of output, piece rate, or coerced labor (serfdom or slavery) consists of the employers paying by sacrifice -- by some measure of the desirable things the employee forgoes to pursue the employer’s objectives. An hour spent at work is an hour not spent partying, playing with the children, etc. For labor, this “opportunity cost” is most easily denominated in time – a day spent working for the employer is a day not spent doing things the employee would, if not for the pay, desire to do.

Time doesn’t specify costs such as effort and danger. These have to be taken into account by an employee or his union when evaluating a job offer. Worker choice, through the ability to switch jobs at much lower costs than with serfdom, allows this crucial quality control to occur.

It’s usually hard to specify customer preferences, or quality, in a production contract. It’s easy to specify sacrifice, if we can measure it. Time is immediately observed; quality is eventually observed. With employment via a time-wage, the costly giving up of other opportunities, measured in time, can be directly motivated (via daily or hourly wages), while quality is motivated in a delayed, discontinuous manner (by firing if employers and/or peers judge that quality of the work is too often bad). Third parties, say the guy who owned the shop across the street, could observe the workers arriving and leaving, and tell when they did so by the time. Common synchronization greatly reduced the opportunities for fraud involving that most basic contractual promise, the promise of time.

Once pay for time is in place, the basic incentives are in place – the employee is, verifiably, on the job for a specific portion of the day – so he might as well work. He might as well do the work, both quantity and quality, that the employer requires. With incentives more closely aligned by the calendar and the city bells measuring the opportunity costs of employment, to be compensated by the employer, the employer can focus observations on verifying the specific quantity and qualities desired, and the employee (to gain raises and avoid getting fired) focuses on satisfying them. So with the time-wage contract, perfected by northern and western Europeans in the late Middle Ages, we have two levels of the protocol in this relationship: (1) the employee trades away other opportunities to commit his time to the employer – this time is measured and compensated, (2) the employee is motivated, by (positively) opportunities for promotions and wage rate hikes and (negatively) by the threat of firing, to use that time, otherwise worthless to both employer and employee, to achieve the quantity and/or quality goals desired by the employer.

(from From "A Measure of Sacrifice")

Proxy measures also explain how money can depend on standards of nature instead of a singular trusted third party:

Although anatomically modern humans surely had conscious thought, language, and some ability to plan, it would have required little conscious thought or language, and very little planning, to generate trades. It was not necessary that tribe members reasoned out the benefits of anything but a single trade. To create this institution it would have sufficed that people follow their instincts to make obtain collectibles with the characteristics outlined below. (as indicated by proxy observations that make approximate estimations for these characteristics).

Utilitarian jewelry: silver shekels from Sumeria. The recipient of a payment would weigh the segment of coil used as payment and could cut it at an arbitrary spot to test its purity. A wide variety of other cultures, from the Hebrews and the famously commercial Phoenicians to the ancient Celtic and Germanic tribes, used such forms of precious metal money long before coinage.

At first, the production of a commodity simply because it is costly seems quite wasteful. However, the unforgeably costly commodity repeatedly adds value by enabling beneficial wealth transfers. More of the cost is recouped every time a transaction is made possible or made less expensive. The cost, initially a complete waste, is amortized over many transactions. The monetary value of precious metals is based on this principle. It also applies to collectibles, which are more prized the rarer they are and the less forgeable this rarity is. It also applies where provably skilled or unique human labor is added to the product, as with art.

(from "Shelling Out: The Origins of Money")

The rational price of such a monetary commodity will be, ignoring more minor effects (e.g. the destruction of monetary units) the lesser of:

(1) the value a monetary unit is expected to add to future transactions, i.e. what it will save in transaction costs

(2) the cost of creating a new monetary unit

Since (1) cannot be objectively measured, but at best can only be intuitively estimated, as long as one is confident that (1)>(2) the cost of creating a monetary unit becomes a good proxy measure of the value of the monetary unit. If monetary units are not fungible, but their costs are comparable, their relative value can be well approximated by their relative costs even if (2)>=(1). If the expected value of a monetary unit drops below the replacement cost, gold miners (for example) stop creating new monetary units (in the case of gold miners, if their production costs exceed this expected value they stop mining gold), holding the money supply steady which minimizes inflation. If it goes above replacement costs more gold mines can be affordably worked, increasing the money supply as the economy grows without inflation. This all depends on having a secure floor to replacement costs, which is well approximated by gold and silver but violated by fiat currencies (as can be seen by for example comparing the rampant inflations of many 20th century fiat currencies against the quite mild, by comparison, inflations that followed discoveries such as Potosi silver in the 16th century and California gold in the 19th).

Sumerian shell money.

We often adapt institutions from prior similar institutions, and often by accident. So, for example, money can evolve from utilitarian commodities that have the best characteristics of money (securely storable, etc.) in a particular environment (e.g. cigarettes in a prison). But humans also have foresight and can reason by analogy, and so can design an exchange to trade new kinds of securities, a new kind of insurance service, or a new kind of currency.

Naturally such designs are still subject to a large degree of trial and error, of the creative destruction of the market. And they are subject to network externalities: to, roughly speaking, Metcalfe's law, which states the value of a network is proportional to the square of the number of its members, and that a network of one is useless. A telephone is useless unless somebody else you want to talk to also has a telephone. Likewise, currency is useless unless one has somebody to buy from or sell to who will take or give that currency. Indeed, with money the situation is even worse than most networks, because unless that person wants to do a corresponding sell or buy back to you at a later date, the money will get "stuck." It will have zero velocity. One needs either specific cycles to keep money circulating (as with the ancient kula ring), or a second currency with such cycles that can be exchanged for the first. Thus, the marketing problem of starting a new currency is formidable. Although Paypal was implementing just a payment system in dollars, not a new currency, it hit on a great strategy of general applicability: target specific communities of people that trade with each other (in Paypal's case, the popular eBay auction site).

But insisting that gold, silver, shells, online payment systems or currencies, etc. must be useful for some other purpose, such as decoration, before they can be used as money, is a terrible confusion, akin to insisting that an insurance service must start out as useful for something else, perhaps for stabling horses, before one can write the insurance contracts. Indeed many of us value precious metals and shells for decoration more for a reverse reason, which I explain in the above-linked essay on the origins of money.

Conclusion

The labor theory of value is wrong. Value is fundamentally a matter of subjective preferences. Nevertheless. Yoram Barzel's crucial idea of proxy measures allows us to understand why measures of labor and more generally measures of cost are so often and so usefully applied as measures of value in institutions, including wage contracts and non-fiat money.

Agricultural consequences of the Black Death

The Black Death, caused (as DNA testing of victims has recently confirmed) by the bubonic plague, Yersina pestis, swept through Europe primarily in the years 1347-1351, with some recurring outbreaks thereafter. This plague not only brought great demographic changes to Europe, killing off probably about half of the population, but substantial changes to the economy and especially to the dominant sector of the economy then, agriculture. The Black Death provides a great empirical test of Malthusian theories of agricultural economies and it is very informative to look at what happened.

Under a naive reading of Malthusian theory, yield (the amount of food or fodder produced per unit area) should have improved after the Black Death: a lower population abandoned marginally productive lands and kept the better yielding ones. Furthermore, the increased ratio of livestock to arable acres should have improved fertilization of the fields, as more nutrients and especially nitrogen moved from wastes (wild lands) and pastures to the arable.

But while marginal lands were indeed abandoned (or at least converted from arable to pasture), and the livestock/arable ratio indeed increased, it's not the case that yield improved after the Black Death. In fact, the overall (average) yield of all arable probably declined slightly, and the yields of particular plots of land that were retained as arable usually declined quite substantially. Yields declined and for the most part stayed low for several centuries thereafter. Other factors must have been work, factors stronger than the abandonment of marginally productive lands and the influence of more livestock on nutrient movement.
Per capita agricultural output (vertical axis) vs. population density (horizontal axis) in England before, during, and after the Black Death and through the agricultural revolutions of the succeeding centuries.

The main such factor is that increasing yield without great technological improvement requires more intensive labor: extra weeding, extra transport of manures and soil conditioners, etc. Contrariwise, a less intensive use of land readily leads to falling yield. After the Black Death, labor was more expensive relative to land, so it made sense to engage in a far less intensive agriculture. The increase of pasture and livestock at the expense of the arable was a part of this. But letting the yield decline was another. What increased quite dramatically in the century after the Black Death was labor productivity and per capita standard of living (see first three points on the diagram, each covering an 80-year period from 1260 to 1499).

Indeed, the whole focus in the literature on improvements in "productivity" as yield in the agricultural revolution has been very misleading. The crucial part of the revolution, and the direct cause of the release of labor from the agricultural to the industrial and transport sectors, was an increase in farm labor productivity: more food grown with less labor. Over short timescales, that is without dramatic technological or institutional progress, farm labor productivity was inversely correlated with yield for the reasons stated. Higher labor productivity, certainly not higher yield, is what distinguished most of Europe for example from East and Southeast Asia, where under intensive rice cultivation, and abundant rain during warm summers, yield had long been far higher than in Europe. Per Malthus, however, population growth in these regions had kept pace with yield, resulting in a highly labor-intensive, low labor productivity style of agriculture, including much less use of draft animals for farm work and transport.

The bubonic plague was spread by rodents, who in turn fed primarily on grain stores. This also had a significant impact on agriculture, because grain-growing farmers and grain-eating populations were disproportionately killed and people fled areas that grew or stored grain for regions specializing in other kinds of agriculture. In coastal regions like Portugal there was a large move from grain farming to fishing. In Northern Europe there was a great move to pasture and livestock, and in particular a relative expansion of the unique stationary pastoralism that had started to develop there in the previous centuries, both due to the relative protection of pastoralists from the plague and the less labor-intensive nature of pastoralism after land had become cheaper relative to labor.

A final interesting effect is that the long trend from about the 11th to the 19th century in England of conversion of draft animals from oxen to horses halted temporarily in the century after the Black Death. The main reason for this was undoubtedly the reversion of arable to pasture or waste (wild lands). Oxen are, to use horse culture terminology, very "easy keepers". As ruminants they can convert cellulose to glucose for energy and horses can't. Oxen, therefore, became relatively cheaper than horses in an era with more wastes and less (or at least much less intensive use of) arable for growing fodder. This kind of effect occurred again much later on the American frontiers where there was a temporary, partial reversion from horses to oxen as draft animals given the abundance of wild lands relative to groomed pasture and arable for growing fodder. For example the famous wagon trains to California and Oregon were pulled primarily by oxen, which could be fed much better than horses in the wild grasslands and arid wastes encountered. However in England itself, the trend towards horses replacing oxen, fueled by the increasing growth of fodder crops, had resumed by 1500 and indeed accelerated in the 16th century, playing a central role in the transportation revolutions of the 15th through 19th centuries.