Thursday, January 9, 2025

Working with metrics that don't tell you much

We spent the whole month of December talking about metrics: how to create them, and how to avoid some common pitfalls associated with their use. Before we leave the subject, I want to address one more topic: What about when your metrics don't give you all that much information?

The first thing is to check how much information you really need. If you obviously need more than you are getting, that's almost like having no metric at all. Then maybe you need to create one, using whatever tools you have available. But not so fast. Sometimes even just a little data can be enough.

My son, Graham Mills, is a soil scientist, and recently he and I were talking about the kinds of measurements commonly used to classify Western rangeland. The Bureau of Land Management (BLM) has defined a strategy called Assessment, Inventory, and Monitoring (AIM). The standard source for this methodology is this publication,* which explains that:

Core methods generate indicators which represent the minimum information necessary to describe three key ecosystem attributes: soil and site stability, watershed function, and biotic integrity …. Nearly everything we value about ecosystems depends on these attributes. These core methods can also be used to generate many additional indicators that directly inform multiple management objectives, such as maintaining wildlife habitat, biodiversity conservation, producing forage, and supporting watershed health. Modifications to the core methods are discouraged as they limit the ability to combine and compare datasets, and thus describe ecosystem attributes at multiple scales.**

So far, so good. The catch, as Graham explained it to me, is that the actual measurements of soil health represent such a small fraction of the total characteristics of the soil that they are still maddeningly vague. It is, for example, not really possible to develop a solid theoretical understanding of the changes that have taken place over the years on a particular stretch of rangeland. To anyone with a scientific background, this limitation is frustrating.

Frustrating but not immobilizing. It turns out that soil scientists can still work with the AIM results.

The key is that the range of possible actions for restoring damaged or depleted rangeland is so very narrow. BLM scientists understand that rangeland is a biological system, and that systems—by definition!—are self-organizing and therefore unpredictable. So there are only a very few interventions permitted at all; and all of them are familiar and well understood. Plant this kind of bush here. Plant that kind of ground cover over there. If there is human garbage clogging a freshwater spring, remove the garbage. And so on. 

The list of approved actions is very short. And therefore a complete quantification of all possible soil characteristics is not needed. If the soil is seriously damaged, do this; if mildly damaged, do that; if already thriving, do a third thing—or maybe nothing at all. It turns out that that's enough to cover it.

As in so many other cases, the first step is to understand what you really need and want. Only then can you set about getting it.     

__________

Herrick, J.E., Jornada Experimental Range, 2018. Monitoring manual for grassland, shrubland, and savanna ecosystems. USDA - ARS Jordana Experimental Range, Las Cruces, NM.

** Ibid., p.1.

Photo credits: G. Mills, May 2021.      

No comments:

Post a Comment

Five laws of administration

It's the last week of the year, so let's end on a light note. Here are five general principles that I've picked up from working ...