Publish or Perish – Another Example of Goodhart’s Law
The backbone of scientific breakthroughs is for scientists to publish their findings so that others can learn from them and expand their knowledge. This is why some books are rightly considered among the most influential math and science books of all time:
Elements, Euclid, c. 300 BC
Physics, Aristotle, ch. 330 BC
On the revolutions of the celestial spheres, Nicolas Copernicus, 1543
Dialogue concerning the two main world systems, Galileo Galileo, 1632
Mathematical Principles of Natural Philosophy, Isaac Newton, 1687
The origin of species, Charles Darwin, 1859
As Newton said, “If I have seen further, it is by standing on the shoulders of the Giants.”
It seems logical to assess the importance of today’s researchers by the number of their publications and the frequency with which their research has been cited by others. Thus, the cruel slogan “publish or perish” has become a brutal fact of life. Promotions, funding, and brand awareness all rely on evidence through publication that a researcher deserves to be promoted, supported and celebrated.
Each job interview, each promotion file, each grant application includes a publication list. Virtually all researchers have a website with a link to their curriculum vitae (CV). The Internet has now made it possible to tabulate quotes.
For example, in 2004, Google launched Google Scholar, a database of hundreds of millions of academic articles found by its crawlers. Next, Google created Scholar Citations, which lists a researcher’s articles and the number of citations found by Googlebots. Soon Google began to compile productivity indices based on the number of articles an author wrote and the number of times those articles were cited. For example, the h-index (named after its creator Jorge E. Hirsch) is equal to the maximum number of articles h which were cited at least h time. My h index is currently 24, which means I have written 24 articles that have each been cited by at least 24 other articles.
Google Quotes is a quick and easy (okay, lazy) way to rate productivity and importance. As Google says, its citation indexes are meant to measure “visibility and influence”. Citation indexes are now considered so important that some researchers include them in their resumes and web pages.
Unfortunately, the culture of publication or destruction has encouraged researchers to play with the system, undermining the usefulness of the number of publications and citations. Here is an example of Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure. The American and British monetary authorities believed that there was a close statistical relationship between the rate of growth of the money supply and the rate of inflation and that they could therefore set a target growth rate for the money supply that would allow the growth of money supply. ‘achieve a desirable rate. inflation.
The problem is that the definition of money supply is ambiguous and the relationship between money and inflation is tenuous. Is money supply just the amount of cash that floats, or does it include current account balances? What about funds in savings accounts and money market funds that can be easily spent? What about cash in brokerage accounts? What about the fact that most purchases are made with credit cards?
Governments have tried several measures of money supply (M0, M1, M2, M3, MZM) but, according to Goodhart’s law, they all failed. Every measure used as a target has ceased to be a good measure. Governments have had to abandon money supply targets and instead focus directly on inflation targets (currently 2% in England and the US).
In the context of publish or perish, the widespread adoption of publication counts and citation indexes to assess visibility and influence has caused publication counts and citation indexes to cease to be a good measure of visibility and influence. The first sign of this collapse was an explosive growth in the number of journals in response to researchers’ insatiable demand for the number of publications. In 2018, it was estimated that over 3 million articles were published in over 42,000 peer-reviewed academic journals. It has been estimated that half of all peer-reviewed articles are not read by anyone other than the author, editor, and reviewers of the journal, although I cannot think of any workable way to ” identify articles that no one has read. It is certainly true, however, that many articles are read by very few, but the incentive to post or perish structure makes it better to post something that no one reads rather than not post at all.
It also encourages a variety of inappropriate tricks to pass peer review and rack up citations.
Next: Peer review is well intentioned, but flawed and vulnerable to gambling.