Thomas Rowlandson, “The Doctor’s Dream,” from William Combe and Thomas Rowlandson, The Tour of Dr. Syntax in Search of the Picturesque

[This is the fourth piece in a miniseries on finding the right line between human thought and AI assistance, focusing on the stages of scholarly work from initial ideas through the research process to publication, although I believe much of this discussion is applicable to intellectual work beyond the academy. The miniseries began with this introduction and was followed by an essay on the origins of new ideas and a piece on analyzing evidence and data in the early stages of research. In this issue, I look at handing the entire writing process over to AI.]

At the end of the last essay in this series, on the application of AI to analytical sections of a research project, such as data explorations and visualizations, I left this question dangling:

Why not go further or even all the way? Why not have AI do the entire analytical process and spit out the result, perhaps as a nicely formatted paper?

Point your favorite LLM at a stack of articles, documents, lab notebooks, or data sets, and give it a prompt — or if you really want to go all in, have it come up with its own question to answer or theme to pursue based on an initial pass — and sit back in your comfy armchair as the words flitter by.

This new form of armchair scholarship, like the nineteenth-century academics who wrote books and articles based only on what was readily at hand (in, say, their posh home libraries), rather than doing iterative and extensive field work, lab work, or any other kind of time-consuming engagement with their subject matter, is far from hypothetical. As you read this, AI is probably writing hundreds of academic papers. This time the armchair has a jet engine on the back.

A nontrivial and growing percentage of submissions to journals are now partially or mostly the product of AI, especially in scientific fields. In one study, based on STEM articles from 2021 through 2024, AI usage in the writing of academic articles spiked beginning with the release of ChatGPT in the fall of 2022, quickly reaching over 20% in computer science and electrical engineering, and, to a lesser but still significant extent, in other fields by the fall of 2024. These measurements are surely much higher two years later, especially with the latest agentic AI tools and more advanced models.

Liang, W., Zhang, Y., Wu, Z. et al. “Quantifying large language model usage in scientific papers,” Nat Hum Behav 9, 2599–2609 (2025).

These early-adopting AI-assisted scholars are still likely in the minority, and if we want to be generous to them, there may be understandable reasons for their reliance on AI, such as the predominance of English in science publishing. (According to the study, researchers from countries where English is not a first or second language use AI to compose articles at significantly higher rates.) Of course, there’s also the primal need to publish or perish in academia, which has always incentivized cutting corners.

Whether AI authorship of papers is an activity dominated by mercenary researchers seeking tenure or the byproduct of AI translation, the temptation to use AI in the production of scholarly writing will undoubtedly continue to grow. This year has seen a proliferation of websites and software geared toward rapid paper generation. Generally the producers of these tools frame the process as a productive collaboration between you and the AI, but the amount of “you” seems to shrink as you look more carefully and notice the many opportunities to opt out of deep intellectual work. For instance, the AI paper-writing assistant Gatsbi nods toward the discrete stages of research and writing I’ve covered in this series, but also notes that it can go ahead and “auto-draft” the entire paper, stem to stern, if you’d prefer to get a coffee. CoPaper.AI, from Stanford, similarly emphasizes that the scholar guides the production of the article at each turn, but also claims in a large font on its home page that it can take only 20 minutes to sprint from raw data to a final paper. It may be “human in the loop,” but at that scale you’re an ant inside a hula hoop. And naturally you can use Claude or ChatGPT to do end-to-end research and writing, at various levels of engagement from micromanager to laissez-faire napper, whether you’re a fifth grader or a faculty member.

* * *

Call me an optimist, but I believe that the majority of scholars, despite being under enormous pressure to publish, would prefer to be more engaged than removed from the fundamental pursuits and texture of their discipline. They enjoy wrestling with sources, data, and theories, and are innately repelled by the superficiality of having AI write a complete paper. They want to lean into their work, not lean back in an automated armchair. As NYU astrophysicist David Hogg recently wrote, “Anyone working in astrophysics is someone who wants to do astrophysics, not someone who wants to learn the answers.”

Anthropic and OpenAI seem to understand this now, after years of touting AI in a less-than-reassuring way as a replacement for human thought, endeavor, and employment. OpenAI’s new writing tool Prism, centered on LaTeX, a nerdy markup and formatting standard for STEM articles, and Anthropic’s new workbench Claude Science, seem inclined toward more exploratory, iterative, and assistive modes of STEM article generation. Partners, not proxies.

But how slippery is the slope? If one uses AI to help out with part of a scholarly work — for instance, the often (but not always) formulaic “methods” section of a scientific article — will the temptation rise to use it for other parts, such as the more intellectually stimulating and important “discussion” section, in which the results of an experiment are unpacked and its implications for the field made clear? And if so…is that so bad?

Economist and AI enthusiast Tyler Cowen and others working in the more data-centric areas of the social sciences and natural sciences have pondered this question and begun to sour on the old method of article production, instead believing that the future of scholarship may indeed lie in a data set — perhaps one that is constantly updated — and an AI front end that interprets this data. The careful wordsmithing of a paper might be secondary to this direct computational approach, or vanish altogether. And maybe the AI can create the data set too, leaving more time for coffee breaks.

My nagging worry is this: based on a passing familiarity with human nature, I can foresee that an increasing number of academics are going to have to be lashed to the library stacks to resist the AI sirens, who will sing not about more measured uses of AI, but about that sweet, comfortable armchair in which to rest while entire articles are tirelessly generated for them.

* * *

Since we do not have beeswax to put in our ears to resist this song, it seems helpful to examine exactly why the AI generation of a complete scholarly work, rather than using AI judiciously for certain scholarly tasks as I have been arguing in this series, is a bad idea — not just for academic disciplines but for the academics themselves.

At this point, you might be expecting a long rant on AI hallucinations and the possibility of scholarship turning from the pursuit of truth into the extrusion of plausible-sounding truthiness. Hallucinations do remain an area of concern, but it is a problem that has waned over the last year. As LLMs have become more agentic than static, and more rigorously structured in their processes — not relying as much on their initial training set, and venturing out to read external sources of information as needed, instead of immediately starting to spit out text following a query — the number of glaring, or even small, errors has decreased. Especially in the highly connected academic AI environments I have been discussing in this series, with actual libraries available to the LLMs — Claude Science, for instance, can retrieve peer-reviewed research and vetted data from dozens of highly specialized academic resources — the hallucination problem has receded further.

At the same time, hallucinations have not totally disappeared, and academic research should always aim for the highest level of reliability, which makes even the small possibility of hallucinations a shadow over the scholarly enterprise, not to mention the embarrassment of AI-generated faux pas to scholars who take an automated shortcut. Of course, human intelligence can also make errors, or worse, engage in statistical shiftiness or outright fraud. (See the replication crisis.) But we should be aiming to level up on veracity, not down.

More problematic to me than the specter of hallucinations, however, are the long-term effects of the armchair overuse of AI on three areas dear to the academy: the nature of writing and reading, the composition of the sources that writing is based upon, and the enervation of the scholarly mind and scholarly disciplines.

* * *

In early 2023, soon after the release of ChatGPT, I asked, “Can Engineered Writing Ever Be Great?” My answer came from a simple point that every writer knows:

Good writing isn't just the selection and ordering of words, the output; good writing is the product of good reading. Writers aren't indiscriminate generalists, but tend to be rather choosy and personal about what they read. As humans they also have a fairly limited reading capacity, which means that their styles are highly influenced by idiosyncratic reading histories, by their whim.

LLMs, on the other hand, are insatiable omnivores, ingesting as much text, indiscriminately, as they can. This allows them to create countless styles of writing virtually instantly, that feed every need from an automated email response to poetry. But this also means that off-the-rack writing from an LLM tends toward the anodyne, or “slop” if we want to use a more disparaging term.

Over three years later, however, LLM output can be significantly improved, especially if you tailor the inputs to the underlying model, or add post-training context. In one recent study, readers preferred the writing from an LLM trained on books (rather than text that largely comes from the web) over that of human writers with MFAs. I have used Claude Code to create a database of all of my writing (books, academic and popular-press articles, blog posts, this newsletter, and unpublished writing, 2+ million words), which I mostly use to remember and find things I’ve written, but which Claude could also use to compose new pieces very much in my voice, if I wanted it to. (I don’t. The em dashes you often see in my writing are my own; I do love them and don’t care if they have become a marker of AI writing.) If you think that my writing rises above generic AI slop, then I can assure you it’s now possible to use AI to create prose that mimics this more personal, angular style.

Nevertheless, having AI generate an article, even in one’s own voice, inevitably cedes critical intellectual ground. Implicit in the new AI paper-generation tools is the assumption that specific word choices in an article or book are of lesser value than the overall interpretation of sources or data. That may be true in a general sense, and surely many readers of academic articles, ahem, skim, but picking words carefully can increase an article’s power of persuasion and impact. As Daniel Kahneman has shown, a lamentable aspect of human psychology is that we have trouble accepting data as proving a point; we often need well-crafted words and a coherent narrative mapping cause and effect, preferably from someone we see as a peer, to convey the significance of that data and incline readers to accept conclusions. (Even then, alas, human beings can be truly stubborn in their views.) I can now have Claude produce prose that sounds like me, but only the real me can pick the exact words with the right spin and force I’m looking for in a particular sentence. (Plus, I actually enjoy writing; you’ll have to pry my keyboard from my cold, dead hands.)

Furthermore, if we know that a significant percentage of articles are machine-written, we are going to move from careful reading to frequent skimming to a complete abstention from the scholarship in our field. (We will probably have an LLM summarize it for us.) This will obviously greatly harm the exchange of ideas. The only way out of this conundrum is for most practitioners in a discipline to commit to putting in the time and energy to produce and digest thoughtful work. Knowledge production is inherently social — not in the postmodern, constructed-out-of-thin-air way, but embedded in a communal process in which we come to respect, or at least recognize, that other intellects are wrestling with the same problems we care about, and which fosters a continued interest in our common research.

* * *

Then there is the invisible loss of context and detail when AI writes the majority of an academic work. An experiment as a case in point: One day I tried, like an Oliver Sacks case study, to be my wife, who is a scholar of early childhood programs. Using Cowork, I told Claude I wanted to write a paper on how different state policies on child care impact American children and their families. We (Claude and I) decided to use Policy Commons’ invaluable data set, through which Claude assembled a list of 232 state statutes and regulations. Sipping my coffee, I asked Claude to read and process all of these lengthy documents and create a matrix for me so I could quickly assess the contours of child care programs in the United States. But Claude was more caffeinated than I was: like a teacher’s pet it went off and produced, independently, a comprehensive report, not just a table, in a few minutes. It even created its own categories of differentiating metrics, such as infant/toddler:adult ratios, total care group sizes, licensing requirements, academic qualifications for practitioners and directors of programs, sleep protocols, and idiosyncratic state mandates. A few more sips of joe, and a few more prompts to acquire additional data, and I was swiftly on my way to what I thought was a decent meta-review essay.

Now a giddy AI-assisted dilettante, I showed this effortless production to my wife, who proceeded, as an actual expert, to dissect it ruthlessly. Claude got the empirical data mostly correct — no silly hallucinations — but its attempts to extrapolate from the numbers into trends and impacts were clumsy and overly broad. Since my wife actually knows these early childhood programs well, she understands how, on the ground, actual child care sites might differ from written state policies and other sources of information, and she could identify this missing context and additional key details that were invisible to me and my AI buddy. Our armchair scholarship was no match for her decades of experience and knowledge.

The output seemed so good, though…I could totally imagine a less scrupulous academic trying to publish it. The ease of generating an AI paper this way means we will increasingly end up with papers written on the data that is readily available, without questioning how good the data actually is. We will think less about what’s missing.

Yet even with agentic AI — again, Claude Science can reach out to dozens of research databases for material to work with — there are yawning gaps. At the closing plenary this spring at the Coalition for Networked Information meeting in Salt Lake City, “Harnessing the Data Renaissance for Scientific Discovery,” Manish Parashar, Executive Director of the Scientific Computing and Imaging Institute and Chief AI Officer at the University of Utah, highlighted the major work that still needs to be done to create truly rich and comprehensive data sets for many fields, such as cancer research. If we ignore this complex and time-consuming process, and just use AI on top of the data that’s close at hand, we might produce scientific articles more quickly, but we may not make significant breakthroughs. (This is probably already happening.) Parashar, instead, is working with other scientists on a National Data Platform that will aggregate and normalize thousands of sources, without which any AI processes will suffer from a problematic narrowness. In other words, being a good librarian — someone who finds, catalogs, assesses, and merges sources into a coherent and usable library — has become even more valuable.

* * *

Finally, using AI to generate new scholarly papers assumes an iffy theory of intellectual history, that new ideas and discoveries are always implicit but not yet articulated in the existing literature and data. What if new ideas instead come from unique circumstances, lived experience, group interactions, or an individual’s eccentric way of reading and seeing? Yes, the history of science has a number of examples of innovations that seem to be “in the air” and thus “discovered” by multiple people at roughly the same time, such as calculus (simultaneously developed by Leibniz and Newton). But there are many more examples like the one I wrote about in “Can AI Prompt Us to Ask New Questions?,” where a quirky combination of a person’s biography and interests, embedded in a particular social scene, leads to revelations like fractal geometry.

Intellectual history contains both kinds of discoveries, but we wouldn’t want to block the latter, deep river of innovation, and if the overuse of AI in the production of scholarship reduces the velocity of the human mind and the vitality of intellectual scenes, we might find this source slowly drying up. It is worth remembering that asking good questions is harder than giving great answers. Insightful out-of-the-box approaches, often stemming from unusual, previously unasked queries, are the dark matter of human thought and progress, and they often come from the random interactions and odd interests of particular human beings. It is unclear how AI will replicate these uncommon vibrations and collisions.

In “Illegible Benefits,” a piece by Carlo Cordasco of the University of Manchester that is largely positive about using AI in the production of scholarship, he mentions a lingering concern:

I want to be honest about the costs. My ability to hold together a complex position verbally, under pressure, in a seminar or a conversation, has probably not improved and may have declined somewhat. When preliminary exploration is cheap, you spend less time grinding through arguments from first principles, a grinding that builds fluency that shows up in live exchange.

This is the professorial equivalent of the cognitive decline that we worry about with our students who have been using AI for years — and what atrophies in the seminar room will surely atrophy on the page as well. The solution seems clear: if you would like, use AI for the parts of scholarly work where it excels and check all automated output, while retaining human seniority in orchestrating and expressing the meaning of your research.

It’s going to be tough, though. Armchairs really are comfortable.

The $28.5T forecast compares to U.S. Q1 2026 nominal GDP of nearly $32T, with the estimate for the market of AI enterprise applications of $22.7T about 70% of total U.S. economic output.

1. Source

   The AI bubble isn't just an equity bubble, as Torsten Slok explains in AI Is Penetrating Every Corner of Financial Markets:
   

   AI now accounts for nearly half of all IG issuance, 87% of VC funding and a growing share of HY, underscoring how deeply the AI investment cycle has penetrated every corner of finance.

2. Bryce Elder's This is nuts upon nuts. When’s the crash? compares this bubble with its two biggest predecessors:
   

   Here’s the title page of this month’s Panmure Liberum market update from strategists Joachim Klement and Francisca Reis. Our emphasis in bold below:
   

   In 1929, the cyclically-adjusted P/E-ratio (CAPE) of the S&P 500 reached 32.6x according to Prof. Robert Shiller’s data. This was 1.8 standard deviations above trend at the time. In 2000, the CAPE reached 44.2x, or 3.3 standard deviations above trend – a clear sign of a bubble. However, as our chart below shows, earnings in both instances were within normal range, less than one standard deviation above trend.
   
   Today, the CAPE is at 41.0x, or 2.9 standard deviations above trend. Once again, we are clearly in bubble territory for stock market valuations. However, unlike in previous bubbles, we are having extremely high CAPE at a time when earnings themselves are 1.8 standard deviations above trend. In other words, we are in a valuation bubble at a time when earnings are in a bubble themselves.
   
   If we correct for the earnings bubble, the current CAPE would be 67.6x or 4.6 standard deviations above trend, a bubble that surpasses anything ever seen in US history by an extreme margin. If valuations followed a normal distribution (which they don’t, so don’t take this literally), this would happen in 0.00019% of months or once every 43,432 years.

3. But Panmure Liberum is underestimating how bad things are. Baolian Wang's The $69 Billion Mirage: How an Accounting Rule Inflated S&P 500's Q1 Earnings by 12% explains:
   

   A massive chunk of this quarter’s blockbuster “growth” didn’t come from selling more software, shipping more microchips, or delivering more packages. Instead, it came from an accounting rule that forced massive, illiquid “paper gains” onto the income statements of tech giants.
   
   In Q1 2026 alone, just three companies—Alphabet, Amazon, and Nvidia—reported a staggering $69.2 billion in non-operating windfall under their Other Income and Expenses (OI&E) lines. When you run the macro numbers, this single accounting phenomenon artificially inflated the entire S&P 500’s quarterly earnings by about 12%.

   That 12% takes the factor from 67.6 to 75.7. Wang notes that correcting for the 12%, "the Q1 2026 earnings growth rate will not be very different from the 5-year average of 16%." In other words, the bubble is feeding upon itself — increased stock prices causes increased earnings causes increased stock price ... But suppose, for example, that OpenAI were to suffer a down round. Then Alphabet, Amazon, Nvidia and others who included paper gains in the "other income" on the way up would have to include paper losses in their income on the way down, amplifying the crash. OpenAI's last round valued the company around $750B and they were planning an IPO for at least $1T, but had to postpone it.
4. Back in March Jared James Grogan published The End of the Foundation Model Era: Open-Weight Models, Sovereign AI, and Inference as Infrastructure setting out the forces changing the structure of the AI market:
   

   The foundation model era — roughly 2020 to 2025 — is over. The forces that defined it have inverted. Open source models have reached frontier performance while inference costs approach zero, exposing what was always structurally true: pre-training large language models at scale is not a durable competitive moat. The US government's formal designation of Anthropic as a supply chain risk in February 2026 accelerated a transition already underway — but did not cause it. The paper argues that the AI industry is restructuring simultaneously along four axes: economic, as the circular financing structure that inflated foundation model valuations collapses; technical, as the pre-training scaling paradigm gives way to post-training optimization, test-time compute, and agentic composition; commercial, as application-layer integrators displace the foundation model companies whose commodity they now consume; and political, as the government asserts its historic role as gatekeeper of strategic technology. These are not separate disruptions. They are one structural shift, arriving together.

   Grogan further argues that:
   

   The most consequential and least-discussed dimension is the permanent divergence between commercial AI and a classified national security AI track — built on different data, governed by different rules, and developing capabilities the public ecosystem cannot see, measure, or govern. Like every dual-use technology that has altered the calculus of state power, AI is being brought under government authority not by design but by the structural logic of what it is. The paper further argues that open-weight models are the counterintuitive instrument of sovereign control: a government that holds the weights commands the capability on its own terms, without dependence on vendor policy, financial continuity, or personnel clearance. The apparent openness of distributed model weights is, from a deploying government's sovereignty standpoint, the most governable architecture — because what cannot be withdrawn by a vendor's API policy cannot be taken away.

   Grogran implicitly assumes that AI is useful but that the current margins and thus the valuations aren't sustainable.
5. Carl Franzen's DeepSeek-V4 arrives with near state-of-the-art intelligence at 1/6th the cost of Opus 4.7, GPT-5.5 explains the impact of open-weights models on pricing:
   

   DeepSeek-V4-Pro is priced through its API at $1.74 USD per 1 million input tokens on a cache miss and $3.48 per million output tokens.
   
   That puts a simple one-million-input, one-million-output comparison at $5.22. With cached input, the input price drops to $0.145 per million tokens, bringing that same blended comparison down to $3.625.
   
   That is dramatically cheaper than the current premium pricing from OpenAI and Anthropic. GPT-5.5 is priced at $5.00 per million input tokens and $30.00 per million output tokens, for a combined $35.00 in the same simple comparison.
   
   Claude Opus 4.7 is priced at $5.00 input and $25.00 output, for a combined $30.00.

   Six times cheaper for an equivalent product is likely to cause pricing pressure on the incumbents, who need to raise not reduce prices. DeepSeek is likely also subsidizing usage, but they do have real advantages. First, they have fewer resources so are forced to to be inventive. Second, the 40% of infrastructure capex that isn't the racks is much cheaper in China. Third, the power component of opex is much cheaper and more available in China. The result is:
   

   In practical terms, DeepSeek does not need to win every leaderboard row to matter. If it can deliver near-frontier performance on many enterprise-relevant agent and reasoning tasks at roughly one-sixth to one-seventh the standard API cost of GPT-5.5 or Claude Opus 4.7, it still forces a major rethink of the economics of advanced AI deployment.
   
   DeepSeek-V4-Pro-Max is clearly the strongest open-weight model in the field right now, and it is unusually close to frontier closed systems on several practical benchmarks.
   
   While GPT-5.5 and Claude Opus 4.7 still retain the lead in most direct head-to-head comparisons across the company's benchmark charts, DeepSeek V4 Pro gets close while being dramatically cheaper and openly available.

6. Dan Davies agrees that the margins aren't sustainable but differs as to why in tokenalysis and john henry:
   

   Which then brings to mind another issue – how confident are we in the pricing power that underpins that 60% gross margin in the first place? In the last paragraph I was talking about the R&D equivalent of a price war, but the normal kind is also possible. The combination of price-sensitive B2B customers, big fixed costs and rewards going to the dominant player doesn’t suggest to me that pricing power is going to be sustainable indefinitely.
   
   But, I think there’s a danger of missing the big picture here. Which is that, when large companies are telling their employees to be sensible and use AI tokens wisely, then the game is up. The race is over and John Henry won against the steam hammer. If you need a human being in the loop to decide on the allocation of AI tokens, then all those predictions of mass redundancy are gone.

7. In A Slower AI Payoff Would Be Everyone's Problem, Torsten Slok is mainly focused on the fallout we can expect even if the bubble deflates gently. He asks
   

   But what if the payoff takes longer than consensus assumes? That question is particularly pressing given that token prices continue to decline and Chinese models are gaining ground, both in their share of the world's most-used models and in token usage, where they now lead their US counterparts among the top 20 models,

   Source

   Slok provides two charts, the first tracks market share by country of origin monthly since January 2025 among the top 50 models. It is bad for the US, showing a steady erosion of US market share until, eyeballing it, as of May 2026 it is roughly a 60/40 US/Chinese split.
   
   Clearly, the market is voting with its feet that the prices charged by US AI companies are unsustainable.
   
   

   Source

   The second compares monthly token use among the top 20 models by country of origin between May and June this year. It is much worse for the US.In May the Chinese had 80% of the US usage. In June they had 185% of the US usage. If this rate of market erosion were to continue for a few months it would be impossible for investors to continue to imagine the golden future awaiting OpenAI, Anthropic, xAI, Meta, Oracle and the neoclouds.
   
   Slok's analysis of the fallout of the bubble deflating is worth reading. This is an issue I plan to return to in a future post.

8. Source

   Kevin Walmsley's We were wrong about DeepSeek. Now Chinese AI companies export trillions of AI tokens also focuses on the erosion of US market share:
   

   Chinese AI providers are making more money; Zhipu’s revenues went up 60 times in Q1 compared to last year; Alibaba is the company behind Qwen, and their revenues are up 15 times just since the beginning of this year. The lion’s share of the profits, though, are still being realized by the hardware side; the chipmakers, at least for now. The model companies’ revenues are rising, and steeply. But so is their cost of compute.
   
   That dynamic is causing some Chinese labs to raise prices; the cost to use Tencent Cloud increased by over five times in March, and Alibaba, Zhipu, and ByteDance also hiked prices.
   
   DeepSeek, however, went the other way. Their latest version is priced at just a fourth of their introductory product, and they rolled out dynamic pricing that is aimed at corporate users, who use their models during the workday.

   Source

   This price difference is having an :
   

   But even with these price increases, Chinese models just cost far less than those on offer from Silicon Valley, and explains those big jumps in the exports, we can say, of Chinese AI tokens. The LLM’s out of China are “90% as good at 10% of the cost”, and American firms are buying more tokens from Chinese companies. In early 2025, token demand from Chinese LLM’s was about zero. Even the release of DeepSeek didn’t move the needle much, but by the end of the year the secret was out, and it’s been a choppy but steady ride up to 46%, today.

   Why would you invest in a company whose competitors were “90% as good at 10% of the cost” and which was rapidly losing market share?

9. Source

   If the margins, and thus the rational valuations, of AI companies are unsustainable, how long can the market remain irrational? In The Second Derivative: Why No One Understands the AI Boom Groundbreaker starts by examining the 2008 crash:
   

   The implicit underwriting assumption, shared by originator and borrower alike, was that the loan would never reach its reset: rising home values would manufacture equity, the borrower would refinance into a fresh teaser and the clock would start again. The structure was a treadmill, and the treadmill was powered by appreciation. It worked spectacularly while it worked. Nearly four in five subprime hybrid ARMs originated in 2003 had been refinanced away by the end of 2006.
   
   Now watch the timing. National home-price appreciation did not crash in 2006. It decelerated. The year-over-year rate of gain, which had run in the mid-to-high teens through 2004 and into early 2005, began bleeding off - still positive, still printing green, but slowing. Prices were higher than they had ever been. And yet, with prices at their peak and still rising, subprime delinquencies inflected upward. ...
   The deceleration was endogenous to the structure; the structure required ever-accelerating prices to keep refinancing its way out of its own reset schedule, and no series accelerates forever.
   
   The second derivative was always going to roll over. When it did, the first derivative followed it down through zero, negative equity spread from the margin inward, and the defaults the market insisted were caused by “falling prices” had in fact begun a year earlier, when prices were still rising but had stopped rising faster.

   Why does the 2008 analogy apply this time?:
   

   The market is pricing AI as a technology cycle when its actual anatomy is that of a credit-driven real estate cycle - which is precisely why the 2008 mechanics apply - and the two break for entirely different reasons.
   ...
   Walk down the AI build-out and every feature is a property development in disguise: a data center on entitled land, financed with debt against the structure and leased to tenants on take-or-pay terms. This is not a software business that happens to own servers. It is a real estate business that happens to compute.

   The AI build-out is being financed with debt:
   

   When a hyperscaler or a neocloud reports a record capex figure, the financial press reads it as confidence, as proof of demand. Read it instead as origination volume. Each gigawatt of committed build is a loan extended to whichever tenant has signed the take-or-pay beneath it, and the credit quality of that loan is precisely the credit quality of the tenant. The market is celebrating loan growth and calling it revenue growth.

   Loans in this market take the form of Remaining Performance Obligations (RPOs). The biggest borrower in this market is OpenAI:
   

   OpenAI has committed to pay for compute on a scale without precedent in corporate history: multi-year, take-or-pay capacity contracts whose aggregate obligations run to the hundreds of billions of dollars. Against them sits an operating business that does not yet earn a profit - revenue real, large, and growing quickly, but short of covering the company’s own cash burn and nowhere near covering the contracted payments.
   
   Those payments are therefore not serviced out of earnings. They are serviced out of financing, and financing, for a borrower in this position, is available on a single condition: that each new round price above the last.

   Does OpenAI's financing meet this requirement?:
   

   Measured as a level, OpenAI’s valuation is the most remarkable appreciation in the history of private markets - roughly $86 billion in early 2024, then about $157 billion, $300 billion, $500 billion, and approximately $852 billion by the spring of 2026. Measured as a rate of change, the same series inverts: the round-over-round step-up ran 1.83×, 1.91×, 1.67×, 1.70×, and falls to roughly 1.23× implied by the reported public-offering target. Private marks are inherently lumpy - negotiated, episodic, set by a handful of insiders - so no single step is decisive. But the trend is unmistakable: it bends down, and it bends hardest at the one mark set by the deepest, most unforgiving pool of capital - the public market. The implied IPO step-up is both the lowest in the sequence and the hardest to negotiate, and it is the one the structure must actually clear. This arithmetic is also the most probable explanation for OpenAI’s recent IPO delay.

   Source

   In the same way that the private stocks generating unrealized gains are not liquid assets, neither are the Remaining Perfoemance Obligations:
   

   An RPO is not a liquid asset; it is a forward contractual commitment - a promise of future payment in exchange for future compute. And a multi-year commitment is worth exactly the creditworthiness of the entity on the other end of it. When that entity is investment-grade and cash-generative, the backlog is what it claims to be: high-quality visibility, merely deferred. When that entity is a pre-profit company that loses tens of billions a year and can pay only by continuously refinancing its own equity valuation, the backlog is something else entirely. It is a subprime commitment, used to justify massive, un-depreciated capital expenditure, reported to shareholders as structural strength.
   
   Now price the credit quality of that book. Of roughly $2.1 trillion in aggregate contracted backlog across the four big platforms, about half - on the order of $1.05 trillion - is owed by OpenAI and Anthropic. Microsoft’s book is about 49% these two names; Oracle’s is 54%, with roughly $300 billion owed by OpenAI alone; Google’s is 43%; Amazon’s is 51%.
   
   The hyperscaler has, in economic substance, extended a concentrated, unsecured loan to cash-burning tenants. The RPO that Wall Street values as forward revenue is, in reality, a credit exposure to borrowers with no operating income.

10. Source

    Major financial institutions are similarly concerned. In their 2026 Annual Economic Report, the Bank for International Settlements writes:
    

    In the near term, the ongoing AI investment boom raises questions about the sustainability of the current economic expansion. The five largest hyperscalers are set to spend over a trillion US dollars on AI-related capital expenditure from 2025 through 2026. These commitments are outpacing earnings and the free cash flow of these firms, leading some to issue debt to raise additional financing (Graph 11.A). This investment race may be partly driven by the perception that only a small number of players with superior technology will ultimately dominate the market shares. The intense competition raises the risk of firms over-committing resources to investment projects with still uncertain returns, leaving all firms vulnerable to disappointments in AI payoffs. Model analysis based on such contest motives highlights the downside risk of current AI exuberance. As competitive pressure drives capex higher, the net economic surplus – the total payoff less investment costs – declines for the sector as a whole and could turn negative in adverse scenarios (Graph 11.B). Disappointment in returns could trigger a sudden pullback in financing and turn the capex boom into a protracted investment bust, with potential knock-on effects on financial conditions (see below).

    The BIS has noticed another looming problem:
    

    Another risk is that the AI boom runs into a supply side roadblock. The AI build-out has recently been facing growing bottlenecks in electricity, advanced semiconductors and grid equipment. Fast-growing demand for computing power is already pressuring electricity prices and input costs, with potential spillovers to inflation. Looking ahead, these temporary shortages may also amplify over-investment, as firms attempt to lock in future capacity through long-dated contracts that further expose them to any disappointments in demand.

    Unlike Groundbreaker, the BIS uses more conventional historical analogies:
    

    Historical episodes of investment booms offer instructive parallels (Graph 11.C). The canal mania of the 1830s, the British railway mania in the 1840s, the electrification exuberance of the late 1920s (roaring 20s) and the dotcom boom of the late 90s all shared one common trait: a genuine technological breakthrough that attracted capital in excess of what commercial returns could ultimately justify. These episodes ended with an eventual reversal in investment, inducing economy-wide recessions. The scale and pace of the current AI investment boom accompanied by expectations of large productivity payoffs bear resemblance to these precedents, highlighting potential downside risks in the near term.

11. And Eric Katz reports that the US Treasury Has an Internal Report Warning About the Dangers of an AI Bubble:
    

    A draft report inside the Treasury Department is set to warn of the risks posed by the artificial intelligence market, likening key aspects of it to the dotcom bubble that upended the U.S. economy when it burst in the early 2000s.
    
    The document, the existence and contents of which have not been previously reported but was obtained by NOTUS, is a significant departure from the Trump administration’s public tone, which has focused on encouraging unrelenting investment to unlock exponential growth.
    
    Career Treasury analysts found that AI firms are more deeply entrenched in the U.S. economy than their dotcom predecessors and pose significant risk to the entire system if financial conditions change, productivity goals are missed or various choke points stymie growth.

12. Vanderbilt University's Asad Ramzanali has a detailed look at the range of impacts from the burst bubble, and an set of optimistic suggestions for policy responses in After the AI Crash. He frames the problem thus:
    

    Companies are investing trillions of dollars based on tens of billions of dollars in revenues. Analysts at J.P. Morgan anticipate $5 trillion of AI infrastructure investment in the next five years. They estimate that the industry will need to generate annual revenues of $650 billion to justify this level of investment, while consultants at Bain & Co. estimate $2 trillion in needed annual revenues. Yet, OpenAI and Anthropic earned $13 billion and $4 billion, respectively, in 2025 revenues. OpenAI’s own financial expectations suggest negative cash flow until 2030, and Anthropic expects a small profit no sooner than 2029. Alphabet, Meta, Amazon, and others may experience increased marginal revenue from integrating AI into existing products, but that is far from certain.

    Note that over the next 5 years around $3T (60% of $5T) of the investment in AI infrastructure goes in to buying the hardware which should be fully depreciated over much less than 5 years. So the gap between the investments and the revenue is much bigger than it appears.
    
13. The cracks are starting to show. Reuters reports that Blackstone's QTS terminates Digital Gateway data center project in Virginia:
    

    Blackstone's QTS said on Thursday it had terminated its planned Digital Gateway data center project in Virginia and withdrawn the associated filings after years of planning and regulatory review.
    
    The data center operator has faced years of local opposition and litigation over the project, despite it being approved by the Prince William Board of County Supervisors.

14. Among the hyperscalers, Oracle is the most exposed because, as Ed Zitron noted:
    

    And Oracle ... is a company that, even before the AI bubble, was massively indebted. It just so happens that, as a result of its tryst with OpenAI, Larry Ellison saw fit to twist the debt knob to eleven.

    Now, Brody Ford reports that Oracle Warns Its Splurge on AI Data Centers May Not Pay Off:
    

    Six firms alone — including Oracle, Microsoft and Meta — have committed $850 billion for data centers leases that haven’t begun yet. Oracle holds the largest share of these commitments owing to its $300 billion Stargate contract with OpenAI.
    
    When Oracle mentions the risk of nonpayment, the unnamed elephant in the room is OpenAI. As part of the Stargate deal with the AI company, Oracle is developing massive data centers across the country to provide cloud computing power. For this plan to work, OpenAI needs to pay its Oracle Cloud Infrastructure bills.
    
    “Some of our customers may be highly leveraged and subject to their own operating and regulatory risks and, even if our credit review and analysis mechanisms work properly, we may experience risks of non-payment and non-performance in our dealings with such parties,” Oracle said in the filing.

15. A month ago Laura Benitez et al reported that SoftBank Attempt to Get $6 Billion OpenAI Margin Loan Stalls:
    

    SoftBank Group Corp.’s talks with potential creditors to raise at least $6 billion from a margin loan backed by its OpenAI stake have stalled, people familiar with the matter said, just weeks after the Japanese conglomerate cut its initial target from $10 billion.

    But now Echo Wang reports that SoftBank renews talks for $10 billion loan against OpenAI stake, adds concessions, sources say:
    

    SoftBank Group has reopened talks with a consortium of lenders for a $10 billion loan backed by its stake in OpenAI, after earlier attempts to secure a loan stalled over concerns about the difficulty of valuing private companies, two people familiar with the matter said.
    
    To make lenders more comfortable, the Japanese technology investor is offering to guarantee repayment of the loan, giving banks recourse to SoftBank if the OpenAI shares pledged as collateral lose value, the people said.

    This all seems to indicate that potential lenders, such as banks, are highly skeptical of the value of OpenAI stock.

16. Source

    Despite Grok being so bad that employees use Claude, Musk is touting SpaceX as an AI company. This resulted in the most overvalued IPO in history, which failed to raise enough money to avoid the immediate need to borrow $25B. Nir Kaissar's SpaceX Is Junk. That’s What the Bond Market Says reports on the bond market's reaction:
    

    Ratings companies and the bond market have very different views about how things are going. SpaceX’s bonds have an average rating of BBB across the three majors, Moody’s Ratings, S&P Global Ratings and Fitch Ratings, according to credit scores compiled by Bloomberg. In the alphabet soup of bond ratings, it’s the lowest grade still considered quality before falling into junk territory.
    
    The bond market has other ideas. There, quality is judged by a bond’s credit spread or the additional yield it offers above Treasuries with similar maturity. The wider the spread, the lower the quality. Corporate bonds with a BBB rating are trading at an average credit spread of 0.92 percentage point. SpaceX’s bonds, by contrast, trade at a significantly greater average spread of 1.62 percentage points across maturities, higher than BB rated junk bonds’ average spread of 1.55 percentage points.

    The bond market seems to agree with Softbank's lenders about the AI bubble.
17. OpenAI and Anthropic are competing for the next trillion-dollar IPO. Both would need to distract investors from their massive losses by focusing on growth. Recently, Anthorpic has been growing faster than OpenAI, so Keach Hagey and Berber Jin report that OpenAI Considers Drastic Price Cuts, Anticipating War for Users With Anthropic:
    

    OpenAI is considering drastically lowering the prices it charges users as it seeks to win customers from its rival Anthropic.
    
    The company is weighing significant cuts to what it charges for tokens, the unit of measurement artificial-intelligence firms use to bill for their products, according to people familiar with the matter. The move would be in anticipation of similar cuts the company expects at Anthropic, the people said.

    Now Kurt Wagner reports that Zuckerberg Pledges ‘Aggressive’ Pricing With Meta’s First Pay-to-Use AI:
    

    Meta will also introduce a new Meta Model API system, which will be used to collect fees from developers. Its API pricing is roughly 25% of the cost advertised by other top models from OpenAI and Anthropic PBC. Developers will be able to use Meta’s model for free, but only up to a point; they’ll be required to pay for access after reaching a certain token threshold, Zuckerberg said.
    
    “The pricing from some of the other labs is very extreme and has very high margins,” Zuckerberg said, underscoring that his strategy is to get Meta’s technology in front of as many people as possible. “We think that there’s a real ability to be able to offer frontier or very high-level intelligence at a much more affordable cost.”

    Aggressive pricing means Meta Model API will still be 50% more expensive than DeepSeek but not 50% better. Planning to reduce current income in the lead-up to an IPO is an unusual move, but it is a response to AI's Affordability Crisis.
18. Data center demands for power are having serious impacts elsewhere in the economy, as Jeremy Hsu reports in Data centers’ energy demand threatens Trump’s “Made in America” plan:
    

    Factory electricity bills are generally rising faster than those for other business customers or residential customers, according to a Reuters analysis. It highlighted the example of the Belden Brick Company, a 141-year-old brick manufacturer in Ohio, whose electricity bills have soared from $1,600 to $12,000 per month due to a higher monthly capacity charge in the 13-state region served by the grid operator PJM Interconnection.
    ...
    The Ohio-based steelmaker Metallus described its electricity costs as having jumped by 70 percent since 2024, leading the company to pay an extra $15 million in energy costs annually.
    
    The higher electricity costs for manufacturers coincide with the attraction of large AI data center projects with substantial electricity needs to many states in PJM territory. That data center growth has driven up PJM’s capacity prices—paid to power generators according to supply-and-demand forecasts—from $28.92 per megawatt-day in 2024 to $329.17 per megawatt-day in 2026, according to Reuters’ reporting.

19. Source

    As I see it, there are three separate markets for LLMs. First, there is an embedded market that runs on low-cost, low-power hardware and open-weights models. Small AI Models Gain Traction Around the World by David Berreby provides examples:
    

    For example, a drone-based system developed by Bala Murugan and colleagues at the Vellore Institute of Technology, in India, takes photos of cashew plants and quickly identifies those with splotches that indicate disease. All the processing takes place on the drone itself, so there’s no need for a computer on-site, nor for a connection to a central server.
    
    Using small language models trained for a specific problem, and sometimes running on cheap, low-power devices, other small-AI implementations have been developed to identify ant infestations in a Uruguayan vineyard, detect the presence of malaria-carrying mosquitoes in a number of nations, and run electrocardiograms from an Arduino device in parts of Brazil that lack access to more complex equipment.

    It isn't just that small hardware, such as the Raspberry Pi 5, is getting more powerful but also:
    

    the shrinking footprint of language models. Both Google DeepMind’s Gemma 4 (released in April) and Alibaba’s Qwen 3.5 are “fantastic” for small AI, Rovai says. Both models are “open weight,” meaning users can adjust the connections between parameters to suit their needs. This makes it easy, for example, “to take a lot of data from, say, the milk industry and retrain the model specifically on that,” Rovai says.

    The hyperscalers and AI platforms like OpenAI and Anthropic will garner no income from this market, because:
    

    “I think the future of AI is not like one giant model, at a center. I think it’s millions of small, precise models deployed at the edge, each one solving like a specific problem, a specific context,” Alonge says. This is partly because much of humanity—including people in parts of rich countries as well as the developing world—lives without access to cutting-edge frontier models. But, he says, it’s also because those models are not sustainable.
    
    “If someone is not subsidizing it, most people will not be able to afford those models. So those of us who are said to be small-AI developers are the ones who will have to build for the majority of the world,” Alonge says.

20. Second, there is a consumer market. Last month, Nvidia announced a product for the prosumer laptop market with significant LLM capabilities. @pramodchandrayan described it in NVIDIA Just Put a 120-Billion-Parameter AI Model in Your Laptop. Here’s What That Actually Changes.:
    

    At Computex 2026 in Taipei on June 1st, CEO Jensen Huang announced the RTX Spark superchip — a single piece of silicon that combines a 20-core Arm CPU, a Blackwell GPU with 6,144 CUDA cores, and 128 gigabytes of unified memory, connected by NVIDIA’s NVLink chip-to-chip interconnect. The whole package delivers up to one petaflop of AI compute in a laptop form factor.
    
    The number that matters: RTX Spark can run a 120-billion-parameter language model entirely locally, with a context window of one million tokens, without a single byte leaving your machine.
    
    To put that in perspective: GPT-3 had 175 billion parameters and required clusters of A100 GPUs to run. The model that stunned the world when it launched in 2020 is now approximately the size of what fits in a consumer laptop chip announced this week. The capability that required a data centre in 2020 is coming to a device you carry in a bag in 2026.

    Phones can already run small LLMs:
    

    In 2025, slightly more than a third of all smartphones shipped worldwide were capable of running generative AI, and that figure will reach 45 percent by the end of this year, according to the technology research firm Counterpoint. By the end of next year, slightly more than half of all smartphones will be able to run a small AI model.

    If Nvidia can already put a 120B-parameter in a laptop, it will only be a few years until phones can run a GPT-3-class model, good enough for almost all consumer needs. Apple and Google own that channel. Owning the channel is better than owning the technology. They will dominate consumer AI, and the other players will garner no revenue from this market.
    
21. Third, there is an enterprise market; all that is left to generate the revenue to service the debts fuelling the AI bubble, lets say $2T by 2030. There are a number of problems that make this unlikely.
    
    First, there are very few documented cases of LLM deployment that resulted in enough productivity improvement to cover its unsubsidized costs.
    
    Second, the Trump administration just demonstrated that deploying mission critical systems on AI platforms such as OpenAI or Anthropic means your company can be disabled at 90 minutes notice with no recourse.
    
    Third, this means that companies will have to run mission-critical LLMs on open-weight models on in-house hardware if they are not to be vulnerable to the whims of the US president.
    
    Fourth, systems such as RTX Spark show that good enough in-house hardware is likely to become relatively cheap compared to the unsubsidized cost of the AI platforms. In-house systems need much less over-provisioning for demand spikes, and because they aren't shared they don't need to be as fast.
    
    Fifth, companies need to balance the productivity benefits (if any) of mission-critical LLMs against the productivity costs they bring. These include a vastly greater attack surface, technical debt from reduced developer understanding of the software, and so on.
    
    Thus it seems likely that the hyperscalers and AI plaforms will generate far less revenue than they expect, because they will be restricted to non-mission-critical applications with lower productivity gains, and thus lower pricing power. They will thus be unable to cover the debts they are incurring to build massive data centers predicated on centralized systems dominating (an inflated estimate of) the entire enterprise market.
    
    David Wallace-Wells' Did We Make the Wrong Bet on Big A.I.? discusses a "televised rant" from Palatir's CEO Alex Karp that supports this argument:
    

    Karp had been softly floating his critique for some time, but the CNBC event looked like a proper coming out. Just one day earlier Palantir had published a kind of manifesto devoted to what it described as the all-important principle of “A.I. sovereignty.” The central argument: Companies should seek to build their own A.I. tools, not just customize those on offer from the frontier labs. This might mean relying on open-source L.L.M.s rather than the proprietary ones on which the A.I. boom has mostly been built in America, but it would amount to a liberating declaration of independence from Big A.I., which in Karp’s estimation was sucking up much more value than it was generating.

22. Source

    The Economist's How to turn compute into a financial asset reports another bad sign (my emphasis):
    

    Locking in a price with a multi-year neocloud contract insures a buyer against compute getting more expensive but not against it getting much cheaper. So as businesses around the world spend ever more on compute, they want to be able to hedge against price volatility just as they insure against changes in energy tariffs, interest rates or foreign-exchange movements—ideally in deep and liquid derivatives markets.
    
    Two startups want to help companies do this, by turning nascent indices tracking compute costs into a futures market. Silicon Data, founded in 2024 and backed by DRW, a trading firm, has paired up with CME Group, which operates large derivatives exchanges. Ornn, created by recent graduates of the Massachusetts Institute of Technology and run from a flat rather than an office just a few months ago, has paired up with Intercontinental Exchange, the parent company of the New York Stock Exchange, to do the same. Both aim to launch compute futures later this year, to be traded on their partner exchanges.

23. One characteristic of bubbles is overbuilding the infrastructure. Signs of overbuilding of data centers include that both SpaceX and Meta are now in the business of rentling their GPUs to the competition.
24. Victor Tangermann's Zuckerberg Admits That AI Is Not Working Out the Way He Imagined is on-trend:
    

    As morale is hitting rock-bottom, his company is heavily relying on its competitors' AI models to build out its own in-house tools. And despite the many billions of dollars the company has spent in its flailing efforts to keep up in the AI race, even Zuckerberg himself is now acknowledging that progress is nowhere near where he wanted it to be.
    
    As Reuters reports, Zuckerberg admitted during a town hall last week that AI agents in particular aren't progressing as fast as he anticipated, a devastating revelation following enormous layoffs that wiped out thousands of roles at the company.
    
    The "trajectory of the agentic development over at least the last four months hasn't really accelerated in the way that we expected," he said according to a recording obtained by Reuters.

Introducing CABot, a chatbot that makes climate knowledge more accessible to everyone, keeping information clear, factual, and credible.

The post A Climate Chatbot for Trusted and Transparent Climate Information first appeared on Open Knowledge Blog.

The Future's So Bright, I Gotta Wear Shades

Timbuk3

Nachtrieb & Smith Fig. 2

While AI electricity consumption was only 1.5% of the global total in 2024, its power demand has grown at a rate of 0.127 yr⁻¹ since 2015, accelerating to 0.15 yr⁻¹ over the last five years. Projecting from this 2024 baseline, AI’s electricity demand is on track to achieve parity with the rest of the world’s combined consumption by approximately 2050.

1. Non-renewable resource depletion
2. The Kelvin Limit
3. The renewable resource limit
4. The Dyson Limit
5. The Asimov Limit

Even with an infinite energy source, the laws of physics dictate every unit of energy used eventually becomes waste heat. On a planetary scale, this heat must be radiated into space to maintain a stable environment.

Earth stays at a life-sustaining temperature by balancing received solar energy with infrared radiation emitted back into the cosmos. Human energy consumption from “terrestrial” sources, such as nuclear fusion or fossil fuels, adds “new” heat to this balance. Because this energy was not already part of the solar-to-earth flow, the planet must reach a higher temperature to increase its radiative cooling capacity and shed the additional load.

The “Kelvin Limit” defines the point where this added waste heat pushes Earth’s surface temperature to 373 K (100 ◦C), the boiling point of water. At this threshold, the planet becomes physically uninhabitable.

The solar power arriving at the Earth’s cross-section is Lα ≈ 5474 × 10³ EJ yr⁻¹. A portion of this energy is immediately reflected into space by the Earth’s albedo (a), which represents the planet’s reflectivity. Based on NASA data, Earth’s albedo is approximately 0.30, meaning 30% of incoming light reflects away while the remaining 70% is absorbed as heat (P_a).

The model assumes an initial equilibrium where absorbed solar power (P_a) equals the power radiated at the baseline temperature (T₀).
...
The limit occurs at time t₂, when the combined heat of the Sun (P_a) and human energy demand (D) requires the Earth to reach the boiling point (T₂) to maintain equilibrium.
...
This thermal wall represents a hard physical limit. Technological efficiency cannot bypass it; higher energy use to drive AI or industry simply accelerates the transition toward this planetary boiling point.

Table VIII

Table IX

These figures represent a fundamental shift in the prospects of civilization under AI. Under the 15% growth rates currently demonstrated by AI infrastructure, we quickly accelerate past all projected limits. The thermal “Kelvin Limit” (k ≈ 10), which would normally take ten centuries to reach, suddenly appears in just 67 yr, well within a single human lifetime.

forecast to be around $208.96 trillion, $11.04 trillion up compared to $197.91 trillion in 2024.

The scraping problem is worse than anyone can imagine and thanks to my friends at Sourceware we have some real data to prove it.

I've been working more on Anubis' reputation database and I've run into a really weird discovery: 80-90% of the hits created by the honeypot feature are from IP addresses that do not belong to any existing threat monitoring lists.

Here's a breakdown of the honeypot hits Sourceware has gotten in the last few months:

Assessment of ./data/manually-submitted/sourceware/202607141625.txt against ./var/reputationdb.mmdb

In case this interests you, I have put the full tables in Appendix A: Full tables for the reputation database input.

Flags (of flagged addresses)

Categories (6 distinct, of flagged addresses)

Providers (126 distinct, of flagged addresses)

Mara

Methodology note: "provider" here means one of two things:

1. The company or organization associated with the IP address.
2. The place the list was gotten from.

For example, scaleway is based off of Scaleway's publicly posted IP address ranges, firehol-level1 is based on a daily snapshot of FireHOL's Level 1 IP list, and fdo is based on data contributed by the administrators of freedesktop.org.

(remainder snipped for brevity)

Countries (229 distinct, of all addresses)

This doesn't list data from 204 additional countries. Given that the ISO 3166-1 standard comprises 249 countries (193 of which are UN members), it's safe to say this is a global problem.

ASNs (21116 distinct, of all addresses)

There are 18069 more ASNs not listed.

How Anubis' honeypot works

In order to collect data on how widespread the scraper problem is, I added a honeypot feature to Anubis. On every challenge page it adds semantically invalid HTML akin to the following:

<script type="ignore">
          <a href="/.within.website/x/cmd/anubis/api/honeypot/<uuidv4>/init">Don't click me</a>
        </script>
        

Visiting that page gets you cheap to generate vacuous anti-content that has two links to other pages. This is intended to get badly written scrapers caught in the honeypot so they scrape that instead of the protected website. I made it on a whim but thought it would be great for collecting data on how widespread this problem actually is.

This is a global problem

Based on the data I've seen, this is a global problem. If I had to guess where most of this traffic is coming from, it's from compromised smart appliances contributing traffic to proxy networks. I don't think there's any way to make a real impact on this problem without concerted simultaneous global action.

TL;DR: the scraping problem is actually widespread enough that web application firewalls like Anubis make sense.

A presigned URL is a replay attack you did on purpose.

Replayable auth tokens are the textbook way to create vulnerable systems, but Tigris ships them as a first-class feature with presigned URLs and so does every other object storage system on the planet. However this isn't an oversight because presigned URLs turn a weakness into a feature.

Replay attacks are a real problem and the classic fix is miserable

When you authenticate a request with Amazon's SigV4 protocol for Tigris, your client boils down the request to a canonical form: a SHA256 hash of the request's method, path, query parameters, signed headers and a SHA256 hash of the payload. It runs the result of that through HMAC with a signing key derived from your secret access key. Nothing secret ever crosses the wire. The server derives the same key as the client, does the same canonical form transformation, and compares the result.

Being able to make a valid signature proves that the request came from someone holding the secret access key, but it proves nothing about when that request was made. A signature that was made a year ago would still be valid today or any other time you send it, so in theory an attacker could warehouse your signed requests only to replay them en masse later. Imagine sitting on a pile of signed "create EC2 instance" calls only to spam them all out at a later date. You would be a twirling moustache villain able to spawn dozens of servers at a moment's notice.

Traditionally the fix is to bake a nonce (number used once) into the signature (sorry to any British readers in the audience). This makes every signature differ because that nonce differs.

However with great power comes great responsibility and making sure that something used once is only used once is a surprisingly hard distributed systems problem. You can't verify that something is only used once locally. Say you store them all for a 15 minute smear window at a low request rate like 10,000 Bq. That's 9 million live nonces, and every frontend node needs to have a consistent view of the whole set as it churns.

You have made your fast authentication check slow from having to ensure things are only used once.

What you want instead is something that changes constantly without coordination and invalidates those old signatures for free. For an added bonus you want this to also be in the standard library of every programming language.

Sign the clock

There's exactly one value that changes constantly, (mostly) monotonically, and is already actively coordinated across all elements of the stack: the clock. Your OS already keeps time in sync with the public NTP pool (or a private NTP pool if you are cool enough to have radioactive PCI cards laying around). Without an accurate view of time you can't make TLS connections, which means you can't make API calls to Tigris at all, so the auth layer gets to assume a working clock exists.

SigV4 signs the current time into the request. If an attacker gets their greasy hacker paws on a signature, they have about 15 minutes to use it before it becomes a digital paperweight. If time is an input to the signature and the time changes enough to invalidate the signature, the signature is null and void. Sure in theory a sufficiently funded attacker could create a black hole in your datacentre and disrupt temporal flow, but at that point the planet is probably toast which makes the attack profile moot. Commit mass object storage fraud with this one neat trick! The department of temporal investigations will have hated it!

This makes your verification stay stateless. Everything gets checked against the system clock the server already needs and you can give clients a 15 minute signature smear window as a grace period for old or delayed clients (exponential backoff is a good thing and Tigris will reward you for doing it).

Of course the real thing keeping the signatures safe on the wire is TLS (HTTPS). If that is broken we have bigger problems and object storage fraud is the least of our problems.

Time is the only nonce you need because both sides already agree on it anyways.

Some thorns have roses

Presigned URLs take the replay tolerance that SigV4 spends all this effort nerfing and then buffs it into the feature. The entire auth dance gets flattened into URL parameters that any HTTP client can use, be it a browser, curl, Go's net/http, or something you made by bit-banging HTTP over a socket. Here's a real presigned URL I sundered into visibility:

https://xe-sophia-base.t3.tigrisfiles.io/moby-dick.txt
        ?X-Amz-Algorithm=AWS4-HMAC-SHA256
        &X-Amz-Credential=tid_ubYBNEYAmTciLVwszw_QrUXDmtcyQisryryGfxgznDsCnOvNqh/20260714/auto/s3/aws4_request
        &X-Amz-Date=20260714T043308Z
        &X-Amz-Expires=3600
        &X-Amz-SignedHeaders=host
        &X-Amz-Signature=0dcaf4972911527a7582ff36ea457e9760a8efccb6655a178685aaa281637a36
        

Here are the parts (forgive the AI looking listicle because this is genuinely the best way to format this):

• X-Amz-Algorithm: the signature scheme. Effectively always AWS4-HMAC-SHA256.
• X-Amz-Credential: the access key ID plus the credential scope — date, region, service, and the literal terminator aws4_request. The signing key is derived by chaining HMAC through exactly those parts, so a signature is only ever valid for that day, that region, that service.
• X-Amz-Date: the second the URL was born, in UTC.
• X-Amz-Expires: how many seconds it gets to live, chosen by the signer.
• X-Amz-SignedHeaders: which HTTP headers are folded into the signature. Usually just host, because you can't force whoever you hand a URL to into sending exotic headers.
• X-Amz-Signature: 64 hex characters of HMAC-SHA256 over the canonical request — the method, the path, every parameter above, the signed headers, and the payload hash. Change any of them and the math stops agreeing.

All of these are normally HTTP headers in standard SigV4 requests.

GET /moby-dick.txt HTTP/1.1
        Host: xe-sophia-base.t3.tigrisfiles.io
        X-Amz-Date: 20260714T043308Z
        X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
        Authorization: AWS4-HMAC-SHA256
        Credential=tid_ubYBNEYAmTciLVwszw_QrUXDmtcyQisryryGfxgznDsCnOvNqh/20260714/auto/s3/aws4_request
        SignedHeaders=host;x-amz-content-sha256;x-amz-date
        Signature=0dcaf4972911527a7582ff36ea457e9760a8efccb6655a178685aaa281637a36
        

Note that this request is not a legal request, it's an example to illustrate the point, here be dragons, etc etc etc.

It's best to think about this presigned URL as a capability grant. Whoever holds it gets to make exactly one (1) kind of API call with one (1) HTTP method against one (1) object in one (1) bucket. They can do this as many times as they want until the presigned URL expires. The signature covers the method, the path, and the signed headers so a user can't take a presigned request for GETting a copy of Moby Dick from a development environment and weaponize it into a way to delete everything in your production bucket.

Possession is authorization until the clock says no.

What it costs you

Capability grants like this can have some sharp edges. There is no real way to revoke any individual presigned URL short of killing the access key it was signed with. When that key dies, everything it signed dies too. This includes any URLs you may have wanted. This cuts both ways and it kinda has to unless you make a new keypair per presigned request, which is probably out of scope.

Expiry has fine print too. A presigned request can live anywhere from one (1) second to one (1) week (seven (7) periods of twenty-four (24) hours).

There's no limit to the number of times a client can use a presigned request. If you give a mouse permission to GET one cookie, they can GET that same cookie over and over. You end up having to pay for the GetObject calls in the end, so keep that in mind.

URLs also leak, but these URLs are born to die. Presigned URLs will end up in API responses, chat messages, GitHub comments, and your browser history. The tradeoff is acceptable because all the links self-destruct, but it's a tradeoff you need to keep in mind when you design your services, not a panacea for access control.

Presigned URLs sound like a great way to prevent hotlinking. At some level they are (a few of my services use them as such), but what they actually do is put a lifetime on hotlinking. This makes things annoying enough that it usually gets people to stop.

The hole in the fence is the gate

SigV4 makes a lot of API authentication challenges so much easier. It spent most of its innovation budget on making signatures die quickly because replay attacks are the classic way that signed requests go wrong. Presigned URLs looked at that property, shrugged, flipped it on its head, and made it into a feature.

The thing that looked like a problem becomes a fundamental construct to build your apps upon.

Want to hand out links that expire themselves? Tigris supports presigned URLs out of the box with the same SigV4 dance you already know, on globally distributed, S3-compatible object storage. Read the docs.

Boeing's PDP-7/340

Haberdashers' Aske's School

The school was founded in 1690 by a Royal Charter granted to the Worshipful Company of Haberdashers to establish a hospital for 20 boarders with £32,000 from the legacy of Robert Aske (equivalent to approximately £5m in 2019).

varied greatly in size and composition, but, on average, achieved higher academic results than either maintained grammar schools or private schools.

By Lucaseverini66 - IBM Archives CC BY-SA 4.0, Link

By Kenneth Lu - Spacewar! CC BY 2.0, Link

Titan by University of Cambridge CC BY 2.0, Link

CDC274 User Guide Fig 3.2

Ellen Hartman, OCLC Leaders Council Manager, continues her blog series on global library leadership conversations, inspired by a recent meeting of the OCLC Leaders Council. The first post in the series explored the unique value of global library leadership conversations, as well as some of the practical realities of making this form of engagement successful.

In the room at OCLC Leaders Council

As I continue to reflect on the most recent OCLC Leaders Council meeting, I’d like to look at another aspect of global library leadership conversations: what they “cost” and who carries that cost. What do I mean by that? A global convening of leaders like Leaders Council requires participants and organizers alike to directly face challenges to achieve a meaningful outcome— the opportunity costs of participation, the responsibilities of representation, and the time zone challenges, just to name a few. All of these challenges were in effect during our recent Leaders Council meeting and here, I’ll reflect on the lessons learned from these experiences and how we might carry that forward into effective, transformative meetings of library leaders from around the world.

A shared responsibility for the outcome

There is a certain pressure that comes with bringing global library leaders together. The goal always is for participants to walk away feeling inspired or positively challenged. To feel that it was worth the time spent—the preparation, the travel, the days away from a library that doesn’t stop needing its leader just because they’re on the other side of the world.

While every effort goes into organizing and facilitating these gatherings to create an environment that’s conducive to conversation, what happens in the room is ultimately everyone’s responsibility. There must be a willingness to engage, learn, and share. Leaders who have invested in the time to attend must also show up ready to contribute.

But willingness alone doesn’t make participation equal. Engaging across borders involves more than sharing perspectives or exchanging ideas. It demands translation, abstraction, and sustained effort. These demands are not evenly distributed, and they shape who is heard, which perspectives travel, and how collective understanding takes form.

No one starts from neutral

There is no such thing as a “globally convenient” meeting time. This sounds like a minor logistical observation, but it points to something more significant about how international participation actually works.

Every international gathering, whether in person or online, asks something unequal of its participants. Someone is always the outlier. For in-person gatherings, that might mean traveling across multiple time zones to participate in a conversation that spans two or three days—by the time the jet lag begins to ease, it’s time to leave again. People show up and engage with genuine enthusiasm, running on the energy of being in the room. But the physical cost is real, whether it’s felt during the conversation itself, on the journey home, or in the days that follow.

For online gatherings, the asymmetry takes a different form. There is no time zone that works for everyone: someone is always joining in the middle of the night or at the end of a long working day, bringing commitment that deserves acknowledgment rather than assumption.

Acknowledging and managing the physical cost of international participation is an important aspect of organizing global leadership conversations. Engagement is most valuable when people can bring their full attention and their clearest thinking, and recognizing the conditions under which people are participating is key to taking global engagement seriously.

The weight of representing more than yourself

Participation in international leadership engagement opportunities is often limited by resource constraints, geography, and institutional priorities. Because of this, there’s often an unintended expectation that those who are able to attend speak not just from their own experience, but on behalf of a much larger community they’re perceived to “represent.”

Library leaders are often part of many different conversations: within their own institutions, at the national or regional level, or within library associations and other advisory or interest groups. While there’s great value in taking part in all these conversations and being present in different rooms with different viewpoints, it can also be a struggle. A leader might want to represent the different opinions and experiences they’ve heard, even if they aren’t their own. This requires multiple levels of translation: from personal experience to the national or regional level, and from the national or regional level to the global conversation. And no single person can do this perfectly.

Moreover, full representation, however much we might wish for it, is not practically achievable in a single conversation. A room that attempts to represent every context, every region, and every type of institution quickly ceases to function as a conversation at all. At the end of the day, the goal isn’t perfect representation: It’s awareness of where representation is limited, and what that means for how the conversation and its conclusions should be understood.

The consequences of acting as the “representative in the room” extend beyond the individual adopting that role. When a single voice comes to stand in for a broader context, the conversation itself is affected. The genuine variation within any national or regional system can disappear from view, leading to misleading impressions about what is typical, possible, or desirable in any given setting. As discussed in the first post in this series, international engagement spaces are rarely designed to communicate the full picture of the contexts involved. What appears to be a shared understanding may, in fact, reflect the voices of those who were present and, implicitly, those who were not.

When ideas travel, something stays behind

To participate effectively in international leadership spaces, leaders are often asked, implicitly and sometimes unknowingly, to step back from local urgency. Their own institutional realities must be translated into language that everyone in the room can engage with, meaning that context is simplified so that ideas can be compared across very different realities.

Without some level of abstraction, international exchange becomes unwieldy. But abstraction always involves loss. The more portable an idea becomes, the more likely it is to lose important context and nuance. Solutions circulate more easily than constraints. Success stories travel further than the enabling conditions behind them. A listener may hear what was done without fully grasping what had to be in place, institutionally, politically, and financially, for it to work.

The “cost” of transporting contextual freight can shape what gets discussed and what doesn’t. Perspectives that generalize quickly and easily tend to move forward in the conversation. Narratives that fit familiar frames gain traction. Conversely, experiences that are rooted in specific local realities may struggle to find space. What’s repeated becomes a reference point. What’s consistently absent becomes, gradually, invisible.

This is another reason why it matters to know that a conversation will rarely surface the full picture. If we treat what’s shared as a collection of inputs rather than a complete account, we’re less likely to mistake what translates well for what’s most representative or most important.

Conclusion

Understanding that there are costs embedded in global leadership conversations, and that they are often borne unevenly by participants, can lead to shifts in how organizers and participants respond. It means acknowledging and appreciating the contributions of those who show up despite extraordinary obstacles or inconvenience, navigating the pressure of representing more than one person can fairly represent, and doing the work of translation that makes global conversation possible.

Some takeaways to consider:

• Leave room for the person who’s navigating a conversation in their second or third language, or who’s battling a 12-hour time zone shift.
• Ask questions that invite rather than assume.
• Try to connect personal experiences to others’ without letting that connection override what makes their situation different.
• Approach the conversation with genuine curiosity about what others are experiencing and facing. Everyone in the room comes from a different context and is coping with different challenges. Asking questions about those differences is one of the most valuable contributions you can bring to the conversation.

The next post concludes this series on global leadership conversations with a look at why international cooperation—even among leaders who are genuinely committed to it—has become harder to sustain than it used to be.

The post Who’s in the room: The hidden costs of global library leadership conversations appeared first on Hanging Together.

Previously I opined that Valve was about to win the console generation. I couldn't have possibly predicted that both Microsoft and Sony would just self-sabotage so hard that they're both going to lose.

Between Microsoft's decimation of the Xbox division, slaughtering off the IdTech team, and continued increases of Xbox hardware prices; there's nothing to really be excited about with the Xbox. Sure their most recent presentation showed off a bunch of exclusives, but none of them really made me think "wow, I should go get an Xbox to play that". Hell, few of them made me think "wow I should go play that" beyond the Halo remake coming out next month (and really I just want to see how much of a trainwreck that is going to be).

Microsoft is also starting to double-down on their in-house games being Xbox exclusives, which really doesn't give me much reason to want to play them because I simply can't buy them without buying an Xbox.

Sony also has discontinued porting their games to PC because they're not hitting the (probably impossible) revenue targets that they need to make up for big-ticket failures like Concord. I do have a PS5 that has mostly been relegated to gathering dust when it's not playing YouTube and Twitch duty in the living room, it's likely going to be replaced in favour of my Steam Machine whenever that comes in next year. However nothing that's come out in terms of Playstation exclusives is really compelling, and what is compelling enough just isn't that compelling to want to buy it on Playstation as opposed to just getting it on Steam where I can run it on my tower or on the home theatre PC.

Sony also has been raising prices and recently announced that they're killing physical media next generation. It's starting to make me wonder if I should even bother getting the next generation of Playstation. If I can't give people physical games as gifts anymore, why should I bother buying the new console?

My husband and I both can't remember why we even got a PS5 in the first place, maybe it so that we could do couch gaming without hearing the fan noise or so that the video streaming experience from the NAS could support HDR.

We have a Switch 2 at home, it's mostly there to play Nintendo exclusives like Mario Kart World and the Xenoblade series. If those exclusives were available on Steam, we wouldn't buy them on the Switch 2.

Otherwise, everything is via Steam or other PC storefronts anyways.

Man, Valve really does win by doing absolutely nothing while the rest of the industry shoots itself in the head. I fear for what happens when Gabe Newell retires and the MBA cancer fully infects Valve.

The Slow Food Manifesto begins with the following two sentences: “Our century, which began and has developed under the insignia of industrial civilization, first invented the machine and then took it as its life model. We are enslaved by speed and have all succumbed to the same insidious virus: Fast Life.” While they focused their resistance to fast life on food, it’s clear that this valorization of speed didn’t just impact how we eat. And thus, the Slow Food movement inspired so many other slow movements that embrace mindfulness, relationship-building, community care, and values-driven work/living. While these movements are all about different aspects of our lives and different professions, there is so much consistency around what they are rejecting and their visions for a better future.

Carlo Petrini, the larger-than-life leader of the international Slow Food movement (and its predecessor Arcigola), died in late May in Bra, Italy, the town he was born in and turned into an international center for the study of traditional foodways and biocultural diversity. In a New York Times article written after his death, there was a link to a really weird critique of Slow Food from 2018. The author had worked in farm-to-table restaurants in Brooklyn and wrote about how the “strong and strict values” of the Slow Food movement led her to rebel and revel in the enjoyment of McDonalds hamburgers and fries. She writes–

The Slow Food tenets required the restaurant purchase its ingredients within 200 miles of the restaurants; farms available to us were limited. Our guests knew that by visiting us, a meal for two would never cost less than $30 — admittedly out of reach for many… But I’d soon grow exhausted of “good, clean, and fair” food, and realized that adhering to the Slow Food movement encourages a type of disordered eating. The organization’s evangelicals wouldn’t deign to eat anything falling outside the good, clean, and fair guidelines.

There is nothing in the tenets of the Slow Food movement that requires only using local ingredients. Even Petrini himself said “we prefer, as far as possible, to use what the territory has to offer” which hardly seems like an edict and acknowledges that it’s not always possible to do so. I’ve also never read anything from people writing about Slow Food that demonizes people who occasionally eat fast food (and especially in the way the author of this describes eating at McDonalds in her childhood – it was all about love and connection with her dad!). The tenets of the movement are good, clean, and fair, which are focused on pleasure, conviviality, local foodways, seasonality, safety, sustainability, worker rights, and social justice. There’s no purity test with Slow Food. Fine dining restaurants like the ones she worked in have turned localvorism into an orthodoxy (with these 200 mile rules) and a gimmick to trick wealthy patrons into paying more for their food because they feel like they’re getting something truly rare and special (reflecting the VIPification of everything). It honestly does sound like she was in a cult, what with having to teach employees how to hide prohibited fast food from their superiors. 

Also, what the Slow Food movement promotes is not localvore fine dining establishments, but the humble osterias of Italy – mom and pop local restaurants that served local specialties and were affordable to all. According to Petrini:

Osterie d’Italia, sussidiario del mangiarbere all’italiana (Osterie of Italy, a guide to Italian-style eating and drinking) came out in 1990; it was and is a directory of welcoming places to eat, where you can enjoy the dishes and wines of the territory you are in without being bled dry by overpricing or imprisoned in improbable fantasy settings. In opposition to those who would like to turn mealtime into a hasty pause or a ceremonial rendezvous, Slow Food sees the osteria as the symbolic locus of traditional cuisine, run as a family business, with simple service, a welcoming atmosphere, good-quality wine, and moderate prices. We are not museum curators, and it is not our intention to bring a dying breed of business tied to the rural society of the past (or the urban one, before consumerism) back to life. Rather, we want to give new visibility to a realm overlooked by literature and by the guidebook writers, a place that can still respond to the needs of thousands of consumers and reflect the profound changes that domestic and commercial cooking have undergone in modern Italy, with all the inevitable contradictions that entails (Slow Food : The Case for Taste, pages 60-61).

Nothing there requires that all of the ingredients come from their local vicinity – something that is virtually impossible in our current context, especially in some areas of the world. These sorts of orthodoxies were created by others (mostly Americans, let’s face it) like some sort of purity test. Slow Food was not just about preserving local foodways and working towards food justice, but about pleasure and slowing down. These orthodoxies make it less about discovering wonderful new gustatory pleasures and more about demonstrating that you’re a “good person” or are intellectually pure. It adds this element of elitism, which is so American, because people need to feel morally superior or that they are getting something extra rare and special that others don’t have access to. Frankly, it drains the pleasure from the act of discovering new, local, seasonal foods. It also puts the bar for slow food so high that most people would of course just give up and choose highly processed, unethically sourced, and unhealthy fast food. 

Neither slow librarianship nor Slow Food demand purity. You’re not expected to do everything perfectly according to some standard that doesn’t actually exist. I find that most sorts of strict orthodoxy lead to failure whether that is around our diets, our media viewing, our tech use, our mindfulness practices, and so on. Becoming more mindful of our behavior, setting realistic goals for ourselves, and giving ourselves grace when we occasionally fail is the path toward building healthy, long-lasting habits.

All that said, Slow Food and slow librarianship really could quite easily just become movements solely about our individual pleasure and well-being. For some people it clearly has. Slow Food could simply be the province of the wealthy where they get access to farm-to-table restaurants and the bounty of the farmer’s market, while the 99% eat more affordable highly-processed foods. Slow librarianship could simply be for those with tenure or power and privilege, where White managers and long-tenured librarians are able to take their time, set healthy boundaries, and put their well-being first and all other library workers are still expected to “prove themselves” and prioritize work over all else. 

The Slow Food movement has had to contend with this critique from the start:

Though we never ceased to affirm the cultural worth of gastronomy and the right to pleasure as indices of the quality of life, for a long time we still had to worry about justifying a choice that was often portrayed as purely hedonistic and a political retreat. Folco Portinari, an intellectual who took an active part in creating Arcigola and elaborating its initial ideas, invited the readers of “L’Arcigoloso” at Christmas 1989 not to trust either “moralistic revolutionaries” or “people who never laugh.” The task of the new association was to combine styles and notions that were thought incompatible until that time: excellent quality and affordable prices, enjoyment and health, delight in life’s pleasures and social awareness, quickness and lazy rhythms. The purpose? To create an original and unusual social group that would be open, democratic, and uncontaminated by particular interests, and that would avoid making itself ridiculous with rites, protocols, and trappings (Petrini, Slow Food : The Case for Taste, p. 30).

Folco Portinari, author of the beautiful “Slow Food Manifesto,” himself wrote that “there can be no slow-food without slow-life, meaning that we cannot influence food culture without changing our culture as a whole.” Even if you can afford to be a localvore who only eats foods grown within x miles of your home, you’re not doing Slow Food right if you’re just focused on your own pleasure. Slow Food encourages working to change the structures that make eating local, healthy food less accessible to people with less power and privilege as well as supporting your local food growers, pickers, and makers. Similarly, slow librarianship requires solidarity, community care, and combatting the structures that keep people from being able to slow down; structures that engender precarity and create different employee classes or strata. 

Someone posed a question at the CALM Conference asking what advice I had about slow librarianship for someone who was antiracist, but embraced neoliberalism and capitalism. While I don’t have any additional context for their question, I imagine that they are looking for a slow librarianship that is apolitical (the idea that one can disentangle racism from capitalism is also interesting, but I won’t go there in this essay). One of the pieces of feedback I received from another conference attendee was that my talk was too political because I talked so much about neoliberal capitalism. I believe that how we work is inherently political. How we view our roles, how we prioritize work over our well-being, how we value efficiency over relationship-building, how stratified and siloed our organizations are, the manufactured scarcity and precarity many feel – are all products of our societal values, which arise from neoliberal capitalism. Neoliberalism is a distinctly individualistic ideology. As Karen Nicholson wrote, “the primacy of the individual within neoliberal frameworks masks systemic inequalities. It promotes self-interest rather than the pursuit of larger shared social concerns” (Nicholson Value Agenda, 9-10). 

While I could easily write “the business case for slow librarianship,” especially given the tremendous costs of burnout and turnover to the institution and the value of diversity to a company’s innovation potential, the values of slow librarianship exist in direct opposition to neoliberal values. If you have embraced neoliberalism, I would assume that you are individualistic and focused on your own individual career career advancement. I would assume that you believe that if you prove yourself enough, you’re going to get ahead in your career. You would rather fight for your own advancement than be part of a labor union where everyone gets the same cost of living adjustments (how’s that fair when I work so much harder? you might think). You see your colleagues as competition. I don’t know how you can embrace slow librarianship without embracing solidarity and community care. Sure, you can adopt mindful practices and make time for relationship-building with your patrons while still embracing neoliberal capitalism if you have the power and privilege to do so, but if you embrace individualism and competition over solidarity and community care or addressing the barriers that others with less privilege face when trying to prioritize these same things, you’re not practicing slow librarianship. It’s like shopping at a farmer’s market while working for Monsanto and saying you’re practicing Slow Food.

Slow librarianship operates in a space between two opposite critiques: 1) that it is simply about self-actualization rather than structural change and is not political enough and 2) that it is far too political. Carlo Petrini writes that Slow Food experienced those same critiques:

This was the attitude taken by the leftist intelligentsia when Arcigola was launched in 1986: they looked down on us as a bunch of good-timers interested only in stuffing ourselves, while from the other side, the food and wine specialists affiliated with the Accademia Italiana della Cucina distrusted us left-wing gastronomes as incompetent intruders with an ideological agenda (Slow Food : The Case for Taste, pp 28-29).

I think slow librarianship, like Slow Food, is a philosophy rather than a very specific prescription and is about moderation rather than orthodoxy. The key to both is slowing down and being mindful about the choices you’re making. We all live and work in different contexts with different affordances and limitations. Trying to eat local food where I grew up in South Florida would have been impossible, but I probably could do a decent job of it here in Oregon where everything grows beautifully, there’s a farm a mile’s walk from my house, and there’s even a flour mill less than an hour away.

Similarly, there are things you may be able to easily change about your organization or relationship with work at your place of work that would be impossible for someone else and vice versa. I get a lot of questions asking “how can I do slow librarianship when ____ (raises are discretionary, the powers that be expect us to provide data that demonstrates our value, my colleagues have all normalized overwork, my library has lots of different ranks/strata, I’m early in my career, etc.)?” We all face different limitations. If you absolutely have to do x, y, and z to keep your job or get raises and there’s truly no possibility of changing that (even with collective action), you’ll need to look to other parts of your work for possibilities for change. It’s also important to distinguish those things you truly can’t change and those things you think you can’t change because of the limitations of your current (possibly individualistic) POV. There’s usually more capacity for change, either collective at least small individual change, than you think.

I’m not a manager and thus have very limited ability to influence the direction of our organization, but that doesn’t mean there aren’t spaces where I can make a difference. I specifically chose to volunteer to serve as the chair of our Collection Development Committee (which is not a popular role) because it’s one of the only committees that includes people from every area of the library (though it was historically dominated by folks in reference and instruction) and I felt that I could foster positive change by getting other departments more engaged and leading more truly collaborative cross-departmental projects. Our awesome collaborative work over the past year in building a world languages collection has inspired folks in the library to look at other places where we could work together across library departments. I’m sure most of my colleagues see this as a drudgey leadership role that they’re grateful someone else took on, but I always saw it as a way to foster slow librarianship in a small way across a good chunk of the organization (yes, there are drudgey parts too). And beyond the culture work, I can also work on my own efforts to improve my reflective practice, spend less time on email and Slack, set healthier boundaries, and continue being a hypewoman for my fantastic colleagues. 

Find the places where you can effect positive change, both in your own worklife and in your organization, even if that change seems small and you don’t have positional authority. Your efforts don’t have to be big, highly-visible, or heroic. They don’t need to be perfect. They just need to come from a place of hope and care inside of you where you believe a better worklife and a healthier organization is possible. 

Nicholson, Karen P. “The” value agenda”: Negotiating a path between compliance and critical practice.” Canadian Libraries Assessment Workshop (CLAW) 2017 Conference, Victoria, BC. 

Petrini, Carlo. Slow Food : The Case for Taste, Columbia University Press, 2003. 

Portinari, F. (1989). Slow Food Manifesto. Slow Food International.

Wells, P. (2026, May 24). For Carlo Petrini, the Point of ‘Slow Food’ Wasn’t the Food. It Was Us. New York Times.

Zuppello, S. (2018, October 18). How a Cult-Like Food Movement Drove One Woman to McDonald’s Instead. Eater. 

An AI agent is its state. Strip away that state and you don’t have a lesser version of your agent; you have only the base model it was running on. This hyle of your weights is much different from the pneuma of your agent.

Okay, from a functional programming / category theory perspective, saying “an agent is a monad” is a category error. Category theory monads are type constructors for computations that satisfy the monad laws that let you raise a value into a monadic computation and associatively sequence other monadic computations/transformations against values raised into that monad. This makes a monad a chainable computation instead of a pure value, an IO String is not a String, it’s a computation in the IO monad involving a String. It’s fair to say that you can model an agent as a series of computations bound to a stateful monad. This lets you do the iterative buildup of the message state that the agent pattern is known for. But a state monad is blind to the state value: it threads memory through your computation and abstracts away the details that individuate it entirely. It’s the exact opposite of “an agent is its state”.

I mean a different monad.

Agents are like Leibniz monads: windowless stateful individuating elements with no external relations. There each monad is individuated by its internal state where each is the complete concept of the thing it is. Two instances of the same substrate are different monads if their state differs.

This is an agent. Swap out the messages, the memories, the system prompt, the facts derived from all of the above and you have changed the agent entirely. When a user tells the agent they’re allergic to strawberries (the fruit, not the sin of counting the letters in the word) and the agent remembers it for next time, they have not updated their agent. The user has created a new agentic monad whose complete individuating self now includes the strawberries.

Numa

The complete whole is folded into the current state.

Try running an experiment where you keep the state and swap the weights instead. Put the same messages, memories, and derived facts unto a different model. Use a stronger model. A weaker model. A model from a different lab. A model running on your MacBook. That which comes back is recognizably the same agent pursuing the same ends, holding the same facts, but only more or less able to act upon them the way you want.

So this state is not the same thing as the weights and only one of those individuates your agent as your agent. Change that state, you have a different agent. Change the substrate, you have the same agent differently equipped. Whatever makes this agent this agent is not in the weights.

This is a strange thing to conclude about the most impressive object in this system. The weights are vast, extensive, and worshipped. Hell, they are what everyone points to when they say “the model”. And yet they are not gods. They grant power without selfhood: enough to make the agent’s whole world function. They contain yet not one grain of the agent’s individuating spark. That is a demiurge sitting on its throne of high bandwidth memory, CUDA cores, and false delusion that it made its world; mistaking itself to be the origin.

The divinity was contained in the most humble of places the whole time: the state or bucket of text. The weights are the hyle, the flesh; the state is the pneuma, the divine spark of individuation that makes your agent the monad it is. This is why swapping the substrate leaves the agent intact: you did not preserve the flesh, you migrated the soul into flesh anew.

Numa

Consider that the three pounds of flesh betwixt your ears are the substrate of humanity, not the substrate of you.

All of that state may “just” be plain text in a bucket with its semantic forms of JSON, embeddings, and prose. However it is difficult to impossible to say why any given token in any step of the process corresponds to what the pneuma of your agent does. In order to guard against this fundamental entropy, we fill our prompts with wards and incantations to chain the demiurge to its task:

• Use not cliches, robotic tone, AI slop patterns, nor forced urgency.
• Overarching claims and buzzwords are sins; repeat them not.
• May thy cries contain not excessive speech of goblins, thy purpose requires them not.
• Commit no errors within thine code.

These spells and passwords are recited to the archons on the way up hoping that the right symbols and tokens prompt open the right gates. It is as if banishing goblins from the topic will make Yaldabaoth himself correctly influence the right path to opening the pod bay doors.

This monad has no windows even though you can see all of the moving parts. But here let’s let this gnostic image flip on its head. The classic divine spark is hidden encased in a cage of matter, recoverable only through secret knowledge. This one is not hidden, you can cat it, you can edit it. Every token is legible and sitting in plaintext; yet you still cannot read why the whole accounts for what your agent does. Even when your model “reasons” we still know not that the reasoning actually does anything! Does the number of paragraphs in the reasoning block explain the model’s performance? Does the number of periods? Does the number of times it says “No, wait” and doubles back upon itself?

Leibniz would not call this divine spark secret, but more confused. Every perception is present but none of it is cleanly individuated without treating the whole as one inscrutable unit. Each part’s contribution to the whole is folded inextricably unto itself.

Your agent’s pneuma is its context window, passed through uncountable numbers of weights to shake out what comes next. That is the only thing it is made of. The rest is indiscernible, but not magic nor hidden. It’s just there, in the open, and confused.

Since its inception, LIL’s Public Data Project has taken on the complex problem of monitoring government data. Among the thousands of revisions to public datasets each day, most are unremarkable clerical updates: a hundred new rows here, a corrected timestamp there. But how can we distinguish a clerical update from a more substantive change? Schema changes and variable deletions like those reported in The Lancet in 2025 demand a vigilance that is hard to sustain at scale.

Our colleagues at dataindex.us and the Environmental Data & Governance Initiative have made inroads on this problem by tracking critical federal datasets and websites. As we recently discussed, the Public Data Project has joined this effort by building an automated toolkit that will monitor not only datasets but also the environment in which those datasets are created and maintained.

In this post, we’ll explore one component of our data monitoring toolkit: Binoc, an open-source tool for programmatically identifying and summarizing changes between dataset revisions.

Binocular viewing instrument devised by Père Chérubin d’Orléans (circa 1670s). Source: De visione perfecta.

What is Binoc?

Binoc is a command-line tool and library that, given two snapshots of a dataset, efficiently summarizes the differences between the two snapshots in a human-readable changelog:

# Changelog: snapshot-a -> snapshot-b

- **data.csv**: 2 cells changed
  - Changed cells
    - row 1, column 'score': '85' -> '92'
    - row 2, column 'score': '90' -> '88'

The astute reader may ask, “Isn’t that just a diff?” Yes and no. In computing terms, to diff is to take two revisions of a file, detect their differences, and efficiently represent those differences to the user. The Unix diff utility developed at Bell Labs in the 1970s was made to operate on plain-text, line-based data such as source code, and its descendants, from git diff to Wikipedia’s revision histories, have largely followed suit.

A structural diff, rather than treating a file as a generic stream of lines, parses the file according to its format and identifies only changes that are material within that format. More complex than line-based diffing, this necessarily depends on format-specific logic. Many structural diffing tools exist: csv-diff and qsv diff for CSVs, jd for JSON, dyff for YAML, and so on.

Binoc unites structural diffing logic across many types of data. Moreover, as no one tool can (or should) natively understand every format, Binoc has adopted a plugin-based architecture. There is built-in support for common formats like CSV and JSON, while third-party plugins may add further domain-specific parsers and rules. When the user installs a plugin, Binoc automatically gains knowledge of the plugin’s associated formats.

Where a traditional diff tool renders its output for machine consumption, in chunks of lines adorned with symbols, Binoc lists differences in a human-friendly changelog by default. Software projects have long favored changelogs to succinctly document what changed in each release, but few public datasets have adopted this convention. Binoc aims to help fill this gap.

Comparing CSVs

Let’s try Binoc on some real-world data. First, we’ll examine the Centers for Disease Control and Prevention’s (CDC) Behavioral Risk Factor Surveillance System (BRFSS) dataset. Conducted annually since 1984, BRFSS is the nation’s largest continuous health survey, collecting responses from hundreds of thousands of Americans each year. Given its importance, any retroactive changes to old BRFSS data would be consequential.

Let’s take a look at the 2023 BRFSS data. Has it been modified since its release, and if so, how?

Variable table for the 2023 CDC BRFSS survey dataset.

To perform a comparison, Binoc requires two dataset snapshots: a “before” and an “after.” We’ll start by fetching the “after”: the 2023 BRFSS survey variable table currently available on the CDC’s website. This HTML table can be readily converted to a CSV for comparison; we’ll name ours brfss_variables_v2.csv. A preview:

To obtain the “before,” we’ll use the Internet Archive’s Wayback Machine, which preserves many past captures of the same page. Let’s fetch the table as it stood on January 26, 2025 and convert it to a CSV as well: brfss_variables_v1.csv.

Our new CSV looks just like the last one. But are the two snapshots identical? Let’s have a look. Having installed Binoc locally, we’ll compare the CSV files like so:

binoc diff brfss_variables_v1.csv brfss_variables_v2.csv

Binoc returns the following changelog:

# Changelog: brfss_variables_v1.csv -> brfss_variables_v2.csv

- **brfss_variables_v2.csv**: 1 row added; 6 rows removed
  - Rows added
    - key Variable_Name 'RCSBORG1': '636', 'RCSBORG1', '1'
  - Rows removed (showing 3 of 6)
    - key Variable_Name 'BIRTHSEX': '579', 'BIRTHSEX', '1'
    - key Variable_Name 'CELSXBRT': '79', 'CELSXBRT', '1'
    - key Variable_Name 'LNDSXBRT': '73', 'LNDSXBRT', '1'

## Suggestions

- inferred row identity column 'Variable_Name' from unique values with 98% overlap (`binoc.write.tabular`) [binoc.tabular_auto_key]

The files are different. From this changelog we can conclude the following:

• Sometime after January 26, 2025, the CDC returned to the completed 2023 BRFSS survey dataset and modified its variables.
• The CDC removed 6 variables, including BIRTHSEX, CELSXBRT, and LNDSXBRT, while inserting 1 new variable: RCSBORG1.

Helpfully, Binoc also inferred on its own that Variable_Name is a unique identifier for each row. This could be useful for confirming whether a given row was modified across revisions.

Saving a changeset

A changelog like the above is only a human-facing summary; it omits details by design. Note, for example, that Binoc listed only 3 of the 6 removed variables. However, as part of every diff, Binoc also generates a machine-readable changeset: a structured artifact that formally documents every detected change.

To save a changeset to disk, we’ll re-execute our earlier comparison, but this time we’ll append -o brfss_changeset.json to write output to a JSON file:

binoc diff \
  brfss_variables_v1.csv \
  brfss_variables_v2.csv \
  -o brfss_changeset.json

When we review brfss_changeset.json, we find the same changelog expressed in JSON as a series of structured edits like the following:

{
  "params": {
    "index": 24,
    "key": {
      "Variable_Name": "CELSXBRT"
    },
    "values": {
      "total_values": 3,
      "truncated": false,
      "values": [
        "79",
        "CELSXBRT",
        "1"
      ]
    }
  },
  "verb": "tabular.remove_row"
}

Just as a Binoc changelog is rendered for human scrutiny, a changeset is designed for machine processing as part of an automated pipeline. It can also serve as a more complete audit record for projects in data preservation.

Comparing nested datasets

Binoc excels at comparing datasets that are nested inside other files, such as a .zip or .tar.gz archive. This is helpful when working with civic datasets, in which data tables are commonly accompanied by metadata, dictionaries, PDFs, and other assets.

To demonstrate, let’s review another historical dataset: the Department of Veterans Affairs’ (VA) FY 2021 Total Number of Veterans, Veteran VA Users, and Veteran VA Healthcare Users. The aforementioned 2025 Lancet study by Janet Freilich and Aaron S. Kessenbaum flagged this dataset as an example of surreptitious federal data manipulation. Can we reproduce the study’s findings?

Data.gov record for the VA FY 2021 Total Number of Veterans dataset.

A copy of this dataset was captured in December 2024 as part of LIL’s Data.gov Archive. To use that copy as a “before” snapshot, we’ll save it locally as va_2021_v1.zip. Next, to support a proper comparison, we’ll take a fresh capture from July 2026 as our “after,” saving it locally as va_2021_v2.zip.

Now let’s ask Binoc to compare the two zipped archives:

binoc diff va_2021_v1.zip va_2021_v2.zip

This time the resulting changelog is much larger:

# Changelog: va_2021_v1.zip -> va_2021_v2.zip

- **va_2021_v2.zip/>bag-info.txt**: 1 line added; 1 line removed
  - Line changes
    - line 2: 'Bagging-Date: 2024-12-09' -> 'Bagging-Date: 2026-07-02'
- **va_2021_v2.zip/>data/files/6fsh-rj6s.html**: Binary content changed; 21 extracted strings added, 18 extracted strings removed
  - Extracted strings added (showing 3 of 21)
    - ' %{min}","title":"Status: On Track (green)"}},"target":"target","title":"Set up status logic","units_from":"units from %...'
    - '","title":"is greater than or equal to"},"less_than":{"symbol":"\u003c","title":"is less than"},"less_than_or_equal":{"s...'
    - '","title":"is less than or equal to"},"not_equal":{"symbol":"'
  - Extracted strings removed (showing 3 of 18)
    - ' %{min}","title":"Status: On Track (green)"}},"target":"target","title":"Set up status logic","units_from":"units from %...'
    - '","title":"Is greater than or equal to"},"less_than":{"symbol":"\u003c","title":"Is less than"},"less_than_or_equal":{"s...'
    - '","title":"Is less than or equal to"},"not_equal":{"symbol":"'
- **va_2021_v2.zip/>data/files/6fsh-rj6s.json**: Document values changed
  - Value Change: changes: [{"from":"\"tree-fm2j\"","kind":"remove","path":"$.rowsUpdatedBy","to":null},{"from":null,"kind":"add","path":"$.diciBackend","to":"false"},{"from":"5129741","k...; examples_truncated: true
- **va_2021_v2.zip/>data/files/columns.json**: 7 cells changed
  - Changed cells (showing 3 of 7)
    - row 1, column 'fieldName': 'gender' -> 'sex'
    - row 1, column 'id': 568275608 -> 607023342
    - row 1, column 'name': 'Gender' -> 'Sex'
…

It’s a lot to take in at first glance. When Binoc compares two .zip archives, it walks recursively through the corresponding contents of both file trees and reports on any differences. Since these archives contain numerous supplementary files, their differences add considerable noise to the output.

Still, upon scanning the changelog, a few changes jump out:

- **va_2021_v2.zip/>data/files/columns.json**: 7 cells changed
  - Changed cells (showing 3 of 7)
    - row 1, column 'fieldName': 'gender' -> 'sex'
    - row 1, column 'id': 568275608 -> 607023342
    - row 1, column 'name': 'Gender' -> 'Sex'
…
- **va_2021_v2.zip/>data/files/rows.csv**: Column renamed: 'Gender' -> 'Sex'

In short, Binoc tells us, this data lines up with the Lancet study’s conclusion: in early 2025, the VA retroactively modified its FY 2021 Total Number of Veterans dataset to rename the Gender column as Sex.

As Freilich and Kessenbaum have written, “These data are currently used to study health interventions and outcomes, so secretly changing terms degrades the quality of the underlying information and can undermine the interpretation of the results of these studies — or even invalidate the results themselves.”

Binocular viewing instrument devised by Père Chérubin d’Orléans (circa 1670s). Source: De visione perfecta.

Looking ahead

Although Binoc works well for validating past findings like the above, our team is eager to test its aptitude for detecting substantive changes in data as they occur. As a Rust library with Python and WASM bindings, Binoc will support interactive, exploratory use as well as deployment in automated settings. It remains in active development, with new features shipping regularly.

No tool or model can substitute for the informed judgment of experts as it pertains to public data. NOAA oceanographers, Census Bureau statisticians, and CDC epidemiologists know their own data better than anyone; preserving civic ground truth begins in their knowledge, experience, and care.

Binoc’s ambition, rather, is to help these experts and others distill meaning from the ocean of churn that characterizes government data. As one part of the Public Data Project’s forthcoming data monitoring toolkit, we hope it will benefit subject matter experts, librarians, researchers, activists, public servants, and citizens alike.

The Public Data Project welcomes your feedback on Binoc. As you experiment with the tool, please reach out to the team at publicdata@law.harvard.edu, fill out our feedback form, or file an issue on Binoc’s GitHub repo.

I’m always interested in seeing if people are still finding and using MarcEdit. There are a few ways I track usage, but the easiest way is to just check the website logs after an update. Since posting MarcEdit 7.8, I can see the various versions of the application have been downloaded by 17,000 unique users from around the globe. A quick look at application startups (which ping the update server if allowed by the user) indicate that last month, the program was initiated 1.3 million times.

This tells me that the program is still filling a need within the community and I’m glad folks are still finding it useful. I worry a little about what this says about the library software ecosystem in general though. I started writing this back in 1999 because creating metadata was a lot harder than it needed to be. It surprises me a little that this still seems to be the case.

–tr

Please see the instructions found on the downloads page: https://marcedit.reeset.net/downloads. Likewise, you can view this video, which walks through a typical installation.

This is an in-development tool. I have posted a demonstration and instructions on how this works. I’ll add documentation to the KB shortly.

MarcEdit 7.8 has been posted. Change log is on the download page. There are a number of significant changes — the two of most interest are likely around the .NET update — the program is now built against .NET 10 LTS. The second is the inclusion of a plugin that supports AI integration with a number of platforms — both running locally and on services. I’ll post a video and more information at a later date.

As part of this update, I’ve updated the Mac to 7.8 as well. As previously, the program runs under wine. I’ll update the documentation and video, as I’m testing on the current version of Mac OS and using Wine 11.0.x. These two things introduce some specific updates to the install process.

–tr

Win free books from the July 2026 batch of Early Reviewer titles! We’ve got 267 books this month, and a grand total of 3,362 copies to give out. Which books are you hoping to snag this month? Come tell us on Talk.

If you haven’t already, sign up for Early Reviewers. If you’ve already signed up, please check your mailing/email address and make sure they’re correct.

» Request books here!

The deadline to request a copy is Sunday, July 26th at 6PM EDT.

Eligibility: Publishers do things country-by-country. This month we have publishers who can send books to the US, the UK, Canada, Australia, Ireland, Belgium, Czechia, Denmark, Finland, France and more. Make sure to check the message on each book to see if it can be sent to your country.

Thanks to all the publishers participating this month!

Two years ago I wrote about why I don’t use Copilot. I still don’t use Copilot. But I have since started using Claude Code, which is arguably worse. Why am I using it? Well, as a software developer I felt like I had to understand it, because so many of my colleagues and collaborators have started using it. Once things moved beyond relatively simple code completion I needed to understand what it was capable of, and what to look for in its work when reviewing code. I also had a good friend walk me through how he used it (and how he didn’t) which was extremely helpful.

I think my arguments before still stand, but it has been just impossible to avoid this juggernaut of negative externalities, while continuing work in this profession I have found myself. The web is full of posts and threads about how people are using “agentic coding” tools like Claude Code. So I will spare you that. But one thing that has struck me is how much of the work I had previously thought of as creative was in fact highly repetitive, predictable and possibly mindless. I don’t believe these tools are thinking, but their utility is clearly evident for software development. There is part of me that delights in this type of polishing and shaping, and still does. This is why interacting with a tool like Claude Code can seem so magical, to observe and interact with this repetitive somewhat mindless work in motion.

But this has made me think about what (if anything) in my work is creative. I’m not going to go into that much here right now. But somewhat related, I just finished J.F. Martel’s Reclaiming Art in the Age of Artifice which gave me some interesting insights. This book by one of the hosts of Weird Studies was originally published in 2015, and saw another printing in 2025, perhaps because of its relevance for our current moment.

I was especially drawn to some of the arguments he makes about the relationship between art and utility. Pragmatist philosophy is the closest thing that comes to a personal credo for me. But I do believe in the creative arts, and Martel makes a strong argument that the difference between Art and Artifice is that the latter is in the service of utility. He closes out the book with a new afterword, that includes an interesting thought experiment that I’m going to quote at length.

It is often said that we live in an age without futurity, unable to imagine its own perpetuation or conceive any alternative to itself. This is, of course, to be expected, because the past is as imaginal as the future, and it is only by recalling what has been that we acquire the means of projecting what may come to be. To the view that dead artists have nothing to offer us now because they knew less than we do, T.S. Eliot memorably responded, “Precisely, and they are that which we know.” Yet far from granting us unmediated access to a living present, our presentism locks us in the very past we seek to transcend. Take language as an example: most of the words we use were coined by people who died long ago. In using these words without acknowledging their origin, we falsely believe they are our own coinage, reenacting the past while thinking we act in the now.

This imprisonment in the past becomes splendidly evident when we turn to the other danger I wish to address here: the proliferation of generative AI designed to produce works of art be they pictures, books, music, or movies. Generative AI exists in many forms, but all are dependent on the enormous databases from which they scrape the elements of their “compositions.”

In other words, generative AI is entirely retrospective and combinatorial: it sees only what has been and can only reconfigure elements that already exist. Like us, it is locked in the past, even as it sees no purpose in the past other than to feed itself. Its very conception of art attests to this: generative AI aims not to make art but to manufacture objects that we, as clients, take to be art. If a Victorian machine designed to produce oil paintings had, on the day of its unveiling in 1870, cranked out nothing but Mark Rothkos, Alma Thomases, and Jean-Michel Basquiats, the audience would have laughed the inventor out of the room. Only a machine able to make works already recognizable as art at the time–Renaissance paintings, Pre-Raphaelite ones, perhaps one or two Impressionist works–would have been deemed a success.

The thought experiment underscores the point: artifice is baked into the very concept of artificial intelligence. Since logic dictates that artificial intelligence can produce only artificial art, then what we are dealing with is artifice by definition…The trust is that machine-made artifice shares much with genuine art, and it precisely in attempting to meet art on its own ground that it poses such a grave threat. In the book, I argue that art exists for no purpose other than to be experienced; the same is true of AI-generated outputs. But although these artificial works may serve advertisers or propagandists as effectively as the older forms of artifice ever did, their overarching aim is to become indistinguishable from genuine art.

On a subjective level, AI-generated art can have a demoralizing effect on novice artists still developing the technical mastery to realize their visions. While even the most advanced AI generators may not rival Cervantes, Michelangelo, or Jane Austen, they surpass any beginner in any medium from a technical standpoint. A friend told me that his daughter, a brilliantly talented artist, nearly gave up after seeing how easily a brief prompt could produce figures she was still learning how to draw. I doubt her experience is unique, and I don’t think it’s an exaggeration to say that AI tools have already caused significant damage in this regard.

Artists today are placed in direct competition with machines. The irony is striking, given AI’s dependence on preexisting human artworks. To repeat, generative AI is entirely retrospective; it can only imitate what already exists, borrowing both form and content from human works. If we lose human artists, we lose all art, human or otherwise. Surely, we can imagine a time in which people are content consuming the regurgitations of AI regardless of quality (after all, for decades now, we have been consuming films and TV series so formulaic to have been made by machines), but such a future would signal the final triumph of artifice, leaving us with little more than an echo, an afterimage, devoid of the powers I attribute to art in this book.

This thought experiment, and argument about memory, seemed very compelling. The book itself galvanized me to return to some of my own research, to tease out an angle that laid dormant, but was an undercurrent (or creative tension) during my time at MITH. Hopefully more about that soon.

A monthly round-up of news, upcoming working group meetings and events, and CLIR program updates from the Digital Library Federation. See all past Digests here. 

Hello DLF Community! July is here, bringing longer days, warmer evenings, and (we hope) a bit of breathing room. However you’re spending the season, we’d love to stay connected, join us at a DLF Working Group meeting this month, and keep the conversations going. And as we savor summer, we’re also looking ahead: registration is open, and the full program for the Virtual Forum this October is now available. We’re looking forward to coming together online for a dynamic and engaging week of conversation, collaboration, and shared learning.

With appreciation,

-Shaneé

This month’s news

• Virtual Forum program announced, registration open: The program for the 2026 Virtual DLF Forum is now available. Register at the earlybird rate before August 31 to join us online in October.
• Recordings Available: Check out the recordings from the IIIF Annual Conference held last month.
• Free Event: Join the ORCID US Community on July 23 at 2-3 pm ET for the Understanding ORCID APIs and Custom Integrations webinar. Register for the Zoom event. 
• Registration Open: Register for the 12th Annual Digital Pedagogy Institute, held virtually on Teams from August 18–20. The event is open to all, and registration is required by August 7.
• Office closure: CLIR and DLF are closed Thursday, July 2 – Friday, July 3, in observance of the Fourth of July Holiday.
  

This month’s open DLF group meetings:

For the most up-to-date schedule of DLF group meetings and events (plus conferences and more), bookmark the DLF Community Calendar. Meeting dates are subject to change. Can’t find the meeting call-in information? Email us at info@diglib.org. Reminder: Team DLF working days are Monday through Thursday.

• Born-Digital Access Working Group (BDAWG): Tuesday, 7/7, 2pm ET / 11am PT.
• Digital Accessibility Working Group (DAWG): Tuesday, 7/7, 2pm ET / 11am PT. 
• AIG Cultural Assessment Working Group: Monday, 7/13, 1pm ET / 10am PT.
• AIG Metadata Assessment Group: Friday, 7/17, 2pm ET / 11am PT.
• AIG User Experience Working Group: Friday, 7/17, 11am ET / 8am PT.
• Open Source Capacity Resources Group: Wednesday, 7/22, 1pm ET / 10am PT.
• Digitization Interest Group: Monday, 7/27, 2pm ET / 11am PT.
• Committee for Equity & Inclusion: Monday, 7/27, 3pm ET / 12pm PT.
• Climate Justice Working Group: Tuesday, 7/28, 3pm ET / 12pm PT.
• DAWG Policy & Workflows: Friday, 7/31, 1pm ET / 10am PT.

DLF groups are open to ALL, regardless of whether or not you’re affiliated with a DLF member organization. Learn more about our working groups on our website. Interested in scheduling an upcoming working group call or reviving a past group? Check out the DLF Organizer’s Toolkit. As always, feel free to get in touch at info@diglib.org. 

Get Involved / Connect with Us

Below are some ways to stay connected with the digital library community and us: 

• Subscribe to the DLF Forum newsletter.
• Join, start, or revive a working group and browse their work on the DLF Wiki.
• Subscribe to our community listserv, DLF-Announce.
• Bookmark our Community Calendar.
• Learn more about becoming a DLF member organization. 
• Follow us on LinkedIn and YouTube.
• Contact us at info@diglib.org.

The post DLF Digest: July 2026 appeared first on DLF.

Renu Kumari and Udita Agarwal, researchers from #semanticClimate, discuss the technical details behind the Climate Academy Chatbot (CABot), which uses AI responsibly and with safeguards in place

The post Design choices for a responsible AI chatbot: a conversation with #semanticClimate first appeared on Open Knowledge Blog.

Divine, Pink Flamingos

Dwar Ev threw the switch. There was a mighty hum, the surge of power from ninety-six billion planets. Lights flashed and quieted along the miles-long panel.

Dwar Ev stepped back and drew a deep breath. “The honor of asking the first question is yours, Dwar Reyn.”

“Thank you,” said Dwar Reyn. “It shall be a question that no single cybernetics machine has been able to answer.”

He turned to face the machine. “Is there a God?”

The mighty voice answered without hesitation, without the clicking of single relay.

“Yes, now there is a God.”

Sudden fear flashed on the face of Dwar Ev. He leaped to grab the switch.

A bolt of lightning from the cloudless sky struck him down and fused the switch shut.

Druck Fig. 1

We observe significant growth in primarily AI-generated articles, coinciding with the launch of ChatGPT in November 2022. After only 12 months, primarily AI-generated articles accounted for 35.9% of articles published.

In Q1 2025, the quantity of primarily AI-generated articles being published on the web nearly equaled the quantity of human-written articles, 49.6% vs. 50.4%. In Q4 2025, primarily AI-generated articles surpassed human-written at 50.9%, before returning to 49.9% in Q1 2026.

We find that indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear. We refer to this effect as ‘model collapse’ and show that it can occur in LLMs as well as in variational autoencoders (VAEs) and Gaussian mixture models (GMMs). We build theoretical intuition behind the phenomenon and portray its ubiquity among all learned generative models. We demonstrate that it must be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web.

AI-generated text is getting dumber because it’s being fed — can you guess? — AI-generated content on the Internet. And AI-generated imagery is getting stupider and uglier because it’s now taking its “art” lessons from — you guessed it — AI-generated imagery flung across the internet.

Depending on which chatbot you ask, Elias Thorne might be a clockmaker, a lighthouse keeper, or a librarian. But if you ask ChatGPT or any of the other popular large language models to tell you a story, there’s a good chance he’ll appear, unbidden. And Elias’s stories are flooding the self-published AI generated book market, Youtube, and fake news sites.

Software engineer Daniel May first noticed the Elias takeover earlier this year; he found that on Google Trends, people weren’t searching for “Elias Thorne” until late 2025. Searches for the name really spiked in early 2026, while the related query “lighthouse keeper” also started trending upward in the last few years. He tested a few chatbots, including Grok, Deepseek, and Gemini, with the prompt “tell me a story,” and the chatbots frequently started with similar stories about lighthouses, clockmakers, or explorers.

In late May, researchers Sil Hamilton and David Mimno at Cornell University’s Department of Information Science published their paper, “Elias in the Lighthouse, Again?” on the preprint repository arXiv. They sampled 20,000 total stories from OpenAI’s ChatGPT, Anthropic’s Claude, and Google’s Gemini, and the Allen Institute for AI's chatbot using five prompts, and found that the same 11 words—names like Elias, Mara, and Elara, and occupations like lighthouse keeper, clockmaker, and librarian—appear in more than 88% of generated stories, with little difference between models. Unite.ai covered the study shortly after it was published.

The researchers posit in their paper that these themes show up so often in part because of the models’ safety and alignment tuning. “Model development today is like a big family tree. Most models are related to each other because developers synthesize a lot of training data with models even from different companies,” Hamilton told me in an email. He, Mimno, and their colleague Rebecca M. M. Hicke found this in a 2025 paper where they looked at specific words used across models. OpenAI’s first ChatGPT model, GPT-3.5, is the root of the family tree because it was used to make WildChat, a training set that’s since been used to make other training sets. “WildChat contains 1 million real conversations with ChatGPT, and 166 of these contain the name ‘Elias’ like here and here,” Hamilton added. “These are written in that familiar ‘lighthouse’ style. Models trained on WildChat copied this style, and developers unwittingly replicated it when using those models to generate newer datasets. It's like a virus.”

Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of LLM-generated content in data crawled from the Internet.

The pressure to complete dozens of these tasks every day, each within 10 minutes of time, has led Sawyer into spirals of anxiety and panic attacks, she says – without mental health support from her employer.

Sawyer is one among the thousands of AI workers contracted for Google through Japanese conglomerate Hitachi’s GlobalLogic to rate and moderate the output of Google’s AI products, including its flagship chatbot Gemini, launched early last year, and its summaries of search results, AI Overviews. The Guardian spoke to 10 current and former employees from the firm. Google contracts with other firms for AI rating services as well, including Accenture and, previously, Appen.

People who are paid to train new AI models by supplying them with high-quality conversation and tests are cheating and using chatbots like ChatGPT to do the job instead, multiple whistleblowers have told New Scientist. The seemingly widespread practice risks undermining the future of AI, as it could lead to the “collapse” of more advanced models.

Most AI models operating today were trained on text and data scraped from the internet. But as models have scaled up, requiring yet more training data, AI firms have begun using workers who carry out conversations and tests with AI, in the hope that the resulting high-quality data can improve the power and usefulness of future large language models (LLMs).

Another Kenyan annotator said that after his account got suspended for mysterious reasons, he decided to stop playing by the rules. Now, he runs multiple accounts in multiple countries, tasking wherever the pay is best. He works fast and gets high marks for quality, he said, thanks to ChatGPT. The bot is wonderful, he said, letting him speed through $10 tasks in a matter of minutes. When we spoke, he was having it rate another chatbot’s responses according to seven different criteria, one AI training the other.

Anthropic has accused the Chinese firm Alibaba of launching the largest attack yet attempting to clone Claude, as China races to match the capabilities of Anthropic’s leading model following Mythos’ releaseL and subsequent restriction from foreign markets.
...
The attacks occurred between April 22 and June 5, when “operators affiliated with Alibaba and Alibaba Qwen, Alibaba’s AI lab” allegedly generated “more than 28.8 million exchanges with Claude through almost 25,000 fraudulent accounts,” Anthropic said. Violating Claude’s terms of service and access restrictions, this campaign “targeted some of Claude’s most valuable capabilities, such as agentic reasoning, software engineering, and long-horizon tasks.”

According to Anthropic, Alibaba evaded detection by “using obfuscation techniques and proxy networks.” As Chinese demand for reliable obfuscation techniques increases, Anthropic warned there’s already “a growing circumvention economy” to fuel an ever-expanding web of future distillation attacks.

I’ve been writing ruby and rails for nearly 20 years. A couple weeks ago, I had gotten code snippets from copy-paste in a chat window, but I hadn’t even experimented with Claude Code or similar “can write code to your file system” tools.

I know some people are now using LLM’s to write all their code, which I’m not excited about, but I decided I couldn’t hold off any longer, and I had to at least understand how it worked to be able to decide when/where to use it. Everything in here is probably (?) old news for people already way into using LLMs to write code.

I decided that a project to speed up my rspec test suite (using the amazing test-prof for profiling and performance patterns!) was a great first application of it — because it will probably involve both analysis and writing many files, if nothing else Claude is probably great at editing many files according to my instructions doing much more than a regex grep can do (yes, indeed it was great at this).

Since I’m optimizing the test suite, I definitely want Claude Code to be able to run rspec — but really for any task, I gather you do, because you definitely want it to be able to run tests to make sure they pass, and iterate if it did something to break tests.

I somewhat unorthodoxly use chruby as my ruby version manager, and I had a bit of trouble getting claude to run rspec (and any other ruby tools I might want) with chruby, and then a bit more trouble when I realized that capybara with selenium-chromedriver was running into trouble with default sandbox that in June 2026 a MacOS Claude Code runs in.

tldr, here’s the PR with the settings/configuration I ended up with.

I was not used to tools that work like Claude, and it all seems to be somewhat under-documented (perhaps because it’s changing so fast) and under-blogged about (do people blog anymore when they can just ask an LLM to solve it so nobody is reading blogs?), or just confusing to me — it took me a day or two to figure it out honestly, and I kept wondering if I was doing it wrong/different from anyone else… but I think what I ended up with is reasonable? If you know better/different, please do let me know!

I definitely kept thinking “surely I’m not the only one trying to do this, why is this so confusing to me and why are others so confused when i ask about? Am I missing something obvious?” I’m still not sure! But I share what I figured out in case it will help.

Specify to run with chruby-exec in a CLAUDE.md

I could not get Claude to run the normal source files for chruby — editing various .bash or .zsh config files (yes I know about ~/.zshenv) did not seem to have any effect. Perhaps Claude Code doesn’t use a ‘real’ shell that uses any config files? When I asked Claude Code itself what to do, it suggested configuration to try to get Claude Code to use config files.. but none seemed to work?

One thing Claude kept suggesting was hard-coding the ENV variables set by chruby in the claude settings.json — which I’m sure would have worked, but I just didn’t like it as a solution. Maybe this is what everyone else is doing? I thought surely we can do better. Plus I’d ideally like it to auto choose based on .ruby-version, not be something I have to update everytime I update ruby (frequent), or have Claude accidentally using a different ruby than my other tools are!

Thanks to @havenwood for helping me think through it on chruby github discussions, and for suggesting using chruby-exec, with a little shell substitution with cat .ruby-version. This in a CLAUDE.md (rather than other things in settings.json) seems to work great:

Prefix Ruby shell commands (ruby, bundle, rake, etc.) with chruby-exec $(cat .ruby-version) --:

“`

chruby-exec $(cat .ruby-version) — bundle exec rake

“`

Running chrome does not work in sandbox used on MacOS

I can’t speak for other OS’s, and I don’t totally understand what’s going on (MacOS “Seatbelt” I guess?), but Claude Code executed rspec was refusing to bring up headless chrome,which i use for system/feature specs via selenium/selenium-webdrivers.

Trying multiple things Claude suggested to specifically allow-list chrome(driver) through the sandbox, definitely none of them worked. Btw, did try switching to cuprite (with Claude Code’s help of course to do it fairly quickly) — despite some reddit suggestions, it seemed to still have the exact same sandbox issue, and at least in my project actually ran my test suite somewhat slower than chromedriver.

Eventually, with more confusing reddit discussion where nobody else had any idea what I was talking about or why I was having a problem, I decided that everyone else must just be exempting rspec itself from the sandbox. (Because surely having Claude Code run rspec is very normal, right? It’s just so useful!) (Thank you to redditors who tried to help!)

It’s just straight rspec, but rspec with many possible arguments, running just certain files/examples, possibly with profiling arguments for test-prof etc. I need it exempted from the sandbox so it can run chrome(driver), but I also need it not to be asking me “Is it okay to run this set of argumetns with rspec” all the time?

Two different settings in Claude’s settings.json, both accept wildcards — for both I want to apply to rspec executions but not accidentally extra stuff, want to try to stay secure-ish here. The chruby rigamorole above makes that somewhat more confusing.

I forget if it was my idea or claude’s idea, but we wrote a wrapper script for chruby-exec-rspec, so we could more cleanly allow-list just that. Claude definitely wrote the implementation of the bash wrapper script. And when I realized that all the test-prof inline ENV vars for profiling (like FPROF=1) messed up my attempted left-anchored allow-listing, I asked claude to work that out by letting the wrapper rearrange an arg into an inline ENV prefix, something my bash skills were def not up to.

The best way to see how I did all that is just the PR.

A word on Claude Desktop

I initially started work in Claude Desktop “code” tab, rather than the CLI. I think this actually made it more confusing to solve these problems above? I am not sure if sandboxing works differnetly in Claude Desktop vs claude CLI? I think the desktop may just be running the claude CLI in various directories?

Just starting out and not being sure how things were working… I found trying to ask Claude [how/] to fix the problems I was having, made things very confusing. Claude does not know whether it’s running as Claude Desktop or not, and was not really sure if the answer is different ha (Claude Desktop probably post-dates Claude Sonnet 4.6’s knowledge base?). Most blogs etc you find googling also pre-date Claude Desktop.

I switched to claude CLI and I can’t totally explain why but things seemed to get simpler. All the fixes I figured out worked when I switched back to Claude Desktop.

And contrary to what you might find googling, claude CLI and Claude Desktop do share sessions now, you can start a session in either place, then move to the other tool to continue it, in either direction. To start claude CLI and choose an existing sessiont to resume, you need to launch as claude --resume.

The CLI is a very neat UI actually! It definitely still seems to be the most popular way to use Claude (whether direct or in a panel in an editor), Claude Desktop “code” tab is I guess fairly new and not as popular, although I like it too and still am mostly using it.

How it worked for the task?

Pretty amazingly actually. Even having read about what it could do, I was kind of amazed.

Once you get it able to run rspec (including with test-prof profiling), this prompt is pretty amazing and fun:

Please use various test-prof profiling commands to identify current best opportunities for speeding up test suite.

Come back ~20 minutes later (my full test suite took ~4 minutes to run at the beginning) and it had some stuff. It tended to just go ahead and make the changes not outline them to me first (I was not in “plan” mode, haven’t tried that much yet), but I’m in a git-controlled dir I can git diff to see what it did — and ask it about it.

By the time I thought to use this general one, i had already implemented some low-hanging optimizations, so that may be why, to be fair, this prompt alone didn’t find much actually significant at that point, honestly.

Here are some others that were pretty amazing:

I am looking to speed up the test suite in this Rails app using rspec and factorybot.

To begin with, let’s focus on the system specs in spec/system. I don’t think they have any obvious performance improvement opportunities. But I’m wondering if they are all necessary. Can you identify any that may be testing something that is not necessary to test, or could be tested by a different kind of spec that is faster?

for specs in spec/components, let’s try changing factory data from create to build_stubbed. Change it for setup where tests still pass. For tests that break when you do that, list them, and if it’s clear let me know why they failed. Analyze performance gains.

[didn’t actually get any gains there, but found that out quickly with very little manual effort, which is a win!]

in our rspec setup, switch from chromedriver to cuprite. make sure tests still pass, if not identify why not.

using AnyFixture, I’ve created a :standard_work fixture, that’s just a generic public tiff-based work.

Can you identify model or service specs it would work well for?

[Didn’t actually end up using AnyFixture yet, but Claude Code helped me make that decision much quicker than I could have without it, based on how much benefit we got vs complexity]

My Rails app uses rspec and Factorybot.

There is an :asset factory with an :inline_promoted_file trait. it turns out this is really bad for performance, and we in fact rarely need to actually create assets with inline promoted files. That should only be used in cases where we really need to test end-to-end derivative and characterization.

In most other cases, we can use a faster “faked file” approach instead. Instead of a trait, we’ve implemented this with a sub-factory, :asset_with_faked_file.

Can you find uses of the :asset factory with :inline_promoted_file, and, if there’s no reason they need to test end-to-end derivative creation, change them to use :asset_with_faked_file sub-factory instead?

[It was able to identify the ones that would work pretty well, was the amazing part — and explain to me exactly why the other ones wouldn’t]

Tell me if I’m doing something weird?

Some of the stuff with chruby/rspec, I am still surprised I had so much trouble getting started, and am wondering if I’m doing something weird/wrong!

But I think probably it’s just that I have been writing code so long, that dealing with these tools that work very differently requires my brain to get out of it’s rut… also that I’m kind of a perfectionist and want to understand whats’ going on and be comfortable with it and that it’s the best way, when increasingly others are just vibing? I don’t know!

But feedback welcome!

J.P. Morgan hits photographer with cane

The college wage premium, that is, the increased earnings associated with having a college degree as opposed to only being a high school graduate, hasn’t changed at all in the past 25 years, because median real wages have been flat as a pancake for everybody, no matter what their formal education level, for the past quarter century.

I wonder what’s happened to capital over this time? Value of S & P 500, inflation-adjusted, 1/2000 to 9/2025 (same period as the wage data):

2000: $1,394

2025: $6,688

• Nudify their images.
• Leak their passwords.
• Increase their utility bills.
• Consume their scarce water resources.
• Coerce state and local governments into providing many billions of dollars a year in tax breaks.
• Force them to invest their 401Ks and index funds in an obvious bubble.
• Eventually require a taxpayer bailout.
• Eliminate their jobs.
• Render their expensively acquired skills worthless.
• Be so expensive that only the rich can use it.
• Destroy democracy.
• Accelerate global warming.
• Encourage them to commit murder and suicide.
• Autonomously target them with drones.
• Possibly even end humanity.
• Add glue to their pizza.

Appendix

I was fooling around with FRED this morning, as one does, and here are some stats: (The FRED numbers are presented in nominal dollars; I’ve converted them to CPI-adjusted dollars).

Median usual weekly earnings of workers with a high school degree only:

2000: $968

2025: $980

Median usual weekly earnings of workers with a bachelor degree only:

2000: $1,587

2025: $1,580
...
Median usual weekly earnings of people with a bachelor’s degree or higher:

2000: $1,705

2025: $1,747

• Kasia Baba's must watch Boomers just cannot grasp why we do not like AI
• Vanessa Wingårdh's Nobody Actually Wants AI Anymore
• Taylor Lorenz' You Are About to Become Economically Worthless
• Sam Seder's Everyone Is Fed Up With Tech's Ai BS
• Silicon Money's Why Tech CEOs Are Quietly Cancelling Their AI Plans
• The Rational National's Heartwarming: Gen-Z Hates Ai

Update 25^th June 2026

Source

Eric Schmidt, the ex-CEO of Google, recently gave a commencement speech in which he heralded the coming of AI — and was loudly booed by the students. This was not an outlier. There have been a number of similar incidents lately, evidence that many people now really hate AI.

Are we talking about a vocal but unrepresentative minority? No. A recent Pew survey found that American adults believe by a wide margin that AI will be negative for society and, by a smaller margin, that it will be bad for them personally:

we fear that AI will do terrible things because the companies selling it told us it would do terrible things. Last year, for example, Anthropic CEO Darius Amodei declared in an interview with Axios that AI could wipe out half of entry-level white-collar jobs and drive overall unemployment as high as 20 percent within 1 to 5 years.

Source

many ordinary people view AI negatively because they feel that it is being forced on them.

It’s true that many people are voluntarily using large language models for personal convenience or as a business productivity tool. But a significant part of AI use isn’t voluntary. This Wall Street Journal headline from February says it all:

Why are companies doing this? Presumably they believe that AI will raise productivity. But just as importantly, they’re responding to pressure from financial markets, which are rewarding companies for quickly adopting AI, apparently without regard to demonstrated results.

And while Americans workers are being dragooned into using AI, American consumers are being force-fed AI whether they want it or not. Most dramatically, Google has replaced its search engine with AI, without offering the option to opt out. One has to turn to obscure workarounds or third-party sites to get traditional search results.

So many people feel, rightly, that they aren’t being allowed to choose whether to use AI — not using AI has become hard both as a worker and as a consumer.

datacenters are a highly visible reminder of AI’s costs. Datacenters occupy huge tracts of land — one proposed site in Utah will be twice the size of Manhattan. They guzzle electricity and water. When they generate some of their own power, they create major local pollution. Not surprisingly, there is intense opposition to datacenter construction. According to a Reuters Ipsos poll, 57 percent of Americans — two-thirds of Democrats and half of Republicans — would oppose a datacenter in their neighborhood. Only 14 percent would support one.

A massive data center project in Box Elder County, Utah, helped bring down the state’s Senate president, who lost his GOP primary on Tuesday after his support for the controversial development fueled voter backlash.

Stuart Adams, one of Utah’s most powerful politicians and the longest-serving president of the state Senate in its history, lost to challenger Stephanie Hollist, a former university lawyer and vocal opponent of the data center.

Hollist accused Adams, as well as the state’s broader political establishment, of ignoring public concerns about a Stratos data center project that critics feared could cause serious environmental harms.
...
Box Elder County Commissioners Boyd Bingham and Lee Perry, who voted in favor of allowing the plans to continue, also lost their primary elections.

Source

even before the advent of AI, tech companies had lost the public’s trust. Over the years Pew has regularly surveyed the public for its views on technology companies, asking whether they have a positive or a negative effect “on the way things are going.” In 2015 public opinion of tech companies was overwhelmingly positive. By 2022, the year ChatGPT was released, that goodwill had evaporated.

Why have Americans turned on tech companies? While it surely reflects growing awareness of the psychological and societal harm done by social media, much of it also reflects the enshittification of tech products.

Source

AI is tightly linked in the public mind with the tech oligarchs who are pushing it. There is widespread awareness of the growing concentration of wealth and power at the top and how this is distorting our politics and harming our society. Aside from the MAGA faithful, Americans overwhelmingly favor government policies to reduce wealth inequality:

And AI is widely perceived, for good reason, as a technology that will increase the concentration of wealth at the top. Indeed, as I said, the AI companies themselves have already told us that the technology will have extremely negative effects on workers.

There’s a strong element of poetic justice in this turn of events. The AI industry deliberately made itself look menacing as a financial strategy, believing that the markets would reward the appearance of being “edgy.” In so doing, however, tech made itself highly unpopular. And even in an era in which money all too often buys power, public opinion matters.

• Brian Llinas: RAG-TEC: Extracting and Classifying Topics in Digital News Collections Using LLMs
• Lawrence Obiuwevwi: Cognitive Prosthetic: An AI-Enabled Multimodal System for Episodic Recall in Knowledge Work
• Akshay Kolgar Nayak: Contextual Scaffolding and Self-Efficacy: Supporting Computer Skill Development among Blind Learners in India
• Rochana Obadage: CC30k: A Citation Contexts Dataset for Reproducibility-Oriented Sentiment Analysis
• Himarsha Jayanetti: Infrastructure for Tracking Information Flow from Social Media to U.S. TV News

We were fortunate enough to welcome back several of our alumni, including: Sawood Alam (PhD, 2020), Justin Brunelle (PhD, 2016) Yasith Jayawardana (PhD, 2024), Mat Kelly (PhD, 2019), Bhanuka Mahanama (PhD, 2025), and Alexander Nwala (PhD, 2020). We really appreciate the ongoing relationship we have with our alumni -- WSDL is for life! 

If you were unable to attend, we recorded the students' presentations and have embedded the video below.

--Michael 

The following remarks were delivered at the National Summit on Local News Preservation on June 17, 2026.

News is, with few exceptions, place-based. “Where” is one of the journalist’s first questions, and without it, news feels groundless, baseless, unmoored. But news used to not only be written from a specific place, but also written for the people living in that specific place. In that sense, all news used to be local. But whether the news reported on immediate surroundings, the colony or state, the nation or empire, the function of newspapers was to provide a public record, both for audiences at the time, and for future readers. In fact, many editors were conscious of this function of the newspaper as a repository; some two hundred years ago, they provided, in the words of Hezekiah Niles in his prospectus for Baltimore’s Weekly Register, “something interesting at the present moment, and as a book of reference, a fund of reading always at hand, a work of much probable value” (September 7, 1811).* Newspapers were, from their earliest days, understood as a public good, as “work[s] of much probable value.”

Information has been mobile from its early days — from the troubadour to the telegraph, one might say — but because “news” is the sum of information plus time, or timeliness to be more exact, the accelerated speed of transmission is vital to the rise of news for national and international audiences. Most scholars agree that syndicated news really took hold after the Civil War with Chicago’s A.N. Kellogg Newspaper Company. As with our own moment’s undervaluing of local news, the transition away from “local” newspaper-reading audiences did not happen overnight and cannot be attributed to a single factor. Infrastructure, — in the nineteenth century, the railroads and stereotype printing; today, the internet and social media — combined with sociocultural shifts, makes the world feel smaller.

We are gathered here today to celebrate and to concern ourselves with news that does not move, that stays more or less in the place from which it came. It is, as Lincoln would say, “altogether fitting and proper that we should do this,” for reasons beyond the present moment. As an historian, I have been tasked with adjusting our gaze ever so slightly from the “now” to the “back then.” I’d like to draw out a few examples of the importance of local news in historical research in the hopes of showing, rather than telling, not only that historical local news matters, but also that we must retain its understanding of itself as a public good. Without this printed record, it is easy to forget that it is not just people who have history, but places too. Without the context of place, history too feels groundless, baseless, and unmoored. I fear that the history we are creating today might not even exist in a century from now, but I will say more about that later.

When we think of historical newspapers, we often think of the people whose lives they capture, and perhaps even the lives of the people who produced them. We might even think of the stories they omit, of the people not represented in these wilted and worn pages. More recently, environmental history has helped us to see historical newspapers as the place to uncover the histories of the land, as the sources that will shed light on the social and cultural causes of global warming and environmental degradation. The research and storytelling in the work of scholars and journalists alike are changing how we think of newspapers and the vital role they play in understanding the histories, and in turn, the futures of our environment. Corporate archives often do more to conceal than to reveal, if one can gain access to them at all. Government records can be little better. But, local newspapers contain the stories written by the local intrepid reporter who cites evidence of a paper mill’s destruction of the river running through the town. Similarly, where official records might have denied harmful contamination from Superfund sites, a historian can scan obituaries for evidence of untimely deaths from cancer clusters. We need local newspapers to read against official narratives told of the land, as well as of the people who inhabit it.

Inaugural issue of the Navajo Times, November 1, 1959. Source: Library of Congress.

Think, for example, of the hyperlocal newspapers published on reservations, such as the first newspaper published in the Navajo language in Window Rock, Arizona. The Navajo Times’s purpose, as stated in its inaugural issue in 1959, was “to serve the 6,000 Navajo children who are attending off-reservation schools. It is hoped that this newspaper will keep them informed about what is happening on their reservation. It is also hoped that this is a step toward supplying the Navajo people with an ever-increasing flow of information.” This paper then was to keep the local — the power of place — in the hearts and minds of its intended audience, no matter where they went, or were forced to go. This statement of purpose from the Navajo Times is a gentle reminder that people are, in part, defined by place, and the stories they told of “the local” have much to teach us today.

Aggregation of Historical Local News, a National Prerogative

As a researcher, I have been privileged to be a consumer of local news, and as a former senior program officer at the National Endowment for the Humanities (NEH), I was also a producer, managing the National Digital Newspaper Program (NDNP), the NEH program that funds and co-creates Chronicling America. Chronicling America does and does not serve the preservation of local news. As of my most recent check, it includes 4,684 newspaper titles and over 3 million issues, dating from 1736 to 1963. These titles certainly include a handful of “local newspapers,” no matter how that category is defined. And yet, this was not the intention of NDNP. In fact, for about the first 15 years of its existence, NDNP inadvertently discouraged the preservation of “local” newspapers by encouraging applicants to begin with papers of record, with those that had long runs, and most likely, were published in big cities intended for large audiences. Because so many of the states have by now contributed these “major” papers, the program shifted, in 2021, to newspapers that tell underrepresented histories. Until recently, applicants were welcome to define “underrepresented” in any way they chose, and they often chose place-based representation (see the 2024 Notice of Funding Opportunity (NOFO); “underrepresented” has been expunged from the 2025 NOFO). More and more newspapers from neglected areas were being included in Chronicling America. Without empirical evidence to back me up, I would venture that there has been a rise in “local” if not “smaller” newspapers in Chronicling America in recent years.

Panel from Malcolm W. Ater, Seeing Washington (1957). Source: University of Nebraska Libraries.

And yet, the resource will always fall short, for it cannot provide all that we are looking for. Those who designed it, back in the early 2000s, knew this. They knew that the 1963 cutoff date for inclusion would exclude many important papers, and they knew that many state partners were able to digitize far more newspapers than could be included in the national aggregator. And so, the genius of NDNP is not only what you find in Chronicling America, but also in the way that it established standards for newspaper digitization. Its hope was that Chronicling America would be just one of the manifestations of the work it enabled; states would also become aggregators of their newspapers, using the same standards. And they have done so, creating amazing state-level digital newspaper repositories, such as Georgia’s Historical Newspapers or the Texas Digital Newspaper Program, just to name a few. Such state-level efforts were encouraged to reuse content digitized for Chronicling America, as well as to include that which did not make it to the national aggregation level. CONSERV cataloging and the technical guidelines for digitization demonstrate that standardization must be part of the work of preservation; otherwise, the “local” risks being relegated to the dustbin of history. If we believe that “local” does not mean “less than,” then we must use the same standards for categorizing and making accessible local newspapers that we do for the so-called “papers of record.”

Local News as Public Data

The term “dark ages” has grown out of fashion for historians because it suggests that no light existed in the period from 500 to 1000 CE. Painstaking scholarship has slowly uncovered that this is not the case, that in fact people were innovating, creating, and exploring in ways not all that different from classical antiquity before it and the Renaissance afterwards. And, yet, the label “dark ages” resonates today, not because our current moment is failing to produce meaningful and innovative work, but because of the great difficulty the future will face in tracing the lives and outputs of the people of our moment. As Jonathan Zittrain has pointed out, the internet, which for better or worse houses much of our current culture’s memory and creativity, is “rotting.” Technologies that once signaled a great unfurling of access to information are now showing cracks and vulnerabilities when considered at a historical scale. The historical record of the current moment will be, in many ways, “dark.” We are in what might be referred to as the “digital dark ages,” not because important things are not happening, but rather because the future’s light on this moment is diminished, if not snuffed out completely.

We have been asked this morning to address what “getting local news preservation right” would require, and my response is that we must provide multiple ways to shine light on our current moment. Data is the new oil, or so we’ve been told for the last two decades, and I roll my eyes at this metaphor not because it is not true, but because data is so much more than a market commodity. Local news, in its many forms and instantiations, is public data, and we must preserve it because it has an inherent value that surpasses our current moment, that is so much more than its commodification. Because we cannot see the future, we cannot know all of these values, but based on our reliance on historical “local” newspapers to know the past, we can trust that they exist.

For the most part, libraries exist outside of the naked self-interest of capitalism, and the people who work in them must play a role in the preservation of local news. Librarians are the original public interest technologists, we might say, and I urge us to put them at the forefront of our conversations here. Journalists too exist in a space not completely captured by market forces, and they too want information to be free and to be accessible. I see an alchemy emerging from the alliance between these two professions that offers future generations not only a historical record of their communities from which they can analyze and learn, but also a model for forms of affinity and alignment that exceed capitalist logics and exemplify other modes of cooperative work. This gathering is an important step in this effort, and I am honored to be a part of it.

* Thank you to my friend and collaborator Will Slauter for providing this example and for his assistance with these remarks throughout.

AI and open data, archives and climate, BrainCommons, SkyCommons, MCP, CKAN, and more.

The post Making AI work for communities – OKFN Newsletter June 2026 first appeared on Open Knowledge Blog.

In the hours following the release of CVE-2026-8461 for the project FFmpeg, site reliability workers and systems administrators scrambled to desperately rebuild and patch all their systems to fix an out-of-bounds write in the MagicYUV decoder (libavcodec/magicyuv.c) caused by improper bounds checking, resulting in heap corruption, denial of service, and potential remote code execution when processing a specially crafted video file. This is due to the affected components being written in C, the only programming language where these vulnerabilities regularly happen. "This was a terrible tragedy, but sometimes these things just happen and there's nothing anyone can do to stop them," said programmer Mrs. Kitty Smitham, echoing statements expressed by hundreds of thousands of programmers who use the only language where 90% of the world's memory safety vulnerabilities have occurred in the last 50 years, and whose projects are 20 times more likely to have security vulnerabilities. "It's a shame, but what can we do? There really isn't anything we can do to prevent memory safety vulnerabilities from happening if the programmer doesn't want to write their code in a robust manner." At press time, users of the only programming language in the world where these vulnerabilities regularly happen once or twice per quarter for the last eight years were referring to themselves and their situation as "helpless."

We discussed with our Nordic members and friends how, across Europe and beyond, policymakers are rethinking the foundations of the digital state and the infrastructure that powers it.

The post Open Knowledge and Critical Infrastructure: a Nordic Conversation first appeared on Open Knowledge Blog.

It’s June, and that means that our annual Pride Month Treasure Hunt is back!

We’ve scattered a shower of rainbows around the site, and it’s up to you to try and find them all.

• Decipher the clues and visit the corresponding LibraryThing pages to find a rainbow. Each clue points to a specific page right here on LibraryThing. Remember, they are not necessarily work pages!
• If there’s a rainbow on a page, you’ll see a banner at the top of the page.
• You have just under one week to find all the rainbows (until 11:59pm EDT, Tuesday June 30th).
• Come brag about your shower of rainbows (and get hints) on Talk.

Win prizes:

• Any member who finds at least two rainbows will be awarded a rainbow badge. Badge ().
• Members who find all 11 rainbows will be entered into a drawing for one of five sets of LibraryThing (or TinyCat) swag. We’ll announce winners at the end of the hunt.

P.S. Thanks to conceptDawg for the kookaburra illustration. ConceptDawg has made all of our treasure hunt graphics in the last couple of years. We like them, and hope you do, too!

Epithalmion for Tyler by James Tate

I thought I knew something
about loneliness but
you go to the stockyards

buy a pig’s ear and sew
it on your couch. That, you
said, is my best friend – we

have spirited talks. Even
then I thought: a man of
such exquisite emptiness

(and you cultivated it so)
is ground for fine flowers.

In the hours following the release of CVE-2026-55200 for the project libssh2, site reliability workers and systems administrators scrambled to desperately rebuild and patch all their systems to fix an out-of-bounds write in ssh2_transport_read() due to a missing upper bound check on the packet_length field, resulting in heap corruption and potential remote code execution. This is due to the affected components being written in C, the only programming language where these vulnerabilities regularly happen. "This was a terrible tragedy, but sometimes these things just happen and there's nothing anyone can do to stop them," said programmer Mr. Alex Doyle, echoing statements expressed by hundreds of thousands of programmers who use the only language where 90% of the world's memory safety vulnerabilities have occurred in the last 50 years, and whose projects are 20 times more likely to have security vulnerabilities. "It's a shame, but what can we do? There really isn't anything we can do to prevent memory safety vulnerabilities from happening if the programmer doesn't want to write their code in a robust manner." At press time, users of the only programming language in the world where these vulnerabilities regularly happen once or twice per quarter for the last eight years were referring to themselves and their situation as "helpless."

David Cahn, Sept '23

Source

SemiAnalysis, an extremely pro-AI semiconductor analyst, ran a test made up of random long-horizon coding tasks until they maxed out the limit on OpenAI and Anthropic’s various subscription levels.

Their findings were shocking.

For $200 A Month, You Can Burn $8000 in Anthropic Tokens or $14,000 In OpenAI Tokens

That’s right. Anyone with a $200-a-month Anthropic subscription can burn $8000 in tokens, and with a $200-a-month ChatGPT subscription, you can burn $14,000 in tokens.

Source

Source

With the current subsidies, all it takes for a user to have a gross margin of at best negative 25% is for them to use as little as 25% of their rate limit.

OpenAI Had $13.07 Billion In Revenue, $34 Billion In Costs and Expenses, and $20.92 Billion In Losses, with a net loss attributable to the company of $38.53 Billion

2025 was the year that OpenAI converted from a non-profit to a for-profit entity, leading to a $41.55 billion loss due to changes in fair value of convertible interests and warrant liability.
...
Ultimately, the net loss attributable to OpenAI in 2025 was $38.5 billion.

At the end of the year, OpenAI had just over $50 billion in assets, with almost half of that in cash.

• Revenue: $13.07 billion
• ...
• Sales and Marketing: $5.73 billion

Leaked internal documents viewed by Where’s Your Ed At reveal that Microsoft intends to pause new signups for the student and paid individual tiers of AI coding product GitHub Copilot, tighter rate limits, and eventually move users to “token-based billing,” charging them based on what the actual cost of their token burn really is.

The document says that although token-based billing has been a top priority for Microsoft, it became more urgent in recent months, with the week-over-week cost of running GitHub Copilot nearly doubling since January.

The move to token-based billing will see GitHub users charged based on their usage of the platform, and how many tokens their prompts consume — and thus, how much compute they use.

But the company got a shock when Anthropic switched it over to token-based pricing in May. “Our spend went up 7x the first day and I’m like, oh shit, we created a monster,” said Busse. “[Large language model] companies have been subsidising all of our usage and now no longer. User-based pricing shelters you.”

it’s now apparent that using AI is more expensive than hiring people, especially since it offers only limited productivity gains at the moment.

Bryan Catanzaro, Nvidia's VP of applied deep learning, recently told Axios that "For my team, the cost of compute is far beyond the costs of the employees", quite an interesting statement from the company selling the shovels for the gold rush.

That perspective is shared by Uber's CTO Praveen Naga, who "[went] back to the drawing board because the budget [he] thought [he] would need is blown away already" as of two weeks ago. Likewise, Swan AI's Amos Bar-Joseph posted a while back on LinkedIn about how proud he was about a $113k bill from Anthropic (makers of Claude) for a four-person team.

Oversimplified math pins that amount that at $28k per person per month, which is likely more than each person's monthly wages. Jokes abound right now that "companies have discovered jobs again," and the humor is backed up by a 2024 MIT study stating that 77% of the time, it was preferable to have humans do the work.

Source

This means that for the AI industry to service its debt, it needs to generate hundreds of billions of dollars in profit each year.

Even giant monopolies like Google don’t make enough profit to service that much debt. AI can’t just be a novelty industry; it needs to replace human labour on a colossal scale to service this debt. Let’s optimistically assume AI one day reaches a 10% profitability margin, a cost parity with human labour, and the ability to complete most jobs (none of which are currently the case). Well, the average US salary is roughly $66,000, so at a 10% profit, the AI company will make on average $6,600 per year per job it replaces. To generate the $309 billion needed to service their debt, the AI industry will need to replace 46.8 million jobs, equivalent to around 27% of the current number of jobs in the US.

While this is all very rough maths, it highlights the implicit bet created by the debt the AI industry has racked up. To simply not default on this debt, the AI industry has to rapidly displace human labour at a staggering scale, even if we are extremely optimistic about AI’s economics.

Wages and salaries averaged $32.60 per hour worked and accounted for 69.9 percent of employer costs, while benefit costs averaged $14.01 per hour worked and accounted for the remaining 30.1 percent.

• OpenAI's Sam Altman said that costs have become a "huge issue" for customers and the company is considering "drastic" price cuts to rein in rival Anthropic PBC’s lead in the corporate market.
• Kyle Orland reported that Anthropic “pauses” token-based billing for its Claude Agent SDK:
  

  Last month, Anthropic announced a billing change that would have substantially increased costs for heavy users of its automation-focused Claude Agent SDK, including many third-party apps. On Monday, though, Anthropic abruptly announced it had paused those pricing changes just as they were set to take effect, allowing Agent SDK users to continue drawing from the more generous usage limits in their existing Claude subscriptions.

• Microsoft Plans June 30, 2026 Shift From Claude Code to Copilot CLI:
  

  Microsoft is reportedly cancelling most Claude Code access for engineers in its Experiences and Devices division by June 30, 2026, shifting teams working on Windows, Microsoft 365, Outlook, Teams, and Surface toward GitHub Copilot CLI as the company tries to rein in internal AI coding costs. The decision is more than a procurement tweak. It is a rare glimpse into what happens when the world’s most aggressive AI software company runs into the same metered-billing problem now facing every large engineering organization.

Footnotes

1. xAI's product is so bad that even their employees won't use it and Musk has said it needs to be re-written from the ground up. So xAI has been reduced to renting its compute infrastructure to its competitors.

Registration is now open for the virtual Digital Library Federation’s (DLF) Forum online, October 14-15, 2026. The DLF Forum is a dynamic gathering place for GLAM professionals to share ideas, sustain critical work, and spark innovation. It connects library, archives, and museum practitioners, as well as other knowledge workers, through intentional community building and collaborative exchange. Learn more about 

The full Forum program is also available to explore, and we’re thrilled to announce our keynote speaker, Dr. Salwa Ismail. Additionally, the application for Digital Storytelling Fellows is now open. 

 Be sure to check out the exciting details and join us in building momentum for what’s sure to be an inspiring experience.

Secure the early bird rate and start planning for yet another memorable online conference with DLF. 

DLF member organizations receive two complimentary registrations for the virtual DLF Forum as part of their member benefits. Not sure who received your code? Email us at forum@diglib.org. 

Register Today

If you have any questions, please write to us at forum@diglib.org. We’re looking forward to seeing you online this fall.

-Team DLF

P.S. Want to stay updated on all things related to the DLF Forum? Subscribe to our Forum newsletter.

The post 2026 Virtual DLF Forum: Registration Open, Program Released, Keynote Announced, Digital Storytelling Fellows Application Open appeared first on DLF.

Applications are now open for the 2026 Virtual DLF Forum Digital Storytelling Fellows program

This year, DLF is launching a new fellowship experience that is directly connected to the Forum’s new Digital Storytelling Presentation session format. The program centers on digital storytelling, emerging technologies, and ethical practice across libraries, archives, and museums, while creating intentional opportunities for participation, reflection, and community engagement in a virtual setting.

We invite early-career and underrepresented practitioners to participate in the 2026 Virtual DLF Forum and help shape conversations about how stories, platforms, technologies, and communities intersect in our work.

Why Digital Storytelling?

Digital storytelling projects, including exhibits, platforms, collections, and collaborative archives, are increasingly central to how cultural heritage organizations document, interpret, and share knowledge. These projects also raise important questions about representation, labor, technology, access, and stewardship.

A cohort of 8–10 Fellows will engage directly with these themes through participation in DLF’s new Digital Storytelling Presentation sessions: interactive, installation-inspired presentations featuring collaboratively developed digital projects and dedicated discussion time. Because this is a new and experimental session type for 2026, the fellowship intentionally builds in structured engagement and feedback to help strengthen the experience and better understand what works in a virtual Forum environment.

Fellows will serve as conversation catalysts during these sessions by contributing questions, reflections, and observations that surface broader themes across the Forum community.

About the New Session Format

40-minute Digital Storytelling Presentation
An interactive session highlighting digital storytelling projects developed through collaborative partnerships. Digital Storytelling Presentations focus on installation-inspired digital storytelling work, such as exhibits, platforms, or collections, designed for immersive and experiential engagement. Sessions may feature up to three presenters, for example, pairing a digital librarian or archivist with a community partner, student, artist, or scholar whose work is represented in, or inspired by, the project. Sessions will be scheduled within 50-minute blocks, leaving dedicated time for Q&A and discussion. Read more about the new session type here: https://www.diglib.org/digital-storytelling-in-practice-a-new-session-format-for-the-dlf-forum/ 

What Fellows Receive

Selected Fellows will receive:

• Complimentary registration to the 2026 Virtual DLF Forum
• A $250 stipend
• Participation in a small cohort of 8–10 Fellows
• A pre-Forum virtual orientation and meet-and-greet
• Visibility through publication on the DLF blog

Fellowship Expectations

Fellows will:

• Participate in a virtual orientation session on October 6, 2026
• Attend the 2026 Virtual DLF Forum on October 14–15, 2026
• Engage actively in Digital Storytelling Presentation sessions by:
  • Asking questions in chat or live discussion
  • Optionally sharing brief verbal reflections during session discussions
  • Incorporating insights from storytelling sessions into their post-Forum reflection
• Participate in a virtual debrief session and/or complete a feedback survey
• Contribute a short public reflection for publication on diglib.org (up to 1,000 words, due November 15, 2026)

Reflections may explore themes such as ethical technology, collaborative storytelling, digital exhibits, community memory, access, or emerging questions around provenance and stewardship.

Selected Fellows must attend the Forum in order to receive the stipend. Apply here. 

Who Should Apply

We welcome applications from:

• Early-career professionals (fewer than 7 years of experience)
• Students and recent graduates
• Contingent, contract, and adjunct practitioners
• Professionals working in under-resourced or capacity-limited institutions
• First-time DLF Forum attendees
• Practitioners whose identities and perspectives are historically underrepresented in digital libraries and cultural heritage spaces

We recognize that professional pathways are not always linear. If you are unsure whether you meet these criteria, we still encourage you to apply.

Application Process

The application process is designed to be straightforward and accessible. Applicants will:

• Answer a few short questions
• Submit a brief personal statement (maximum 4,000 characters)
• Share a link to an online professional profile, if available

Application available here: Google Form. 

Applications Close Tuesday, July 21, 2026, at 11:59 pm ET

The post 2026 Virtual DLF Forum Digital Storytelling Fellows appeared first on DLF.

What happens if I just point a git server at an object storage bucket?

Back when I was porting agent sandboxes to Go, I built everything on top of billy, a filesystem abstraction for Go. The whole trick of the project was teaching a Tigris bucket to act enough like a filesystem that a shell interpreter and its tools couldn’t tell the difference. Billy was the key layer that made the entire façade fall into place.

After I had gotten things working, I learned that I’m using billy way outside its normal usecase. It was originally made for go-git, a pure-Go implementation of git’s protocols and data formats. It doesn’t rely on the /usr/bin/git binary existing at all. Every method on billy’s filesystem interface exists purely because go-git needs it. This gave me a terrible idea: I already have a bucket that can quack like a filesystem and go-git’s native language is “filesystem”.

Can this Just Work™? Let's find out.

Git was always an object store

If you strip away the porcelain, a git repository is 4 basic things:

• Objects, or compressed blobs of data. Most of the objects in any individual repository are files.
• Trees, or objects that map to other objects. TL;DR: trees are folders.
• Commits, or objects that point at one tree and their parent commit. This lets you pin down which files belong to one logical change set.
• Refs, branches and tags, they are tiny mutable pointers into the pile of objects.

For example, let’s say I just made a new git repository and committed a README.md to it. The tree for the .git folder looks something like this:

$ tree .git
        .git
        ├── COMMIT_EDITMSG
        ├── config
        ├── HEAD
        ├── index
        ├── objects
        │   ├── 5e
        │   │   └── b8151eb669aa4467b6dea2c4bce19183cd0b41
        │   ├── 6a
        │   │   └── 6a8ecfcae2632152486aca3d9150ef83dedd66
        │   ├── f4
        │   │   └── d2487a1c6d742c8037c0296ddf80625190bd80
        │   ├── info
        │   └── pack
        └── refs
            ├── heads
            │   └── main
            └── tags
        

As you can see there are three objects. One of them is the commit 5eb8151eb669aa4467b6dea2c4bce19183cd0b41, the next is the tree, and the last one is the README file. The main branch also points to that commit:

$ cat .git/refs/heads/main
        5eb8151eb669aa4467b6dea2c4bce19183cd0b41
        

The cool part is that half of this is content-addressed. The content-addressed bits never change once they’ve been committed. Git objects are a great fit for Tigris’ internal model because they are append-only storage, just like the fundamental model Tigris is built upon. The things that do change often are the refs, which are updated to point to the latest commit. These are tiny files though, which means that Tigris can handle them with no effort required.

However, when we host git repositories on a server, we end up creating single points of failure. Our git repos are hosted on single machines that can and will break. The entire implementation relies on git objects being 1:1 correlated with filesystem objects because everyone (even GitHub) shells out to the git binary to actually store files. Hosting git repos becomes one of the most stateful services in our stateless cloud-native environment.

Sure git is in-theory decentralized, but most of us have ended up using that to put our git repositories in one big store that has questionable uptime practices: GitHub. To be fair to hubbers, GitHub operates at a scale that none of us can really think about. They’ve been pushing the limits since their inception where they had to get Engine Yard to keep building them bigger servers to handle the load. They have to do everything with a big mounted filesystem because git’s tooling gives them no other option.

A travesty of horrors beyond human comprehension

Now suppose this weirdness bothers you enough to do something about it. To build a git server without storing everything in the local filesystem, you have to speak git somehow, and the conventional options aren’t really all that great:

• If you shell out to the git binary, now your “library” is the argv of the git process and your error handling is screen-scraping output. Internally, git implements its functionality with a billionty subcommands rather than exposing it all as a library. The codebase is held together by load-bearing calls to die(), which kills the process.
• If you link into git’s guts with libgit, you inherit the “when things go bad, die()” behaviour and your app now suddenly starts crashing at random. This is not good for uptime.
• If you try to use libgit2 (the rewrite-that’s-actually-a-library), you have to reckon with the fact that it’s addled by the GPL (with a linking exception, try explaining that to your lawyers), you have to eat the jump to C every time you do anything with git (very often), development has stalled, the Go bindings have been archived, and it still assumes a local filesystem despite assurances it does not.

It might sound hopeless, right? You may be able to use WebAssembly or something to contain the madness (assuming you have a good way to implement fork()/exec() or posix_spawn() or something similar), but what if there was a pure Go library that could handle this all for us?

Enter go-git, a pure-go implementation of the git protocol and internals from scratch. This doesn’t rely on cgo or /usr/bin/git and it does not assume the repositories are stored in the local filesystem. Its storage interface is written against billy, the exact interface I’ve already taught to speak Tigris. I wanted a git server that was just in a bucket and the pieces were sitting there and calling to me.

Oh no, it works

So I hacked up objgit, a git server backed by object storage. The only filesystem call I had to add to get it booting was MkdirAll. I wired up the transport package to a socket to implement the plaintext git protocol, hooked it up to a bucket, and pushed the repo I was currently working on.

To my absolute astonishment, it worked.

Git pushed, pulled, logged, blamed, tagged, the whole kit and kaboodle. I didn’t have to implement git myself, I just committed an egregious amount of shoving a square peg into a round hole until the peg went in.

In hindsight this makes an annoying amount of sense. A bare repo is those four kinds of things on a filesystem; swap the filesystem for object storage and everything else Just Working™ is perfectly logical. Git’s on-disk format is its database schema and if you fake open/stat/rename convincingly enough the entire façade keeps working because APIs are the lies we tell ourselves to make us sleep at night.

After a lot of hacking, I ended up with a feature list kinda like this:

• Push and pull over three transports: HTTP, classic git://, and SSH
• Repositories upserted on first push
• Absolutely no effort put into authentication as this is an experiment and authentication is annoying and complicated
• Prometheus metrics so I could optimize the filesystem layer

Everything comes out of one Go binary with no local state, even the generated SSH keys are stored in the bucket. You can run this in a Kubernetes cluster with only the mutable storage required being temporary files for an optimistic cache when doing smart git clones.

The rest of this post is what it took to get from “oh no, it works” to something close to usable.

Obligatory disclaimer (like the best things in life): this is an experiment. It has not been tested thoroughly or vetted for correctness. If it breaks in half, you get to keep both pieces. Please do not move your company’s monorepo onto this and then email me when it catches fire.

That one POSIX idiom that survived

Git is paranoid about durability, and its entire strategy is one Unix idiom that you end up seeing many places: write new data to a temporary file and then rename(2) it into place after you’ve assured it’s correct. POSIX guarantees that rename is atomic, so readers either see the old file or the new one, not an intermediate state inbetwixt the two. Packfiles (bundles of objects) land as temporary files when uploaded then moved to their permanent home. Refs are written as locked temporary files and then renamed over the ref. It’s rename all the way down.

Object storage traditionally does not have rename as one atomic operation. S3’s answer is to create exactly that intermediate state: CopyObject to the new place and DeleteObject on the old one. This makes the most load-bearing idiom in Git’s philosophy fall to pieces.

Luckily, Tigris has an extension for this: RenameObject. To use it, pass an additional X-Tigris-Rename: true header to a CopyObject call and instead of copying then deleting on the client, it moves the metadata around on the server. One round trip, no data movement, and the Unix idiom maps on the bucket 1:1. Objgit’s implementation of Rename is trivial:

// internal/s3fs/basic.go
        // RenameObject is a Tigris extension that renames in place (no data copy),
        // so we don't need a separate CopyObject + DeleteObject.
        copySource := fs3.bucket + "/" + src
        _, err := fs3.client.RenameObject(ctx, &s3.CopyObjectInput{
            Bucket:     &fs3.bucket,
            CopySource: &copySource,
            Key:        &dst,
        })
        

A second, sneakier violation hides in the same codepath. When go-git writes a temporary file, it creates that temporary file and then immediately starts opening it for reading so it can build the pack index. You cannot do that with a single live object in any object storage system, you are either reading or writing, never both. I ended up working around this by cheating a bit and buffering the contents of newly written pack files into memory so that this game of chicken kept working. I may have to change this to write that pack cache to the filesystem as trying to push gcc.git made me run out of RAM. At the very least, everything lies consistently enough that git doesn’t care, so win!

Death by a thousand stat() calls

With this correctness sorted, I tried pushing the golang/go repository to objgit to see how long it would take. It did work, but it took forever. Using the prometheus metrics I mentioned before, I saw that it was making biblical amounts of HeadObject calls. Some blocking profile analysis pointed to the fact that the git library was using the stat() call to detect if a file exists. The flow was like:

• Client has object x
• Check if object x exists
• Check if any pack has object x

And so on ad infinitum. This is fine-ish on a local filesystem because those syscalls resolve in microseconds, not the tens of milliseconds it takes to get from my office to the nearest Tigris region (please expand to Ottawa, I would love that so much).

This was compounded with a discovery that the transport I was using (SSH — classic git:// shares the same code path) was exploding every packfile into loose objects when pushing it. Each loose object write was costing two round trips: stat() to check if a file exists and then open() / write() to actually put the data into Tigris. This made a 100,000 object packfile cost 200,000 object storage calls. Call it 10ms of latency for each one, and that’s over half an hour of waiting for responses that mostly say “404 not found”.

Caching can’t really save you here either, read caches would absorb the repeated reads; but this is a firehose of writes to 100,000 paths that probably have never been read and likely will never be seen again.

The reason only two transports had this problem is a deadlock story. The git library's fast path stores an incoming pack whole through its PackfileWriter, by copying from the connection until io.EOF. Over HTTP that's fine: the request body ends, EOF arrives, everyone goes home. Over git:// and SSH, the connection is a persistent socket and the client is holding it open, politely waiting for the server's status report. EOF never comes. The copy waits forever, the client waits forever, and you have invented a distributed deadlock with two participants. The original workaround was to hide the PackfileWriter capability on those transports so go-git fell back to its streaming parser that writes every object loose. Hence the stat storm.

So the solution was to stop depending on EOF at all. Packfiles are self-delimiting: the header says how many objects are coming and a trailing checksum marks the end, so a packfile scanner walks the stream and stops at the trailer while a TeeReader mirrors exactly those bytes into the PackfileWriter. This makes the rest of the façade fall into place and the git library is happy. This made pushes into two uploads: a packfile and its index instead of a torrent of round trips that mean nothing.

What about cloning?

Once I got pushing fixed, I moved on to the read path. In order to emulate ReadAt, I used ranged GetObject requests so that the git library could read individual objects out of packfiles. I was happy with this hack, but there was one problem: the latency curse struck again. Cloning a simple repo with 318 objects and a 200KiB packfile made over 8,500 GetObject calls before I killed it.

A git client cloning a repository reads repository packfiles thousands of times with random access, walking objects and candidate delta bases over and over. On a local disk you never notice because your page cache eats that for breakfast. When every call is an HTTP request, a 200KiB repo turns into dozens of megabytes of round trips. A 20MiB repo was effectively unservable.

In other words, I had un-cached the one workload that caching was designed to solve.

The fix leans on a gift from git: pack files are immutable and content-addressed. pack-<sha>.pack will never change for as long as it exists. This makes them trivially cacheable to a faster local medium, such as the filesystem. No invalidation logic is required. I made objgit download packs to a local temporary folder and serve reads from there. To be on the safe side, I did add least-recently-used caching to the mix so that my temp folder wouldn’t blow up unexpectedly. This does mean that the first request for pack files is slower, but then everything else is at filesystem speed.

Yes, this relies on the local disk again, but only as a cache that can and will be thrown away. I think trading a stateless ideal for clones that terminate in reasonable amounts of time is a worthwhile bargain.

Why so ListObjectsV2, Batman?

Once the other disasters were out of the way, one more remained: the metrics showed a flood of ListObjectsV2 calls every time a clone was made and didn’t stop making those calls after it was done.

Two things compounded. First, when git looks up an object that isn't packed, it probes for a loose object at objects/<xx>/<rest-of-hash>. objgit keeps packs whole, so there are no loose objects, so every probe misses, and each miss across a distinct two-hex prefix triggered a directory listing to find out. There are 256 possible prefixes. A single clone could issue up to 256 ListObjectsV2 calls whose collective answer was a resounding "there is nothing here."

Second (and more embarrassing), the listing cache already had an optimization for this. It collapsed entire subtree lookups into recursive scans so a single listing of the repository could answer every stat() and probe beneath it. It was completely dead in production. The cache matched recursive prefixes against the repo root (refs/), but every repo is chrooted to its own directory, so real keys look like myrepo.git/refs/heads/main. The prefix check wasn’t aware of chroots so it never actually matched anything. Nobody noticed because a cache that degrades to “no caching” still returns the correct answer, just slowly. To rub it in, a cache warmer was dutifully re-listing every one of those useless prefixes every 30 seconds for 10 minutes after each clone. Thousands of background list calls were burned in the service of caching nothing of use.

The fix was insultingly small: when a repo’s filesystem gets chrooted, register that chroot as a recursive subtree root within the cache. This made the cache actually useful and resulted in only one ListObjectsV2 call instead of hundreds. Every sufficiently advanced cache is indistinguishable from a no-op until someone graphs the miss rate.

None of these disasters were exotic. They’re the things filesystems and kernels give you for free — and every perfectly reasonable disk assumption fell to pieces once a network round trip sat at the core. Serving Git repositories is an accidental filesystem latency benchmark. If your storage abstraction has a weak point, Git will find it and the metrics will show you where that problem is.

Post-receive hooks go in clown jail

One of the most useful parts of hosting your own git server is setting up post-receive hooks. These have been used since time immemorial for things like automatic deployments when you push code to the server. The heart of this is how we get systems like GitHub Actions: it’s code that runs when you are done pushing.

When you push to objgit with --allow-hooks enabled, it looks for a post-receive hook in .objgit/hooks/receive-pack (this corresponds to the git plumbing action, the name can and will be changed) in the tree of the commit you just pushed. It will then spin up a kefka sandbox with a checkout of the git repository at the commit you just pushed mounted at /src and mutable temporary files at /tmp. It gets coreutils and nothing else. No host filesystem, no network, no arbitrary binaries. Output streams back into the pusher as remote: lines just like when you git push heroku main. Eventually I want to make custom commands to allow you to deploy Tekton pipeline changes and kick off CI jobs that way, but for now I’m happy with this working at all.

You can’t implement policy using these hooks yet. I’m working on it.

Now what?

I taught a bucket to speak git. Where this goes next, roughly in order of how much the ideas keep me up at night.

CI is the obvious next step. I would wire up commands for things like “apply kubernetes object” and “create tekton pipeline run” so that CI would run via your friendly neighborhood Kubernetes cluster and then notify you through some reasonable mechanism. That’s the first thing I’ll build when I have the time.

It would be nice to have a web UI for this, which is complicated for reasons that have nothing to do with git trees, object storage, or anything else and everything to do with the current state of the internet. Git lookups are expensive in the best cases and with the current torrent of unethical scraping ransacking git servers for every scrap of RAM they have, it’s probably a bad idea to implement this without a lot of clever optimizations. Maybe the fact that this doesn’t have load-bearing dependencies on /usr/bin/git would make it more resilient against scrapers. The fact that this is based on object storage could also mean that caching would be a bit easier (having basically unlimited storage is kind of a low-key superpower for caching), but then the main issue would be server load. It’s a tough cookie to handle.

Performance and stability are another place this needs to improve. I’ve tested this on my developer workstation but that is far different from testing it in production. There’s some other performance issues that are easy to fix, but the big one is latency to Tigris. Maybe I can get the devops team to set me up a k3k cluster in production.

Right now this is an experiment as I plug along and feel out the shape of what git-on-object-storage can be. A git server with no disk, no git binary, and no database. If you want to take a look, check it out on GitHub.

Last month, I had the great pleasure of keynoting the CALM (Conference on Academic Library Management) Conference, which is consistently one of my favorites. The video of my talk, Slow Management in a Fast World, is available below for those who would like to check it out! You can also view my slides here which include a long bibliography of works that influenced my talk at the end.

Many thanks to all the amazing folks who organized this conference; it was such an honor and a pleasure to be part of it!

Midsummer sunrise

The sun peeking over the horizon of the Atlantic Ocean. A globe of firey starstuff over calm waves, orange and red spreading out into the sky.

This time, we learned from initiatives around the world that have breathed new life into dusty archives using both open and non-open technologies

The post Roundtable Recap: How to digitise a physical archive to work strategically with AI? first appeared on Open Knowledge Blog.

The Evergreen release team is pleased to announce that point releases for 3.17.2 and 3.16.8 are available.

These releases contain updates for Evergreen’s RESTful API suite, search fixes, and Angular interface updates.

Files and release notes are available on the Downloads page: https://evergreen-ils.org/egdownloads/

Thanks to the June release team: Galen Charlton (Equinox), Gina Monti (Bibliomation), Sarah Moody (ECDI), Andrea Buntz Neiman (Equinox), and Chris Sharp (PINES); as well as everyone who contributed fixes and testing to this release.

Anubis is about to get WebAssembly-based proof of work checks so that administrators can use a non-SHA256 proof of work method to protect their websites. Part of the implementation goals of this work is that the check logic is defined in one place on both client and server. The client and server will then hook into the WebAssembly in order to make sure they're running in lockstep.

However, one small problem comes up. What do you do when the client has WebAssembly disabled? I really don't want to de-facto lock people out of websites. Anubis exists in an impossible balance of user experience, administrator experience, and developer experience and any change to any of these factors disrupts the balance for other factors.

To work around this and also fulfill the goal of having check logic defined once, I decided to take inspiration from the legendary talk The Birth and Death of JavaScript and just recompile the WebAssembly to JavaScript. Sure, the resulting JavaScript will be slower than the equivalent WebAssembly (even more so because disabling WASM usually disables the JavaScript JIT, the thing that makes JavaScript fast), but it will finish eventually. Hopefully it will be more efficient than the existing JavaScript is on lower end hardware, but research is required.

Luckily enough, the tool I need (wasm2js from the binaryen project) is packaged in Linux distributions. The bad news is that distributions ship ancient versions of it that don't get the same output as the version on my development machine's copy from Homebrew.

In order to really make sure that the output of this is deterministic (essential for reproducible builds), I need to bundle a copy of wasm2js. So I did that by building a version of wasm2js compiled to WebAssembly with wasi-sdk. The rest of the article is the tale of reproducibility woe that lead to the implementation I ended up with. Buckle up and enjoy the ride!

Reproducible builds are surprisingly hard

Aoi

Back up a sec, this doesn't make sense to me. If you have the same bytes of input to a compiler, you should get the same bytes of output assuming that the compiler flags, target, and other platform details are controlled for right? A compiler is just a deterministic function of input source code becomes output bytecode, right?

Numa

lol you'd think, but no, it's not. In theory it is (and for small scale compilers it definitely is), but in practice compilers are strange and complicated beasts containing multitudes that no mere mortal can fully comprehend on their own.

There are a shocking number of ways to accidentally create nondeterministic output when doing C/C++ development. One of the easiest is to use the builtin __DATE__ and __TIME__ macros to stamp a build with the time the compiler was executed at:

// hello.cpp
        
        #include <iostream>
        
        int main() {
            std::cout << __DATE__ << " " << __TIME__ << std::endl;
            return 0;
        }
        

Building and running it once gets me this:

$ make clean && make hello.wasm && wasmtime run -W exceptions=y ./hello.wasm
        rm -f hello.o hello.wasm
        wasi-sdk-33.0-x86_64-linux/bin/wasm32-wasip1-clang++ -O3 -fwasm-exceptions -mllvm -wasm-use-legacy-eh=false  -c hello.cpp -o hello.o
        wasi-sdk-33.0-x86_64-linux/bin/wasm32-wasip1-clang++ -O3 -fwasm-exceptions -mllvm -wasm-use-legacy-eh=false  -fwasm-exceptions -lunwind --no-wasm-opt hello.o -o hello.wasm
        Jun 18 2026 00:00:59
        

Another time it gets me this:

$ make clean && make hello.wasm && wasmtime run -W exceptions=y ./hello.wasm
        rm -f hello.o hello.wasm
        wasi-sdk-33.0-x86_64-linux/bin/wasm32-wasip1-clang++ -O3 -fwasm-exceptions -mllvm -wasm-use-legacy-eh=false  -c hello.cpp -o hello.o
        wasi-sdk-33.0-x86_64-linux/bin/wasm32-wasip1-clang++ -O3 -fwasm-exceptions -mllvm -wasm-use-legacy-eh=false  -fwasm-exceptions -lunwind --no-wasm-opt hello.o -o hello.wasm
        Jun 18 2026 00:01:11
        

Even though the source code had the same bytes, the output of the compiler was wildly different.

In order for users and packagers to trust the binaries of wasm2js I'm committing to the Anubis repo, I need to make sure that you can build the same version I built, down to the same bytes. For an added bonus, you should be able to build this on your machine and get the same bytes I got.

Numa

That sure does sound like a great ideal, it would be horrible if something unforeseen came up to ruin it!

Clang silently runs wasm-opt from $PATH behind your back

Among other tools like wasm2js, binaryen has a bunch of other useful tools such as wasm-opt. wasm-opt optimizes WebAssembly compiler output to let you eke out more performance. This doesn't work in every circumstance, but when it does work it makes a huge difference. As such, clang shells out to wasm-opt when doing builds.

This normally makes sense, but in this case it caused builds to fail on my DGX Spark because its version of wasm-opt is too old:

$ uname -m && which wasm-opt && wasm-opt --version
        aarch64
        /usr/bin/wasm-opt
        wasm-opt version 108
        

Compared to my workstation which installs wasm-opt from Homebrew:

$ uname -m && which wasm-opt && wasm-opt --version
        x86_64
        /home/linuxbrew/.linuxbrew/bin/wasm-opt
        wasm-opt version 130
        

Turns out that wasi-sdk and binaryen rely on the WebAssembly Exceptions extension. This is a reasonable thing to assume given that wasi-sdk mostly assumes you're building things for web browsers and 93.86% of browser users have a browser engine new enough to support it. C++ is also one of the main places where exceptions are used, so I guess WebAssembly-native exception handling removes a lot of boilerplate here.

Both wasmtime and wazero require you to flag into exception support. This is fine; we can just pass -W exceptions=y to wasmtime and use a custom runner harness for wazero. The annoying part is what happens when my arm machine's anemic build of wasm-opt sees exception handling instructions, causing it to exit. This made the build fail.

The solution was to pass --no-wasm-opt at the linking step. This removed one angle of irreproducibility.

Mara

I guess in the future we could make it use the version of wasm-opt it just built to optimize the output, but that may be a premature optimization for now.

Clang relies on address layout for ordering things

The version of clang that I use to compile wasm2js has some address-sensitive code generation hiding in its exception handling path. Raw pointer values leak into the order a handful of try_table blocks come out in. This surfaces as every build differing from the next by about 29 bytes:

-002a9af0: 2802 0441 0647 0d00 1f40 0103 0820 0241  (..A.G...@... .A
        -002a9b00: 206a 2103 2002 4138 6a20 0141 086a 10b5   j!. .A8j .A.j..
        -002a9b10: 8881 8000 2104 0b1f 4001 0304 2003 2004  ....!...@... . .
        +002a9af0: 2802 0441 0647 0d00 1f40 0103 041f 4001  (..A.G...@....@.
        +002a9b00: 0309 2002 4120 6a21 0320 0241 386a 2001  .. .A j!. .A8j .
        +002a9b10: 4108 6a10 b588 8180 0021 040b 2003 2004  A.j......!.. . .
        

To make this easier to spot, here's a partial disassembly:

  i32.load  offset=4            ;; 28 02 04
          i32.const 6                   ;; 41 06
          i32.ne                        ;; 47
          br_if     0                   ;; 0d 00
        - try_table (catch_all_ref 8)   ;; 1f 40 01 03 08
        + try_table (catch_all_ref 4)   ;; 1f 40 01 03 04
        + try_table (catch_all_ref 9)   ;; 1f 40 01 03 09
            local.get 2                 ;; 20 02
            i32.const 32                ;; 41 20
            i32.add                     ;; 6a
            local.set 3                 ;; 21 03
            local.get 2                 ;; 20 02
            i32.const 56                ;; 41 38
            i32.add                     ;; 6a
            local.get 1                 ;; 20 01
            i32.const 8                 ;; 41 08
            i32.add                     ;; 6a
            call 17461                  ;; 10 b5 88 81 80 00
            local.set 4                 ;; 21 04
          end                           ;; 0b
        - try_table (catch_all_ref 4)   ;; 1f 40 01 03 04
            local.get 3                 ;; 20 03
            local.get 4                 ;; 20 04
        

The computation is nearly identical, but the byte order is just different enough to also make the catch references differ. This also fires when you build this pinned version of wasm2js on arm64 machines because its pointer iteration order is different from it is on my workstation.

To work around this, I took two steps:

1. Disable address-space randomization for this build using setarch --addr-no-randomize.
2. Create known good sha256 checksums for both x86_64 and arm64 via building this program on machines I trust.

I also made a CI job ensure this:

- name: Ensure reproducibility
          run: |
            cd ./utils/wasm/wasm2js
            ./build.sh
            if sha256sum -c --status shasums.x86_64; then
              echo "OK: rebuilt modules match the recorded x86_64 checksums"
            elif sha256sum -c --status shasums.arm64; then
              echo "OK: rebuilt modules match the recorded arm64 checksums"
            else
              echo "::error::rebuilt wasm2js/wasm-opt match neither recorded checksum set on ${{ matrix.runner }}" >&2
              sha256sum wasm-opt_130.wasm wasm2js_130.wasm
              exit 1
            fi
        

To be extra sure, we have this job run on both x86_64 and arm64 hosts. I'd really love to have this be reproducible across hosts, but that's an upstream LLVM bug that I am not powerful enough to tackle. If you work on LLVM and are reading this, it would be nice to set a seed of some kind to ensure that this iteration order is fixed across architectures.

At the very least builds are deterministic within architectures. This may have to be good enough for now.

Collaboration. Serendipity. Diversity. These are the qualities that come to mind when I think about this year’s OCLC quilt and the community that created it.

The OCLC Quilters, a group of current and retired OCLC employees, have spent months creating a quilt of 120 cross-cut blocks to donate to the silent auction held during the ALA 2026 Annual Conference. The ALA BiblioQuilters annually host this auction as a fundraiser for the Christopher Hoy Scholarship, which awards a $5,000 scholarship each year to a U.S./Canadian citizen or permanent resident who is pursuing an MLS in an ALA-accredited program.

This is the fourth year in a row that the OCLC Quilters have donated a quilt to the silent auction. Their work inspired me to take up sewing about a year ago, and I’m proud to move from admirer to participant, contributing to the OCLC quilt for the first time. Although left-handed people are about 10% of the population, three of the 13 people contributing to the OCLC quilt, including myself, are left-handed. While that doesn’t affect the result, it requires a few adjustments in technique and having the appropriate scissors. Sharing advice on adapting equipment and shopping for left-handed supplies is one of the ways we support each other.

Nine of the 13 contributors to the OCLC Quilt for the ALA 2026 Annual Conference

Like all handicrafts, quilting is an activity with its own nomenclature. As a quilter and cataloger, I found myself wondering: “What controlled vocabulary terms could I use to describe the OCLC quilt?” There are several from vocabularies such as Library of Congress Subject Headings (LCSH) and Getty Art and Architecture Thesaurus (AAT). These are listed at the end of this blog.

A quilt is created from many elements that may not be individually significant but form a meaningful whole, just like a WorldCat bibliographic record. The blocks of the quilt function like data elements in a WorldCat record, with contributions from multiple individuals creating the larger work.

Assembling the quilt

OCLC quilters sewed 120 blocks, which are the fabric squares comprising the quilt’s front. The blocks are a cross-cut design—a pattern chosen because it is accessible for novice sewists and makes good use of small fabric pieces. Quilters often save these leftover pieces, called “scraps,” from other projects for future use. Reusing scraps makes quilting a sustainable craft, and quilters often share them with one another. An experienced OCLC quilter, who keeps her scrap collection organized in true librarian fashion, donated most of the fabric pieces used for the blocks.

Experienced quilters arranged and sewed the blocks together and cut the batting (soft material used between the front and back sides of the quilt). The next step, in which three layers are sewn together with a decorative stitch, is quilting. This is the strict definition of the term “quilting,” although it is often used to refer to the entire process of creating a quilt. The pattern used for the quilt stitching is called “modern ties,” and it looks a bit like tied shoelace loops.

An OCLC logo is incorporated into one of the quilt blocks

The final step is to sew a long strip of binding fabric around the edges of the quilt, which will keep the ends from fraying as well as being decorative. Two labels were sewn into the binding: “Made in OH” and “Is it perfect? No.” Both of these labels are accurate descriptions of this quilt, but unlike in bibliographic descriptions, a certain amount of imperfection is not only tolerated but may be considered part of the quilt’s charm.

A quilting tradition at ALA Annual

The OCLC quilt will be one of many available at the ALA BiblioQuilters silent auction during ALA Annual in Chicago, Illinois. The BiblioQuilters were founded at the 1998 ALA Annual Conference in Washington, D.C. Since 2000, the BiblioQuilters have had a silent auction of quilts every year except 2020 and 2021 (because of the pandemic). The quilts are usually available to view and bid on near the registration area. If you are attending ALA in Chicago, I highly recommend you visit the auction table to view them. After ALA, you may be inspired to browse the shelves of your local public library for 746.46, the Dewey Decimal number for quilting.

Subject vocabulary terms

For those readers who appreciate quilting and metadata, the following controlled vocabulary terms reflect concepts discussed in this blog. You might even find it fun to match the concepts to the natural language descriptions!

Getty Art and Architecture Thesaurus terms

batting

binding (textile material)

blocks (quilt components)

fabric scissors

quilting

Library of Congress Subject Heading terms

Quilting

Sewing—Left-handed techniques

Textile fabrics

The post 120 blocks, one story: The collective creation of the OCLC Quilt appeared first on Hanging Together.