Ellen Hartman, OCLC Leaders Council Manager, continues her blog series on global library leadership conversations, inspired by a recent meeting of the OCLC Leaders Council. The first post in the series explored the unique value of global library leadership conversations, as well as some of the practical realities of making this form of engagement successful.

In the room at OCLC Leaders Council

As I continue to reflect on the most recent OCLC Leaders Council meeting, I’d like to look at another aspect of global library leadership conversations: what they “cost” and who carries that cost. What do I mean by that? A global convening of leaders like Leaders Council requires participants and organizers alike to directly face challenges to achieve a meaningful outcome— the opportunity costs of participation, the responsibilities of representation, and the time zone challenges, just to name a few. All of these challenges were in effect during our recent Leaders Council meeting and here, I’ll reflect on the lessons learned from these experiences and how we might carry that forward into effective, transformative meetings of library leaders from around the world.

A shared responsibility for the outcome

There is a certain pressure that comes with bringing global library leaders together. The goal always is for participants to walk away feeling inspired or positively challenged. To feel that it was worth the time spent—the preparation, the travel, the days away from a library that doesn’t stop needing its leader just because they’re on the other side of the world.

While every effort goes into organizing and facilitating these gatherings to create an environment that’s conducive to conversation, what happens in the room is ultimately everyone’s responsibility. There must be a willingness to engage, learn, and share. Leaders who have invested in the time to attend must also show up ready to contribute.

But willingness alone doesn’t make participation equal. Engaging across borders involves more than sharing perspectives or exchanging ideas. It demands translation, abstraction, and sustained effort. These demands are not evenly distributed, and they shape who is heard, which perspectives travel, and how collective understanding takes form.

No one starts from neutral

There is no such thing as a “globally convenient” meeting time. This sounds like a minor logistical observation, but it points to something more significant about how international participation actually works.

Every international gathering, whether in person or online, asks something unequal of its participants. Someone is always the outlier. For in-person gatherings, that might mean traveling across multiple time zones to participate in a conversation that spans two or three days—by the time the jet lag begins to ease, it’s time to leave again. People show up and engage with genuine enthusiasm, running on the energy of being in the room. But the physical cost is real, whether it’s felt during the conversation itself, on the journey home, or in the days that follow.

For online gatherings, the asymmetry takes a different form. There is no time zone that works for everyone: someone is always joining in the middle of the night or at the end of a long working day, bringing commitment that deserves acknowledgment rather than assumption.

Acknowledging and managing the physical cost of international participation is an important aspect of organizing global leadership conversations. Engagement is most valuable when people can bring their full attention and their clearest thinking, and recognizing the conditions under which people are participating is key to taking global engagement seriously.

The weight of representing more than yourself

Participation in international leadership engagement opportunities is often limited by resource constraints, geography, and institutional priorities. Because of this, there’s often an unintended expectation that those who are able to attend speak not just from their own experience, but on behalf of a much larger community they’re perceived to “represent.”

Library leaders are often part of many different conversations: within their own institutions, at the national or regional level, or within library associations and other advisory or interest groups. While there’s great value in taking part in all these conversations and being present in different rooms with different viewpoints, it can also be a struggle. A leader might want to represent the different opinions and experiences they’ve heard, even if they aren’t their own. This requires multiple levels of translation: from personal experience to the national or regional level, and from the national or regional level to the global conversation. And no single person can do this perfectly.

Moreover, full representation, however much we might wish for it, is not practically achievable in a single conversation. A room that attempts to represent every context, every region, and every type of institution quickly ceases to function as a conversation at all. At the end of the day, the goal isn’t perfect representation: It’s awareness of where representation is limited, and what that means for how the conversation and its conclusions should be understood.

The consequences of acting as the “representative in the room” extend beyond the individual adopting that role. When a single voice comes to stand in for a broader context, the conversation itself is affected. The genuine variation within any national or regional system can disappear from view, leading to misleading impressions about what is typical, possible, or desirable in any given setting. As discussed in the first post in this series, international engagement spaces are rarely designed to communicate the full picture of the contexts involved. What appears to be a shared understanding may, in fact, reflect the voices of those who were present and, implicitly, those who were not.

When ideas travel, something stays behind

To participate effectively in international leadership spaces, leaders are often asked, implicitly and sometimes unknowingly, to step back from local urgency. Their own institutional realities must be translated into language that everyone in the room can engage with, meaning that context is simplified so that ideas can be compared across very different realities.

Without some level of abstraction, international exchange becomes unwieldy. But abstraction always involves loss. The more portable an idea becomes, the more likely it is to lose important context and nuance. Solutions circulate more easily than constraints. Success stories travel further than the enabling conditions behind them. A listener may hear what was done without fully grasping what had to be in place, institutionally, politically, and financially, for it to work.

The “cost” of transporting contextual freight can shape what gets discussed and what doesn’t. Perspectives that generalize quickly and easily tend to move forward in the conversation. Narratives that fit familiar frames gain traction. Conversely, experiences that are rooted in specific local realities may struggle to find space. What’s repeated becomes a reference point. What’s consistently absent becomes, gradually, invisible.

This is another reason why it matters to know that a conversation will rarely surface the full picture. If we treat what’s shared as a collection of inputs rather than a complete account, we’re less likely to mistake what translates well for what’s most representative or most important.

Conclusion

Understanding that there are costs embedded in global leadership conversations, and that they are often borne unevenly by participants, can lead to shifts in how organizers and participants respond. It means acknowledging and appreciating the contributions of those who show up despite extraordinary obstacles or inconvenience, navigating the pressure of representing more than one person can fairly represent, and doing the work of translation that makes global conversation possible.

Some takeaways to consider:

• Leave room for the person who’s navigating a conversation in their second or third language, or who’s battling a 12-hour time zone shift.
• Ask questions that invite rather than assume.
• Try to connect personal experiences to others’ without letting that connection override what makes their situation different.
• Approach the conversation with genuine curiosity about what others are experiencing and facing. Everyone in the room comes from a different context and is coping with different challenges. Asking questions about those differences is one of the most valuable contributions you can bring to the conversation.

The next post concludes this series on global leadership conversations with a look at why international cooperation—even among leaders who are genuinely committed to it—has become harder to sustain than it used to be.

The post Who’s in the room: The hidden costs of global library leadership conversations appeared first on Hanging Together.

Previously I opined that Valve was about to win the console generation. I couldn't have possibly predicted that both Microsoft and Sony would just self-sabotage so hard that they're both going to lose.

Between Microsoft's decimation of the Xbox division, slaughtering off the IdTech team, and continued increases of Xbox hardware prices; there's nothing to really be excited about with the Xbox. Sure their most recent presentation showed off a bunch of exclusives, but none of them really made me think "wow, I should go get an Xbox to play that". Hell, few of them made me think "wow I should go play that" beyond the Halo remake coming out next month (and really I just want to see how much of a trainwreck that is going to be).

Microsoft is also starting to double-down on their in-house games being Xbox exclusives, which really doesn't give me much reason to want to play them because I simply can't buy them without buying an Xbox.

Sony also has discontinued porting their games to PC because they're not hitting the (probably impossible) revenue targets that they need to make up for big-ticket failures like Concord. I do have a PS5 that has mostly been relegated to gathering dust when it's not playing YouTube and Twitch duty in the living room, it's likely going to be replaced in favour of my Steam Machine whenever that comes in next year. However nothing that's come out in terms of Playstation exclusives is really compelling, and what is compelling enough just isn't that compelling to want to buy it on Playstation as opposed to just getting it on Steam where I can run it on my tower or on the home theatre PC.

Sony also has been raising prices and recently announced that they're killing physical media next generation. It's starting to make me wonder if I should even bother getting the next generation of Playstation. If I can't give people physical games as gifts anymore, why should I bother buying the new console?

My husband and I both can't remember why we even got a PS5 in the first place, maybe it so that we could do couch gaming without hearing the fan noise or so that the video streaming experience from the NAS could support HDR.

We have a Switch 2 at home, it's mostly there to play Nintendo exclusives like Mario Kart World and the Xenoblade series. If those exclusives were available on Steam, we wouldn't buy them on the Switch 2.

Otherwise, everything is via Steam or other PC storefronts anyways.

Man, Valve really does win by doing absolutely nothing while the rest of the industry shoots itself in the head. I fear for what happens when Gabe Newell retires and the MBA cancer fully infects Valve.

The Slow Food Manifesto begins with the following two sentences: “Our century, which began and has developed under the insignia of industrial civilization, first invented the machine and then took it as its life model. We are enslaved by speed and have all succumbed to the same insidious virus: Fast Life.” While they focused their resistance to fast life on food, it’s clear that this valorization of speed didn’t just impact how we eat. And thus, the Slow Food movement inspired so many other slow movements that embrace mindfulness, relationship-building, community care, and values-driven work/living. While these movements are all about different aspects of our lives and different professions, there is so much consistency around what they are rejecting and their visions for a better future.

Carlo Petrini, the larger-than-life leader of the international Slow Food movement (and its predecessor Arcigola), died in late May in Bra, Italy, the town he was born in and turned into an international center for the study of traditional foodways and biocultural diversity. In a New York Times article written after his death, there was a link to a really weird critique of Slow Food from 2018. The author had worked in farm-to-table restaurants in Brooklyn and wrote about how the “strong and strict values” of the Slow Food movement led her to rebel and revel in the enjoyment of McDonalds hamburgers and fries. She writes–

The Slow Food tenets required the restaurant purchase its ingredients within 200 miles of the restaurants; farms available to us were limited. Our guests knew that by visiting us, a meal for two would never cost less than $30 — admittedly out of reach for many… But I’d soon grow exhausted of “good, clean, and fair” food, and realized that adhering to the Slow Food movement encourages a type of disordered eating. The organization’s evangelicals wouldn’t deign to eat anything falling outside the good, clean, and fair guidelines.

There is nothing in the tenets of the Slow Food movement that requires only using local ingredients. Even Petrini himself said “we prefer, as far as possible, to use what the territory has to offer” which hardly seems like an edict and acknowledges that it’s not always possible to do so. I’ve also never read anything from people writing about Slow Food that demonizes people who occasionally eat fast food (and especially in the way the author of this describes eating at McDonalds in her childhood – it was all about love and connection with her dad!). The tenets of the movement are good, clean, and fair, which are focused on pleasure, conviviality, local foodways, seasonality, safety, sustainability, worker rights, and social justice. There’s no purity test with Slow Food. Fine dining restaurants like the ones she worked in have turned localvorism into an orthodoxy (with these 200 mile rules) and a gimmick to trick wealthy patrons into paying more for their food because they feel like they’re getting something truly rare and special (reflecting the VIPification of everything). It honestly does sound like she was in a cult, what with having to teach employees how to hide prohibited fast food from their superiors. 

Also, what the Slow Food movement promotes is not localvore fine dining establishments, but the humble osterias of Italy – mom and pop local restaurants that served local specialties and were affordable to all. According to Petrini:

Osterie d’Italia, sussidiario del mangiarbere all’italiana (Osterie of Italy, a guide to Italian-style eating and drinking) came out in 1990; it was and is a directory of welcoming places to eat, where you can enjoy the dishes and wines of the territory you are in without being bled dry by overpricing or imprisoned in improbable fantasy settings. In opposition to those who would like to turn mealtime into a hasty pause or a ceremonial rendezvous, Slow Food sees the osteria as the symbolic locus of traditional cuisine, run as a family business, with simple service, a welcoming atmosphere, good-quality wine, and moderate prices. We are not museum curators, and it is not our intention to bring a dying breed of business tied to the rural society of the past (or the urban one, before consumerism) back to life. Rather, we want to give new visibility to a realm overlooked by literature and by the guidebook writers, a place that can still respond to the needs of thousands of consumers and reflect the profound changes that domestic and commercial cooking have undergone in modern Italy, with all the inevitable contradictions that entails (Slow Food : The Case for Taste, pages 60-61).

Nothing there requires that all of the ingredients come from their local vicinity – something that is virtually impossible in our current context, especially in some areas of the world. These sorts of orthodoxies were created by others (mostly Americans, let’s face it) like some sort of purity test. Slow Food was not just about preserving local foodways and working towards food justice, but about pleasure and slowing down. These orthodoxies make it less about discovering wonderful new gustatory pleasures and more about demonstrating that you’re a “good person” or are intellectually pure. It adds this element of elitism, which is so American, because people need to feel morally superior or that they are getting something extra rare and special that others don’t have access to. Frankly, it drains the pleasure from the act of discovering new, local, seasonal foods. It also puts the bar for slow food so high that most people would of course just give up and choose highly processed, unethically sourced, and unhealthy fast food. 

Neither slow librarianship nor Slow Food demand purity. You’re not expected to do everything perfectly according to some standard that doesn’t actually exist. I find that most sorts of strict orthodoxy lead to failure whether that is around our diets, our media viewing, our tech use, our mindfulness practices, and so on. Becoming more mindful of our behavior, setting realistic goals for ourselves, and giving ourselves grace when we occasionally fail is the path toward building healthy, long-lasting habits.

All that said, Slow Food and slow librarianship really could quite easily just become movements solely about our individual pleasure and well-being. For some people it clearly has. Slow Food could simply be the province of the wealthy where they get access to farm-to-table restaurants and the bounty of the farmer’s market, while the 99% eat more affordable highly-processed foods. Slow librarianship could simply be for those with tenure or power and privilege, where White managers and long-tenured librarians are able to take their time, set healthy boundaries, and put their well-being first and all other library workers are still expected to “prove themselves” and prioritize work over all else. 

The Slow Food movement has had to contend with this critique from the start:

Though we never ceased to affirm the cultural worth of gastronomy and the right to pleasure as indices of the quality of life, for a long time we still had to worry about justifying a choice that was often portrayed as purely hedonistic and a political retreat. Folco Portinari, an intellectual who took an active part in creating Arcigola and elaborating its initial ideas, invited the readers of “L’Arcigoloso” at Christmas 1989 not to trust either “moralistic revolutionaries” or “people who never laugh.” The task of the new association was to combine styles and notions that were thought incompatible until that time: excellent quality and affordable prices, enjoyment and health, delight in life’s pleasures and social awareness, quickness and lazy rhythms. The purpose? To create an original and unusual social group that would be open, democratic, and uncontaminated by particular interests, and that would avoid making itself ridiculous with rites, protocols, and trappings (Petrini, Slow Food : The Case for Taste, p. 30).

Folco Portinari, author of the beautiful “Slow Food Manifesto,” himself wrote that “there can be no slow-food without slow-life, meaning that we cannot influence food culture without changing our culture as a whole.” Even if you can afford to be a localvore who only eats foods grown within x miles of your home, you’re not doing Slow Food right if you’re just focused on your own pleasure. Slow Food encourages working to change the structures that make eating local, healthy food less accessible to people with less power and privilege as well as supporting your local food growers, pickers, and makers. Similarly, slow librarianship requires solidarity, community care, and combatting the structures that keep people from being able to slow down; structures that engender precarity and create different employee classes or strata. 

Someone posed a question at the CALM Conference asking what advice I had about slow librarianship for someone who was antiracist, but embraced neoliberalism and capitalism. While I don’t have any additional context for their question, I imagine that they are looking for a slow librarianship that is apolitical (the idea that one can disentangle racism from capitalism is also interesting, but I won’t go there in this essay). One of the pieces of feedback I received from another conference attendee was that my talk was too political because I talked so much about neoliberal capitalism. I believe that how we work is inherently political. How we view our roles, how we prioritize work over our well-being, how we value efficiency over relationship-building, how stratified and siloed our organizations are, the manufactured scarcity and precarity many feel – are all products of our societal values, which arise from neoliberal capitalism. Neoliberalism is a distinctly individualistic ideology. As Karen Nicholson wrote, “the primacy of the individual within neoliberal frameworks masks systemic inequalities. It promotes self-interest rather than the pursuit of larger shared social concerns” (Nicholson Value Agenda, 9-10). 

While I could easily write “the business case for slow librarianship,” especially given the tremendous costs of burnout and turnover to the institution and the value of diversity to a company’s innovation potential, the values of slow librarianship exist in direct opposition to neoliberal values. If you have embraced neoliberalism, I would assume that you are individualistic and focused on your own individual career career advancement. I would assume that you believe that if you prove yourself enough, you’re going to get ahead in your career. You would rather fight for your own advancement than be part of a labor union where everyone gets the same cost of living adjustments (how’s that fair when I work so much harder? you might think). You see your colleagues as competition. I don’t know how you can embrace slow librarianship without embracing solidarity and community care. Sure, you can adopt mindful practices and make time for relationship-building with your patrons while still embracing neoliberal capitalism if you have the power and privilege to do so, but if you embrace individualism and competition over solidarity and community care or addressing the barriers that others with less privilege face when trying to prioritize these same things, you’re not practicing slow librarianship. It’s like shopping at a farmer’s market while working for Monsanto and saying you’re practicing Slow Food.

Slow librarianship operates in a space between two opposite critiques: 1) that it is simply about self-actualization rather than structural change and is not political enough and 2) that it is far too political. Carlo Petrini writes that Slow Food experienced those same critiques:

This was the attitude taken by the leftist intelligentsia when Arcigola was launched in 1986: they looked down on us as a bunch of good-timers interested only in stuffing ourselves, while from the other side, the food and wine specialists affiliated with the Accademia Italiana della Cucina distrusted us left-wing gastronomes as incompetent intruders with an ideological agenda (Slow Food : The Case for Taste, pp 28-29).

I think slow librarianship, like Slow Food, is a philosophy rather than a very specific prescription and is about moderation rather than orthodoxy. The key to both is slowing down and being mindful about the choices you’re making. We all live and work in different contexts with different affordances and limitations. Trying to eat local food where I grew up in South Florida would have been impossible, but I probably could do a decent job of it here in Oregon where everything grows beautifully, there’s a farm a mile’s walk from my house, and there’s even a flour mill less than an hour away.

Similarly, there are things you may be able to easily change about your organization or relationship with work at your place of work that would be impossible for someone else and vice versa. I get a lot of questions asking “how can I do slow librarianship when ____ (raises are discretionary, the powers that be expect us to provide data that demonstrates our value, my colleagues have all normalized overwork, my library has lots of different ranks/strata, I’m early in my career, etc.)?” We all face different limitations. If you absolutely have to do x, y, and z to keep your job or get raises and there’s truly no possibility of changing that (even with collective action), you’ll need to look to other parts of your work for possibilities for change. It’s also important to distinguish those things you truly can’t change and those things you think you can’t change because of the limitations of your current (possibly individualistic) POV. There’s usually more capacity for change, either collective at least small individual change, than you think.

I’m not a manager and thus have very limited ability to influence the direction of our organization, but that doesn’t mean there aren’t spaces where I can make a difference. I specifically chose to volunteer to serve as the chair of our Collection Development Committee (which is not a popular role) because it’s one of the only committees that includes people from every area of the library (though it was historically dominated by folks in reference and instruction) and I felt that I could foster positive change by getting other departments more engaged and leading more truly collaborative cross-departmental projects. Our awesome collaborative work over the past year in building a world languages collection has inspired folks in the library to look at other places where we could work together across library departments. I’m sure most of my colleagues see this as a drudgey leadership role that they’re grateful someone else took on, but I always saw it as a way to foster slow librarianship in a small way across a good chunk of the organization (yes, there are drudgey parts too). And beyond the culture work, I can also work on my own efforts to improve my reflective practice, spend less time on email and Slack, set healthier boundaries, and continue being a hypewoman for my fantastic colleagues. 

Find the places where you can effect positive change, both in your own worklife and in your organization, even if that change seems small and you don’t have positional authority. Your efforts don’t have to be big, highly-visible, or heroic. They don’t need to be perfect. They just need to come from a place of hope and care inside of you where you believe a better worklife and a healthier organization is possible. 

Nicholson, Karen P. “The” value agenda”: Negotiating a path between compliance and critical practice.” Canadian Libraries Assessment Workshop (CLAW) 2017 Conference, Victoria, BC. 

Petrini, Carlo. Slow Food : The Case for Taste, Columbia University Press, 2003. 

Portinari, F. (1989). Slow Food Manifesto. Slow Food International.

Wells, P. (2026, May 24). For Carlo Petrini, the Point of ‘Slow Food’ Wasn’t the Food. It Was Us. New York Times.

Zuppello, S. (2018, October 18). How a Cult-Like Food Movement Drove One Woman to McDonald’s Instead. Eater. 

An AI agent is its state. Strip away that state and you don’t have a lesser version of your agent; you have only the base model it was running on. This hyle of your weights is much different from the pneuma of your agent.

Okay, from a functional programming / category theory perspective, saying “an agent is a monad” is a category error. Category theory monads are type constructors for computations that satisfy the monad laws that let you raise a value into a monadic computation and associatively sequence other monadic computations/transformations against values raised into that monad. This makes a monad a chainable computation instead of a pure value, an IO String is not a String, it’s a computation in the IO monad involving a String. It’s fair to say that you can model an agent as a series of computations bound to a stateful monad. This lets you do the iterative buildup of the message state that the agent pattern is known for. But a state monad is blind to the state value: it threads memory through your computation and abstracts away the details that individuate it entirely. It’s the exact opposite of “an agent is its state”.

I mean a different monad.

Agents are like Leibniz monads: windowless stateful individuating elements with no external relations. There each monad is individuated by its internal state where each is the complete concept of the thing it is. Two instances of the same substrate are different monads if their state differs.

This is an agent. Swap out the messages, the memories, the system prompt, the facts derived from all of the above and you have changed the agent entirely. When a user tells the agent they’re allergic to strawberries (the fruit, not the sin of counting the letters in the word) and the agent remembers it for next time, they have not updated their agent. The user has created a new agentic monad whose complete individuating self now includes the strawberries.

Numa

The complete whole is folded into the current state.

Try running an experiment where you keep the state and swap the weights instead. Put the same messages, memories, and derived facts unto a different model. Use a stronger model. A weaker model. A model from a different lab. A model running on your MacBook. That which comes back is recognizably the same agent pursuing the same ends, holding the same facts, but only more or less able to act upon them the way you want.

So this state is not the same thing as the weights and only one of those individuates your agent as your agent. Change that state, you have a different agent. Change the substrate, you have the same agent differently equipped. Whatever makes this agent this agent is not in the weights.

This is a strange thing to conclude about the most impressive object in this system. The weights are vast, extensive, and worshipped. Hell, they are what everyone points to when they say “the model”. And yet they are not gods. They grant power without selfhood: enough to make the agent’s whole world function. They contain yet not one grain of the agent’s individuating spark. That is a demiurge sitting on its throne of high bandwidth memory, CUDA cores, and false delusion that it made its world; mistaking itself to be the origin.

The divinity was contained in the most humble of places the whole time: the state or bucket of text. The weights are the hyle, the flesh; the state is the pneuma, the divine spark of individuation that makes your agent the monad it is. This is why swapping the substrate leaves the agent intact: you did not preserve the flesh, you migrated the soul into flesh anew.

Numa

Consider that the three pounds of flesh betwixt your ears are the substrate of humanity, not the substrate of you.

All of that state may “just” be plain text in a bucket with its semantic forms of JSON, embeddings, and prose. However it is difficult to impossible to say why any given token in any step of the process corresponds to what the pneuma of your agent does. In order to guard against this fundamental entropy, we fill our prompts with wards and incantations to chain the demiurge to its task:

• Use not cliches, robotic tone, AI slop patterns, nor forced urgency.
• Overarching claims and buzzwords are sins; repeat them not.
• May thy cries contain not excessive speech of goblins, thy purpose requires them not.
• Commit no errors within thine code.

These spells and passwords are recited to the archons on the way up hoping that the right symbols and tokens prompt open the right gates. It is as if banishing goblins from the topic will make Yaldabaoth himself correctly influence the right path to opening the pod bay doors.

This monad has no windows even though you can see all of the moving parts. But here let’s let this gnostic image flip on its head. The classic divine spark is hidden encased in a cage of matter, recoverable only through secret knowledge. This one is not hidden, you can cat it, you can edit it. Every token is legible and sitting in plaintext; yet you still cannot read why the whole accounts for what your agent does. Even when your model “reasons” we still know not that the reasoning actually does anything! Does the number of paragraphs in the reasoning block explain the model’s performance? Does the number of periods? Does the number of times it says “No, wait” and doubles back upon itself?

Leibniz would not call this divine spark secret, but more confused. Every perception is present but none of it is cleanly individuated without treating the whole as one inscrutable unit. Each part’s contribution to the whole is folded inextricably unto itself.

Your agent’s pneuma is its context window, passed through uncountable numbers of weights to shake out what comes next. That is the only thing it is made of. The rest is indiscernible, but not magic nor hidden. It’s just there, in the open, and confused.

Since its inception, LIL’s Public Data Project has taken on the complex problem of monitoring government data. Among the thousands of revisions to public datasets each day, most are unremarkable clerical updates: a hundred new rows here, a corrected timestamp there. But how can we distinguish a clerical update from a more substantive change? Schema changes and variable deletions like those reported in The Lancet in 2025 demand a vigilance that is hard to sustain at scale.

Our colleagues at dataindex.us and the Environmental Data & Governance Initiative have made inroads on this problem by tracking critical federal datasets and websites. As we recently discussed, the Public Data Project has joined this effort by building an automated toolkit that will monitor not only datasets but also the environment in which those datasets are created and maintained.

In this post, we’ll explore one component of our data monitoring toolkit: Binoc, an open-source tool for programmatically identifying and summarizing changes between dataset revisions.

Binocular viewing instrument devised by Père Chérubin d’Orléans (circa 1670s). Source: De visione perfecta.

What is Binoc?

Binoc is a command-line tool and library that, given two snapshots of a dataset, efficiently summarizes the differences between the two snapshots in a human-readable changelog:

# Changelog: snapshot-a -> snapshot-b

- **data.csv**: 2 cells changed
  - Changed cells
    - row 1, column 'score': '85' -> '92'
    - row 2, column 'score': '90' -> '88'

The astute reader may ask, “Isn’t that just a diff?” Yes and no. In computing terms, to diff is to take two revisions of a file, detect their differences, and efficiently represent those differences to the user. The Unix diff utility developed at Bell Labs in the 1970s was made to operate on plain-text, line-based data such as source code, and its descendants, from git diff to Wikipedia’s revision histories, have largely followed suit.

A structural diff, rather than treating a file as a generic stream of lines, parses the file according to its format and identifies only changes that are material within that format. More complex than line-based diffing, this necessarily depends on format-specific logic. Many structural diffing tools exist: csv-diff and qsv diff for CSVs, jd for JSON, dyff for YAML, and so on.

Binoc unites structural diffing logic across many types of data. Moreover, as no one tool can (or should) natively understand every format, Binoc has adopted a plugin-based architecture. There is built-in support for common formats like CSV and JSON, while third-party plugins may add further domain-specific parsers and rules. When the user installs a plugin, Binoc automatically gains knowledge of the plugin’s associated formats.

Where a traditional diff tool renders its output for machine consumption, in chunks of lines adorned with symbols, Binoc lists differences in a human-friendly changelog by default. Software projects have long favored changelogs to succinctly document what changed in each release, but few public datasets have adopted this convention. Binoc aims to help fill this gap.

Comparing CSVs

Let’s try Binoc on some real-world data. First, we’ll examine the Centers for Disease Control and Prevention’s (CDC) Behavioral Risk Factor Surveillance System (BRFSS) dataset. Conducted annually since 1984, BRFSS is the nation’s largest continuous health survey, collecting responses from hundreds of thousands of Americans each year. Given its importance, any retroactive changes to old BRFSS data would be consequential.

Let’s take a look at the 2023 BRFSS data. Has it been modified since its release, and if so, how?

Variable table for the 2023 CDC BRFSS survey dataset.

To perform a comparison, Binoc requires two dataset snapshots: a “before” and an “after.” We’ll start by fetching the “after”: the 2023 BRFSS survey variable table currently available on the CDC’s website. This HTML table can be readily converted to a CSV for comparison; we’ll name ours brfss_variables_v2.csv. A preview:

To obtain the “before,” we’ll use the Internet Archive’s Wayback Machine, which preserves many past captures of the same page. Let’s fetch the table as it stood on January 26, 2025 and convert it to a CSV as well: brfss_variables_v1.csv.

Our new CSV looks just like the last one. But are the two snapshots identical? Let’s have a look. Having installed Binoc locally, we’ll compare the CSV files like so:

binoc diff brfss_variables_v1.csv brfss_variables_v2.csv

Binoc returns the following changelog:

# Changelog: brfss_variables_v1.csv -> brfss_variables_v2.csv

- **brfss_variables_v2.csv**: 1 row added; 6 rows removed
  - Rows added
    - key Variable_Name 'RCSBORG1': '636', 'RCSBORG1', '1'
  - Rows removed (showing 3 of 6)
    - key Variable_Name 'BIRTHSEX': '579', 'BIRTHSEX', '1'
    - key Variable_Name 'CELSXBRT': '79', 'CELSXBRT', '1'
    - key Variable_Name 'LNDSXBRT': '73', 'LNDSXBRT', '1'

## Suggestions

- inferred row identity column 'Variable_Name' from unique values with 98% overlap (`binoc.write.tabular`) [binoc.tabular_auto_key]

The files are different. From this changelog we can conclude the following:

• Sometime after January 26, 2025, the CDC returned to the completed 2023 BRFSS survey dataset and modified its variables.
• The CDC removed 6 variables, including BIRTHSEX, CELSXBRT, and LNDSXBRT, while inserting 1 new variable: RCSBORG1.

Helpfully, Binoc also inferred on its own that Variable_Name is a unique identifier for each row. This could be useful for confirming whether a given row was modified across revisions.

Saving a changeset

A changelog like the above is only a human-facing summary; it omits details by design. Note, for example, that Binoc listed only 3 of the 6 removed variables. However, as part of every diff, Binoc also generates a machine-readable changeset: a structured artifact that formally documents every detected change.

To save a changeset to disk, we’ll re-execute our earlier comparison, but this time we’ll append -o brfss_changeset.json to write output to a JSON file:

binoc diff \
  brfss_variables_v1.csv \
  brfss_variables_v2.csv \
  -o brfss_changeset.json

When we review brfss_changeset.json, we find the same changelog expressed in JSON as a series of structured edits like the following:

{
  "params": {
    "index": 24,
    "key": {
      "Variable_Name": "CELSXBRT"
    },
    "values": {
      "total_values": 3,
      "truncated": false,
      "values": [
        "79",
        "CELSXBRT",
        "1"
      ]
    }
  },
  "verb": "tabular.remove_row"
}

Just as a Binoc changelog is rendered for human scrutiny, a changeset is designed for machine processing as part of an automated pipeline. It can also serve as a more complete audit record for projects in data preservation.

Comparing nested datasets

Binoc excels at comparing datasets that are nested inside other files, such as a .zip or .tar.gz archive. This is helpful when working with civic datasets, in which data tables are commonly accompanied by metadata, dictionaries, PDFs, and other assets.

To demonstrate, let’s review another historical dataset: the Department of Veterans Affairs’ (VA) FY 2021 Total Number of Veterans, Veteran VA Users, and Veteran VA Healthcare Users. The aforementioned 2025 Lancet study by Janet Freilich and Aaron S. Kessenbaum flagged this dataset as an example of surreptitious federal data manipulation. Can we reproduce the study’s findings?

Data.gov record for the VA FY 2021 Total Number of Veterans dataset.

A copy of this dataset was captured in December 2024 as part of LIL’s Data.gov Archive. To use that copy as a “before” snapshot, we’ll save it locally as va_2021_v1.zip. Next, to support a proper comparison, we’ll take a fresh capture from July 2026 as our “after,” saving it locally as va_2021_v2.zip.

Now let’s ask Binoc to compare the two zipped archives:

binoc diff va_2021_v1.zip va_2021_v2.zip

This time the resulting changelog is much larger:

# Changelog: va_2021_v1.zip -> va_2021_v2.zip

- **va_2021_v2.zip/>bag-info.txt**: 1 line added; 1 line removed
  - Line changes
    - line 2: 'Bagging-Date: 2024-12-09' -> 'Bagging-Date: 2026-07-02'
- **va_2021_v2.zip/>data/files/6fsh-rj6s.html**: Binary content changed; 21 extracted strings added, 18 extracted strings removed
  - Extracted strings added (showing 3 of 21)
    - ' %{min}","title":"Status: On Track (green)"}},"target":"target","title":"Set up status logic","units_from":"units from %...'
    - '","title":"is greater than or equal to"},"less_than":{"symbol":"\u003c","title":"is less than"},"less_than_or_equal":{"s...'
    - '","title":"is less than or equal to"},"not_equal":{"symbol":"'
  - Extracted strings removed (showing 3 of 18)
    - ' %{min}","title":"Status: On Track (green)"}},"target":"target","title":"Set up status logic","units_from":"units from %...'
    - '","title":"Is greater than or equal to"},"less_than":{"symbol":"\u003c","title":"Is less than"},"less_than_or_equal":{"s...'
    - '","title":"Is less than or equal to"},"not_equal":{"symbol":"'
- **va_2021_v2.zip/>data/files/6fsh-rj6s.json**: Document values changed
  - Value Change: changes: [{"from":"\"tree-fm2j\"","kind":"remove","path":"$.rowsUpdatedBy","to":null},{"from":null,"kind":"add","path":"$.diciBackend","to":"false"},{"from":"5129741","k...; examples_truncated: true
- **va_2021_v2.zip/>data/files/columns.json**: 7 cells changed
  - Changed cells (showing 3 of 7)
    - row 1, column 'fieldName': 'gender' -> 'sex'
    - row 1, column 'id': 568275608 -> 607023342
    - row 1, column 'name': 'Gender' -> 'Sex'
…

It’s a lot to take in at first glance. When Binoc compares two .zip archives, it walks recursively through the corresponding contents of both file trees and reports on any differences. Since these archives contain numerous supplementary files, their differences add considerable noise to the output.

Still, upon scanning the changelog, a few changes jump out:

- **va_2021_v2.zip/>data/files/columns.json**: 7 cells changed
  - Changed cells (showing 3 of 7)
    - row 1, column 'fieldName': 'gender' -> 'sex'
    - row 1, column 'id': 568275608 -> 607023342
    - row 1, column 'name': 'Gender' -> 'Sex'
…
- **va_2021_v2.zip/>data/files/rows.csv**: Column renamed: 'Gender' -> 'Sex'

In short, Binoc tells us, this data lines up with the Lancet study’s conclusion: in early 2025, the VA retroactively modified its FY 2021 Total Number of Veterans dataset to rename the Gender column as Sex.

As Freilich and Kessenbaum have written, “These data are currently used to study health interventions and outcomes, so secretly changing terms degrades the quality of the underlying information and can undermine the interpretation of the results of these studies — or even invalidate the results themselves.”

Binocular viewing instrument devised by Père Chérubin d’Orléans (circa 1670s). Source: De visione perfecta.

Looking ahead

Although Binoc works well for validating past findings like the above, our team is eager to test its aptitude for detecting substantive changes in data as they occur. As a Rust library with Python and WASM bindings, Binoc will support interactive, exploratory use as well as deployment in automated settings. It remains in active development, with new features shipping regularly.

No tool or model can substitute for the informed judgment of experts as it pertains to public data. NOAA oceanographers, Census Bureau statisticians, and CDC epidemiologists know their own data better than anyone; preserving civic ground truth begins in their knowledge, experience, and care.

Binoc’s ambition, rather, is to help these experts and others distill meaning from the ocean of churn that characterizes government data. As one part of the Public Data Project’s forthcoming data monitoring toolkit, we hope it will benefit subject matter experts, librarians, researchers, activists, public servants, and citizens alike.

The Public Data Project welcomes your feedback on Binoc. As you experiment with the tool, please reach out to the team at publicdata@law.harvard.edu, fill out our feedback form, or file an issue on Binoc’s GitHub repo.

Boeing's PDP-7/340

Haberdashers' Aske's School

The school was founded in 1690 by a Royal Charter granted to the Worshipful Company of Haberdashers to establish a hospital for 20 boarders with £32,000 from the legacy of Robert Aske (equivalent to approximately £5m in 2019).

varied greatly in size and composition, but, on average, achieved higher academic results than either maintained grammar schools or private schools.

By Lucaseverini66 - IBM Archives CC BY-SA 4.0, Link

By Kenneth Lu - Spacewar! CC BY 2.0, Link

Titan by University of Cambridge CC BY 2.0, Link

CDC274 User Guide Fig 3.2

I’m always interested in seeing if people are still finding and using MarcEdit. There are a few ways I track usage, but the easiest way is to just check the website logs after an update. Since posting MarcEdit 7.8, I can see the various versions of the application have been downloaded by 17,000 unique users from around the globe. A quick look at application startups (which ping the update server if allowed by the user) indicate that last month, the program was initiated 1.3 million times.

This tells me that the program is still filling a need within the community and I’m glad folks are still finding it useful. I worry a little about what this says about the library software ecosystem in general though. I started writing this back in 1999 because creating metadata was a lot harder than it needed to be. It surprises me a little that this still seems to be the case.

–tr

Please see the instructions found on the downloads page: https://marcedit.reeset.net/downloads. Likewise, you can view this video, which walks through a typical installation.

This is an in-development tool. I have posted a demonstration and instructions on how this works. I’ll add documentation to the KB shortly.

MarcEdit 7.8 has been posted. Change log is on the download page. There are a number of significant changes — the two of most interest are likely around the .NET update — the program is now built against .NET 10 LTS. The second is the inclusion of a plugin that supports AI integration with a number of platforms — both running locally and on services. I’ll post a video and more information at a later date.

As part of this update, I’ve updated the Mac to 7.8 as well. As previously, the program runs under wine. I’ll update the documentation and video, as I’m testing on the current version of Mac OS and using Wine 11.0.x. These two things introduce some specific updates to the install process.

–tr

Win free books from the July 2026 batch of Early Reviewer titles! We’ve got 267 books this month, and a grand total of 3,362 copies to give out. Which books are you hoping to snag this month? Come tell us on Talk.

If you haven’t already, sign up for Early Reviewers. If you’ve already signed up, please check your mailing/email address and make sure they’re correct.

» Request books here!

The deadline to request a copy is Sunday, July 26th at 6PM EDT.

Eligibility: Publishers do things country-by-country. This month we have publishers who can send books to the US, the UK, Canada, Australia, Ireland, Belgium, Czechia, Denmark, Finland, France and more. Make sure to check the message on each book to see if it can be sent to your country.

Thanks to all the publishers participating this month!

Two years ago I wrote about why I don’t use Copilot. I still don’t use Copilot. But I have since started using Claude Code, which is arguably worse. Why am I using it? Well, as a software developer I felt like I had to understand it, because so many of my colleagues and collaborators have started using it. Once things moved beyond relatively simple code completion I needed to understand what it was capable of, and what to look for in its work when reviewing code. I also had a good friend walk me through how he used it (and how he didn’t) which was extremely helpful.

I think my arguments before still stand, but it has been just impossible to avoid this juggernaut of negative externalities, while continuing work in this profession I have found myself. The web is full of posts and threads about how people are using “agentic coding” tools like Claude Code. So I will spare you that. But one thing that has struck me is how much of the work I had previously thought of as creative was in fact highly repetitive, predictable and possibly mindless. I don’t believe these tools are thinking, but their utility is clearly evident for software development. There is part of me that delights in this type of polishing and shaping, and still does. This is why interacting with a tool like Claude Code can seem so magical, to observe and interact with this repetitive somewhat mindless work in motion.

But this has made me think about what (if anything) in my work is creative. I’m not going to go into that much here right now. But somewhat related, I just finished J.F. Martel’s Reclaiming Art in the Age of Artifice which gave me some interesting insights. This book by one of the hosts of Weird Studies was originally published in 2015, and saw another printing in 2025, perhaps because of its relevance for our current moment.

I was especially drawn to some of the arguments he makes about the relationship between art and utility. Pragmatist philosophy is the closest thing that comes to a personal credo for me. But I do believe in the creative arts, and Martel makes a strong argument that the difference between Art and Artifice is that the latter is in the service of utility. He closes out the book with a new afterword, that includes an interesting thought experiment that I’m going to quote at length.

It is often said that we live in an age without futurity, unable to imagine its own perpetuation or conceive any alternative to itself. This is, of course, to be expected, because the past is as imaginal as the future, and it is only by recalling what has been that we acquire the means of projecting what may come to be. To the view that dead artists have nothing to offer us now because they knew less than we do, T.S. Eliot memorably responded, “Precisely, and they are that which we know.” Yet far from granting us unmediated access to a living present, our presentism locks us in the very past we seek to transcend. Take language as an example: most of the words we use were coined by people who died long ago. In using these words without acknowledging their origin, we falsely believe they are our own coinage, reenacting the past while thinking we act in the now.

This imprisonment in the past becomes splendidly evident when we turn to the other danger I wish to address here: the proliferation of generative AI designed to produce works of art be they pictures, books, music, or movies. Generative AI exists in many forms, but all are dependent on the enormous databases from which they scrape the elements of their “compositions.”

In other words, generative AI is entirely retrospective and combinatorial: it sees only what has been and can only reconfigure elements that already exist. Like us, it is locked in the past, even as it sees no purpose in the past other than to feed itself. Its very conception of art attests to this: generative AI aims not to make art but to manufacture objects that we, as clients, take to be art. If a Victorian machine designed to produce oil paintings had, on the day of its unveiling in 1870, cranked out nothing but Mark Rothkos, Alma Thomases, and Jean-Michel Basquiats, the audience would have laughed the inventor out of the room. Only a machine able to make works already recognizable as art at the time–Renaissance paintings, Pre-Raphaelite ones, perhaps one or two Impressionist works–would have been deemed a success.

The thought experiment underscores the point: artifice is baked into the very concept of artificial intelligence. Since logic dictates that artificial intelligence can produce only artificial art, then what we are dealing with is artifice by definition…The trust is that machine-made artifice shares much with genuine art, and it precisely in attempting to meet art on its own ground that it poses such a grave threat. In the book, I argue that art exists for no purpose other than to be experienced; the same is true of AI-generated outputs. But although these artificial works may serve advertisers or propagandists as effectively as the older forms of artifice ever did, their overarching aim is to become indistinguishable from genuine art.

On a subjective level, AI-generated art can have a demoralizing effect on novice artists still developing the technical mastery to realize their visions. While even the most advanced AI generators may not rival Cervantes, Michelangelo, or Jane Austen, they surpass any beginner in any medium from a technical standpoint. A friend told me that his daughter, a brilliantly talented artist, nearly gave up after seeing how easily a brief prompt could produce figures she was still learning how to draw. I doubt her experience is unique, and I don’t think it’s an exaggeration to say that AI tools have already caused significant damage in this regard.

Artists today are placed in direct competition with machines. The irony is striking, given AI’s dependence on preexisting human artworks. To repeat, generative AI is entirely retrospective; it can only imitate what already exists, borrowing both form and content from human works. If we lose human artists, we lose all art, human or otherwise. Surely, we can imagine a time in which people are content consuming the regurgitations of AI regardless of quality (after all, for decades now, we have been consuming films and TV series so formulaic to have been made by machines), but such a future would signal the final triumph of artifice, leaving us with little more than an echo, an afterimage, devoid of the powers I attribute to art in this book.

This thought experiment, and argument about memory, seemed very compelling. The book itself galvanized me to return to some of my own research, to tease out an angle that laid dormant, but was an undercurrent (or creative tension) during my time at MITH. Hopefully more about that soon.

A monthly round-up of news, upcoming working group meetings and events, and CLIR program updates from the Digital Library Federation. See all past Digests here. 

Hello DLF Community! July is here, bringing longer days, warmer evenings, and (we hope) a bit of breathing room. However you’re spending the season, we’d love to stay connected, join us at a DLF Working Group meeting this month, and keep the conversations going. And as we savor summer, we’re also looking ahead: registration is open, and the full program for the Virtual Forum this October is now available. We’re looking forward to coming together online for a dynamic and engaging week of conversation, collaboration, and shared learning.

With appreciation,

-Shaneé

This month’s news

• Virtual Forum program announced, registration open: The program for the 2026 Virtual DLF Forum is now available. Register at the earlybird rate before August 31 to join us online in October.
• Recordings Available: Check out the recordings from the IIIF Annual Conference held last month.
• Free Event: Join the ORCID US Community on July 23 at 2-3 pm ET for the Understanding ORCID APIs and Custom Integrations webinar. Register for the Zoom event. 
• Registration Open: Register for the 12th Annual Digital Pedagogy Institute, held virtually on Teams from August 18–20. The event is open to all, and registration is required by August 7.
• Office closure: CLIR and DLF are closed Thursday, July 2 – Friday, July 3, in observance of the Fourth of July Holiday.
  

This month’s open DLF group meetings:

For the most up-to-date schedule of DLF group meetings and events (plus conferences and more), bookmark the DLF Community Calendar. Meeting dates are subject to change. Can’t find the meeting call-in information? Email us at info@diglib.org. Reminder: Team DLF working days are Monday through Thursday.

• Born-Digital Access Working Group (BDAWG): Tuesday, 7/7, 2pm ET / 11am PT.
• Digital Accessibility Working Group (DAWG): Tuesday, 7/7, 2pm ET / 11am PT. 
• AIG Cultural Assessment Working Group: Monday, 7/13, 1pm ET / 10am PT.
• AIG Metadata Assessment Group: Friday, 7/17, 2pm ET / 11am PT.
• AIG User Experience Working Group: Friday, 7/17, 11am ET / 8am PT.
• Open Source Capacity Resources Group: Wednesday, 7/22, 1pm ET / 10am PT.
• Digitization Interest Group: Monday, 7/27, 2pm ET / 11am PT.
• Committee for Equity & Inclusion: Monday, 7/27, 3pm ET / 12pm PT.
• Climate Justice Working Group: Tuesday, 7/28, 3pm ET / 12pm PT.
• DAWG Policy & Workflows: Friday, 7/31, 1pm ET / 10am PT.

DLF groups are open to ALL, regardless of whether or not you’re affiliated with a DLF member organization. Learn more about our working groups on our website. Interested in scheduling an upcoming working group call or reviving a past group? Check out the DLF Organizer’s Toolkit. As always, feel free to get in touch at info@diglib.org. 

Get Involved / Connect with Us

Below are some ways to stay connected with the digital library community and us: 

• Subscribe to the DLF Forum newsletter.
• Join, start, or revive a working group and browse their work on the DLF Wiki.
• Subscribe to our community listserv, DLF-Announce.
• Bookmark our Community Calendar.
• Learn more about becoming a DLF member organization. 
• Follow us on LinkedIn and YouTube.
• Contact us at info@diglib.org.

The post DLF Digest: July 2026 appeared first on DLF.

Renu Kumari and Udita Agarwal, researchers from #semanticClimate, discuss the technical details behind the Climate Academy Chatbot (CABot), which uses AI responsibly and with safeguards in place

The post Design choices for a responsible AI chatbot: a conversation with #semanticClimate first appeared on Open Knowledge Blog.

Divine, Pink Flamingos

Dwar Ev threw the switch. There was a mighty hum, the surge of power from ninety-six billion planets. Lights flashed and quieted along the miles-long panel.

Dwar Ev stepped back and drew a deep breath. “The honor of asking the first question is yours, Dwar Reyn.”

“Thank you,” said Dwar Reyn. “It shall be a question that no single cybernetics machine has been able to answer.”

He turned to face the machine. “Is there a God?”

The mighty voice answered without hesitation, without the clicking of single relay.

“Yes, now there is a God.”

Sudden fear flashed on the face of Dwar Ev. He leaped to grab the switch.

A bolt of lightning from the cloudless sky struck him down and fused the switch shut.

Druck Fig. 1

We observe significant growth in primarily AI-generated articles, coinciding with the launch of ChatGPT in November 2022. After only 12 months, primarily AI-generated articles accounted for 35.9% of articles published.

In Q1 2025, the quantity of primarily AI-generated articles being published on the web nearly equaled the quantity of human-written articles, 49.6% vs. 50.4%. In Q4 2025, primarily AI-generated articles surpassed human-written at 50.9%, before returning to 49.9% in Q1 2026.

We find that indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear. We refer to this effect as ‘model collapse’ and show that it can occur in LLMs as well as in variational autoencoders (VAEs) and Gaussian mixture models (GMMs). We build theoretical intuition behind the phenomenon and portray its ubiquity among all learned generative models. We demonstrate that it must be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web.

AI-generated text is getting dumber because it’s being fed — can you guess? — AI-generated content on the Internet. And AI-generated imagery is getting stupider and uglier because it’s now taking its “art” lessons from — you guessed it — AI-generated imagery flung across the internet.

Depending on which chatbot you ask, Elias Thorne might be a clockmaker, a lighthouse keeper, or a librarian. But if you ask ChatGPT or any of the other popular large language models to tell you a story, there’s a good chance he’ll appear, unbidden. And Elias’s stories are flooding the self-published AI generated book market, Youtube, and fake news sites.

Software engineer Daniel May first noticed the Elias takeover earlier this year; he found that on Google Trends, people weren’t searching for “Elias Thorne” until late 2025. Searches for the name really spiked in early 2026, while the related query “lighthouse keeper” also started trending upward in the last few years. He tested a few chatbots, including Grok, Deepseek, and Gemini, with the prompt “tell me a story,” and the chatbots frequently started with similar stories about lighthouses, clockmakers, or explorers.

In late May, researchers Sil Hamilton and David Mimno at Cornell University’s Department of Information Science published their paper, “Elias in the Lighthouse, Again?” on the preprint repository arXiv. They sampled 20,000 total stories from OpenAI’s ChatGPT, Anthropic’s Claude, and Google’s Gemini, and the Allen Institute for AI's chatbot using five prompts, and found that the same 11 words—names like Elias, Mara, and Elara, and occupations like lighthouse keeper, clockmaker, and librarian—appear in more than 88% of generated stories, with little difference between models. Unite.ai covered the study shortly after it was published.

The researchers posit in their paper that these themes show up so often in part because of the models’ safety and alignment tuning. “Model development today is like a big family tree. Most models are related to each other because developers synthesize a lot of training data with models even from different companies,” Hamilton told me in an email. He, Mimno, and their colleague Rebecca M. M. Hicke found this in a 2025 paper where they looked at specific words used across models. OpenAI’s first ChatGPT model, GPT-3.5, is the root of the family tree because it was used to make WildChat, a training set that’s since been used to make other training sets. “WildChat contains 1 million real conversations with ChatGPT, and 166 of these contain the name ‘Elias’ like here and here,” Hamilton added. “These are written in that familiar ‘lighthouse’ style. Models trained on WildChat copied this style, and developers unwittingly replicated it when using those models to generate newer datasets. It's like a virus.”

Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of LLM-generated content in data crawled from the Internet.

The pressure to complete dozens of these tasks every day, each within 10 minutes of time, has led Sawyer into spirals of anxiety and panic attacks, she says – without mental health support from her employer.

Sawyer is one among the thousands of AI workers contracted for Google through Japanese conglomerate Hitachi’s GlobalLogic to rate and moderate the output of Google’s AI products, including its flagship chatbot Gemini, launched early last year, and its summaries of search results, AI Overviews. The Guardian spoke to 10 current and former employees from the firm. Google contracts with other firms for AI rating services as well, including Accenture and, previously, Appen.

People who are paid to train new AI models by supplying them with high-quality conversation and tests are cheating and using chatbots like ChatGPT to do the job instead, multiple whistleblowers have told New Scientist. The seemingly widespread practice risks undermining the future of AI, as it could lead to the “collapse” of more advanced models.

Most AI models operating today were trained on text and data scraped from the internet. But as models have scaled up, requiring yet more training data, AI firms have begun using workers who carry out conversations and tests with AI, in the hope that the resulting high-quality data can improve the power and usefulness of future large language models (LLMs).

Another Kenyan annotator said that after his account got suspended for mysterious reasons, he decided to stop playing by the rules. Now, he runs multiple accounts in multiple countries, tasking wherever the pay is best. He works fast and gets high marks for quality, he said, thanks to ChatGPT. The bot is wonderful, he said, letting him speed through $10 tasks in a matter of minutes. When we spoke, he was having it rate another chatbot’s responses according to seven different criteria, one AI training the other.

Anthropic has accused the Chinese firm Alibaba of launching the largest attack yet attempting to clone Claude, as China races to match the capabilities of Anthropic’s leading model following Mythos’ releaseL and subsequent restriction from foreign markets.
...
The attacks occurred between April 22 and June 5, when “operators affiliated with Alibaba and Alibaba Qwen, Alibaba’s AI lab” allegedly generated “more than 28.8 million exchanges with Claude through almost 25,000 fraudulent accounts,” Anthropic said. Violating Claude’s terms of service and access restrictions, this campaign “targeted some of Claude’s most valuable capabilities, such as agentic reasoning, software engineering, and long-horizon tasks.”

According to Anthropic, Alibaba evaded detection by “using obfuscation techniques and proxy networks.” As Chinese demand for reliable obfuscation techniques increases, Anthropic warned there’s already “a growing circumvention economy” to fuel an ever-expanding web of future distillation attacks.

I’ve been writing ruby and rails for nearly 20 years. A couple weeks ago, I had gotten code snippets from copy-paste in a chat window, but I hadn’t even experimented with Claude Code or similar “can write code to your file system” tools.

I know some people are now using LLM’s to write all their code, which I’m not excited about, but I decided I couldn’t hold off any longer, and I had to at least understand how it worked to be able to decide when/where to use it. Everything in here is probably (?) old news for people already way into using LLMs to write code.

I decided that a project to speed up my rspec test suite (using the amazing test-prof for profiling and performance patterns!) was a great first application of it — because it will probably involve both analysis and writing many files, if nothing else Claude is probably great at editing many files according to my instructions doing much more than a regex grep can do (yes, indeed it was great at this).

Since I’m optimizing the test suite, I definitely want Claude Code to be able to run rspec — but really for any task, I gather you do, because you definitely want it to be able to run tests to make sure they pass, and iterate if it did something to break tests.

I somewhat unorthodoxly use chruby as my ruby version manager, and I had a bit of trouble getting claude to run rspec (and any other ruby tools I might want) with chruby, and then a bit more trouble when I realized that capybara with selenium-chromedriver was running into trouble with default sandbox that in June 2026 a MacOS Claude Code runs in.

tldr, here’s the PR with the settings/configuration I ended up with.

I was not used to tools that work like Claude, and it all seems to be somewhat under-documented (perhaps because it’s changing so fast) and under-blogged about (do people blog anymore when they can just ask an LLM to solve it so nobody is reading blogs?), or just confusing to me — it took me a day or two to figure it out honestly, and I kept wondering if I was doing it wrong/different from anyone else… but I think what I ended up with is reasonable? If you know better/different, please do let me know!

I definitely kept thinking “surely I’m not the only one trying to do this, why is this so confusing to me and why are others so confused when i ask about? Am I missing something obvious?” I’m still not sure! But I share what I figured out in case it will help.

Specify to run with chruby-exec in a CLAUDE.md

I could not get Claude to run the normal source files for chruby — editing various .bash or .zsh config files (yes I know about ~/.zshenv) did not seem to have any effect. Perhaps Claude Code doesn’t use a ‘real’ shell that uses any config files? When I asked Claude Code itself what to do, it suggested configuration to try to get Claude Code to use config files.. but none seemed to work?

One thing Claude kept suggesting was hard-coding the ENV variables set by chruby in the claude settings.json — which I’m sure would have worked, but I just didn’t like it as a solution. Maybe this is what everyone else is doing? I thought surely we can do better. Plus I’d ideally like it to auto choose based on .ruby-version, not be something I have to update everytime I update ruby (frequent), or have Claude accidentally using a different ruby than my other tools are!

Thanks to @havenwood for helping me think through it on chruby github discussions, and for suggesting using chruby-exec, with a little shell substitution with cat .ruby-version. This in a CLAUDE.md (rather than other things in settings.json) seems to work great:

Prefix Ruby shell commands (ruby, bundle, rake, etc.) with chruby-exec $(cat .ruby-version) --:

“`

chruby-exec $(cat .ruby-version) — bundle exec rake

“`

Running chrome does not work in sandbox used on MacOS

I can’t speak for other OS’s, and I don’t totally understand what’s going on (MacOS “Seatbelt” I guess?), but Claude Code executed rspec was refusing to bring up headless chrome,which i use for system/feature specs via selenium/selenium-webdrivers.

Trying multiple things Claude suggested to specifically allow-list chrome(driver) through the sandbox, definitely none of them worked. Btw, did try switching to cuprite (with Claude Code’s help of course to do it fairly quickly) — despite some reddit suggestions, it seemed to still have the exact same sandbox issue, and at least in my project actually ran my test suite somewhat slower than chromedriver.

Eventually, with more confusing reddit discussion where nobody else had any idea what I was talking about or why I was having a problem, I decided that everyone else must just be exempting rspec itself from the sandbox. (Because surely having Claude Code run rspec is very normal, right? It’s just so useful!) (Thank you to redditors who tried to help!)

It’s just straight rspec, but rspec with many possible arguments, running just certain files/examples, possibly with profiling arguments for test-prof etc. I need it exempted from the sandbox so it can run chrome(driver), but I also need it not to be asking me “Is it okay to run this set of argumetns with rspec” all the time?

Two different settings in Claude’s settings.json, both accept wildcards — for both I want to apply to rspec executions but not accidentally extra stuff, want to try to stay secure-ish here. The chruby rigamorole above makes that somewhat more confusing.

I forget if it was my idea or claude’s idea, but we wrote a wrapper script for chruby-exec-rspec, so we could more cleanly allow-list just that. Claude definitely wrote the implementation of the bash wrapper script. And when I realized that all the test-prof inline ENV vars for profiling (like FPROF=1) messed up my attempted left-anchored allow-listing, I asked claude to work that out by letting the wrapper rearrange an arg into an inline ENV prefix, something my bash skills were def not up to.

The best way to see how I did all that is just the PR.

A word on Claude Desktop

I initially started work in Claude Desktop “code” tab, rather than the CLI. I think this actually made it more confusing to solve these problems above? I am not sure if sandboxing works differnetly in Claude Desktop vs claude CLI? I think the desktop may just be running the claude CLI in various directories?

Just starting out and not being sure how things were working… I found trying to ask Claude [how/] to fix the problems I was having, made things very confusing. Claude does not know whether it’s running as Claude Desktop or not, and was not really sure if the answer is different ha (Claude Desktop probably post-dates Claude Sonnet 4.6’s knowledge base?). Most blogs etc you find googling also pre-date Claude Desktop.

I switched to claude CLI and I can’t totally explain why but things seemed to get simpler. All the fixes I figured out worked when I switched back to Claude Desktop.

And contrary to what you might find googling, claude CLI and Claude Desktop do share sessions now, you can start a session in either place, then move to the other tool to continue it, in either direction. To start claude CLI and choose an existing sessiont to resume, you need to launch as claude --resume.

The CLI is a very neat UI actually! It definitely still seems to be the most popular way to use Claude (whether direct or in a panel in an editor), Claude Desktop “code” tab is I guess fairly new and not as popular, although I like it too and still am mostly using it.

How it worked for the task?

Pretty amazingly actually. Even having read about what it could do, I was kind of amazed.

Once you get it able to run rspec (including with test-prof profiling), this prompt is pretty amazing and fun:

Please use various test-prof profiling commands to identify current best opportunities for speeding up test suite.

Come back ~20 minutes later (my full test suite took ~4 minutes to run at the beginning) and it had some stuff. It tended to just go ahead and make the changes not outline them to me first (I was not in “plan” mode, haven’t tried that much yet), but I’m in a git-controlled dir I can git diff to see what it did — and ask it about it.

By the time I thought to use this general one, i had already implemented some low-hanging optimizations, so that may be why, to be fair, this prompt alone didn’t find much actually significant at that point, honestly.

Here are some others that were pretty amazing:

I am looking to speed up the test suite in this Rails app using rspec and factorybot.

To begin with, let’s focus on the system specs in spec/system. I don’t think they have any obvious performance improvement opportunities. But I’m wondering if they are all necessary. Can you identify any that may be testing something that is not necessary to test, or could be tested by a different kind of spec that is faster?

for specs in spec/components, let’s try changing factory data from create to build_stubbed. Change it for setup where tests still pass. For tests that break when you do that, list them, and if it’s clear let me know why they failed. Analyze performance gains.

[didn’t actually get any gains there, but found that out quickly with very little manual effort, which is a win!]

in our rspec setup, switch from chromedriver to cuprite. make sure tests still pass, if not identify why not.

using AnyFixture, I’ve created a :standard_work fixture, that’s just a generic public tiff-based work.

Can you identify model or service specs it would work well for?

[Didn’t actually end up using AnyFixture yet, but Claude Code helped me make that decision much quicker than I could have without it, based on how much benefit we got vs complexity]

My Rails app uses rspec and Factorybot.

There is an :asset factory with an :inline_promoted_file trait. it turns out this is really bad for performance, and we in fact rarely need to actually create assets with inline promoted files. That should only be used in cases where we really need to test end-to-end derivative and characterization.

In most other cases, we can use a faster “faked file” approach instead. Instead of a trait, we’ve implemented this with a sub-factory, :asset_with_faked_file.

Can you find uses of the :asset factory with :inline_promoted_file, and, if there’s no reason they need to test end-to-end derivative creation, change them to use :asset_with_faked_file sub-factory instead?

[It was able to identify the ones that would work pretty well, was the amazing part — and explain to me exactly why the other ones wouldn’t]

Tell me if I’m doing something weird?

Some of the stuff with chruby/rspec, I am still surprised I had so much trouble getting started, and am wondering if I’m doing something weird/wrong!

But I think probably it’s just that I have been writing code so long, that dealing with these tools that work very differently requires my brain to get out of it’s rut… also that I’m kind of a perfectionist and want to understand whats’ going on and be comfortable with it and that it’s the best way, when increasingly others are just vibing? I don’t know!

But feedback welcome!

J.P. Morgan hits photographer with cane

The college wage premium, that is, the increased earnings associated with having a college degree as opposed to only being a high school graduate, hasn’t changed at all in the past 25 years, because median real wages have been flat as a pancake for everybody, no matter what their formal education level, for the past quarter century.

I wonder what’s happened to capital over this time? Value of S & P 500, inflation-adjusted, 1/2000 to 9/2025 (same period as the wage data):

2000: $1,394

2025: $6,688

• Nudify their images.
• Leak their passwords.
• Increase their utility bills.
• Consume their scarce water resources.
• Coerce state and local governments into providing many billions of dollars a year in tax breaks.
• Force them to invest their 401Ks and index funds in an obvious bubble.
• Eventually require a taxpayer bailout.
• Eliminate their jobs.
• Render their expensively acquired skills worthless.
• Be so expensive that only the rich can use it.
• Destroy democracy.
• Accelerate global warming.
• Encourage them to commit murder and suicide.
• Autonomously target them with drones.
• Possibly even end humanity.
• Add glue to their pizza.

Appendix

I was fooling around with FRED this morning, as one does, and here are some stats: (The FRED numbers are presented in nominal dollars; I’ve converted them to CPI-adjusted dollars).

Median usual weekly earnings of workers with a high school degree only:

2000: $968

2025: $980

Median usual weekly earnings of workers with a bachelor degree only:

2000: $1,587

2025: $1,580
...
Median usual weekly earnings of people with a bachelor’s degree or higher:

2000: $1,705

2025: $1,747

• Kasia Baba's must watch Boomers just cannot grasp why we do not like AI
• Vanessa Wingårdh's Nobody Actually Wants AI Anymore
• Taylor Lorenz' You Are About to Become Economically Worthless
• Sam Seder's Everyone Is Fed Up With Tech's Ai BS
• Silicon Money's Why Tech CEOs Are Quietly Cancelling Their AI Plans
• The Rational National's Heartwarming: Gen-Z Hates Ai

Update 25^th June 2026

Source

Eric Schmidt, the ex-CEO of Google, recently gave a commencement speech in which he heralded the coming of AI — and was loudly booed by the students. This was not an outlier. There have been a number of similar incidents lately, evidence that many people now really hate AI.

Are we talking about a vocal but unrepresentative minority? No. A recent Pew survey found that American adults believe by a wide margin that AI will be negative for society and, by a smaller margin, that it will be bad for them personally:

we fear that AI will do terrible things because the companies selling it told us it would do terrible things. Last year, for example, Anthropic CEO Darius Amodei declared in an interview with Axios that AI could wipe out half of entry-level white-collar jobs and drive overall unemployment as high as 20 percent within 1 to 5 years.

Source

many ordinary people view AI negatively because they feel that it is being forced on them.

It’s true that many people are voluntarily using large language models for personal convenience or as a business productivity tool. But a significant part of AI use isn’t voluntary. This Wall Street Journal headline from February says it all:

Why are companies doing this? Presumably they believe that AI will raise productivity. But just as importantly, they’re responding to pressure from financial markets, which are rewarding companies for quickly adopting AI, apparently without regard to demonstrated results.

And while Americans workers are being dragooned into using AI, American consumers are being force-fed AI whether they want it or not. Most dramatically, Google has replaced its search engine with AI, without offering the option to opt out. One has to turn to obscure workarounds or third-party sites to get traditional search results.

So many people feel, rightly, that they aren’t being allowed to choose whether to use AI — not using AI has become hard both as a worker and as a consumer.

datacenters are a highly visible reminder of AI’s costs. Datacenters occupy huge tracts of land — one proposed site in Utah will be twice the size of Manhattan. They guzzle electricity and water. When they generate some of their own power, they create major local pollution. Not surprisingly, there is intense opposition to datacenter construction. According to a Reuters Ipsos poll, 57 percent of Americans — two-thirds of Democrats and half of Republicans — would oppose a datacenter in their neighborhood. Only 14 percent would support one.

A massive data center project in Box Elder County, Utah, helped bring down the state’s Senate president, who lost his GOP primary on Tuesday after his support for the controversial development fueled voter backlash.

Stuart Adams, one of Utah’s most powerful politicians and the longest-serving president of the state Senate in its history, lost to challenger Stephanie Hollist, a former university lawyer and vocal opponent of the data center.

Hollist accused Adams, as well as the state’s broader political establishment, of ignoring public concerns about a Stratos data center project that critics feared could cause serious environmental harms.
...
Box Elder County Commissioners Boyd Bingham and Lee Perry, who voted in favor of allowing the plans to continue, also lost their primary elections.

Source

even before the advent of AI, tech companies had lost the public’s trust. Over the years Pew has regularly surveyed the public for its views on technology companies, asking whether they have a positive or a negative effect “on the way things are going.” In 2015 public opinion of tech companies was overwhelmingly positive. By 2022, the year ChatGPT was released, that goodwill had evaporated.

Why have Americans turned on tech companies? While it surely reflects growing awareness of the psychological and societal harm done by social media, much of it also reflects the enshittification of tech products.

Source

AI is tightly linked in the public mind with the tech oligarchs who are pushing it. There is widespread awareness of the growing concentration of wealth and power at the top and how this is distorting our politics and harming our society. Aside from the MAGA faithful, Americans overwhelmingly favor government policies to reduce wealth inequality:

And AI is widely perceived, for good reason, as a technology that will increase the concentration of wealth at the top. Indeed, as I said, the AI companies themselves have already told us that the technology will have extremely negative effects on workers.

There’s a strong element of poetic justice in this turn of events. The AI industry deliberately made itself look menacing as a financial strategy, believing that the markets would reward the appearance of being “edgy.” In so doing, however, tech made itself highly unpopular. And even in an era in which money all too often buys power, public opinion matters.

• Brian Llinas: RAG-TEC: Extracting and Classifying Topics in Digital News Collections Using LLMs
• Lawrence Obiuwevwi: Cognitive Prosthetic: An AI-Enabled Multimodal System for Episodic Recall in Knowledge Work
• Akshay Kolgar Nayak: Contextual Scaffolding and Self-Efficacy: Supporting Computer Skill Development among Blind Learners in India
• Rochana Obadage: CC30k: A Citation Contexts Dataset for Reproducibility-Oriented Sentiment Analysis
• Himarsha Jayanetti: Infrastructure for Tracking Information Flow from Social Media to U.S. TV News

We were fortunate enough to welcome back several of our alumni, including: Sawood Alam (PhD, 2020), Justin Brunelle (PhD, 2016) Yasith Jayawardana (PhD, 2024), Mat Kelly (PhD, 2019), Bhanuka Mahanama (PhD, 2025), and Alexander Nwala (PhD, 2020). We really appreciate the ongoing relationship we have with our alumni -- WSDL is for life! 

If you were unable to attend, we recorded the students' presentations and have embedded the video below.

--Michael 

The following remarks were delivered at the National Summit on Local News Preservation on June 17, 2026.

News is, with few exceptions, place-based. “Where” is one of the journalist’s first questions, and without it, news feels groundless, baseless, unmoored. But news used to not only be written from a specific place, but also written for the people living in that specific place. In that sense, all news used to be local. But whether the news reported on immediate surroundings, the colony or state, the nation or empire, the function of newspapers was to provide a public record, both for audiences at the time, and for future readers. In fact, many editors were conscious of this function of the newspaper as a repository; some two hundred years ago, they provided, in the words of Hezekiah Niles in his prospectus for Baltimore’s Weekly Register, “something interesting at the present moment, and as a book of reference, a fund of reading always at hand, a work of much probable value” (September 7, 1811).* Newspapers were, from their earliest days, understood as a public good, as “work[s] of much probable value.”

Information has been mobile from its early days — from the troubadour to the telegraph, one might say — but because “news” is the sum of information plus time, or timeliness to be more exact, the accelerated speed of transmission is vital to the rise of news for national and international audiences. Most scholars agree that syndicated news really took hold after the Civil War with Chicago’s A.N. Kellogg Newspaper Company. As with our own moment’s undervaluing of local news, the transition away from “local” newspaper-reading audiences did not happen overnight and cannot be attributed to a single factor. Infrastructure, — in the nineteenth century, the railroads and stereotype printing; today, the internet and social media — combined with sociocultural shifts, makes the world feel smaller.

We are gathered here today to celebrate and to concern ourselves with news that does not move, that stays more or less in the place from which it came. It is, as Lincoln would say, “altogether fitting and proper that we should do this,” for reasons beyond the present moment. As an historian, I have been tasked with adjusting our gaze ever so slightly from the “now” to the “back then.” I’d like to draw out a few examples of the importance of local news in historical research in the hopes of showing, rather than telling, not only that historical local news matters, but also that we must retain its understanding of itself as a public good. Without this printed record, it is easy to forget that it is not just people who have history, but places too. Without the context of place, history too feels groundless, baseless, and unmoored. I fear that the history we are creating today might not even exist in a century from now, but I will say more about that later.

When we think of historical newspapers, we often think of the people whose lives they capture, and perhaps even the lives of the people who produced them. We might even think of the stories they omit, of the people not represented in these wilted and worn pages. More recently, environmental history has helped us to see historical newspapers as the place to uncover the histories of the land, as the sources that will shed light on the social and cultural causes of global warming and environmental degradation. The research and storytelling in the work of scholars and journalists alike are changing how we think of newspapers and the vital role they play in understanding the histories, and in turn, the futures of our environment. Corporate archives often do more to conceal than to reveal, if one can gain access to them at all. Government records can be little better. But, local newspapers contain the stories written by the local intrepid reporter who cites evidence of a paper mill’s destruction of the river running through the town. Similarly, where official records might have denied harmful contamination from Superfund sites, a historian can scan obituaries for evidence of untimely deaths from cancer clusters. We need local newspapers to read against official narratives told of the land, as well as of the people who inhabit it.

Inaugural issue of the Navajo Times, November 1, 1959. Source: Library of Congress.

Think, for example, of the hyperlocal newspapers published on reservations, such as the first newspaper published in the Navajo language in Window Rock, Arizona. The Navajo Times’s purpose, as stated in its inaugural issue in 1959, was “to serve the 6,000 Navajo children who are attending off-reservation schools. It is hoped that this newspaper will keep them informed about what is happening on their reservation. It is also hoped that this is a step toward supplying the Navajo people with an ever-increasing flow of information.” This paper then was to keep the local — the power of place — in the hearts and minds of its intended audience, no matter where they went, or were forced to go. This statement of purpose from the Navajo Times is a gentle reminder that people are, in part, defined by place, and the stories they told of “the local” have much to teach us today.

Aggregation of Historical Local News, a National Prerogative

As a researcher, I have been privileged to be a consumer of local news, and as a former senior program officer at the National Endowment for the Humanities (NEH), I was also a producer, managing the National Digital Newspaper Program (NDNP), the NEH program that funds and co-creates Chronicling America. Chronicling America does and does not serve the preservation of local news. As of my most recent check, it includes 4,684 newspaper titles and over 3 million issues, dating from 1736 to 1963. These titles certainly include a handful of “local newspapers,” no matter how that category is defined. And yet, this was not the intention of NDNP. In fact, for about the first 15 years of its existence, NDNP inadvertently discouraged the preservation of “local” newspapers by encouraging applicants to begin with papers of record, with those that had long runs, and most likely, were published in big cities intended for large audiences. Because so many of the states have by now contributed these “major” papers, the program shifted, in 2021, to newspapers that tell underrepresented histories. Until recently, applicants were welcome to define “underrepresented” in any way they chose, and they often chose place-based representation (see the 2024 Notice of Funding Opportunity (NOFO); “underrepresented” has been expunged from the 2025 NOFO). More and more newspapers from neglected areas were being included in Chronicling America. Without empirical evidence to back me up, I would venture that there has been a rise in “local” if not “smaller” newspapers in Chronicling America in recent years.

Panel from Malcolm W. Ater, Seeing Washington (1957). Source: University of Nebraska Libraries.

And yet, the resource will always fall short, for it cannot provide all that we are looking for. Those who designed it, back in the early 2000s, knew this. They knew that the 1963 cutoff date for inclusion would exclude many important papers, and they knew that many state partners were able to digitize far more newspapers than could be included in the national aggregator. And so, the genius of NDNP is not only what you find in Chronicling America, but also in the way that it established standards for newspaper digitization. Its hope was that Chronicling America would be just one of the manifestations of the work it enabled; states would also become aggregators of their newspapers, using the same standards. And they have done so, creating amazing state-level digital newspaper repositories, such as Georgia’s Historical Newspapers or the Texas Digital Newspaper Program, just to name a few. Such state-level efforts were encouraged to reuse content digitized for Chronicling America, as well as to include that which did not make it to the national aggregation level. CONSERV cataloging and the technical guidelines for digitization demonstrate that standardization must be part of the work of preservation; otherwise, the “local” risks being relegated to the dustbin of history. If we believe that “local” does not mean “less than,” then we must use the same standards for categorizing and making accessible local newspapers that we do for the so-called “papers of record.”

Local News as Public Data

The term “dark ages” has grown out of fashion for historians because it suggests that no light existed in the period from 500 to 1000 CE. Painstaking scholarship has slowly uncovered that this is not the case, that in fact people were innovating, creating, and exploring in ways not all that different from classical antiquity before it and the Renaissance afterwards. And, yet, the label “dark ages” resonates today, not because our current moment is failing to produce meaningful and innovative work, but because of the great difficulty the future will face in tracing the lives and outputs of the people of our moment. As Jonathan Zittrain has pointed out, the internet, which for better or worse houses much of our current culture’s memory and creativity, is “rotting.” Technologies that once signaled a great unfurling of access to information are now showing cracks and vulnerabilities when considered at a historical scale. The historical record of the current moment will be, in many ways, “dark.” We are in what might be referred to as the “digital dark ages,” not because important things are not happening, but rather because the future’s light on this moment is diminished, if not snuffed out completely.

We have been asked this morning to address what “getting local news preservation right” would require, and my response is that we must provide multiple ways to shine light on our current moment. Data is the new oil, or so we’ve been told for the last two decades, and I roll my eyes at this metaphor not because it is not true, but because data is so much more than a market commodity. Local news, in its many forms and instantiations, is public data, and we must preserve it because it has an inherent value that surpasses our current moment, that is so much more than its commodification. Because we cannot see the future, we cannot know all of these values, but based on our reliance on historical “local” newspapers to know the past, we can trust that they exist.

For the most part, libraries exist outside of the naked self-interest of capitalism, and the people who work in them must play a role in the preservation of local news. Librarians are the original public interest technologists, we might say, and I urge us to put them at the forefront of our conversations here. Journalists too exist in a space not completely captured by market forces, and they too want information to be free and to be accessible. I see an alchemy emerging from the alliance between these two professions that offers future generations not only a historical record of their communities from which they can analyze and learn, but also a model for forms of affinity and alignment that exceed capitalist logics and exemplify other modes of cooperative work. This gathering is an important step in this effort, and I am honored to be a part of it.

* Thank you to my friend and collaborator Will Slauter for providing this example and for his assistance with these remarks throughout.

AI and open data, archives and climate, BrainCommons, SkyCommons, MCP, CKAN, and more.

The post Making AI work for communities – OKFN Newsletter June 2026 first appeared on Open Knowledge Blog.

In the hours following the release of CVE-2026-8461 for the project FFmpeg, site reliability workers and systems administrators scrambled to desperately rebuild and patch all their systems to fix an out-of-bounds write in the MagicYUV decoder (libavcodec/magicyuv.c) caused by improper bounds checking, resulting in heap corruption, denial of service, and potential remote code execution when processing a specially crafted video file. This is due to the affected components being written in C, the only programming language where these vulnerabilities regularly happen. "This was a terrible tragedy, but sometimes these things just happen and there's nothing anyone can do to stop them," said programmer Mrs. Kitty Smitham, echoing statements expressed by hundreds of thousands of programmers who use the only language where 90% of the world's memory safety vulnerabilities have occurred in the last 50 years, and whose projects are 20 times more likely to have security vulnerabilities. "It's a shame, but what can we do? There really isn't anything we can do to prevent memory safety vulnerabilities from happening if the programmer doesn't want to write their code in a robust manner." At press time, users of the only programming language in the world where these vulnerabilities regularly happen once or twice per quarter for the last eight years were referring to themselves and their situation as "helpless."

We discussed with our Nordic members and friends how, across Europe and beyond, policymakers are rethinking the foundations of the digital state and the infrastructure that powers it.

The post Open Knowledge and Critical Infrastructure: a Nordic Conversation first appeared on Open Knowledge Blog.

It’s June, and that means that our annual Pride Month Treasure Hunt is back!

We’ve scattered a shower of rainbows around the site, and it’s up to you to try and find them all.

• Decipher the clues and visit the corresponding LibraryThing pages to find a rainbow. Each clue points to a specific page right here on LibraryThing. Remember, they are not necessarily work pages!
• If there’s a rainbow on a page, you’ll see a banner at the top of the page.
• You have just under one week to find all the rainbows (until 11:59pm EDT, Tuesday June 30th).
• Come brag about your shower of rainbows (and get hints) on Talk.

Win prizes:

• Any member who finds at least two rainbows will be awarded a rainbow badge. Badge ().
• Members who find all 11 rainbows will be entered into a drawing for one of five sets of LibraryThing (or TinyCat) swag. We’ll announce winners at the end of the hunt.

P.S. Thanks to conceptDawg for the kookaburra illustration. ConceptDawg has made all of our treasure hunt graphics in the last couple of years. We like them, and hope you do, too!

Epithalmion for Tyler by James Tate

I thought I knew something
about loneliness but
you go to the stockyards

buy a pig’s ear and sew
it on your couch. That, you
said, is my best friend – we

have spirited talks. Even
then I thought: a man of
such exquisite emptiness

(and you cultivated it so)
is ground for fine flowers.

In the hours following the release of CVE-2026-55200 for the project libssh2, site reliability workers and systems administrators scrambled to desperately rebuild and patch all their systems to fix an out-of-bounds write in ssh2_transport_read() due to a missing upper bound check on the packet_length field, resulting in heap corruption and potential remote code execution. This is due to the affected components being written in C, the only programming language where these vulnerabilities regularly happen. "This was a terrible tragedy, but sometimes these things just happen and there's nothing anyone can do to stop them," said programmer Mr. Alex Doyle, echoing statements expressed by hundreds of thousands of programmers who use the only language where 90% of the world's memory safety vulnerabilities have occurred in the last 50 years, and whose projects are 20 times more likely to have security vulnerabilities. "It's a shame, but what can we do? There really isn't anything we can do to prevent memory safety vulnerabilities from happening if the programmer doesn't want to write their code in a robust manner." At press time, users of the only programming language in the world where these vulnerabilities regularly happen once or twice per quarter for the last eight years were referring to themselves and their situation as "helpless."

David Cahn, Sept '23

Source

SemiAnalysis, an extremely pro-AI semiconductor analyst, ran a test made up of random long-horizon coding tasks until they maxed out the limit on OpenAI and Anthropic’s various subscription levels.

Their findings were shocking.

For $200 A Month, You Can Burn $8000 in Anthropic Tokens or $14,000 In OpenAI Tokens

That’s right. Anyone with a $200-a-month Anthropic subscription can burn $8000 in tokens, and with a $200-a-month ChatGPT subscription, you can burn $14,000 in tokens.

Source

Source

With the current subsidies, all it takes for a user to have a gross margin of at best negative 25% is for them to use as little as 25% of their rate limit.

OpenAI Had $13.07 Billion In Revenue, $34 Billion In Costs and Expenses, and $20.92 Billion In Losses, with a net loss attributable to the company of $38.53 Billion

2025 was the year that OpenAI converted from a non-profit to a for-profit entity, leading to a $41.55 billion loss due to changes in fair value of convertible interests and warrant liability.
...
Ultimately, the net loss attributable to OpenAI in 2025 was $38.5 billion.

At the end of the year, OpenAI had just over $50 billion in assets, with almost half of that in cash.

• Revenue: $13.07 billion
• ...
• Sales and Marketing: $5.73 billion

Leaked internal documents viewed by Where’s Your Ed At reveal that Microsoft intends to pause new signups for the student and paid individual tiers of AI coding product GitHub Copilot, tighter rate limits, and eventually move users to “token-based billing,” charging them based on what the actual cost of their token burn really is.

The document says that although token-based billing has been a top priority for Microsoft, it became more urgent in recent months, with the week-over-week cost of running GitHub Copilot nearly doubling since January.

The move to token-based billing will see GitHub users charged based on their usage of the platform, and how many tokens their prompts consume — and thus, how much compute they use.

But the company got a shock when Anthropic switched it over to token-based pricing in May. “Our spend went up 7x the first day and I’m like, oh shit, we created a monster,” said Busse. “[Large language model] companies have been subsidising all of our usage and now no longer. User-based pricing shelters you.”

it’s now apparent that using AI is more expensive than hiring people, especially since it offers only limited productivity gains at the moment.

Bryan Catanzaro, Nvidia's VP of applied deep learning, recently told Axios that "For my team, the cost of compute is far beyond the costs of the employees", quite an interesting statement from the company selling the shovels for the gold rush.

That perspective is shared by Uber's CTO Praveen Naga, who "[went] back to the drawing board because the budget [he] thought [he] would need is blown away already" as of two weeks ago. Likewise, Swan AI's Amos Bar-Joseph posted a while back on LinkedIn about how proud he was about a $113k bill from Anthropic (makers of Claude) for a four-person team.

Oversimplified math pins that amount that at $28k per person per month, which is likely more than each person's monthly wages. Jokes abound right now that "companies have discovered jobs again," and the humor is backed up by a 2024 MIT study stating that 77% of the time, it was preferable to have humans do the work.

Source

This means that for the AI industry to service its debt, it needs to generate hundreds of billions of dollars in profit each year.

Even giant monopolies like Google don’t make enough profit to service that much debt. AI can’t just be a novelty industry; it needs to replace human labour on a colossal scale to service this debt. Let’s optimistically assume AI one day reaches a 10% profitability margin, a cost parity with human labour, and the ability to complete most jobs (none of which are currently the case). Well, the average US salary is roughly $66,000, so at a 10% profit, the AI company will make on average $6,600 per year per job it replaces. To generate the $309 billion needed to service their debt, the AI industry will need to replace 46.8 million jobs, equivalent to around 27% of the current number of jobs in the US.

While this is all very rough maths, it highlights the implicit bet created by the debt the AI industry has racked up. To simply not default on this debt, the AI industry has to rapidly displace human labour at a staggering scale, even if we are extremely optimistic about AI’s economics.

Wages and salaries averaged $32.60 per hour worked and accounted for 69.9 percent of employer costs, while benefit costs averaged $14.01 per hour worked and accounted for the remaining 30.1 percent.

• OpenAI's Sam Altman said that costs have become a "huge issue" for customers and the company is considering "drastic" price cuts to rein in rival Anthropic PBC’s lead in the corporate market.
• Kyle Orland reported that Anthropic “pauses” token-based billing for its Claude Agent SDK:
  

  Last month, Anthropic announced a billing change that would have substantially increased costs for heavy users of its automation-focused Claude Agent SDK, including many third-party apps. On Monday, though, Anthropic abruptly announced it had paused those pricing changes just as they were set to take effect, allowing Agent SDK users to continue drawing from the more generous usage limits in their existing Claude subscriptions.

• Microsoft Plans June 30, 2026 Shift From Claude Code to Copilot CLI:
  

  Microsoft is reportedly cancelling most Claude Code access for engineers in its Experiences and Devices division by June 30, 2026, shifting teams working on Windows, Microsoft 365, Outlook, Teams, and Surface toward GitHub Copilot CLI as the company tries to rein in internal AI coding costs. The decision is more than a procurement tweak. It is a rare glimpse into what happens when the world’s most aggressive AI software company runs into the same metered-billing problem now facing every large engineering organization.

Footnotes

1. xAI's product is so bad that even their employees won't use it and Musk has said it needs to be re-written from the ground up. So xAI has been reduced to renting its compute infrastructure to its competitors.

Registration is now open for the virtual Digital Library Federation’s (DLF) Forum online, October 14-15, 2026. The DLF Forum is a dynamic gathering place for GLAM professionals to share ideas, sustain critical work, and spark innovation. It connects library, archives, and museum practitioners, as well as other knowledge workers, through intentional community building and collaborative exchange. Learn more about 

The full Forum program is also available to explore, and we’re thrilled to announce our keynote speaker, Dr. Salwa Ismail. Additionally, the application for Digital Storytelling Fellows is now open. 

 Be sure to check out the exciting details and join us in building momentum for what’s sure to be an inspiring experience.

Secure the early bird rate and start planning for yet another memorable online conference with DLF. 

DLF member organizations receive two complimentary registrations for the virtual DLF Forum as part of their member benefits. Not sure who received your code? Email us at forum@diglib.org. 

Register Today

If you have any questions, please write to us at forum@diglib.org. We’re looking forward to seeing you online this fall.

-Team DLF

P.S. Want to stay updated on all things related to the DLF Forum? Subscribe to our Forum newsletter.

The post 2026 Virtual DLF Forum: Registration Open, Program Released, Keynote Announced, Digital Storytelling Fellows Application Open appeared first on DLF.

Applications are now open for the 2026 Virtual DLF Forum Digital Storytelling Fellows program

This year, DLF is launching a new fellowship experience that is directly connected to the Forum’s new Digital Storytelling Presentation session format. The program centers on digital storytelling, emerging technologies, and ethical practice across libraries, archives, and museums, while creating intentional opportunities for participation, reflection, and community engagement in a virtual setting.

We invite early-career and underrepresented practitioners to participate in the 2026 Virtual DLF Forum and help shape conversations about how stories, platforms, technologies, and communities intersect in our work.

Why Digital Storytelling?

Digital storytelling projects, including exhibits, platforms, collections, and collaborative archives, are increasingly central to how cultural heritage organizations document, interpret, and share knowledge. These projects also raise important questions about representation, labor, technology, access, and stewardship.

A cohort of 8–10 Fellows will engage directly with these themes through participation in DLF’s new Digital Storytelling Presentation sessions: interactive, installation-inspired presentations featuring collaboratively developed digital projects and dedicated discussion time. Because this is a new and experimental session type for 2026, the fellowship intentionally builds in structured engagement and feedback to help strengthen the experience and better understand what works in a virtual Forum environment.

Fellows will serve as conversation catalysts during these sessions by contributing questions, reflections, and observations that surface broader themes across the Forum community.

About the New Session Format

40-minute Digital Storytelling Presentation
An interactive session highlighting digital storytelling projects developed through collaborative partnerships. Digital Storytelling Presentations focus on installation-inspired digital storytelling work, such as exhibits, platforms, or collections, designed for immersive and experiential engagement. Sessions may feature up to three presenters, for example, pairing a digital librarian or archivist with a community partner, student, artist, or scholar whose work is represented in, or inspired by, the project. Sessions will be scheduled within 50-minute blocks, leaving dedicated time for Q&A and discussion. Read more about the new session type here: https://www.diglib.org/digital-storytelling-in-practice-a-new-session-format-for-the-dlf-forum/ 

What Fellows Receive

Selected Fellows will receive:

• Complimentary registration to the 2026 Virtual DLF Forum
• A $250 stipend
• Participation in a small cohort of 8–10 Fellows
• A pre-Forum virtual orientation and meet-and-greet
• Visibility through publication on the DLF blog

Fellowship Expectations

Fellows will:

• Participate in a virtual orientation session on October 6, 2026
• Attend the 2026 Virtual DLF Forum on October 14–15, 2026
• Engage actively in Digital Storytelling Presentation sessions by:
  • Asking questions in chat or live discussion
  • Optionally sharing brief verbal reflections during session discussions
  • Incorporating insights from storytelling sessions into their post-Forum reflection
• Participate in a virtual debrief session and/or complete a feedback survey
• Contribute a short public reflection for publication on diglib.org (up to 1,000 words, due November 15, 2026)

Reflections may explore themes such as ethical technology, collaborative storytelling, digital exhibits, community memory, access, or emerging questions around provenance and stewardship.

Selected Fellows must attend the Forum in order to receive the stipend. Apply here. 

Who Should Apply

We welcome applications from:

• Early-career professionals (fewer than 7 years of experience)
• Students and recent graduates
• Contingent, contract, and adjunct practitioners
• Professionals working in under-resourced or capacity-limited institutions
• First-time DLF Forum attendees
• Practitioners whose identities and perspectives are historically underrepresented in digital libraries and cultural heritage spaces

We recognize that professional pathways are not always linear. If you are unsure whether you meet these criteria, we still encourage you to apply.

Application Process

The application process is designed to be straightforward and accessible. Applicants will:

• Answer a few short questions
• Submit a brief personal statement (maximum 4,000 characters)
• Share a link to an online professional profile, if available

Application available here: Google Form. 

Applications Close Tuesday, July 21, 2026, at 11:59 pm ET

The post 2026 Virtual DLF Forum Digital Storytelling Fellows appeared first on DLF.

What happens if I just point a git server at an object storage bucket?

Back when I was porting agent sandboxes to Go, I built everything on top of billy, a filesystem abstraction for Go. The whole trick of the project was teaching a Tigris bucket to act enough like a filesystem that a shell interpreter and its tools couldn’t tell the difference. Billy was the key layer that made the entire façade fall into place.

After I had gotten things working, I learned that I’m using billy way outside its normal usecase. It was originally made for go-git, a pure-Go implementation of git’s protocols and data formats. It doesn’t rely on the /usr/bin/git binary existing at all. Every method on billy’s filesystem interface exists purely because go-git needs it. This gave me a terrible idea: I already have a bucket that can quack like a filesystem and go-git’s native language is “filesystem”.

Can this Just Work™? Let's find out.

Git was always an object store

If you strip away the porcelain, a git repository is 4 basic things:

• Objects, or compressed blobs of data. Most of the objects in any individual repository are files.
• Trees, or objects that map to other objects. TL;DR: trees are folders.
• Commits, or objects that point at one tree and their parent commit. This lets you pin down which files belong to one logical change set.
• Refs, branches and tags, they are tiny mutable pointers into the pile of objects.

For example, let’s say I just made a new git repository and committed a README.md to it. The tree for the .git folder looks something like this:

$ tree .git
        .git
        ├── COMMIT_EDITMSG
        ├── config
        ├── HEAD
        ├── index
        ├── objects
        │   ├── 5e
        │   │   └── b8151eb669aa4467b6dea2c4bce19183cd0b41
        │   ├── 6a
        │   │   └── 6a8ecfcae2632152486aca3d9150ef83dedd66
        │   ├── f4
        │   │   └── d2487a1c6d742c8037c0296ddf80625190bd80
        │   ├── info
        │   └── pack
        └── refs
            ├── heads
            │   └── main
            └── tags
        

As you can see there are three objects. One of them is the commit 5eb8151eb669aa4467b6dea2c4bce19183cd0b41, the next is the tree, and the last one is the README file. The main branch also points to that commit:

$ cat .git/refs/heads/main
        5eb8151eb669aa4467b6dea2c4bce19183cd0b41
        

The cool part is that half of this is content-addressed. The content-addressed bits never change once they’ve been committed. Git objects are a great fit for Tigris’ internal model because they are append-only storage, just like the fundamental model Tigris is built upon. The things that do change often are the refs, which are updated to point to the latest commit. These are tiny files though, which means that Tigris can handle them with no effort required.

However, when we host git repositories on a server, we end up creating single points of failure. Our git repos are hosted on single machines that can and will break. The entire implementation relies on git objects being 1:1 correlated with filesystem objects because everyone (even GitHub) shells out to the git binary to actually store files. Hosting git repos becomes one of the most stateful services in our stateless cloud-native environment.

Sure git is in-theory decentralized, but most of us have ended up using that to put our git repositories in one big store that has questionable uptime practices: GitHub. To be fair to hubbers, GitHub operates at a scale that none of us can really think about. They’ve been pushing the limits since their inception where they had to get Engine Yard to keep building them bigger servers to handle the load. They have to do everything with a big mounted filesystem because git’s tooling gives them no other option.

A travesty of horrors beyond human comprehension

Now suppose this weirdness bothers you enough to do something about it. To build a git server without storing everything in the local filesystem, you have to speak git somehow, and the conventional options aren’t really all that great:

• If you shell out to the git binary, now your “library” is the argv of the git process and your error handling is screen-scraping output. Internally, git implements its functionality with a billionty subcommands rather than exposing it all as a library. The codebase is held together by load-bearing calls to die(), which kills the process.
• If you link into git’s guts with libgit, you inherit the “when things go bad, die()” behaviour and your app now suddenly starts crashing at random. This is not good for uptime.
• If you try to use libgit2 (the rewrite-that’s-actually-a-library), you have to reckon with the fact that it’s addled by the GPL (with a linking exception, try explaining that to your lawyers), you have to eat the jump to C every time you do anything with git (very often), development has stalled, the Go bindings have been archived, and it still assumes a local filesystem despite assurances it does not.

It might sound hopeless, right? You may be able to use WebAssembly or something to contain the madness (assuming you have a good way to implement fork()/exec() or posix_spawn() or something similar), but what if there was a pure Go library that could handle this all for us?

Enter go-git, a pure-go implementation of the git protocol and internals from scratch. This doesn’t rely on cgo or /usr/bin/git and it does not assume the repositories are stored in the local filesystem. Its storage interface is written against billy, the exact interface I’ve already taught to speak Tigris. I wanted a git server that was just in a bucket and the pieces were sitting there and calling to me.

Oh no, it works

So I hacked up objgit, a git server backed by object storage. The only filesystem call I had to add to get it booting was MkdirAll. I wired up the transport package to a socket to implement the plaintext git protocol, hooked it up to a bucket, and pushed the repo I was currently working on.

To my absolute astonishment, it worked.

Git pushed, pulled, logged, blamed, tagged, the whole kit and kaboodle. I didn’t have to implement git myself, I just committed an egregious amount of shoving a square peg into a round hole until the peg went in.

In hindsight this makes an annoying amount of sense. A bare repo is those four kinds of things on a filesystem; swap the filesystem for object storage and everything else Just Working™ is perfectly logical. Git’s on-disk format is its database schema and if you fake open/stat/rename convincingly enough the entire façade keeps working because APIs are the lies we tell ourselves to make us sleep at night.

After a lot of hacking, I ended up with a feature list kinda like this:

• Push and pull over three transports: HTTP, classic git://, and SSH
• Repositories upserted on first push
• Absolutely no effort put into authentication as this is an experiment and authentication is annoying and complicated
• Prometheus metrics so I could optimize the filesystem layer

Everything comes out of one Go binary with no local state, even the generated SSH keys are stored in the bucket. You can run this in a Kubernetes cluster with only the mutable storage required being temporary files for an optimistic cache when doing smart git clones.

The rest of this post is what it took to get from “oh no, it works” to something close to usable.

Obligatory disclaimer (like the best things in life): this is an experiment. It has not been tested thoroughly or vetted for correctness. If it breaks in half, you get to keep both pieces. Please do not move your company’s monorepo onto this and then email me when it catches fire.

That one POSIX idiom that survived

Git is paranoid about durability, and its entire strategy is one Unix idiom that you end up seeing many places: write new data to a temporary file and then rename(2) it into place after you’ve assured it’s correct. POSIX guarantees that rename is atomic, so readers either see the old file or the new one, not an intermediate state inbetwixt the two. Packfiles (bundles of objects) land as temporary files when uploaded then moved to their permanent home. Refs are written as locked temporary files and then renamed over the ref. It’s rename all the way down.

Object storage traditionally does not have rename as one atomic operation. S3’s answer is to create exactly that intermediate state: CopyObject to the new place and DeleteObject on the old one. This makes the most load-bearing idiom in Git’s philosophy fall to pieces.

Luckily, Tigris has an extension for this: RenameObject. To use it, pass an additional X-Tigris-Rename: true header to a CopyObject call and instead of copying then deleting on the client, it moves the metadata around on the server. One round trip, no data movement, and the Unix idiom maps on the bucket 1:1. Objgit’s implementation of Rename is trivial:

// internal/s3fs/basic.go
        // RenameObject is a Tigris extension that renames in place (no data copy),
        // so we don't need a separate CopyObject + DeleteObject.
        copySource := fs3.bucket + "/" + src
        _, err := fs3.client.RenameObject(ctx, &s3.CopyObjectInput{
            Bucket:     &fs3.bucket,
            CopySource: &copySource,
            Key:        &dst,
        })
        

A second, sneakier violation hides in the same codepath. When go-git writes a temporary file, it creates that temporary file and then immediately starts opening it for reading so it can build the pack index. You cannot do that with a single live object in any object storage system, you are either reading or writing, never both. I ended up working around this by cheating a bit and buffering the contents of newly written pack files into memory so that this game of chicken kept working. I may have to change this to write that pack cache to the filesystem as trying to push gcc.git made me run out of RAM. At the very least, everything lies consistently enough that git doesn’t care, so win!

Death by a thousand stat() calls

With this correctness sorted, I tried pushing the golang/go repository to objgit to see how long it would take. It did work, but it took forever. Using the prometheus metrics I mentioned before, I saw that it was making biblical amounts of HeadObject calls. Some blocking profile analysis pointed to the fact that the git library was using the stat() call to detect if a file exists. The flow was like:

• Client has object x
• Check if object x exists
• Check if any pack has object x

And so on ad infinitum. This is fine-ish on a local filesystem because those syscalls resolve in microseconds, not the tens of milliseconds it takes to get from my office to the nearest Tigris region (please expand to Ottawa, I would love that so much).

This was compounded with a discovery that the transport I was using (SSH — classic git:// shares the same code path) was exploding every packfile into loose objects when pushing it. Each loose object write was costing two round trips: stat() to check if a file exists and then open() / write() to actually put the data into Tigris. This made a 100,000 object packfile cost 200,000 object storage calls. Call it 10ms of latency for each one, and that’s over half an hour of waiting for responses that mostly say “404 not found”.

Caching can’t really save you here either, read caches would absorb the repeated reads; but this is a firehose of writes to 100,000 paths that probably have never been read and likely will never be seen again.

The reason only two transports had this problem is a deadlock story. The git library's fast path stores an incoming pack whole through its PackfileWriter, by copying from the connection until io.EOF. Over HTTP that's fine: the request body ends, EOF arrives, everyone goes home. Over git:// and SSH, the connection is a persistent socket and the client is holding it open, politely waiting for the server's status report. EOF never comes. The copy waits forever, the client waits forever, and you have invented a distributed deadlock with two participants. The original workaround was to hide the PackfileWriter capability on those transports so go-git fell back to its streaming parser that writes every object loose. Hence the stat storm.

So the solution was to stop depending on EOF at all. Packfiles are self-delimiting: the header says how many objects are coming and a trailing checksum marks the end, so a packfile scanner walks the stream and stops at the trailer while a TeeReader mirrors exactly those bytes into the PackfileWriter. This makes the rest of the façade fall into place and the git library is happy. This made pushes into two uploads: a packfile and its index instead of a torrent of round trips that mean nothing.

What about cloning?

Once I got pushing fixed, I moved on to the read path. In order to emulate ReadAt, I used ranged GetObject requests so that the git library could read individual objects out of packfiles. I was happy with this hack, but there was one problem: the latency curse struck again. Cloning a simple repo with 318 objects and a 200KiB packfile made over 8,500 GetObject calls before I killed it.

A git client cloning a repository reads repository packfiles thousands of times with random access, walking objects and candidate delta bases over and over. On a local disk you never notice because your page cache eats that for breakfast. When every call is an HTTP request, a 200KiB repo turns into dozens of megabytes of round trips. A 20MiB repo was effectively unservable.

In other words, I had un-cached the one workload that caching was designed to solve.

The fix leans on a gift from git: pack files are immutable and content-addressed. pack-<sha>.pack will never change for as long as it exists. This makes them trivially cacheable to a faster local medium, such as the filesystem. No invalidation logic is required. I made objgit download packs to a local temporary folder and serve reads from there. To be on the safe side, I did add least-recently-used caching to the mix so that my temp folder wouldn’t blow up unexpectedly. This does mean that the first request for pack files is slower, but then everything else is at filesystem speed.

Yes, this relies on the local disk again, but only as a cache that can and will be thrown away. I think trading a stateless ideal for clones that terminate in reasonable amounts of time is a worthwhile bargain.

Why so ListObjectsV2, Batman?

Once the other disasters were out of the way, one more remained: the metrics showed a flood of ListObjectsV2 calls every time a clone was made and didn’t stop making those calls after it was done.

Two things compounded. First, when git looks up an object that isn't packed, it probes for a loose object at objects/<xx>/<rest-of-hash>. objgit keeps packs whole, so there are no loose objects, so every probe misses, and each miss across a distinct two-hex prefix triggered a directory listing to find out. There are 256 possible prefixes. A single clone could issue up to 256 ListObjectsV2 calls whose collective answer was a resounding "there is nothing here."

Second (and more embarrassing), the listing cache already had an optimization for this. It collapsed entire subtree lookups into recursive scans so a single listing of the repository could answer every stat() and probe beneath it. It was completely dead in production. The cache matched recursive prefixes against the repo root (refs/), but every repo is chrooted to its own directory, so real keys look like myrepo.git/refs/heads/main. The prefix check wasn’t aware of chroots so it never actually matched anything. Nobody noticed because a cache that degrades to “no caching” still returns the correct answer, just slowly. To rub it in, a cache warmer was dutifully re-listing every one of those useless prefixes every 30 seconds for 10 minutes after each clone. Thousands of background list calls were burned in the service of caching nothing of use.

The fix was insultingly small: when a repo’s filesystem gets chrooted, register that chroot as a recursive subtree root within the cache. This made the cache actually useful and resulted in only one ListObjectsV2 call instead of hundreds. Every sufficiently advanced cache is indistinguishable from a no-op until someone graphs the miss rate.

None of these disasters were exotic. They’re the things filesystems and kernels give you for free — and every perfectly reasonable disk assumption fell to pieces once a network round trip sat at the core. Serving Git repositories is an accidental filesystem latency benchmark. If your storage abstraction has a weak point, Git will find it and the metrics will show you where that problem is.

Post-receive hooks go in clown jail

One of the most useful parts of hosting your own git server is setting up post-receive hooks. These have been used since time immemorial for things like automatic deployments when you push code to the server. The heart of this is how we get systems like GitHub Actions: it’s code that runs when you are done pushing.

When you push to objgit with --allow-hooks enabled, it looks for a post-receive hook in .objgit/hooks/receive-pack (this corresponds to the git plumbing action, the name can and will be changed) in the tree of the commit you just pushed. It will then spin up a kefka sandbox with a checkout of the git repository at the commit you just pushed mounted at /src and mutable temporary files at /tmp. It gets coreutils and nothing else. No host filesystem, no network, no arbitrary binaries. Output streams back into the pusher as remote: lines just like when you git push heroku main. Eventually I want to make custom commands to allow you to deploy Tekton pipeline changes and kick off CI jobs that way, but for now I’m happy with this working at all.

You can’t implement policy using these hooks yet. I’m working on it.

Now what?

I taught a bucket to speak git. Where this goes next, roughly in order of how much the ideas keep me up at night.

CI is the obvious next step. I would wire up commands for things like “apply kubernetes object” and “create tekton pipeline run” so that CI would run via your friendly neighborhood Kubernetes cluster and then notify you through some reasonable mechanism. That’s the first thing I’ll build when I have the time.

It would be nice to have a web UI for this, which is complicated for reasons that have nothing to do with git trees, object storage, or anything else and everything to do with the current state of the internet. Git lookups are expensive in the best cases and with the current torrent of unethical scraping ransacking git servers for every scrap of RAM they have, it’s probably a bad idea to implement this without a lot of clever optimizations. Maybe the fact that this doesn’t have load-bearing dependencies on /usr/bin/git would make it more resilient against scrapers. The fact that this is based on object storage could also mean that caching would be a bit easier (having basically unlimited storage is kind of a low-key superpower for caching), but then the main issue would be server load. It’s a tough cookie to handle.

Performance and stability are another place this needs to improve. I’ve tested this on my developer workstation but that is far different from testing it in production. There’s some other performance issues that are easy to fix, but the big one is latency to Tigris. Maybe I can get the devops team to set me up a k3k cluster in production.

Right now this is an experiment as I plug along and feel out the shape of what git-on-object-storage can be. A git server with no disk, no git binary, and no database. If you want to take a look, check it out on GitHub.

Last month, I had the great pleasure of keynoting the CALM (Conference on Academic Library Management) Conference, which is consistently one of my favorites. The video of my talk, Slow Management in a Fast World, is available below for those who would like to check it out! You can also view my slides here which include a long bibliography of works that influenced my talk at the end.

Many thanks to all the amazing folks who organized this conference; it was such an honor and a pleasure to be part of it!

Midsummer sunrise

The sun peeking over the horizon of the Atlantic Ocean. A globe of firey starstuff over calm waves, orange and red spreading out into the sky.

This time, we learned from initiatives around the world that have breathed new life into dusty archives using both open and non-open technologies

The post Roundtable Recap: How to digitise a physical archive to work strategically with AI? first appeared on Open Knowledge Blog.

The Evergreen release team is pleased to announce that point releases for 3.17.2 and 3.16.8 are available.

These releases contain updates for Evergreen’s RESTful API suite, search fixes, and Angular interface updates.

Files and release notes are available on the Downloads page: https://evergreen-ils.org/egdownloads/

Thanks to the June release team: Galen Charlton (Equinox), Gina Monti (Bibliomation), Sarah Moody (ECDI), Andrea Buntz Neiman (Equinox), and Chris Sharp (PINES); as well as everyone who contributed fixes and testing to this release.

Anubis is about to get WebAssembly-based proof of work checks so that administrators can use a non-SHA256 proof of work method to protect their websites. Part of the implementation goals of this work is that the check logic is defined in one place on both client and server. The client and server will then hook into the WebAssembly in order to make sure they're running in lockstep.

However, one small problem comes up. What do you do when the client has WebAssembly disabled? I really don't want to de-facto lock people out of websites. Anubis exists in an impossible balance of user experience, administrator experience, and developer experience and any change to any of these factors disrupts the balance for other factors.

To work around this and also fulfill the goal of having check logic defined once, I decided to take inspiration from the legendary talk The Birth and Death of JavaScript and just recompile the WebAssembly to JavaScript. Sure, the resulting JavaScript will be slower than the equivalent WebAssembly (even more so because disabling WASM usually disables the JavaScript JIT, the thing that makes JavaScript fast), but it will finish eventually. Hopefully it will be more efficient than the existing JavaScript is on lower end hardware, but research is required.

Luckily enough, the tool I need (wasm2js from the binaryen project) is packaged in Linux distributions. The bad news is that distributions ship ancient versions of it that don't get the same output as the version on my development machine's copy from Homebrew.

In order to really make sure that the output of this is deterministic (essential for reproducible builds), I need to bundle a copy of wasm2js. So I did that by building a version of wasm2js compiled to WebAssembly with wasi-sdk. The rest of the article is the tale of reproducibility woe that lead to the implementation I ended up with. Buckle up and enjoy the ride!

Reproducible builds are surprisingly hard

Aoi

Back up a sec, this doesn't make sense to me. If you have the same bytes of input to a compiler, you should get the same bytes of output assuming that the compiler flags, target, and other platform details are controlled for right? A compiler is just a deterministic function of input source code becomes output bytecode, right?

Numa

lol you'd think, but no, it's not. In theory it is (and for small scale compilers it definitely is), but in practice compilers are strange and complicated beasts containing multitudes that no mere mortal can fully comprehend on their own.

There are a shocking number of ways to accidentally create nondeterministic output when doing C/C++ development. One of the easiest is to use the builtin __DATE__ and __TIME__ macros to stamp a build with the time the compiler was executed at:

// hello.cpp
        
        #include <iostream>
        
        int main() {
            std::cout << __DATE__ << " " << __TIME__ << std::endl;
            return 0;
        }
        

Building and running it once gets me this:

$ make clean && make hello.wasm && wasmtime run -W exceptions=y ./hello.wasm
        rm -f hello.o hello.wasm
        wasi-sdk-33.0-x86_64-linux/bin/wasm32-wasip1-clang++ -O3 -fwasm-exceptions -mllvm -wasm-use-legacy-eh=false  -c hello.cpp -o hello.o
        wasi-sdk-33.0-x86_64-linux/bin/wasm32-wasip1-clang++ -O3 -fwasm-exceptions -mllvm -wasm-use-legacy-eh=false  -fwasm-exceptions -lunwind --no-wasm-opt hello.o -o hello.wasm
        Jun 18 2026 00:00:59
        

Another time it gets me this:

$ make clean && make hello.wasm && wasmtime run -W exceptions=y ./hello.wasm
        rm -f hello.o hello.wasm
        wasi-sdk-33.0-x86_64-linux/bin/wasm32-wasip1-clang++ -O3 -fwasm-exceptions -mllvm -wasm-use-legacy-eh=false  -c hello.cpp -o hello.o
        wasi-sdk-33.0-x86_64-linux/bin/wasm32-wasip1-clang++ -O3 -fwasm-exceptions -mllvm -wasm-use-legacy-eh=false  -fwasm-exceptions -lunwind --no-wasm-opt hello.o -o hello.wasm
        Jun 18 2026 00:01:11
        

Even though the source code had the same bytes, the output of the compiler was wildly different.

In order for users and packagers to trust the binaries of wasm2js I'm committing to the Anubis repo, I need to make sure that you can build the same version I built, down to the same bytes. For an added bonus, you should be able to build this on your machine and get the same bytes I got.

Numa

That sure does sound like a great ideal, it would be horrible if something unforeseen came up to ruin it!

Clang silently runs wasm-opt from $PATH behind your back

Among other tools like wasm2js, binaryen has a bunch of other useful tools such as wasm-opt. wasm-opt optimizes WebAssembly compiler output to let you eke out more performance. This doesn't work in every circumstance, but when it does work it makes a huge difference. As such, clang shells out to wasm-opt when doing builds.

This normally makes sense, but in this case it caused builds to fail on my DGX Spark because its version of wasm-opt is too old:

$ uname -m && which wasm-opt && wasm-opt --version
        aarch64
        /usr/bin/wasm-opt
        wasm-opt version 108
        

Compared to my workstation which installs wasm-opt from Homebrew:

$ uname -m && which wasm-opt && wasm-opt --version
        x86_64
        /home/linuxbrew/.linuxbrew/bin/wasm-opt
        wasm-opt version 130
        

Turns out that wasi-sdk and binaryen rely on the WebAssembly Exceptions extension. This is a reasonable thing to assume given that wasi-sdk mostly assumes you're building things for web browsers and 93.86% of browser users have a browser engine new enough to support it. C++ is also one of the main places where exceptions are used, so I guess WebAssembly-native exception handling removes a lot of boilerplate here.

Both wasmtime and wazero require you to flag into exception support. This is fine; we can just pass -W exceptions=y to wasmtime and use a custom runner harness for wazero. The annoying part is what happens when my arm machine's anemic build of wasm-opt sees exception handling instructions, causing it to exit. This made the build fail.

The solution was to pass --no-wasm-opt at the linking step. This removed one angle of irreproducibility.

Mara

I guess in the future we could make it use the version of wasm-opt it just built to optimize the output, but that may be a premature optimization for now.

Clang relies on address layout for ordering things

The version of clang that I use to compile wasm2js has some address-sensitive code generation hiding in its exception handling path. Raw pointer values leak into the order a handful of try_table blocks come out in. This surfaces as every build differing from the next by about 29 bytes:

-002a9af0: 2802 0441 0647 0d00 1f40 0103 0820 0241  (..A.G...@... .A
        -002a9b00: 206a 2103 2002 4138 6a20 0141 086a 10b5   j!. .A8j .A.j..
        -002a9b10: 8881 8000 2104 0b1f 4001 0304 2003 2004  ....!...@... . .
        +002a9af0: 2802 0441 0647 0d00 1f40 0103 041f 4001  (..A.G...@....@.
        +002a9b00: 0309 2002 4120 6a21 0320 0241 386a 2001  .. .A j!. .A8j .
        +002a9b10: 4108 6a10 b588 8180 0021 040b 2003 2004  A.j......!.. . .
        

To make this easier to spot, here's a partial disassembly:

  i32.load  offset=4            ;; 28 02 04
          i32.const 6                   ;; 41 06
          i32.ne                        ;; 47
          br_if     0                   ;; 0d 00
        - try_table (catch_all_ref 8)   ;; 1f 40 01 03 08
        + try_table (catch_all_ref 4)   ;; 1f 40 01 03 04
        + try_table (catch_all_ref 9)   ;; 1f 40 01 03 09
            local.get 2                 ;; 20 02
            i32.const 32                ;; 41 20
            i32.add                     ;; 6a
            local.set 3                 ;; 21 03
            local.get 2                 ;; 20 02
            i32.const 56                ;; 41 38
            i32.add                     ;; 6a
            local.get 1                 ;; 20 01
            i32.const 8                 ;; 41 08
            i32.add                     ;; 6a
            call 17461                  ;; 10 b5 88 81 80 00
            local.set 4                 ;; 21 04
          end                           ;; 0b
        - try_table (catch_all_ref 4)   ;; 1f 40 01 03 04
            local.get 3                 ;; 20 03
            local.get 4                 ;; 20 04
        

The computation is nearly identical, but the byte order is just different enough to also make the catch references differ. This also fires when you build this pinned version of wasm2js on arm64 machines because its pointer iteration order is different from it is on my workstation.

To work around this, I took two steps:

1. Disable address-space randomization for this build using setarch --addr-no-randomize.
2. Create known good sha256 checksums for both x86_64 and arm64 via building this program on machines I trust.

I also made a CI job ensure this:

- name: Ensure reproducibility
          run: |
            cd ./utils/wasm/wasm2js
            ./build.sh
            if sha256sum -c --status shasums.x86_64; then
              echo "OK: rebuilt modules match the recorded x86_64 checksums"
            elif sha256sum -c --status shasums.arm64; then
              echo "OK: rebuilt modules match the recorded arm64 checksums"
            else
              echo "::error::rebuilt wasm2js/wasm-opt match neither recorded checksum set on ${{ matrix.runner }}" >&2
              sha256sum wasm-opt_130.wasm wasm2js_130.wasm
              exit 1
            fi
        

To be extra sure, we have this job run on both x86_64 and arm64 hosts. I'd really love to have this be reproducible across hosts, but that's an upstream LLVM bug that I am not powerful enough to tackle. If you work on LLVM and are reading this, it would be nice to set a seed of some kind to ensure that this iteration order is fixed across architectures.

At the very least builds are deterministic within architectures. This may have to be good enough for now.

Collaboration. Serendipity. Diversity. These are the qualities that come to mind when I think about this year’s OCLC quilt and the community that created it.

The OCLC Quilters, a group of current and retired OCLC employees, have spent months creating a quilt of 120 cross-cut blocks to donate to the silent auction held during the ALA 2026 Annual Conference. The ALA BiblioQuilters annually host this auction as a fundraiser for the Christopher Hoy Scholarship, which awards a $5,000 scholarship each year to a U.S./Canadian citizen or permanent resident who is pursuing an MLS in an ALA-accredited program.

This is the fourth year in a row that the OCLC Quilters have donated a quilt to the silent auction. Their work inspired me to take up sewing about a year ago, and I’m proud to move from admirer to participant, contributing to the OCLC quilt for the first time. Although left-handed people are about 10% of the population, three of the 13 people contributing to the OCLC quilt, including myself, are left-handed. While that doesn’t affect the result, it requires a few adjustments in technique and having the appropriate scissors. Sharing advice on adapting equipment and shopping for left-handed supplies is one of the ways we support each other.

Nine of the 13 contributors to the OCLC Quilt for the ALA 2026 Annual Conference

Like all handicrafts, quilting is an activity with its own nomenclature. As a quilter and cataloger, I found myself wondering: “What controlled vocabulary terms could I use to describe the OCLC quilt?” There are several from vocabularies such as Library of Congress Subject Headings (LCSH) and Getty Art and Architecture Thesaurus (AAT). These are listed at the end of this blog.

A quilt is created from many elements that may not be individually significant but form a meaningful whole, just like a WorldCat bibliographic record. The blocks of the quilt function like data elements in a WorldCat record, with contributions from multiple individuals creating the larger work.

Assembling the quilt

OCLC quilters sewed 120 blocks, which are the fabric squares comprising the quilt’s front. The blocks are a cross-cut design—a pattern chosen because it is accessible for novice sewists and makes good use of small fabric pieces. Quilters often save these leftover pieces, called “scraps,” from other projects for future use. Reusing scraps makes quilting a sustainable craft, and quilters often share them with one another. An experienced OCLC quilter, who keeps her scrap collection organized in true librarian fashion, donated most of the fabric pieces used for the blocks.

Experienced quilters arranged and sewed the blocks together and cut the batting (soft material used between the front and back sides of the quilt). The next step, in which three layers are sewn together with a decorative stitch, is quilting. This is the strict definition of the term “quilting,” although it is often used to refer to the entire process of creating a quilt. The pattern used for the quilt stitching is called “modern ties,” and it looks a bit like tied shoelace loops.

An OCLC logo is incorporated into one of the quilt blocks

The final step is to sew a long strip of binding fabric around the edges of the quilt, which will keep the ends from fraying as well as being decorative. Two labels were sewn into the binding: “Made in OH” and “Is it perfect? No.” Both of these labels are accurate descriptions of this quilt, but unlike in bibliographic descriptions, a certain amount of imperfection is not only tolerated but may be considered part of the quilt’s charm.

A quilting tradition at ALA Annual

The OCLC quilt will be one of many available at the ALA BiblioQuilters silent auction during ALA Annual in Chicago, Illinois. The BiblioQuilters were founded at the 1998 ALA Annual Conference in Washington, D.C. Since 2000, the BiblioQuilters have had a silent auction of quilts every year except 2020 and 2021 (because of the pandemic). The quilts are usually available to view and bid on near the registration area. If you are attending ALA in Chicago, I highly recommend you visit the auction table to view them. After ALA, you may be inspired to browse the shelves of your local public library for 746.46, the Dewey Decimal number for quilting.

Subject vocabulary terms

For those readers who appreciate quilting and metadata, the following controlled vocabulary terms reflect concepts discussed in this blog. You might even find it fun to match the concepts to the natural language descriptions!

Getty Art and Architecture Thesaurus terms

batting

binding (textile material)

blocks (quilt components)

fabric scissors

quilting

Library of Congress Subject Heading terms

Quilting

Sewing—Left-handed techniques

Textile fabrics

The post 120 blocks, one story: The collective creation of the OCLC Quilt appeared first on Hanging Together.

LibraryThing is pleased to sit down this month with groundbreaking author and poet Cynthia Pelayo, who in 2022 became the first Puerto Rican and first Latina to win a Bram Stoker Award after her Crime Scene took the prize in the Poetry Collection category. Her Into The Forest And All The Way Through was a 2020 nominee, also in the Poetry Collection category, while her Children of Chicago was a 2021 nominee in the Novel category. Pelayo earned a BA in Journalism from Columbia College Chicago, a MS from Roosevelt University, and a MFA in Writing from the School of the Art Institute of Chicago. She is currently pursuing a PhD in English. Her MFA writing thesis, Lotería, was republished in 2023, winning an International Latino Book Award Silver Medal in the Best Collection of Short Stories category. A co-publisher of Burial Day Books, which focuses on horror writing, she is the author of numerous other books, stories and poems, including novels such as The Shoemaker’s Magician (2023), Forgotten Sisters (2024), and Vanishing Daughters (2025). Her new novel, It Came from Neverland, a work of horror inspired by the classic Peter Pan, was published by Crooked Lane Books earlier this month. Pelayo sat down with Abigail this month to discuss her new book.

Tell us a little bit about It Came from Neverland. How did the idea for the story first come to you?

Like many people, I grew up watching the Disney version of Peter Pan, and then I remember watching Hook with Robin Williams and being captivated, seeing Peter Pan as an adult who had to remember who he was. There was an older Wendy in that film, and that always stayed with me because I really wanted to know what Wendy’s story looked like aged into young adulthood.

When I went back to J.M. Barrie’s Peter and Wendy and the original play, I worked out that Wendy would be in her early twenties at the start of the First World War. And then I learned that many young men at that time lied about their age to enlist, some of them were barely more than boys. It all felt like a perfect juxtaposition, these boys going off to war and Wendy’s trauma round caring for the Lost Boys from Neverland.

A woman of that time period would certainly not be believed if she tried to tell the truth about what she experienced as a child in Neverland, and so that certainly played into her experience. Then I thought well, Wendy at this age would certainly be in a position that reflected her character, and schoolteacher fit perfectly. The story wrote itself, because I knew that Wendy would do all that she could to protect those children and I also knew Peter Pan would surely return to whisk more children away to Neverland. So this story is that tale, what does she do to stop Peter Pan.

What drew you to Peter Pan, and what made you feel it was horror?

Peter Pan without Wendy Darling is just a boy screaming into the dark. The story only works because Wendy agrees to go with him to Neverland, and she goes because she is sweet and kind and she believes him. Wendy’s only failure here was that she had a good heart, which is just so sad because her being nice is why she was taken advantage of. She trusted someone who was manipulating her. Peter told her she was special, but what he really meant was that she was useful. She was given the role of mother to the Lost Boys, not because she was truly loved or valued, but because someone needed to do the mending. This is not fantasy. This is a domestic horror story dressed in fairy dusty.

In Peter and Wendy we’re also essentially told that growing up is a curse, but I push back on that. Growing up is the adventure, becoming yourself, and gaining autonomy is the gift.

Peter’s entire pitch was stay here, never change, never leave me. Shrink. Lost yourself to praise me. That’s not love. That’s control.

The horror was always there. I just removed the glitter.

What is it about fairy tales that speaks to you?

Fairy tales were the very first stories I was told as a child. “Little Red Riding Hood,” “Hansel and Gretel,” “Cinderella,” more. I hold all of them dear, even though many of them have a thread of terror, but I suppose that’s why I’m the writer I am today.

What I’ve come to understand is that fairy tales are an early societal warning system, in a way. They prepared us for danger, and that’s why they still work. Little Red Riding Hood is everyone who has been told not to talk to strangers on their way home. Bluebeard is everyone who has been warned to be cautious with suitors. Snow White is every person who has fallen victim to the cruelty of jealousy. These stories survived centuries because beneath them there is some truth that can be applied to many of our experiences today. They encode the things that we should say out loud, but don’t because of all of those strange polite society rules, things like – don’t trust the stranger who flatters you, the beautiful thing is likely the trap, or even, the person who promises you forever can very well be the one who seeks to destroy you.

When looking at all of these through a horror lens, they echo to the horror writer what is our job – and that is to tell the truth. Horror is the genre of truth, to highlight the danger, to be a witness to survival, more. So much of what fairy tales do speak to this.

Are there other classic works you’re interested in transforming?

Yes, and the one upcoming is a Frankenstein retelling titled Everina from Union Square & Co. I have more, but I can’t mention those quite yet.

Tell us a little bit about your writing process.

I generally write in the early morning. In the evenings that’s when I tend to answer email or work on lectures for any workshops I’m teaching or work on any of my own homework.

In terms of my actual writing sessions, I read before I write, generally I will read some poetry and then start writing. If you’re asking about the big questions regarding how do I create something, I think I’m a pretty methodical writer in that yes, I allow discovery to happen, but as of recent, there is a lot of researching, planning, and pre-writing that goes into my actual writing. Then there is editing, and that is an entirely different process. I tell writers to think of these processes as three separate demands, research and preparation requires one aspect of your brain, editing requires a different aspect of your brain and the actual writing, which is a completely different process. When you are writing you are creating, allow yourself to have fun and explore when you’re actually writing.

What comes next for you?

Something Followed Us Home: Tales of Latiné Horror comes out September 29 from Simon & Schuster. It’s an anthology I edited that features Mariana Enríquez, Agustina Bazterrica, Mónica Ojeda, Isabel Cañas, Daniel José Older, Zoraida Córdova, and others, with a foreword by Brenda Lozano. Latiné horror isn’t a subgenre, it’s a tradition that I’m grateful to have had the opportunity to share with readers.

After that, Everina, which is my Frankenstein retelling.

Tell us about your library. What’s on your own shelves?

I read a lot of poetry and classics. My bookshelves reflect that, so we have Lorca, Vallejo, Pizarnik, Anne Sexton, Gwendolyn Brooks, Adrienne Rich, Dickens, Dostoevsky, Homer, Steinbeck, Tolstoy, more. Yes, there’s horror as well, Shirley Jackson, Angela Carter, Mariana Enríquez, Carmen Maria Machado, Daphne du Maurier. And, of course fairy tales, so many fairy tales and non-fiction texts analyzing fairy tales.

What have you been reading lately, and what would you recommend to other readers?

I am reading Piranesi by Susanna Clarke. I love world building, and here, we have a world that operates on its own logic, the architecture of the space, and the feelings it evokes. Clarke gives us infinite halls that become both prison and sanctuary, and it’s this tension that I’m drawn to as both a reader and writer.

Join OKFN and the UN Big Data Regional Hub to learn about our ongoing experiments integrating AI to public open data portals

The post [Webinar] Traceable AI for Public Data: An Open Model for Latin America first appeared on Open Knowledge Blog.

Source

Rockets, unlike other anthropogenic pollution sources, emit gaseous and solid chemicals directly into the upper atmosphere. We compile inventories of these chemicals from rocket launches in 2019 and projections of future growth and speculative space tourism activity. We incorporate these in a 3D atmospheric chemistry model to simulate the impact on climate and the protective stratospheric ozone layer. We find that loss of ozone due to current rockets is small, but that routine space tourism launches may undermine progress made by the Montreal Protocol in reversing ozone depletion in the Arctic springtime upper stratosphere. The BC (or soot) particles from rockets are also of great concern, as these are almost five hundred times more efficient at warming the atmosphere than all other sources of soot combined.

So far, models of spacecraft reentry have focused on understanding the hazard presented by objects that survive to the surface rather than on the fate of the metals that vaporize. Here, we show that metals that vaporized during spacecraft reentries can be clearly measured in stratospheric sulfuric acid particles. Over 20 elements from reentry were detected and were present in ratios consistent with alloys used in spacecraft. The mass of lithium, aluminum, copper, and lead from the reentry of spacecraft was found to exceed the cosmic dust influx of those metals. About 10% of stratospheric sulfuric acid particles larger than 120 nm in diameter contain aluminum and other elements from spacecraft reentry. Planned increases in the number of low earth orbit satellites within the next few decades could cause up to half of stratospheric sulfuric acid particles to contain metals from reentry.

Most of the meteoric mass is deposited at altitudes between 75 and 110 km by a very large number of sub-millimeter meteoroids. Reentering spacecraft, which are larger and moving more slowly, ablate between 40 and 70 km over a ~300 km long footprint

This paper investigates the oxidation process of the satellite's aluminum content during atmospheric reentry utilizing atomic-scale molecular dynamics simulations. We find that the population of reentering satellites in 2022 caused a 29.5% increase of aluminum in the atmosphere above the natural level, resulting in around 17 metric tons of aluminum oxides injected into the mesosphere. The byproducts generated by the reentry of satellites in a future scenario where mega-constellations come to fruition can reach over 360 metric tons per year. As aluminum oxide nanoparticles may remain in the atmosphere for decades, they can cause significant ozone depletion.

we find that these reentry byproducts may take up to 30 years to settle from the top of the mesosphere into the stratospheric ozone layer. Upon reaching an altitude of about 40 km, aluminum oxides catalyze chlorine activation which promotes ozone depletion. This suggests that concentrations of aluminum oxide compounds may start increasing in the mesosphere well before reaching the stratospheric ozone layer. This would introduce a noticeable delay between the beginning of the injection process when orbiting bodies are decommissioned and the eventual ozone-depletion consequences in the stratosphere.

A lack of observations and validated models of reentry demise limits our ability to simulate the complex aerosols associated with reentry, which makes estimating the climate impacts difficult. Aluminum is a primary satellite component and will likely be emitted during reentry vaporization in the form of alumina. Unmodified alumina is a useful approximation for metallic reentry aerosol. In this study, we simulate a potential yearly emission of 10,000 metric tons of alumina from reentering space debris. We investigate how the location of atmospheric accumulation, aerosol size distribution, and radiative properties of reentry alumina impacts the middle atmosphere. We find that 20,000–40,000 metric tons of alumina accumulates at high latitudes between 10 and 30 km in both hemispheres. Small changes in mesospheric heating rates lead to 1.5-K temperature anomalies in the middle atmosphere at high latitudes. These temperature anomalies are accompanied by changes in wind speed in the polar vortex.

To understand if significant ozone losses could occur as the launch industry grows, we examine two scenarios. Our ‘ambitious’ scenario (2040 launches/year) yields a −0.29% depletion in annual-mean, near-global total column ozone in 2030. Antarctic springtime ozone decreases by 3.9%. Our ‘conservative’ scenario (884 launches/year) yields −0.17% annual, near-global depletion; current licensing rates suggest this scenario may be exceeded before 2030. Ozone losses are driven by the chlorine produced from solid rocket motor propellant, and black carbon which is emitted from most propellants. The ozone layer is slowly healing from the effects of CFCs, yet global-mean ozone abundances are still 2% lower than measured prior to the onset of CFC-induced ozone depletion. Our results demonstrate that ongoing and frequent rocket launches could delay ozone recovery. Action is needed now to ensure that future growth of the launch industry and ozone protection are mutually sustainable.

To achieve that, they would need to launch 120,000 satellites per year. Over the 15 years, they would launch 1.8 million satellites, but 800,000 of them would fail (as part of our 9% failure rate), leaving a total operational fleet of one million satellites. This equates to 3,158 Starship launches per year, or nearly nine launches per day. For some context, the current launch rate for Starship is just five per year.
...
In order to keep a million satellites in the constellation, it needs to be maintained. So, each year, SpaceX would have to launch 90,000 AI Sat Minis to replace the roughly 9% of the constellation that failed. That equates to 2,368 Starship launches per year, or 6.4 per day.

A 10-fold enhancement of lithium atoms was detected at 96 km altitude by a resonance lidar at Kühlungsborn, Germany, approximately 20 hours after the uncontrolled re-entry of a Falcon 9 upper stage. The upper-atmospheric extension of the ICON general circulation model, nudged to ECMWF, was used to calculate winds. Backwards trajectories, including wind variability as measured by radar, traced air masses to the Falcon 9 re-entry path at 100 km altitude, west of Ireland. This study presents the first measurement of upper-atmospheric pollution resulting from space debris re-entry and the first observational evidence that the ablation of space debris can be detected by ground-based lidar. The analysis of geomagnetic conditions, atmospheric dynamics, and ionospheric measurements supports the claim that the enhancement was not of natural origin. Our findings demonstrate that identifying pollutants and tracing them to their sources is achievable, with significant implications for monitoring and mitigating space emissions in the atmosphere.

We use a global inventory of launch and re-entry emissions covering the onset of the megaconstellation era (2020–2022), and project these to 2029 based on 2020–2022 growth rates. We implement this inventory into a 3D atmospheric chemistry model to determine the impacts of megaconstellations on the ozone layer and climate. We find that global stratospheric ozone depletion from all mission types is relatively small compared to surface sources and megaconstellation missions only account for about one-tenth of this depletion. This is because rockets launching megaconstellations almost all use kerosene, a large source of black carbon or soot particles, but not of chemicals such as chlorine that directly destroy ozone. Soot from rockets absorbs sunlight, warming the upper layers of the atmosphere and decreasing the amount of sunlight reaching Earth's lower atmosphere, causing it to cool. Megaconstellation missions are responsible for about half of this climate effect. In this regard, rockets launching megaconstellations and other missions are like small-scale stratospheric aerosol injection experiments without forethought for potential unintended consequences.

The open-source movement emphasizes the power of freely modifiable, flexible code to support transparency, collaboration, and building outside vendor lock-in. Open hardware extends that logic to the physical layer: chips you can read, modify, and build on. All software runs on hardware, and over the past few years, the ground under the hardware industry has been shifting. Since 2022, the United States, the Netherlands, and Japan have progressively tightened export controls on advanced chips and the equipment used to manufacture them; China has responded with a state-backed effort to reproduce every layer of that supply chain at home. Policy analysts now routinely describe the trajectory as a “fragmentation” or “decoupling” of the global semiconductor market into separate technology spheres. Hardware costs are climbing across the board, so accessing open hardware feels all the more relevant for a group building open-source software.

Open hardware doesn’t make the chips any cheaper. What it changes is what a chip you already own is allowed to become or what kinds of application-specific chips you can create. A single FPGA (Field-Programmable Gate Array) on a shelf can be a video codec today, a custom search accelerator the next, and a faithful copy of a decommissioned architecture the day after that. The unit cost is what it is; the value you can extract from that unit is no longer fixed by a vendor. For institutions whose time horizons stretch over decades, like libraries, archives, and public-interest research groups, that reconfigurability is the core of the open-hardware argument that compounds.

There are no known open-source hardware implementations of Google’s TPU, the in-house chip family Google designed to accelerate neural network math. The silicon is proprietary and not directly purchasable by consumers except in edge TPU form (e.g., Google Coral). This field note reports a small experiment porting the OpenTPU project, a Python simulation of Google’s TPU published by UCSB’s ArchLab, to an inexpensive FPGA board to explore the pros and cons of using more open hardware. To keep the experiment lightweight, most of the code was written by AI coding agents, with a human directing the work.

Why FPGAs, For A Lab Like This One

An FPGA (Field-Programmable Gate Array) is a chip whose internal logic can be reconfigured. You describe the circuit in a hardware description language, SystemVerilog, compile it to a bitstream, and load it onto the board. A CPU runs your program. An FPGA becomes your program. FPGAs are the obvious place to start, because they’re the one piece of reconfigurable silicon a small lab can actually buy and program today.

The practical difference lies in the shape of the problems they solve. A CPU is a generalist reading a manual one step at a time. A GPU is a factory floor of thousands executing the exact same standard math in unison. An FPGA is a machine whose gears are configured to fit exactly one algorithm. They excel at problems with strange shapes. If your workload involves multiplying massive, uniform matrices to train a language model, you want a GPU. But if your work requires parsing millions of irregular text strings as they stream, finding exact bit-level matches across a sprawling archive, or piping data through a custom hash without ever pausing to fetch instructions from memory, an FPGA is arguably the more elegant approach. A dataset is sometimes only as legible as the software that reads it, and that software is only as runnable as the hardware underneath. An open FPGA design can preserve a faithful copy of obsolete circuitry, thereby preserving the means to read data, not just the data itself.

Source: Wevolver

For an institution like the lab, that distinction matters. Libraries, archives, public-interest research groups, and smaller labs deal with workloads and questions that are awkwardly sized: too large for a laptop, too specialized to deserve a recurring cloud bill, too long-running for any one grant cycle. For a well-resourced lab, the answer is cloud GPUs. For everyone else, the answer has historically been to wait or to scale down the question. That is why open hardware matters here: it gives smaller institutions a path to specialized computing without waiting for ideal market conditions.

A $300 board on a desk can run a custom-designed circuit, tailored to one workload, indefinitely, without anyone’s permission and without a metered bill. A physical circuit sheds the overhead of an operating system and the constant fetch-and-exec cycle, drawing a fraction of the power of a standard CPU. For a public-interest lab, this efficiency makes running some workloads both financially and environmentally sustainable.

TPU-style architectures are particularly well-suited to this kind of work because of the systolic array at their core. A systolic array is a grid of small multiply-add units that pass partial results to their neighbors on every clock tick, making it efficient for matrix math because data flows through the grid rather than being fetched repeatedly from memory. That structure is designed for exactly the dense-matrix operations that underpin embedding generation, similarity search, and the neural network inference behind modern document analysis. This is the kind of regular, spatial structure FPGAs are built to host. A grid of identical small units, wired to their neighbors, all ticking together, is close to a literal description of what an FPGA’s fabric already is.

Designs published openly in SystemVerilog are reusable across institutions, as open code is. What has kept this out of reach was never the silicon; it was the labor: months of vendor-tool learning, a small group of knowledgeable practitioners, and debugging cycles unique to physical systems. That cost is what AI coding agents have started to chip away at, making the open-hardware case more practical.

Porting OpenTPU and the Silicon Boundary

OpenTPU is a published academic re-implementation of Google’s first-generation TPU implemented in Python code that models the hardware’s behavior, not software meant to run on it. The hardware target of this experiment was the Alchitry Pt v2, a roughly $300 board built on an Xilinx Artix-7 chip, with an add-on that exposes USB 3.0 to a host PC. I worked with AI coding agents to describe goals in plain English, review edits, and iterate. A sandbox repository came first: blink an LED, echo bytes, talk to the USB chip. That groundwork paid off within hours of starting the real port. The OpenTPU translation itself — the systolic matrix unit, the weight memory, the instruction decoder, the activation logic, and the host interface — went through seven planned phases. A few days of wall-clock time later, the SystemVerilog testbench ran the same matrix-multiply program as the original Python simulator and produced the same answer.

Source: Jenevieve Haggard, HLS LIL

Translating Python that describes hardware into SystemVerilog that synthesizes hardware turns out to be something AI agents are somewhat capable of. The source is unambiguous, and the testbenches give instant feedback. The hard part was always the silicon boundary: a physical board, a vendor toolchain with subtle caching behavior, a USB chip with several gotchas. What worked was a scaffold around the agent that provided deterministic simulation tests for everything testable in software, plus on-board LEDs wired to specific finite-state-machine states, so a human eye could see what the testbench could not. A small notes tool called “cq” (by Mozilla) recorded what each session learned and made future sessions read those notes first. By the end, the store held 48 entries, almost all painfully, expensively earned.

Performance Realities and the USB Latency Bottleneck

By the end, the FPGA produced output that matched a NumPy reference lane-for-lane on a 200-vector benchmark. End-to-end, it was about 2x faster than the original Python simulator; on compute alone, about 4x. It was also comfortably slower than a CPU running optimized BLAS on small inputs. The point of the result is not that the FPGA wins everywhere, but that it shows where open hardware can be useful.

At this workload size, USB round-trip latency dominates the FPGA’s time budget. FPGAs win when data is moved to the device once, stays there, and is processed many times. Tiny inputs in, tiny outputs out, on every call, is the worst case for this design. We are running the worst case since we’re just at the “make it work” stage. PCIe-attached FPGAs sit much closer to the CPU and avoid this bottleneck entirely; with batched workloads on that kind of board, the compute-side 4x advantage we already see should carry through to the end-to-end number. They’re the natural next step for any workload where the dev-board numbers are encouraging enough to justify the cost.

Open Hardware and the Lowered Cost of Specialization

The lab occasionally runs computations large enough to be uncomfortable: searches over case law corpora and analyses across millions of documents. A pattern for designing a small piece of custom hardware that does those tasks well is a real option for problems that do not fit on a laptop and do not deserve a cloud bill.

More than preservation, open hardware offers freedom of access and the power to control your own collections, your own work, and your own computing. That opens up questions we are only beginning to ask. As archival media like silica and DNA-based storage mature, could open hardware lower the cost or complexity of building the readers required for those formats? As AI model architectures keep shifting, could a reconfigurable board keep pace in a way fixed silicon can’t? FPGAs already take on inference tasks at places like CERN (see their FPGA Developers Forum), adapting to software advances with matching hardware; could FPGAs do the same for our tasks?

AI coding agents are reducing the cost of specialized work that used to require a dedicated team. Custom hardware has historically been among the most specialized. If a small library research lab can produce a working SystemVerilog port of a real architecture in a few weeks, the question of what else has quietly come into reach is suddenly much broader.