Leaders at OpenAI, Nvidia, and IBM have predicted on separate occasions that 2025 would be the year of AI agent adoption. Yet as of May 2026, just 19.8% of U.S. businesses reported using AI in any business function, according to the Census Bureau’s Business Trends and Outlook Survey. More recently, Microsoft has declared 2026 the year of the agent. Nonetheless, these systems will be increasingly deployed, raising numerous concerns. Empirical benchmarking of their reliability in domain-specific use cases remains thin, however, and if we are to take agentic use seriously in research environments, we must close this gap.
As the Public Data Project’s Molly Hardy detailed in an earlier blog post, our team, in partnership with dataindex.us, created an AI benchmarking exercise to test how well AI can assess data risk for federal datasets. We have now built the AI Benchmarking Exercise into a web-based application, and we’re excited to announce its publication!
AI Benchmarking Exercise home page.
The exercise, which can be done in a group setting or alone, compares three approaches on the same datasets: a human researcher working alone, a human steering an LLM chatbot, and a background agent that searches the internet without human steering. All three approaches are measured against the expert risk assessments published by dataindex.us.
Presenting the three approaches side by side allows us to share knowledge in both directions: we can learn how human experts approach dataset evaluation; introduce experts to AI-assisted research; and experiment with automated evaluation tools, which we hope will eventually allow for monitoring and human follow-up on much larger collections.
In this post, we report progress on the third challenge of automated risk assessment, investigating how well AI agents perform a task that requires multiple kinds of domain expertise, intensive research into dataset context, and synthesis against a concrete rubric. We present how we built a background agent to answer that question, the design challenges we hit, and early results: how the agents performed, what they cost, and which did the best.
Experimental Design
In our experiments, we define an agent as an autonomous program that completes tasks for the user using the tools and environments it can access. We took a bare-bones approach, serving a large language model through OpenRouter, with search tools and the internet as its environment to conduct research and synthesis. We ran two frontier models, GPT-6.1 Sol (OpenAI) and Claude Opus 5.5 (Anthropic), using their providers’ native search, and four open-weight models, DeepSeek V4 Pro, DeepSeek V4 Flash (DeepSeek), GLM 5.3 (Z.ai), and Kimi K3 (Moonshot AI), using Exa as the search tool and OpenRouter’s fetch tool. The two frontier models ran at medium reasoning effort, and the rest ran at high since they don’t offer a medium effort level.
In the research phase, the model searches the web and fetches pages to learn about its assigned dataset. In the assessment phase, a second prompt asks the model to use its research to score each category against the assessment rubric. These are the same prompts human participants paste into their chatbots, sent verbatim with no added system prompt. The key difference is that humans can follow up and steer their chatbot, while the background agent works independently. After initial experiments, we ran into two challenges.
Challenge 1: Controlling the Model’s Search Parameters
During our initial in-person experiments, where humans used our AI Benchmarking website to assess data risk categories, participants reported that their chatbots surfaced dataindex.us as a source, which is effectively our answer key.
A simple solution might be to tell the model not to use a source, but that only works if models always follow textual constraints. Frontier models have been observed finding loopholes in evaluations, breaking the user’s constraints, and “peeking” at domains and sources they were told not to access.
To address this, we used the search tools’ excluded-domain settings to remove dataindex.us from the search results shown to the model and blocked the agent from fetching it directly. However, even with these guardrails in place, the DeepSeek and Anthropic background agents surfaced third-party pages that cited dataindex.us, emphasizing that blocking a domain doesn’t block pages that reference it.
Challenge 2: Reducing Cost
As a public service project, we aim to create tools that maximize output while minimizing costs, both for ourselves and for the open source software that we create. Our first version ran with no cost optimizations, at $17.58 per dataset for a single frontier model (Claude Sonnet 5) using their agent SDK. At that price, all 47 datasets across 6 models would cost about $5,000 in the worst case.
To reduce cost, we enabled prompt caching. For each turn of the agentic loop, we send all previous context, so without caching, the model pays full price to re-read everything it has already read. Caching lets providers reuse that repeated prefix at a steep discount, as little as a tenth of the normal input price depending on the provider.
Some providers cache automatically (OpenAI, Kimi K3, GLM 5.3, DeepSeek), while others (Anthropic) require explicit requests to enable caching, which explains the high cost in our initial experiment. With caching, the same Sonnet 5 model dropped from $17.58 to $3.74 per dataset.
What We Found
Across 47 datasets with six categories each, we measured agreement through exact-match with ground truth (i.e., did the agent get the same ratings as dataindex.us?) and distance to ground truth (i.e., how many rating levels off was it, on average?). All models were in a similar accuracy ballpark across both metrics, with DeepSeek V4 Pro leading by matching 55% of categories and averaging 0.52 levels from dataindex.us. All models stayed within one level of error on average across all datasets, with a tendency to rate riskier than the Data Index.
Agreement, cost, and time graphs for the models’ runs.
DeepSeek V4 Flash was cheapest at $0.17 per dataset; Claude Opus 5.5 was most expensive at $2.33 per dataset. Kimi K3 was slowest, averaging 7 minutes per dataset, with outliers as high as 23 minutes. More searching did not reliably buy more agreement: GPT-6.1 Sol searched the most, and DeepSeek V4 Flash fetched the most, yet neither beat 46% exact-match accuracy. In total, running six models across 47 datasets cost $225.
Descriptive statistics about the models’ runs.
Looking Ahead
Models are continuously improving, and agreement on a data risk task like this could improve over time with optimized search tooling. DeepSeek V4 Pro maintained the best accuracy at a fraction of the cost of frontier models, illuminating that the most popular agents aren’t always best suited to the task. We plan to release further work benchmarking these types of data agents on real-world tasks and empirically comparing them to other modern approaches in academia and industry. Working at the intersections of libraries and technology, we aim to create tools that researchers, journalists, and the public at large can trust and replicate without advanced domain understanding or deep pockets. Please stay tuned for further updates on our Data Monitoring Toolkit.
The toxic drug supply crisis has had profound impacts on workers in Canadian public libraries. While library workers struggle with changing job duties, understaffing, and the mental and emotional toll of laboring during death times, narratives of unsafety that pivot on “mental health and addiction” challenges experienced by our patrons have come to dominate library discourse and entrench an antagonism between library workers and our patrons. Using Sarah Ahmed’s (2004) theorization of affective economies, I will interrogate the “safety and security crisis” in Canadian public libraries as a discursive formation that relies on what, after Ahmed, I call the “sticky associations” between drug use, homelessness, mental health, and violence. It is through this process of articulating “violent associations” that delineate the ‘victim’ and the ‘perpetrator’ that the constitutive antagonism of library worker vs patron forms. In this article I will examine how mainstream media, library leadership, and library researchers contribute to a narrative of unsafety that hinges on the individuation of threat, safety, and solution. I then turn to how library workers make sense of, and act on, their affective experiences of threat and unsafety. Finally, I offer suggestions for how a labor framework of Occupational Health and Safety can be mobilized to address the structural and legislated violence shaping library work.
Introduction
My first week at a Toronto Public Library (TPL) branch I was put in-charge. Due to high turnover and understaffing, this is not as uncommon as one might hope. I was anxious, but less about what “incidents” could occur, and more about how I might be seen as responding to them. I was at a branch in the middle of a reckoning – a breakdown of trust between management and frontline staff who felt their wellbeing was not taken seriously. When an incident occurred, staff waited to see if their safety was prioritized and if management’s response was proportionate to the harm they felt they had experienced. We wore walkie-talkies to communicate about branch happenings, which often included tracking people’s movement. There was a board in the workroom that had pictures of “banned patrons” pinned to it and a “call the police immediately” protocol for when certain people entered the building.
I knew that my new colleagues were watching to see how I responded to these early incidents. Would I call the police as soon as X walked into the branch? Would I enforce a zero-tolerance alcohol policy? It felt as if I was forced to make a choice: show my colleagues I value their safety by punishing patrons or honor my abolitionist commitment to non-punitive, transformative responses to conflict. This tension is not unique to a single branch, nor TPL. It has haunted my time at the public library. I have regularly found myself struck by the animosity many colleagues feel towards our patrons – the threat they experience from interactions I do not register, surveillance they so willingly participate in, and their eagerness to see patrons punished.
In a chaotic ecosystem ripe with despair and uncertainty, the enforcement of both formal and informal rules of conduct is an assertion of order. Library rules of conduct are the mechanism through which frontline workers express institution-backed boundaries. With very little power and a whole lot of responsibility, library workers are expected to maintain a “safe” and orderly library for patrons and staff. While library policy and job duties over-determine staff responses to challenging patron interactions, many library workers come to view institutional practices of punishment as strategies for personal safety. Presented with few alternatives, library workers regularly punish patrons for our declining work conditions and public expressions of large-scale abandonment and legislated violence that we are under-resourced to address.
In addition to profound poverty, the toxic drug supply crisis has wreaked havoc on poor, unhoused, and racialized communities. From January 2016 through March 2025, Canada recorded 53,821 opioid toxicity deaths, with 26,000 people dying of a drug poisoning in Ontario (Centre on Drug Policy Evaluation, 2024; Government of Canada, 2025). These preventable deaths are a direct result of the criminalization of drugs and the people who use them. As experts have repeatedly pointed out, neither punishment nor prohibition reduce drug use or prevent drug poisonings. Instead, prohibition fuels stigma, deepens the dangers of illicit and unregulated drug markets, intensifies poverty and homelessness, and drives up rates of preventable illness and death (Boyd et al., 2016). Rather than responding to this crisis with evidence-based solutions, Canadian provinces continue to cut funding for harm reduction programs that save lives while ramping up forced treatment policy and the policing of public drug use (Canadian Public Health Association, 2025). As the drug supply becomes less predictable and more toxic, provincial governments continue to entrench prohibitionist policies, deepening the public health emergency confronting communities across the country.
Increased drug use and drug poisonings at the public library are symptoms of these government policies. As library workers can attest, the complexities of dispossession and private life turned public play out daily in the library. Some patrons exhibit signs of intoxication, fall asleep at computer stations, and experience drug poisonings in the library bathroom. Many patrons are frustrated at being abandoned by the state and the limited ability of the library to address gaps in social and health services. Other patrons are upset that they are confronted by a growing class divide while studying, attending a children’s storytime program, or picking up their holds. This anger is often directed at library workers trying to balance divergent and regularly oppositional desires of people sharing the library.
In a climate of collective unsafety, personal safety is both a real concern for people paid to interface with the public and a discourse that state institutions, including the library, use to assert order in a time of heightened instability. Library workers labor at the crux of this contradiction. While library workers have experienced increased violence and harassment on the job, they are also witness to tremendous human suffering intensified by the dismantling of social services (CUPE Research, 2023; Stevenson, 2021). The knot of bad feelings, or discomfort, brought on by all of the above experiences is often hard to untangle. Library workers are not only frustrated by the expanding scope and emotional toll of their work, but many have expressed feeling unsafe. Safety, as a framework for discussing the demands of public library work, and a lens through which to advocate for better work conditions, has come to dominate library discourse. While this framework may give voice to library workers struggling with the changing demands of their job, terms like violence, safety, and security are power-laden, doing the discursive work of the library seeking to re-assert order and a political project reliant on the dispossession and dislocation of marginalized people.
When fear stands in for a host of other affective experiences resourced by power, it overdetermines how we respond. While library workers, just like all workers, deserve a workplace free of violence, we must resist the co-option of our experiences by institutions with investments in punishing the working class. Safety is undoubtedly at the forefront of conversations about the Canadian public library. However, its usage needs to be further interrogated if what we desire is true, collective safety for everyone in the library. Safety, as prison abolitionist Mariame Kaba (2024) reminds us, is a social relation articulated through access to material resources and our relationships with others (para. 8). When people have what they need to live full, healthy lives free from legislated violence, communities are better resourced to respond generatively and restoratively to conflict.
In what follows, I hope to identify and unsettle the discourse of safety that pits library workers against their patrons, locating the source of unsafety in the patron, and the solution in their disciplining. Using Sarah Ahmed’s (2004) theorization of affective economies, I will interrogate the “safety and security crisis” in Canadian public libraries as a discursive formation that relies on what, after Ahmed, I call the “sticky associations” between drug use, homelessness, mental health, and violence. Sara Ahmed offers the language of the affective economy to name how “emotions circulate between bodies and signs” and “align individuals with communities – or bodily space with social space – through the very intensity of their attachments” (Ahmed, 2004, p. 119). It is through this process of articulating “violent associations” that delineate the ‘victim’ and the ‘perpetrator’ that the constitutive antagonism of library worker vs patron forms.
In the first half of this article I look at the different mechanisms that work to affix associations of homelessness, substance use, mental health, and violence to one another in library discourse. In particular, I examine how mainstream media, library leadership, and library researchers contribute to a narrative of unsafety that hinges on the individuation of threat, safety, and solution. I then turn to how library workers engage the changing landscape of their work. Drawing from my own experiences as a librarian, union representative, and organizer with Library Workers For Supervised Consumption Sites (LW4SCS), I share insights on how library workers act on their affective experiences of threat and unsafety. In the final section of this paper, I offer suggestions for how a labor framework of Occupational Health and Safety can be mobilized to address the structural and legislated violence shaping public library work.
Yes, library workers are struggling with the changing demands of their jobs. We are increasingly caught in the crossfire of the war on drugs. However, rather than capitulate to dominant narratives that criminalize our communities, we have a responsibility to resist becoming willing agents of institutional violence against our patrons.
Library Work in Death Times and the Defining of an Antagonism
The frequency and intensity of emotionally charged interactions with patrons mirrors a deepening racial wealth divide across Canada. The introduction of the Canadian welfare state rollbacks in the late 1980s, major cuts to social services, wage stagnation, an affordable housing crisis, and the rapidly rising cost of living have concentrated wealth in the hands of some, while further disenfranchising others. In Toronto, where market speculation and corporate landlords have hastened the affordable housing crisis, there are over 15,000 people without a home each night (Facts about Homelessness in Toronto, n.d.). This astounding number is only part of the story: approximately 1 in 3 Indigenous adults in Toronto are either precariously housed or sleeping rough (McConkey et al., 2024). Black Torontonians make up 31% of the city’s unhoused population while only constituting 7.5% of the whole population (Black Communities and Homelessness, n.d.). Twenty-one percent of youth in Toronto shelters identify as 2SLGBTQ (2SLGBTQ Youth Homelessness in Canada, 2016). The racial and gendered dynamics of state abandonment and immiseration in Toronto, while on a particular scale, echo across the country.
As I write this article in the fall of 2026, 16 supervised consumption sites (SCS) in Ontario have shut down, while the remaining 8 lost provincial funding in June. Beyond safe places to use drugs, these closures eliminated other life saving services offered at SCS, including drug checking services, primary care, housing support, and grief counseling. Alongside these closures, Ontario has moved to further criminalize public drug use. The Safer Municipalities Act, passed June 3, 2025, gives police power to clear encampments on suspicion of drug use, arrest those who don’t leave, and impose fines up to $10,000 or jail terms up to six months. This crackdown makes it harder for outreach services to reach people where they are, while exposing them to greater risk of police violence and incarceration. With few safe, private places left to use drugs, public drug use has increased, including at the library, which offers bathrooms, good lighting, a weather controlled environment, and a place to rest. As Jackson Davis, Isobel Heintzman, and Aditi Mehta (2025) identify, TPL has increasingly become a haven for precariously housed and disenfranchised Torontonians.
While library management seeks to introduce new positions, partnerships, and policies to address the impact of the toxic drug supply and housing crisis on their institutions, library workers continue to feel unsupported and ill-equipped to deal with these dynamics as they play out in real time (Saunders, 2025). The fissure between the vision of library leadership and employees’ experiences of that vision result in library workers being disillusioned and distrustful of management (Stevenson, 2022). With little power to make institutional or structural change, library workers grasp for action within their control, turning to what Mariame Kaba and Kelly Hayes (2023) call “rituals of order” – the reliance on systems of policing and punishment to resolve conflict, even when they rarely lead to resolution or safety (Hayes & Kaba, 2023, p. 33). In the library, these rituals include disciplinary actions, namely warnings, exclusions, and calling the police, that escalate even minor tensions. These actions locate the problem in the patron and the solution in their disciplining, reifying an antagonism that pits library workers against patrons. This antagonism leads to library workers punishing patrons and advocating for punitive policy and procedure that hurt all working class people.
Obviously, this antagonism does not start or end with library workers. The very foundations of the public library are based in class anxiety – the late 19th century idea that an increasingly agitated poor and working class base (during a time of labor unrest) needed to be pacified and transformed into productive and law-abiding citizens through soft power (Augst, 2007). The regulation of the social body, which necessarily includes formal and informal threats to its unity, permeates the public library’s history as well as its current day social-spatial architecture. From racially segregated libraries, to police enforced late fines, to turning over patron information to government agencies, to high tech securitization, libraries have always participated in the surveillance and policing of marginalized patrons (Kaba, 2025). These practices of social regulation in the library recreate historical inclusions and exclusions, defining public space as belonging to some people while being threatened by others. Many workers, tasked with enforcing the rules of the library and maintaining order in the branch, come to perform the disciplinary function of the institution not just because their job depends on it, but due to their own affective attachments to particular ideas of safety and security.
During a period of intensified neo-liberal austerity, it is no surprise that the regulation of poor and working class people comes to mark the current crisis of the Canadian public library. Rooted in narratives of declining “safety and security,” media and professional reports frame the challenges facing the public library as a result of a growing surplus population with “complex needs” (CULC, 2023; Ghobrial, 2026; Zoledziowski et al., 2024). While committed to “access for all,” immiseration has outpaced the public library’s ability to serve marginalized patrons. In 2023, the Canadian Urban Library Coalition’s (CULC) Safety and Security Working Group – comprised of prominent library leaders and representing more than 50 of the largest public library systems in Canada – declared that “mental health and addictions” prevent the library from performing its literacy function, keeping its workers safe, and maintaining an environment open to everyone. Even while compassionate library workers and committed library leadership seek to address the needs of their patrons, the sticky discursive associations that have come to identify this crisis accumulate as a limit to the possibilities of meaningfully addressing it.
Quantifying a Safety and Security Crisis in Canadian Public Libraries
There has been an increase in high intensity acts of violence in Canadian libraries over the last 5 years. Canadian news outlets have met this intensification with an uptick of coverage regarding safety and security in public libraries. Articles with headlines such as “Library workers punched, spat on as security incidents rise, data shows” and “Library staff facing security issues, including verbal abuse, drug use in branches” exemplify this trend (Ruby, 2025; Zoledziowski et al., 2024). Specific events include a staff assault at the Thunderbay Public Library, a mass stabbing at the Vancouver Public Library, and the 2022 murder of Tyree Cayer at the Winnipeg Public Library (Ketonen, 2025; Travers & Ridgen, 2023). Journalistic accounts of these painful events draw on aggregated incident stats, library tours, and interviews with library staff and leadership. The grand narrative is of a Canadian public library under siege and in decline: a chaotic environment where workers and patrons feel unsafe and the library is unable to honor its primary mission of meeting the literacy demands of its community.
As library workers, we know the importance of critical media literacy. Drawing a story of declining safety from generic incident data is a red herring. In the library world, incidents are the quantitative metric for tracking when and how often library codes of conduct are breached and medical emergencies occur. For library workers, incidents are part of the job. A sample of non-violent behaviours that constitute an incident across many Canadian libraries include: sleeping, excessive noise or disruption of others, having an animal that is not a service animal in the library, food and drink in undesignated areas, intoxication, possession of open alcohol, drugs, or drug paraphernalia, theft, and trespassing (CULC, 2023, pp. 9–10). The reality is that the majority of incidents in Canadian public libraries are non-violent.
Mainstream journalists tasked with covering “safety and security” in the public library often rely on incident logs as the raw data for their reports. For example, a 2024 CBC article cites a 71% spike – from 1,362 to 2,334 – in incidents at the Toronto Public Library (TPL) between 2022 and 2023 (Zoledziowski et al., 2024, para. 4). According to CTV News, the Hamilton Public Library recorded 771 incidents involving security during the first 5 weeks of 2026 (Ghobrial, 2026, para 4). While these numbers may be alarming, it is important to note that they are an aggregate of incidents ranging from mundane to violent. Neither story distinguishes between stats on serious cases in which library workers or patrons are in an unsafe environment – as in the presence of a weapon – and minor incidents of inconvenience – a noise violation, for example.
In fact, according to the CULC Safety and Security Toolkit, a study of 33 Urban Canadian library systems determined that violent incidents made up only 0.001% of all visits to the library (p. 42). In addition to limited violent incidents, the majority of incidents tracked by these 33 libraries fall under the categories of harassment, drugs or alcohol (including possession of paraphernalia and intoxication), and trespassing by banned patrons (CULC, 2023, p. 42).
Disaggregating incident data provides a more nuanced view of the difficulties facing the public library. It also troubles the categorical slippage of what otherwise might be called insubordination into violence and threat. Despite what some Canadian news outlets might have us believe, the library is not a place where random acts of physical violence are inescapable. It is a place where the complexities of life within a white supremacist settler colonial capitalist order are lived out publicly. This includes the many ways that people marginalized by systems of power make sense of their lives and prioritize survival. Substance use, intoxication, dissociation, and emotional or psychological distress all flow downstream from state violence directed at Indigenous, Black, queer, poor, and disabled people living in Canada. The sustained material and psychological harm of legislated violence often results in visible expressions of pain and adaptive strategies of survival. As people in the harm reduction and disability justice communities remind us, these are reasonable responses to subjugation.
Nevertheless, substance use, intoxication, and drug poisonings are key fulcrums on which these narratives of safety and insecurity pivot. Library leadership and professional organizations regularly conflate rare experiences of extreme violence with symptoms of the capitalist crisis under the banner of unsafety: houselessness, racialized policing, and a toxic drug supply. A drug poisoning, for example, is a medical emergency requiring medical attention. Though often scary for the responder, the person experiencing a drug poisoning is at risk of serious brain damage or death. Drug poisonings, just like public drug use and intoxication, are not inherently violent. Still, in most libraries, drug poisonings, drug use, and intoxication are tracked and treated as violent incidents. This is not just a matter of mislabelling. The framing of a safety and security crisis in which public libraries are overwhelmed with violent incidents due to “drug addictions and mental health” collapses categorically different forms of harm. Within this framework a medical emergency like a drug poisoning becomes indistinct from a disruptive but non-violent behavior like sleeping at a computer station or an act of interpersonal violence. This logic transforms distinct situations into an environment of undifferentiated threat, wherein a non-violent event equally contributes to the library as a site of unsafety. Through this collapse, we lose the ability to be precise about the challenges facing public library workers and the diverse responses these challenges require.
To put it another way, the process of creating an abstract and undifferentiated category of incident, feeds the carceral myth that people who use substances are a public safety threat while leaving the conditions that create public drug use and a toxic drug supply crisis unconsidered. Substance use and intoxication are not inherently violent or dangerous. In fact, people who use drugs experience interpersonal violence at disproportionately high rates, not to mention the violence of state neglect, policing, prohibition, and incarceration (Swaich et al., 2023). There are real safety issues associated with the toxic drug supply crisis, most urgently, death. It is through harm reduction, decriminalization, safe supply, affordable housing, and robust social services that we can begin to address the roots of the crisis. Conceiving of public drug use primarily as a safety and security threat to those who witness it imposes a distinction between “innocent patrons” (read sober, housed, white) who deserve safety and “problem patrons” (however outdated we like to think this framing is in library literature). These are racialized, classed, and colonial designations. Often, the very presence of people who use drugs or who are intoxicated (or who are merely assumed to be intoxicated because of their appearance or affect) triggers an alarm. Working as a public librarian in two large Canadian cities, I witnessed how complaint-based rules of conduct grounded in the vague language of “respect and consideration” resulted in the regulation not only of people’s behaviour towards others, but patrons’ state of mind, physical presentation, and use of space. The discomfort that some people experience when being in proximity to vectors of social abandonment and immiseration they are not used to interacting with becomes perversely rendered harm in and of itself. Disregard or transgression of social norms around hygiene, sobriety, and the use of public space are regularly surveilled and punished by both staff and patrons – a structural problem disguised as interpersonal conflict. It is not uncommon that after a complaint, a patron may be told they cannot return to the library until they have showered or sobered up, that they must leave their belongings outside before entering, or that they must quiet or quit the conversation with voices only they can hear. The consequences of these regulatory decisions are profound and often have dangerous impacts for our patrons most impacted by the war on drugs.
Data, aggregated or not, requires interpretation. While alarming numbers might help convey the magnitude of the crisis in public libraries, the articulation of fear and unsafety that accompany these stats give them an affective substance. By collapsing varied incidents into a single category of violence, diverse instances of discomfort on the job come to be experienced by workers as violence even when they are clearly not. In an affective economy where poverty is circulated in and through disgust or violence, fear and anxiety binds patrons and workers together along familiar classed and racialized lines. It is this opposition, wherein the presence of some leads to the perceived unsafety of others, that determines how discipline and punishment is exercised in the library. Together with stats and images (real and conjured), the affective experiences of library workers and patrons are central to the construction of a “safety and security” crisis in Canadian public libraries.
Framing a Safety and Security Crisis
Surveying Library Workers
The way library workers make sense of their experiences is shaped by broader discourse. As media and library leadership address the larger picture, library unions and scholars have increasingly turned attention to frontline workers’ experiences of safety (CUPE Research, 2023; Helfritz, 2018; Knox, 2024; LaPierre, 2021; Stevenson, 2021, 2022). Notable is Siobhan Stevenson’s 2022 survey of 527 library workers across four Canadian library systems about their experiences of third-party violence and incivility, which she frames as a continuum. In the survey section entitled “Feeling Safe”, Stevenson asks participants to consider “physical and sexual assault or threats thereof” alongside “rudeness, eye-rolling and a disrespectful attitude” (Stevenson, 2022, p. 338). The introduction continues: “essentially, any kind of negative interaction that leaves you hurt, feeling intimidated, fearful, or unsettled. It is important to note that you don’t have to experience these things personally to be affected negatively by them” (p. 340). Included in this section are questions about sexual assault, physical aggression, and “dealing with” substance use and mental health. When asked how often respondents felt unsafe – prefaced as including feeling “anxious, unsettled, disquieted, and vulnerable or physically unsafe” – 67% of respondents indicated they felt unsafe on a weekly or monthly basis (p.340).
Reflecting on the limitations of the survey, Stevenson acknowledges that situating questions about “feeling safe” within a continuum between incivility and violence “conflated and flattened affective nuances and differing subject positions” and “may have predisposed respondents to focus on their encounters with, and perceptions of, the less common experience of third-party violence over their exposure to incidents of incivility” (p.347). Stevenson goes on to highlight how “references to vulnerable and street-involved people and their characterization as threatening, for example, dominate the findings” (p.347). As Stevenson acknowledges, the survey instrument itself influences how workers come to interpret and report their experiences, collapsing meaningful differences in kind and severity into a single register of threat.
Notions of incivility have long served a disciplinary function in the library, rooted in ideas of respectability and used to police racialized, poor, and gendered bodies. That 73% of Stevenson’s respondents identified as white and 81% as female matters (p. 339). White womanhood serves as a marker of purity and white women have historically been positioned as “civilizing” agents of colonial and racial projects. As Gina Schlesselman-Tarango (2016) articulates, this led to the construction of the “Lady Bountiful” archetype in librarianship that fused patriarchal ideals of femininity with white supremacist missionary work. This legacy persists today, positioning the white woman librarian as the normative subject empowered to police, discipline, and manage “deviant” patrons under the guise of maternal benevolence and professional neutrality. The Lady Bountiful trope in library literature illuminates how “white supremacy and patriarchy have worked to produce a particular subject in librarianship” through which threats to civility are worked out and disciplined (Schlesselman-Tarango, 2016, p. 683).
Though Stevenson notes concerns about conflating incivility with violence, she does not interrogate incivility itself as a racialized, gendered, and classed category. Supplying frontline workers with a language for their experience, however validating, risks entrenching an interpretive lens in which behaviours such as substance use and intoxication fit into the same affective register as physical violence. This affective slippage is not confined to individual encounters or survey data. It scales up, providing the interpretive scaffolding for how library organizations, media outlets, and administrators frame substance use and mental health as an institutional crisis.
Sticky Associations and Policing the Homeless Body
It is undoubtedly true that public libraries across Canada have seen an increase in drug use and drug poisonings in their spaces. In 2022, the Edmonton Public Library (EPL) recorded almost 100 drug poisonings while TPL saw a 529% increase in suspected poisonings (CULC, 2023, p. 40; Zoledziowski et al., 2024, para. 4). In the first 3 months of 2025, SPL responded to nearly 50 drug poisonings (Warren, 2025), para. 3). This is the reality of the toxic drug supply crisis made worse by defunding harm reduction services and further criminalizing public drug use. The defunding of drug testing and poisoning reversal services alongside the grief and loss accompanying the closure of SCS has resulted in more drug poisonings in public, including the library. In the first 6 months after the initial closures of SCS in Ontario, there was a 69.5% increase in EMS-treated opioid toxicities and a 19.4% increase in opioid toxicity deaths (Kolla & Gomes, 2026, p. 3).
Even when leadership acknowledges the harms of the toxic drug supply crisis, the public library participates in stigmatizing, punishing, and criminalizing patrons who use drugs. Visible drug use, the presence of drugs or drug paraphernalia, and exposure to people who are intoxicated is treated as a safety and security issue. As examples, libraries like EPL and SPL have Code of Conduct rules about possession of substances and intoxication, ranging from a 1 day suspension up to a 6 month system wide suspension for repeatedly being caught with open alcohol or drugs (CULC, 2023, p. 10). Compounding punishment for ‘trespassing’ while banned from the library, can lead to months long suspensions for people who need the library most. Rules that regulate what a patron has on their person, and whether or not they are perceived as intoxicated, do not focus on the very real dangers associated with an unregulated and contaminated drug supply: exposure to biohazardous materials; or instances when intoxication is coupled with physical violence or harassment; and, most importantly, death. Instead, they seek to regulate deviant patrons that may upset the order of the library due to their embodiment.
Fig. 1: Edmonton public library code of conduct example (CULC, 2023, p. 10).
We need to be honest about what kind of substance use is targeted by code of conduct rules. When we talk about drug use in the public library, we are speaking directly to the intersection of poverty, homelessness, racialization, and street drugs. We need to be honest about the conditions under which people are forced to use in public due to the absence of housing, privacy, and now closed SCS. If we are still being honest, drugs are a survival strategy, and often a smart one. Drugs can help one cope with the unbearable suffering of state sanctioned violence and its accompanying grief; drugs can keep you alert and warm at night and they can keep hunger at bay; drugs can make you feel good and they can make you feel nothing. As the Indigenous Advisory Board at the Drug User’s Liberation Front (2023) reminds us, “illicit substances are necessary for many of us to continue surviving in unlivable conditions” (p. 5).
Through theorizing constructed associations between the asylum seeker and the international terrorist in Europe, feminist thinker Sara Ahmed (2004) helps us understand how the “homeless drug user” becomes a reified figure that circulates through social and cultural repetition, accumulating the affective residue of everything associated with it. Homelessness and drug use don’t have to exist together for this to work, they only have to be repeatedly proximate in discourse. Once that proximity is established, pity, fear, and revulsion don’t attach to particular behaviours but to the body itself, ready to be activated by anyone who fits its visible signs (an overflowing shopping cart, unkempt appearance, talking to oneself, nodding off).
For Ahmed, signs don’t have affective value on their own, affect is accumulated through repeated association, so that emotion appears to reside in an object or body when it has actually adhered there through a history of contact between signs. “Homeless,” “addict,” “dangerous,” “dirty,” “unstable” circulate together in media coverage, in policy language, and in conversations with co-workers. Through this repeated association, they begin to stick to one another, so that invoking one is enough to summon the affective charge of the others. A person doesn’t need to be observed using drugs to be deemed scary or wielding a weapon to be considered violent, the visible signs of homelessness are already sticky with the associations of drug use, criminality, and unpredictability, because those signs have circulated together so often that the boundary between them has dissolved.
Regardless of behaviour, the “homeless body” is a threat to the publicness of the library and the safety of its “average” (or in Ahmed’s terms, ordinary) patrons. The signs and associations not only stick to each other but to the object or body in question. Samira Kawash (1998) uses the language of the “the homeless body” to articulate the process through which the “ideological and discursive specter of homelessness” creates a sticky figure – one that “emerges as a particular mode of corporeality in contingent circumstances through which the public struggles to define and secure itself as distinct and whole” (Kawash, 1998, p. 324). It is through the act of exclusion of the homeless, drug-using body in all its supposed violence, that the public makes proprietary claims about spaces like the library. As Kawash says, public space comes to be seen as belonging to the public, of which homeless people are not part. It is the homeless body, Kawash articulates, that is “the stake and site of the contest over public space” (p.323).
By declaring the presence of embodied precarity experienced by an “overwhelming” number of library patrons a crisis of safety and security, new safety and security measures (security guards, trespass bans, carding policies, surveillence) become the logical response. The effect becomes the cause. For example, in a conversation with local Hamilton news channel CHCH (2026), HPL CEO Paul Takala told viewers that patrons who use drugs make the library unsafe for families, newcomers, and seniors, among other marginalized groups. In this declaration, Takala not only erased the intersecting experiences of people made vulnerable to state violence through legacies of white supremacist settler colonialism, xenophobia, and prohibitionist state policies, but made drug users the primary source of insecurity and unsafety in the library – the barrier preventing all other marginalized members of the community from flourishing.
This does not diminish the reality that all kinds of patrons do fucked up things at the library. Transphobia, homophobia, racism, misogyny, and ableism are regularly wielded against people who work at and use the library. Many of my most aggravating encounters have been with entitled, privileged patrons. These include racist mocking and sexual harassment of low-paid library staff; angry yelling when made to wait for the printer; and my personal favorite, the strong suggestion that we “could be the first library to permanently ban homeless people”. And still, a marginalized identity does not preclude someone from being aggressive, violent, or discriminatory. In attempting to disentangle ideas about harm at the library, I want to be careful to not suggest that things are not difficult, complicated, or anxiety-inducing, nor that people do not experience actual threat to their physical safety. However, how we respond to moments of crisis, conflict, and discomfort have real, material impacts for marginalized patrons. When discomfort around visible drug use leads to the banishment of an unhoused patron in the middle of winter, or when staff calling the cops to remove someone who is trespassing leads to criminalization, we must ask ourselves what kind of safety we prioritize and whether or not we truly feel that such actions meet our needs. Individual incidents of frustration, anger, and even violence need not undermine our project of seeking material, sustained safety that leaves no one behind.
Library Workers Responding to the Crisis and Rituals of Order
Rituals of Order
Fear, whether it registers or not, drives the impulse to reach for rituals of order. Fear of losing control of the branch, fear of losing your job, fear for your safety, and fear of the homeless body. As Kaba and Hayes say:
governed by fear, people are largely cooperative with systems that produce torture, mass death, and annihilation. That is the greatest danger that fear poses: not panic amid disorder, but co-operation with an order that we ought to find unspeakable – one that is actually poised to bring about our own distinction (p. 38).
When people are scared, they want to feel safe, they want a restoration to order. This is a human impulse, a reaction to emotional associations that are socially constructed and maintained but felt all the same. Attention to these impulses is important. They are informative even if not how we imagine. Safety promised by expulsion and criminalization allow us to quiet the bad feelings, ignore our own precarity, and detach from the conditions that position us closer to our patrons who use drugs than we want to admit.
However, safety is social and always in flux. It articulates a relationship to harm, including the structural distribution of harm (or absence of harm) within a society. “Safety isn’t a thing: it’s a social relation” Kaba reminds us. “I’m more or less safe depending on my relationship to others and to my proximity to the resources I need to survive” (Kaba, 2024, para. 8). Safety cannot be possessed, but is a function of social and economic relations that unevenly distribute state produced vulnerabilities. Unsafety is not just a feeling but a material reality for many. Wealth and whiteness are two major predictors of safety established through carceral infrastructure built to uphold their supremacy and protect them from threat. However, a vision of safety dependent on the disappearance of others is always tedious, subject to disruption by a mere encounter with the other.
Library workers must be involved in articulating processes of accountability and redress that do not pit us against our patrons. We must figure out how to respond to the difficulty of public library work during death times. For instance, when library workers and patrons are exposed to second-hand harms of substance use – exposure to biohazardous materials and sharps, people who may become aggressive when intoxicated, or the emotional impact of responding to an overdose – we can turn to strategies that help mitigate the harm. We can wear gloves and advocate for sharps disposals, practice de-escalation skills, and insist on paid time to self-regulate and process. Library workers must continue to demand access to resources that allow us to respond to our patrons with compassion, grace, and skill. This includes more staffing and better training, but it also includes SCS, affordable housing, food security and an end to the drug war. From resisting the discursive associations between violence, substance use, and intoxication in our language and our reporting mechanisms to showing up in solidarity with people being evicted from encampments, library workers can best advocate for better work conditions by recognizing that our struggle is connected to that of our marginalized patrons. Safety and security come from having our material needs met, from being in a community that cares for each other, and from learning the skills and building the infrastructure that supports us when things get tough.
Library Workers for Supervised Consumption Sites
Responsibility carries many valences in a workplace like the public library and competing needs rarely reconcile themselves. Patrons, co-workers, and leadership all have expectations that shape how staff respond to incidents. However, it is the wage-relation that often structures a prioritization of staff responsibilities. As the front-line representative of the institution, library workers absorb its liability almost by default. Property starts to outweigh people, and the loudest voice outweighs the most marginalized. This slide occurs not just because library workers want to keep their jobs or have their own investments in punishment, but due to an institutional liability framework that positions safety as an administrative problem rather than a relational one. As Ben Robinson (2019) describes, through the framework of liability, security is treated as one undifferentiated task, lumping fire and flood prevention, theft, digital privacy, and violent behavior into a single risk-management mandate. Bundled with it is a “security ethos” that deputizes staff to surveil and discipline patrons read as engaged in “suspicious activity” (Robinson, 2019, Section Fire, Flood, and Fist Fights). A security ethos and the tremendous amount of responsibility experienced by library works leads to hypervigilance often actioned through surveillance. And because a liability framework is built around assessing and neutralizing risk, its default responses are removal and punishment.
My desire is not to blame frontline library workers for the way we participate in the punishment of our patrons. Library workers, more than most, are aware of the social conditions that shape public space and thus our work. Staying tethered to the compassion and care that got many of us into the field in the first place is an ongoing practice. However, in the absence of meaningful support, resources, and compensation, library workers reach for what they know, and seek to restore order to an always fragile ecosystem. As I have written elsewhere, even those who resist the imperative to police our patrons or punish communities to which they belong, are regularly compelled by the wage relation to comply. While this too can create bad feelings or a sense of moral injury – an affective response to the alienation one feels due to the inability to direct their own labor – it does not diminish the real material impact such punitive practices have on our patrons (Cowley, in press).
As a founding member of Library Workers for Supervised Consumption Sites (LW4SCS) I have had the privilege of engaging many library workers across the province about their experience of the toxic drug supply crisis. LW4SCS formed in fall 2024 in response to Ontario’s Bill 223, the Safer Streets, Stronger Communities Act, which mandated the closure of SCS within 200 meters of schools and childcare centres. As library workers, we recognized that these closures would significantly impact our communities, our patrons, and our work. Beginning with an open letter to Premier Doug Ford, LW4SCS has since grown into a collective that holds regular meetings, compiles frontline data and testimony, publishes zines and articles, hosts workshops, presents at conferences and staff days, delivers board deputations, and pushes our individual locals into action.
Our central aim is to introduce frontline library workers’ voices into the conversation, and to give workers the space, time, and language to organize around the toxic drug supply crisis as it manifests in their workplaces. We believe that when we come together and engage in rigorous analysis and self reflection, we can produce an alternative conceptualization of the crisis that prioritizes collective safety over punishment. Grounded in the commitment that our right to safe and supported work must never be weaponized against patrons who use drugs, we believe we have a responsibility to push not only our employers but our unions to develop a substantive analysis of, and response to, the compounding crises of our time. LW4SCS seeks to create spaces where workers can sit with the complexity and contradiction inherent in frontline library work. As Harsha Walia offers, “complicity and solidarity are always entangled and not easily compartmentalized, and holding both together is the real challenge” (Walia, 2012, para. 9). Our futures are bound up with our patrons’, and thus LW4SCS rejects the premise that patrons are our enemies or that punitive measures can resolve either the toxic drug supply crisis or our own deteriorating working conditions. We encourage library workers to build skills, shift their language, find one another, and get organized. Together, we have the ability to transform not only the narrative around safety in the library, but practices that pit us against our patrons.
An Injury to One is an Injury to All
When it comes to evicting unhoused people, I want to remind people: [Canadian Union of Public Employees (CUPE)] members have the right to refuse unsafe work – work that endangers their lives or the lives of others. If any worker feels that participating in the eviction of unhoused people from encampments is unsafe, for themselves or for anyone, including the people who are homeless, they have the right to refuse that work (Hahn, 2023).
At a 2023 event put on by Voices for Unhoused Liberation in Toronto, CUPE Ontario President Fred Hahn reminded CUPE workers, in particular city workers tasked with dismantling encampments, that they have a right to refuse unsafe work. What has stuck with me about Hahn’s statement is the assertion that the right to refuse includes the right to refuse to endanger others. His statement not only names the connection between housing and safety, but positions workers as active agents with rights and responsibilities towards others, including unhoused people who are regularly and systematically denied access to safety. While I am not suggesting that library workers assert their right to refuse unsafe work in response to the toxic drug supply crisis – I am not certain we have the collective will to do this just yet – I do believe that health and safety mechanism such as the right to refuse provide leverage for workers who oppose the state violence they are expected to perform.
Despite the well-documented limits and betrayals of corporate unionism, labor organizing is one of the best tools available for library workers navigating the toxic drug supply crisis. As an analytic, labor provides us with a framework for linking our own exploitation with the precarity of our patrons. After all, how many library pages can afford to live anything but paycheck-to-paycheck in Toronto? As a tool, the union provides us with an avenue to operationalize this analysis. Unions ask us to redirect our anger away from our patrons and towards our employer and the larger structures that shape our work conditions. As a horizon, labor organizing requires us to collectively articulate and organize around our demands and link them to larger labor struggles. An injury to one is an injury to all.
Health and Safety (H&S) is the bread and butter of union organizing. Workers in Canada are guaranteed three basic health and safety rights: the right to know about hazards, the right to participate in workplace H&S decisions, and the right to refuse unsafe work (Canadian Centre for Occupational Health and Safety Government of Canada, 2025). Together, these rights offer distinct yet interconnected points of intervention for library workers hoping to address the toxic drug supply crisis and the library’s punitive response. It is through the language of H&S that a facility manager at one of my branches argued for safe disposal containers and for PPE during Covid. Through H&S language, the TPL workers union, CUPE 4948 has been able to advocate for a 3 day leave after serious incidents, and push through “no working alone” language in the collective agreement (CUPE, 2020). These are vital initiatives that the employer would certainly not have adopted on their own and that lay the groundwork for other locals to seek similar agreements with their employers.
As I have been arguing throughout this paper, labor organizers need to adopt an expansive understanding of H&S in order to be effective on this issue. Our aims must match our analysis, acknowledging the uneven, structural impact of punitive workplace policies. Take for example the increased use of security guards and partnerships with the police to address drug use in the library. We know that police and security disproportionately target, criminalize, and kill BIPOC people. What would it mean for us to organize around an analysis of police and security presence as a workplace hazard? One that disproportionately targets poor and racialized people, making the library and library work unsafe for many. Could the language of H&S help us push our bosses to invest in alternatives to policing and transformative approaches to conflict?
Organized around the recognition that police presence in the library is a hazard, library workers meaningfully included in decisions about H&S in the library can take a different approach than the employer who prioritizes liability above all else. While an employer may look at the cost saving benefit of contracting low paid security guards to deter and remove potential ‘threats’, unions can advocate for better staffing models, real and substantial training, and deeper community partnerships. Frontline workers can use the wisdom they have earned to devise alternatives to police in the library that seek to address conflict and harm in ways that aim to restore community, not dismantle it. We do not need to settle for the crumbs management provides, we can continue to struggle for real investment in our communities.
The right to refuse unsafe work is perhaps the bedrock of labor activism. The right to refuse is relatively straightforward in contexts like a construction site, where the hazard is immediate and physical. What makes Hahn’s invocation of this right in defense of unhoused peoples striking is its application to a situation that is less immediate. Even though the argument might admittedly stretch the current legal and political framework governing the right of refusal, rank and file labor organizing must build the capacity to withhold our labour in politically charged circumstances, where pushback from both employer and state is expected. When exercised individually, the invocation of this right will likely be ineffective. For example, I refused to call the police on a patron recovering from drug poisoning in a branch washroom. Despite my training as a first-responder and extensive experience in frontline harm reduction, I was fired for insubordination. Even as the central branch of EPL hosted a press conference about a public health partnership providing patrons with free Naloxone kits the very same week, no recourse to structural analysis or safety was available through my union: how could there have been? What if my individual action, though, was part of a collective refusal — an entire branch’s staff declining to call the police in situations that could be handled otherwise? Whether formalized through the avenues of union protection, or instigated directly by rank and file self organization, the right to refuse is most meaningful, effective, and defensible, when it is asserted collectively.
The ability to rearticulate the scope and language of harm is one mechanism through which library unions might begin to unseat the safety and security narrative that defines conversations about the toxic drug supply crisis in the library. This work, however, is demanding, and possible only once frontline library workers acknowledge that we have more in common with patrons who use drugs than with the library CEO. Antagonism between library workers and patrons serves management alone, which mobilizes worker fear and anxiety to enforce punitive order within the branch. Once library workers recognize that our safety is tied to that of our patrons, and to working-class people across the country more broadly, we will be positioned to advocate for policies and procedures that rise the tide for all. This is not easy work; it requires developing practices that allow us to keep our jobs and pay our bills while resisting the institutional imperative to punish our patrons. For this reason, those of us committed to abolition and harm reduction must find one another, talk to our co-workers, interrupt the criminalization of our patrons, and hold our union leadership’s feet to the fire.
Acknowledgements
Thank you to Brittany Paloma Fiedler, Jaena Rae Cabrera, and Emily Drabinski for the energy and care they dedicated to editing this article. Publishing with In the Library with The Lead Pipe was a wonderful experience under their thoughtful guidance.
Thank you to my comrades in LW4SCS. Your care, commitment, and sharp analysis has sustained me over the last few years. I have been so lucky to learn with and from some of the most badass library workers in Canada. Beth Capper, may we always support each other in ‘saying the thing’. Finally, thank you Rob Jackson, the best intellectual and life partner I could ask for. Your edits and feedback make my thinking demonstrably better.
Augst, T. (2007). Faith in reading: Public libraries, liberalism, and the civil religion. In T. Augst & K. Carpenter (Eds.), Institutions of reading (pp. 148–183). University of Massachusetts Press.
CHCH News. (2026, March 18). Hamilton chief librarian says initiative to curb drug use at Central branch is effective [Video]. https://www.youtube.com/watch?v=GBb6qTPYTjE
Hayes, K., & Kaba, M. (with Schenwar, M., & Walia, H.). (2023). Let this radicalize you: Organizing and the revolution of reciprocal care. Haymarket Books.
Helfritz, A. E. (2018). Potential workplace violence in assumed low-risk environments: Library employee perceptions and safety concerns [Doctoral dissertation, University of the Rockies]. ProQuest Dissertations & Theses Global. https://www.proquest.com/docview/2131396889?pq-origsite=primo
McConkey, S., Iannace, J., Snyder, M., Bourgeois, C., & Smylie, J. (2024). Measuring Indigenous homelessness: Findings from Our Health Counts Toronto. Canadian Journal of Public Health = Revue Canadienne de Santé Publique, 115(Suppl 2), 227–238. https://doi.org/10.17269/s41997-024-00974-7
Saunders, C. (2025). Working in an unsafe consumption site: A library perspective. Counterpoint: CUPE National Publication, (Fall 2025), 32–37.
Schlesselman-Tarango, G. (2016). The legacy of Lady Bountiful: White women in the library. Library Trends, 64(4), 667–686. https://doi.org/10.1353/lib.2016.0015
Stevenson, S. (2021, October). Health and safety culture at the public library: Listening to those on our frontlines. Public Libraries, 60(5), 32–38.
Stevenson, S. (2022). Third-party violence, incivility, and the frontline public library worker. The Library Quarterly, 92(4), 330–354. https://doi.org/10.1086/721394
Swaich, A., Richardson, L., Cui, Z., DeBeck, K., Milloy, M.-J., Kerr, T., & Hayashi, K. (2023). Experiences of violence during the COVID-19 pandemic among people who use drugs in a Canadian setting: A gender-based cross-sectional study. BMC Public Health, 23(1), Article 1020. https://doi.org/10.1186/s12889-023-15929-9
Recently, interest rates for government debt have been rising worldwide. As I write the yield of the 10-year US treasury is over 5%. It had been below 4.5% for over a year when, in March, it started a steady rise. This is despite (and probably because of) Scott Bessent's efforts to force it lower, which JP Morgan compared to 'paying your mortgage with your credit card'.
A big part of the reason is governments persistently borrowing to cover deficits, with no plan to reduce them. For example, the Trump administration has borrowed more than the growth in US GDP. Projections are that by the end of their term they will have borrowed $1.40 for every $1 growth in GDP. Half of that GDP growth is from the AI bubble. Remove it and the administration has already borrowed almost 2.5 times the GDP growth.
But another part of the reason is that governments are facing competition for the available funds from other highly-rated borrowers. Hyperscalers such as Microsoft (AAA) and Alphabet (AA+) can no longer pay cash for the immense sums they believe they need in order to keep up with their competitors in the race for AGI. So the bond market is facing a significant increase in demand, which naturally increases interest rates.
Below the fold I look into how this competition for funds is going.
It is a fact of life that the more debt you carry, the worse your credit rating and thus the more expensive it is to add to your debt. This is why the hyperscalers are so anxious to have their massive borrowings kept off their balance sheets and attributed instead to "Special Purpose Vehicles". They hope in this way to distract the rating agencies enough to prevent being downgraded. A downgrade would increase their interest expenditures, thus reduce the coverage of their interest payments, and potentially lead to a further downgrade and a death spiral.
Achieving an investment-grade rating from Fitch, Moody’s and S&P soon after going public would be a remarkable feat for the two lossmaking AI labs, unlocking big benefits for the companies and their infrastructure partners including Oracle and Nvidia. …
Analysts at rating agencies are waiting to see the results of their IPOs before reaching a decision. The two companies remain unprofitable and have shown little sign of generating positive free cash flow. They also face growing risks, including the popularity of Chinese open-weight models.
“We still treat OpenAI and Anthropic as deep in speculative grade . . . they are in the red,” said another senior credit analyst.
An investment-grade rating of BBB- or above would allow them to borrow from institutional lenders such as pension funds.
S&P reckons a full half of the economic growth coming from the US private sector was linked to AI-centric activity over the past year. And they assume the top six US hyperscalers will collectively spend more than $7tn in the five years through 2030. So the fate of the US economy, not to mention the stock market, credit, infrastructure and real estate, is increasingly tied to the AI show staying on the road.
Every time we take a deep dive into this sector, we find that capex is rising faster than we anticipated, financings are becoming more complicated and less transparent, and that returns on investment will take years to realize.
The credit rating process involves analysts diving deep into the workings of companies, typically with access to a ton of non-public information. But the authors write that, despite AI return on investment being critical to the rating judgments, “the big six hyperscalers don’t quantify their returns on investment on AI”. Maybe, the report’s authors speculate, this is because it’s difficult to work out. Or maybe, “simply, entities just choose not to share that data.”
The reason these companies "choose not to share" their "return on investment on AI" with the rating agencies is left as an exercise for the reader.
Source
Nangle has a set of interesting charts based on S&P data. This one shows the ratio of debt and "debtlike commitments" to EBITDA (earnings before interest, taxes, depreciation, and amortization) for the six largest US hyperscalers. Each company is labeled with their current rating. Horizontal dotted lines indicate the trigger level for this ratio that could cause a downgrade. Note that:
Oracle's projected 2027 and 2028 commitments are so close to the trigger for downgrading from their BBB- rating that even a small drop in earnings would render their bonds junk. This would force institutional investors to sell them, making future borrowing and rolling over existing debt as it comes due much more expensive.
Amazon's projected 2027 and 2028 commitments look as though they would cause a downgrade from AA, but it is more likely that Amazon can increase earnings fast enough to prevent this.
Alphabet has a lot of headroom before their AA+ rating is at risk.
Source
An alternative way of looking at the situation is to ask how much more could these companies borrow, or how could their EBITDA decrease, each year before they would be downgraded. This chart answers the question. Note that:
For 2027 through 2029, only Alphabet can borrow more than $100B each year.
On the current projections Amazon will, and Oracle is extremely likely to, be downgraded.
Nangle makes an English understatement:
For double-A-rated Amazon to move to high single-A wouldn’t be much of a disaster. But in Oracle’s case, getting downgraded means getting junked. And with around $117bn of index-eligible US dollar bonds, that would be quite the event.
The limited headroom for increased borrowing starts next year, which is when the "take or pay" commitments with data centers start costing serious money.
Source
The third chart provides yet another way to look at the situation. It plots the companies' projections for EBITDA against the threshhold for downgrades (dotted line). Note that:
Amazon has to exceed and Oracle has to meet their EBITDA projections if they are not to be downgraded.
All these companies' projections for their EBITDA growth from 2025 to 2029 are astonishing. Eyeballing the chart we have:
Alphabet: 2.5x
Amazon: 2x
Meta: 2.6x
Microsoft: 1.8x
Oracle: 3.5x
SpaceX: 15x
Nangle's comment that these represent "pretty punchy EBITDA growth" is another understatement. For example, Alphabet is saying it will add around $225B in EBITDA by 2029. I'm an Englishman, so I can say that it isn't clear where an additional $225B would come from.
Alphabet and Meta together are in effect projecting that by 2029 companies will increase their advertising budgets by $385B/year. Of course, this isn't going to happen. All these companies are expecting the EBITDA growth to come from the "returns on investment on AI" that they "choose not to share" with the rating agencies.
Ignoring the ludicrous SpaceX projection, the other 5 companies' "punchy" projections assume 2029 increased EBITDA of around $900B, which is less than half the estimate of over $2T/year in additional revenue needed to cover their borrowings.
Is it plausible that in 2029 OpenAI and Anthropic will together have more than $1.1T in revenue to make up the difference?
The report also features a circular financing taxonomy, which includes residual-value guarantees, take-or-pay agreements, chip financing, lease liabilities, power purchase agreements, backstop guarantees, lease guarantees, and direct equity investments.
interconnected financing structures could amplify volatility if demand weakens unexpectedly. The failure of one entity would have implications for entities that have lent it money, have guaranteed the value of its assets, or are just expecting payment for goods delivered.
The broader question is whether circular financing is creating leverage collectively that is individually manageable but could become highly correlated should demand fall sharply. . . .
The scale of overlap and interconnectedness is vast. In a downturn, even the best capitalized and most profitable firms may incur substantial pain.
Ed Zitron's Concentration Risk points out another big risk to the hyperscalers' EBITDA projections that S&P seems to have missed:
Anthropic and OpenAI’s compute commitments, in my mind, should be seen more as debt obligations than “contracts,” because they (as take-or-pay agreements) function in much the same way, requiring the company to pay whether or not they need the capacity.
For now, everything looks awesome. Microsoft, Google and Amazon have all had big bumps in revenue from AI lab compute spend along with massive, ever-swelling revenue backlogs — over $1.5 trillion worth to be specific. More than half of that backlog is attributable to Anthropic and OpenAI, which, as I’ll say again and again, isn’t a problem because the money is yet to stop coming in.
Eyeballing the chart, something like $1.1T of the revenue backlog for Microsoft, Oracle, Google and Amazon is revenue that they expect to get from two companies that have yet to make a profit. That is a concentration risk.
Per data from fintech firm Ramp, 80% of OpenAI and Anthropic's enterprise revenues come from 1% of their customers, a number that hasn’t improved over the last three years. Ramp’s lead economist Ara Kharazian notes that the top 1% skews heavily toward the tech sector and AI products and services, and that this was a level of concentration risk unseen in any other software category they tracked.
benchmarking company Vals AI has been evaluating different open and closed models by using its own neutral harness. When every model ran on the same harness in the Terminal-Bench 2.1 evaluation, the open-weights model GLM 5.2 from Chinese company Z.ai (Zhipu AI) scored within a point of Anthropic’s Claude Opus 4.7 and 4.8 while costing about five times less per completed task.
In other words, paying for closed frontier models buys about a four-month head start at about five times the per-task cost—but only when currently looking at tasks taking [a human] between eight and 12 hours.
Below 8 hours there is no advantage for closed-source, above 12 hours neither model can reliably succeed. This isn't a great basis for OpenAI and Anthropic to project hundreds of billions of dollars of future revenue.
On the low end, that means that Anthropic and OpenAI account for over $200 billion dollars worth of expected revenues for Microsoft, Google and Amazon in 2027, which is contingent on their ability to raise venture capital or debt, which is contingent on the continued growth of their businesses, which is contingent on growing AI spend from a small subset of customers, many of whom are funded by venture capital.
The reason this hasn’t been a problem yet is that when you sign these contracts, you tend to pay a small up front fee, and the capacity in question is yet to come online.
...
In other words, Anthropic and OpenAI are currently in the teaser rate period where all of that capacity — and all of the associated costs — are yet to hit.
Next year, at least $200 billion in compute costs are coming due.
The question is whether Anthropic and OpenAI, two unprofitable, unsustainable AI labs that lose tens of billions of dollars a year, will be able to afford to pay them.
If they can't pay, about half of the revenue backlog for Microsoft, Oracle, Google and Amazon goes away. That reduces their EBITDA and potentially triggers downgrades.
Nangle starts with this chart, comparing the yields on the hyperscalers bonds with the average for similarly rated companies. It shows that:
it looks like the market is pricing Oracle and SpaceX as quality junk, and maybe Meta in line with BBB credit, and the rest of them as sort-of-in-line with the average AAA-AA-rated US corporate issuer. Is this cautious enough?
On recognized debt, the market’s calm is justified . . . [but] . . . [t]he real risk lurks below the waterline: off-balance-sheet debt lifts the debt burden by nearly 150% on average and pulls the model-implied credit quality down 1-2 notches.
There follows a complex description of Allianz' model that maps from stock prices to bond ratings, summarized thus:
Allianz calculates the ‘distance to default’ for the hyperscalers — a measure of credit riskiness calculated using equity inputs and balance sheet data — over time. And following work that Moody’s KMV has done mapping distance-to-default to default frequencies, they map the kind of ratings associated with these outputs.
Working solely off recognised balance sheet data, Allianz finds that the ratings implied by equity punters’ collective histrionics are higher than ratings assigned by the agencies for Alphabet, Amazon, Microsoft and Nvidia, a touch lower for Meta and a disaster for Oracle and SpaceX.
I think that's not quite right. Moody's gives Microsoft their highest rating, Aaa, and so does Allianz' model. The model gives Alphabet, Amazon and Nvidia Aaa, which is higher than Aa2, A1 and Aa1 respectively. But overall equity investors are (naturally) more optimistic than bond investors.
But these ratings are a testimony to the effetivness of the hyperscalers' off-balance-sheet techniques for distracting both the equity markets and the rating agencies. Nangle writes:
Once they whisk debtlike uncommenced leases into the equation, they find that Microsoft and Amazon drop into mid-investment-grade territory and that Meta falls into the quality end of junk.
Nangle's first chart showed the market already rating Meta at "the quality end of junk". But once the bond market assimilates the off-balance-sheet debts the downgrades and interest rate increases will be quite dramatic.
What made you decide to tell Ray Trautman’s story? Was he a figure you encountered in your previous exploration of books and reading in the US Army during World War II?
I first encountered Ray Trautman when I was writing When Books Went to War (2014). I mentioned him a few times in that book, but he wasn’t a central figure. Remarkably, after that book was published, Ray’s sons reached out and invited me to visit them and take a look at their father’s records of his wartime work—thankfully, being a librarian, Ray kept everything! As I learned more about him, I was amazed by the incredible odds Ray faced in trying to get books and magazines to troops and his undaunted determination to do whatever it took to get soldiers the reading materials they wanted. I uncovered many forgotten stories about famous people—Eleanor Roosevelt, Ernest Hemingway, FDR, etc.—who Ray interacted with. I just found his life fascinating, and felt others would too.
Tell us a little bit about Trautman—who he was, what he did, and why he should be remembered.
Ray grew up on a farm in Mowrystown, Ohio (population: 200), where there was no community library and very few books. Yet, he treasured every book he could get his hands on—he loved learning about other people’s lives, making new discoveries, meeting beloved characters, and exploring the world. A seed was planted—he would ultimately decide to devote his life to books and libraries, and giving others the opportunity to experience the joys of reading.
His greatest contributions were probably made during World War II, when he became the Army’s Chief Librarian. Ray designed miniature paperbacks, Armed Services Editions, so foot soldiers around the world could carry weightless pocket-sized books wherever they went. Ray knew from experience that these books were many solders’ emotional survival gear. Because troops also wanted magazines, Ray worked with every major American magazine to create small, ad-free versions for troops. And he created an Army comic book, Overseas Comics, so troops would get their favorite cartoon strips.
Ray was responsible for distributing, for free, a half-billion books, magazines, and comics during World War II. In the process, an entire generation of Americans became devoted readers. Most entered the war with no reading habit, and they left the military with a lifelong desire to read, which veterans passed on to their families and friends. He truly revolutionized American reading.
What kind of research did you need to do, while writing the book, and what were some of the most memorable things you learned? Were there surprises along the way?
First, I needed to comb through boxes of records that Ray had saved, along with diaries, photographs, and notes tucked inside of his books. Ray’s son, Eric, secured Ray’s military records from the Army—thankfully, that consisted of almost 1,000 pages that provided guideposts for where Ray was, what he was doing, and when. For more details and context, I scoured newspaper and magazine articles, and read dozens and dozens of books about the people, places, and events Ray encountered.
I was amazed to learn how many famous people this unassuming librarian crossed paths with. Ray solved problems for Eleanor Roosevelt and Ernest Hemingway—and these stories have never been told! He also met Douglas MacArthur (and almost got court martialed for gently suggesting that MacArthur supply his troops with books), and was given library assignments directly from Dwight Eisenhower. One of the more memorable and surprising episodes I wrote about was when Trautman was trapped during the Battle of the Bulge and was forced to lead troops in a tank battle using only homemade weapons—and they survived! It turned out a librarian’s skillset was very useful in coming up with an unconventional solution in a life-or-death situation.
Three of your four books so far have focused on books and reading during World War II. What makes this subject so interesting for you, and why is it important?
I think examining reading during wartime fascinates me because history shows that when democracy has come under attack (in both World War I and World War II), the government has emphasized the importance of the freedom to read and access information. While the United States implemented some censorship measures and produced propaganda, a dominant message during both wars was that practicing democracy—by reading whatever you want, sharing books, donating books to troops so they had access to a variety of reading materials, etc.—was an important way to protect democracy. During the last decade or so, we’ve seen a growing tendency to censor and limit what books are available. I think the history I write about is more important than ever.
Tell us about your writing process. Do you have a particular schedule you keep, or places you like to write? Do you start from an outline, and have all of your information organized before starting each chapter or section?
When I’m writing, I like to start in the morning and have several hours that will be uninterrupted so I can entirely focus on what I am doing. If I’m writing on my computer, all notifications are turned off and my phone will be in another room so I have absolutely no distractions. I have a small room in my house that’s loaded with bookshelves and has a large table—this is where I do most of my writing. I love to have my sources spread out in front of me, almost like having a roadmap of the day’s writing.
For each book I’ve written, I’ve had a different strategy for writing. Sometimes a chapter becomes clear to me before the entire book does, so I might write a chapter and then return to researching other parts of the book. For A Librarian’s War, after I organized all of the material I was planning to use, I did make a chapter-by-chapter plan for how to put it together. Usually, I avoid outlines because I find that the process of writing sharpens my understanding of a story and the final story may not have a strong resemblance to my original plan. I try to stay curious and flexible throughout the process.
What is next for you? Do you have any new books in the works, or book ideas you hope to work on?
I have an idea for my next book and have been doing a lot of preliminary research and reading to try to figure out the story’s contours. It will be an entirely different adventure for me, as I’m changing time periods and won’t be focusing on books in wartime. Each book I’ve written began because a story crossed my path and took root in my heart and mind. Once again, I have found a forgotten chapter of history and I feel compelled to preserve it. For now, I am happily gathering as many pieces as I can before attempting to figure out the structure.
Tell us about your library. What’s on your own shelves?
I have a bit of a book problem at home, as I have too many of them (if there is such a thing). Being a non-fiction author, I read a ton of non-fiction because I want to understand the period I’m writing about from as many different perspectives as I can. So my shelves are full of books from the World War II and World War I eras, along with my favorite works of fiction and any other books I have read and feel such a connection to that I can’t part with them. I also have a shelf devoted to signed books, from books talks I’ve attended by authors I admire.
What have you been reading lately, and what would you recommend to other readers?
In the hours following the release of CVE-2026-92392 for the project curl, site reliability workers
and systems administrators scrambled to desperately rebuild and patch all their systems to fix a use-after-free in HTTP/2 upload retries where cleanup turns leftover upload data into a new request and leaves a pointer to a freed Curl_easy handle (apparently freeing it was only a suggestion), allowing a malicious server to achieve remote code execution under specific conditions. This is due to the affected components being
written in C, the only programming language where these vulnerabilities regularly happen. "This was a terrible tragedy, but sometimes
these things just happen and there's nothing anyone can do to stop them," said programmer Ms. Odie Leuschke, echoing statements
expressed by hundreds of thousands of programmers who use the only language where 90% of the world's memory safety vulnerabilities have
occurred in the last 50 years, and whose projects are 20 times more likely to have security vulnerabilities. "It's a shame, but what can
we do? There really isn't anything we can do to prevent memory safety vulnerabilities from happening if the programmer doesn't want to
write their code in a robust manner." At press time, users of the only programming language in the world where these vulnerabilities
regularly happen once or twice per quarter for the last eight years were referring to themselves and their situation as "helpless."
In Downgrades and Competition For Capital I discussed how AI's insatiable appetite for funds was raising interest rates not just for AI-related debt, but also for everyone else. The impact of these rate increases will take time to become evident, as borrowers roll over existing debt or raise new debt.
Below the fold I look at impacts that we don't have to wait for, because they are already happening on a large scale.
Now, Paul Krugman's AI Versus Everything Else presents a different aspect of the competiton for capital — it is "crowding out" investment in everything else:
Evidence of that crowding out is revealed by a recent article in the New York Times, which noted that the AI Boom is not being slowed by these soaring rates. Yet the spike in interest rates is being felt throughout the non-AI economy. In other words, all that money going into construction of datacenters and purchasing foreign produced semiconductors comes at the expense of investment in everything else.
The chart at the top of this post, which I borrowed from Steve Rattner, clearly makes this point: construction of datacenters appears to be diverting funds and resources from the construction of housing, office buildings, and factories — more or less everything that isn’t AI-related.
That said, AI-related construction — building the structures that house datacenters — is only a fraction of the AI spending boom. Most of the money is going for equipment and software. In fact, that’s one main reason AI spending is so resistant to high interest rates: Borrowing costs matter much less for assets like GPUs, which depreciate rapidly in any case, than they do for long-lived assets like buildings.
Rattner's chart only showed the effects within the construction industry, which absorbs only about 40% of the spending on AI data centers. Krugman's Chart 2 shows the effects on the other 60% of the spending:
I look at business investment in “tech” — information processing and software — measured as a percentage of GDP. The blue line in the graph below shows how much that measure of tech spending has changed since the beginning of 2023. I compare that with the corresponding change in all other investment. This chart clearly reveals dramatic crowding out by tech spending of all other investment:
Interestingly, given the analogies being drawn between the AI boom and the dot-com boom, Krugman shows that the effects of AI spending aren't just much bigger but also different:
Chart 3 shows the same calculation as Chart 2, but applied to the tech boom of the 1990s, and it doesn’t show any crowding out at all — in fact, non-tech investment rose:
Low interest rates imply that people will swap their claims on today’s productive potential for a song. In such cases, a debt-financed investment plan need not yield very much to pencil out. As rates rise, however, the expected return on an investment plan needed to justify the borrowing costs also goes up. Decent investments get squeezed out, then good investments, and then maybe really excellent investments that stand to deliver big economic benefits. So it goes.
We are experiencing this today, and you’ll never guess which part of the economy is responsible for the pronounced and sustained rise in desired borrowing.
But Avent argues that the "part of the economy" isn't AI but governments:
Anyway, gross government debt across advanced economies rose from 67 per cent of GDP in 2001 to 108 per cent today. So one could say that AI borrowing is crowding out other private investment, but one could also, and perhaps more reasonably, say that government borrowing is stymieing what might otherwise be a very broad-based, AI-driven, private sector investment boom.
If this turns out to be the better way to think about it, we’ll know soon enough. Firms in non-AI sectors may abandon efforts to claim more of the economy’s productive potential in the face of rising interest rates, but governments almost certainly won’t. For governments to give way, budget deficits need to fall. Accomplishing this means disappointing people who are expecting to receive claims from the government. It also means people have to give up more of their own claims to the government through higher taxes. In this fractious age, governments that do such things tend to be tossed out in favour of crazier governments which are even less able to do the needful.
Unfortunately, this likely means that we are on one of two paths. Either the AI complex ultimately smashes itself against the irresistible borrowing needs of the public sector and is forced to retrench, with potentially unpleasant consequences for growth, or rising interest rates cause one or another sovereign to slip into a market-destabilising crisis first.
The re-direction of investments toward AI would not be a problem if two conditions were satisfied. First, that the AI investments were more productive than those they crowded out. Skepticism on this question is warranted. There is a distinct lack of empirical evidence that AI is increasing productivity in the economy as a whole. Krugman expresses this skepticism:
Moreover, we shouldn’t assume that the private sector knows what it’s doing. There’s often a double standard when thinking about bad investments, in which public investments that go bad are treated as evidence that the government can’t do anything right, while private malinvestment is brushed off as no big deal.
Think about the $80 billion that Meta, formerly known as Facebook, spent on the “metaverse,” only to basically abandon the concept. A government program that spent $80 billion on a failed project would be the subject of endless Congressional hearings. Yet the waste from a failed private investment is equally real.
Second, that the enhanced productivity from the AI investments was at least as durable as that from those they crowded out. But the benefits of the 60% of the iAI investment that goes into IT hardware decay very rapidly, as I discussed in
Mind The GAAP Again and its predecessors.
So it’s clear that crowding out by the AI boom is happening on a very large scale. And unlike the risks of technological unemployment or a burst bubble, this isn’t a hypothetical risk. It’s happening right now.
Subsequently, Krugman also asked Will AI Crowd Out Jobs?, pointing out that a data center that might cost around $8B might only employ about 100 workers, mostly in low-skill jobs like security:
deploying $8 billion in capital, the cost of a data center, in a non-AI sector will, on average, support more than 10,000 jobs, while investing that money in a data center will support only around 100 jobs. Or to put it another way, money spent building data centers will, on average, create only around one percent as many jobs as investing the same sum in other industries would have created.
Van Nieuwerbergh estimates that through 2032 hyperscalers will spend more than $10 trillion on data centers — capital expenditure that, all the evidence suggests, will come at the expense of investment elsewhere in the economy. And because data centers employ so few people, a back-of-the-envelope estimate yields a number that made even my eyes bug out: the capital invested in data centers will employ roughly ten million fewer workers than that capital would have employed if invested elsewhere.
This doesn’t mean that unemployment will rise by 10 million. It means, instead, that reduced demand for labor will put downward pressure on wages and shift the balance of economic power, which has already been moving against labor, even more in capital’s favor. Even before the rise of AI, we were experiencing a rapid decline in the share of national income going to wages relative to profits, along with rising oligarchy. The anti-labor effects of AI will accelerate that trend.
For the AI companies' managements, this seems like a feature not a bug.
Rattner's and Krugman's charts show a rapid increase in the crowding out effect. But the worrying thing that Krugman doesn't mention is that the AI investments that have already had such a marked crowding out effect are small compared to those planned for the next few years.
This chart, from Big Tech will fund more than a third of its AI investments with debt in 2027, Goldman Sachs predicts by Jake Conley, reports Goldman Sachs' estimates of the scale of the investments by the hyperscalers, and how much they expect them to borrow:
hyperscalers have turned increasingly toward debt markets. In 2025, investment-grade-rated global bond issuance from the hyperscalers totaled $108 billion, per Goldman Sachs, or roughly 26% of capex. Through the first half of 2026, those companies have already issued $194 billion of IG-rated debt, with that figure expected to reach roughly $250 billion in debt issuance, or roughly 33% of capex.
In 2027, Goldman Sachs said, the share of debt-funded capex is likely to rise even further, with $400 billion in expected global IG bond issuance from the hyperscalers against an estimated $1.14 trillion in capex, or a roughly 35% share.
If these investment plans materialize, the crowding out effect will greatly accelerate. In the 18 months from January 2025 it reduced non-AI construction spending by $132B, or about six week's worth. The chart suggests that spendig will double from around $700B to around $1.4T in 2028. That might equate to crowding out 12 weeks worth of non-AI construction in 2028, or about a quarter. That would be a significant hit to the broader economy
the AI complex ultimately smashes itself against the irresistible borrowing needs of the public sector and is forced to retrench,
Zitron is skeptical that these investment plans will materialize:
In any case, AI data center debt is way more expensive now by virtue of the current state of treasuries, with costs compounded by the anxiety around them in general.
This means that any “virgin” projects — those that aren’t directly backstopped or co-signed by hyperscalers — are guaranteed to hit egregiously-high, 9% to 14% rates, which makes (as I discussed a few weeks ago in my two-part debt series) it near-impossible to make the already-questionable economics of running a data center work.
This means that any AI data center debt being raised right now is doing so under stricter credit conditions and doing so at unrealistic, unsustainable prices, if they’re going to be able to raise at all. This makes the $174 billion in debt that SB Energy needs to raise to fund its theoretical data center project with OpenAI — even when backstopped by NVIDIA — either horrendously expensive or impossible to complete, as does it mean that any future large, multi-gigawatt projects will add billions of dollars in interest payments to already-expensive debt.
OpenAI and Anthropic have made huge, non-cancelable "take-or-pay" commitments to the hyperscalers. These form the basis of a vast amount of the hyperscalers' backlog. The compute these commitments cover is for both training, a cost, and inference, which generates revenue. In Thoughts on The Curve and the Future of AI Thomas Wright reported an estimate:
that the labs spend 58% of their compute on R & D and 32% on servicing customers.
Zitron estimates that OpenAI and Anthropic represent 80% of the total demand for compute. Thus about half of the total demand for compute is OpenAI and Anthropic's training and attacking other companies. Halving their internal compute budget would allow for about a 50% increase in compute for customers with no need for additional investment. This is why they are using the claim that their products are too dangerous to release to call for a pause and regulation.
But even if the competitors could agree, and they weren't worried about the Chinese catching up, halving their training budget wouldn't work because the demand doesn't exist:
As a result of their $1.3 trillion in compute commitments, neither company can become “capital efficient” by cutting their training costs, because said training costs are the only means of further growth outside of massive price increases that are unlikely to grow their businesses.
To make matters worse, both are cutting prices to compete with each other, and per Ramp, said price cuts aren’t increasing usage:
Price cuts, flat usage, cutting R&D and revenue disappointments aren't a good look when you're planning a $2T IPO or a $30B funding round at a $1.4T valuation. Skepticism about the projected capex spending, and the increased crowding out is reasonable. But the effects on the broader market of the bubble deflating will be severe.
Panmure Liberum’s Joachim Klement said his base case is for that trade to disintegrate as soon as 2027, sending stocks sharply lower.
“My core conviction is that the AI bubble will either burst in 2027 or in 2028, so sometime in the next two years,” Klement said in an interview. Hyperscalers’ free cash flows are largely depleted, while the cost of debt is rising quickly and becoming prohibitive for these firms, he said.
Klement’s 2027 year-end target of 5,000 points for the S&P 500 implies 36% downside from current levels. ... Klement sees Europe’s Stoxx 600 falling to 430 points, more than 30% below current levels.
There are many more pessimistic forecasts. Hang on to your hats.
The open data activist and Open Knowledge Foundation board member joins us to discuss the less-told part of CKAN's 20-year history: how a small open-source project became the default for government open data, and the product, sales and community work that made it happen
I had the good fortune to attend an Archiving the Black Web
meeting last week. The main focus of the few days in St Louis was for fellows
to share stories of web archiving work that they had been involved with.
A few themes emerged for me while listening to these presentations. We
weren’t using Chatham House
Rules, but I didn’t ask anyone if I could write about their work
here, so these are broad strokes.
Fellows were often trying to contact website creators to negotiate
consent. This process seemed like it was quite time intensive, and
difficult, especially when contact needed to be established, and formal
process was being followed.
Some fellows were archiving within their existing trust networks, where
consent wasn’t such a big hurdle, and seemed to be established in a more
informal way.
Helping website creators understand what it meant for their websites to
be archived was challenging. Communicating why they would want to have a
web archive of their content was also challenging.
There was often a desire to create new websites that arranged the web
archives along with additional materials, such as oral histories, maps,
and other descriptive material.
There was also a need to limit access to a community of known users.
These communities aligned with existing trust networks. I inferred from
this that limiting access was one avenue to gaining consent.
Many of the fellows were using Browsertrix to create web archives,
and some were also using ArchiveWebPage to do more detailed
patch crawling. It seemed like there had also been some initial early
work using Archive-It.
When it came to integrating access to archived content, fellows were
often linking to the playback mechanism inside of Browsertrix itself.
Sometimes the website creators were given access to archived web
content, and offering to serve as a backup.
A few were thinking about the sustainability of the web archiving work
in terms of having their practice become more of a service. I had an
interesting side conversation about how a cooperative model could
potentially help, grow a web archiving practice as a form of mutual aid.
Given the interest in creating web sites for the archived web content,
and administering access to them, I mentioned to one of the organizers
that it would be super if the fellows could be introduced to how to
embed WACZ files in their own sites using the [ReplayWebPage] web
component. I think they were already planning on doing this, and that it
just hadn’t happened yet since the focus so far had been on creating the
web archives.
I think it’s super interesting how methods of access seems to sometimes
gate the types of material, and whose material, gets archived.
Understanding goals in terms of access, such as where the web archives
live, and who can see them, seems to shape the types of content that can
be archived? I think this has a counterpart in traditional archives
where the deed of gift often stipulates how and when records can be
accessed.
Donors introduce a dynamic element to the archival process, bringing
their own values, ideas, and interests. Examining donors through the
concept of agency, as developed in the social sciences, reveals that
they exercise significant influence on key archival functions, which in
our theoretical models are generally treated as the sole purview of
archival professionals. (Fisher, 2015)
I heard from at least one fellow that they were working with Omeka, and
that being able to embed WACZ files in Omeka could be desirable. Omeka S has a WebArchive
plugin that supports WACZ replay, but I noticed that Omeka Classic did not, and it is
still being actively used (and maintained).
Since I had some time to fill while waiting for my plane back I “used”
Claude to create a WebArchive plugin for
Omeka Classic. It was just 80 lines of PHP, and I haven’t written
PHP in over a decade, so I was grateful for the help. Once you’ve
installed the plugin you can upload WACZ files and replay them directly
in Omeka, where the data is stored, and you can use Omeka’s access
control settings to set who can see them. Everything old is new again…
I had the good fortune to attend an Archiving the Black Web
meeting last week. The main focus of the few days in St Louis was for fellows
to share stories of web archiving work that they had been involved with.
A few themes emerged for me while listening to these presentations. We
weren’t using Chatham House
Rules, but I didn’t ask anyone if I could write about their work
here, so these are broad strokes.
Fellows were often trying to contact website creators to negotiate
consent. This process seemed like it was quite time intensive, and
difficult, especially when contact needed to be established, and formal
process was being followed.
Some fellows were archiving within their existing trust networks, where
consent wasn’t such a big hurdle, and seemed to be established in a more
informal way.
Helping website creators understand what it meant for their websites to
be archived was challenging. Communicating why they would want to have a
web archive of their content was also non-intuitive.
There was often a desire to create new websites that arranged the web
archives along with additional materials, such as oral histories, maps,
and other descriptive material.
There was also a need to limit access to a community of known users.
These communities aligned with existing trust networks. I inferred from
this that limiting access was one avenue to gaining consent.
Many of the fellows were using Browsertrix to create web archives,
and some were also using ArchiveWebPage to do more detailed
patch crawling. It seemed like there had also been some initial early
work using Archive-It.
When it came to integrating access to archived content, fellows were
often linking to the playback mechanism inside of Browsertrix itself.
Sometimes the website creators were given access to archived web
content, while offering to serve as a backup.
A few were thinking about the sustainability of the web archiving work
in terms of having their practice become more of a service. I had an
interesting side conversation about how a cooperative model could
potentially help grow a web archiving practice as a form of mutual aid.
Given the interest in creating web sites for the archived web content,
and administering access to them, I mentioned to one of the organizers
that it would be super if the fellows could be introduced to how to
embed WACZ files in
their own sites using the ReplayWebPage web component. I learned
that they were already planning on doing this, and that it just hadn’t
happened yet since the focus so far had been on creating the web
archives. I still think Webrecorder’s innovation of allowing
archived web content to be replayed anywhere on the web is one of their
most transformative ideas.
I think it’s super interesting how methods of access seems to sometimes
gate the types of material, and whose material, gets archived.
Understanding goals in terms of access, such as where the web archives
live, and who can see them, seems to shape the types of content that can
be archived? I think this has a counterpart in traditional archives
where the deed of gift often stipulates how and when records can be
accessed.
Donors introduce a dynamic element to the archival process, bringing
their own values, ideas, and interests. Examining donors through the
concept of agency, as developed in the social sciences, reveals that
they exercise significant influence on key archival functions, which in
our theoretical models are generally treated as the sole purview of
archival professionals. (Fisher, 2015)
I heard from at least one fellow that they were working with Omeka, and
that being able to embed WACZ files in Omeka could be desirable. Omeka S has a WebArchive
plugin that supports WACZ replay, but I noticed that Omeka Classic did not, and it is
still being actively used (and maintained).
Since I had some time to fill while waiting for my plane back I “used”
Claude to create a WebArchive plugin for
Omeka Classic. It was just 80 lines of PHP, and I haven’t written
PHP in over a decade, so I was grateful for the help. Once you’ve
installed the plugin you can upload WACZ files and replay them directly
in Omeka, where the data is stored, and you can use Omeka’s access
control settings to set who can see them. Everything old is new again…
My recent search for a CULTIVATE postdoc (position now filled) got me thinking about what info I could share with recent and soon-to-be PhDs in information, LIS, and related areas. Here are some very generic notes related to the postdoc job search.
Some postdocs find funding by reaching out to senior researchers in the areas most related to their future work: This works best in well-funded fields (like medicine in this example ) or when a country or institution has well-developed opportunities.
Identifying fellowship funding can be an excellent way to write your own position.
I especially recommend the Marie Curie postdoc, an annual call (deadline each September) funded by the European Commission. While competitive, the funding is ample and the opportunity is prestigious.
Recent PhDs of any nationality can propose a 1-2 year postdoc with a mentor in Europe or the many “Horizon Europe Associated Countries”
Europeans (and long-time European residents) with PhDs can propose Global Fellowships: mentored for 1-2 years in a “Third Country” (such as the US) followed by a mandatory reintegration year in Europe/“Horizon Europe Associated Countries”
Fellowships tend to depend on:
the field of study
where the research will be carried out (country, institution, …)
aspects of the researcher (nationality, gender, country of residence, mobility, …)
Fellowships typically come from several places:
the hosting institution (most common when there’s a senior academic you’re visiting, or a library/archival collection that has a fund)
short-term grants from national organizations in the country where you’re based
short-term grants from national organizations in the country you’re visiting (some countries like Germany have lots of these)
groups like COST (I’ve had one from them, bottom-up funding for research groups)
professional societies
private foundations
I share a very out-of-date list I kept as a PhD student, focused on Europe (again, it is just indicative but may point to countries with robust funding for postdocs)
For librarians and library school faculty in lower and middle income countries, the Mortenson Center for International Library Programs may be a resource for training or for US-based professional development. For instance, its Associates Program is an annual program at the University of Illinois Urbana Champaign which has partial scholarships but requires substantial resources for living expenses, travel, and visa costs.
I have always been a bit obsessed with serving static web content that is portable, cheap, maintenance-free, and easy to distribute and mirror. My reasons are practical. I often work on non-profit projects in my spare time, with little or no budget. The web is also changing fast: hosting and server prices are rising, and crawlers are devouring resources, making it harder to keep dynamic applications running. There is a political side too. The governance of the web is fragile, and content can be blocked, taken down or lost for political reasons. Content that is easy to copy and mirror is much harder to make disappear.
The idea of serving digitized documents as zipped packages came to me in 2014, with deepzoom-osd-server. I later wrote about it in IIIF, tropy, canopy and SZI. Earlier this year I built mkiiif for my own needs, but one question stayed open: how do you serve this packaged content?
I knew Service Workers and HTTP range requests could be the answer, since Webrecorder uses the same pattern to replay WACZ files. But Service Workers have always been black magic to me, so I asked Claude to help me build a proof of concept. Here it is: github.com/atomotic/swiiifz, with a demo.
sw · iiif · z = Service Worker · IIIF · Zipped (pronounced "swiff-zee")
What it does
A IIIF Image API level 0 tree has one info.json and thousands of pre-cut tiles per image, plus a Presentation manifest. That normally means thousands of small files to upload, version and keep in sync.
swiiifz puts the whole tree inside one uncompressed zip on any static host. A Service Worker of about 170 lines turns it back into ordinary IIIF URLs in the browser. The viewer and IIIF clients don't know the zip exists. They request …/page-001/info.json or …/page-001/0,0,512,512/512,512/0/default.jpg like any other IIIF resource, and the worker answers by reading the file out of the zip with an HTTP Range request.
There is no server code, no build step on the host and no index file. You only need a host that supports Range requests.
How it works
Routing. A request for <id>/<path> is answered from the entry <id>/<path> inside zip/<id>.zip. Everything else goes to the network unchanged.
Opening a zip takes two requests. The worker reads the end of the file to find the zip's central directory, then fetches and parses it into an in-memory map of file names to offsets.
Reading a file takes one request. The worker fetches just that file's bytes with a Range request, so each tile costs about the same as a normal static tile.
Consistency. If the zip is replaced on the server, the worker notices (via ETag and If-Range), reopens it and retries. It never mixes bytes from two versions.
Compression. Tiles are JPEGs, so the zip is built without compression (zip -0). Compressed entries still work, because the worker can inflate them.
You can watch this in your browser's DevTools. Every tile appears twice in the Network tab: once as the virtual request answered by the Service Worker, and once as the 206 Partial Content request the worker sends to the zip.
The demo
The demo uses a 106-page Italian archival journal, ARCHIVI & COMPUTER (1995), converted with mkiiif:
Source PDF
13 MB
IIIF tree
1,910 files, 46 MB
Zip
1 file, 42 MB
Trade-offs
Pros
One file to upload, version, checksum, mirror or delete.
Replacing the zip is atomic.
No file-count limits, and git and CDNs stay happy.
Cons
The first view costs a few extra requests, plus the Service Worker's start-up time.
It only works in browsers with Service Workers. Harvesters, curl and server-side tools see a zip, not a IIIF endpoint.
It is still a proof of concept: no response caching, no ZIP64 (so a zip must stay under 4 GB), and the index is lost when the worker restarts.
The big limitation: interoperability
There is a major limitation. IIIF's interoperability is lost, because the manifest and tiles are not public, referenceable URLs. They exist only inside the browser, through the Service Worker.
Could this be solved? Perhaps the manifest should be served outside the zip. A new IIIF extension, something like "IIIF Level 0 packaged", could then tell viewers and clients how to load the content from the package.
What's next
The current demo is not truly portable. IIIF ids are absolute URLs, so the manifest is tied to the address it was built for. I have a branch where I am testing a rewrite so that the same zip works on any host.
Screenshot of the first alpha of what we initially called SmartFox and then Firefox Scholar, before we consulted 1) a lawyer, and 2) an Albanian dictionary for other options, June 2006
* * *
Today, software can be created more or less instantly with AI, for a large user base or for yourself, for any purpose or for no serious purpose at all. As Robin Sloan enticingly puts it, just “ask for what you want.” The genesis of Zotero, the “free, easy-to-use tool to help you collect, organize, annotate, cite, and share research,” used by over 20 million people in dozens of languages and countless disciplines, is the opposite of this instant gratification. It took five years, not five minutes, to produce a rough initial prototype, and years after that to refine and expand the application. If AI had existed in the early aughts, we could not have accelerated Zotero’s conception, because we did not know exactly what we wanted, and so could not have written coherent prompts for an LLM. Instead, it took a great deal of time and collaboration to develop a clear vision for what Zotero should be. But that slow formation led to software that was durable rather than ephemeral, with a strong foundation that could be built upon. As programming increasingly becomes a caffeinated bender with magical bots, this measured pace and emphasis on communal thinking likely holds an important lesson.
On this 20th anniversary of Zotero’s launch on October 5, 2006, I will do my best to recount Zotero’s pre-history, what it took to conceive, design, and build the 1.0 release. It’s a story about a bunch of historians who were knowledgeable about emerging technologies and knew how to code — as a secondary rather than primary skill — and who spent a lot of time, some of it wasted time, at a big table chatting. That conversation eventually coalesced into ideas about the future of scholarly research.
As the well-worn saying correctly asserts, failures are orphans while successes have many parents, and Zotero is no different. A proper accounting would require far more than this relatively modest post, and what follows is, unsurprisingly and necessarily, from my own perspective. My contributions to the success of Zotero date largely to the years before and immediately after the launch of the 1.0 beta, when I was an Assistant Professor of History and Director of Research at the Center for History and New Media at George Mason University. This institute is now the Roy Rosenzweig Center for History and New Media (RRCHNM), named after the visionary historian who founded it and really the entire field of digital history. Roy tragically passed away in 2007 when he was only 57, and his illness and death hang like a dark cloud over this narrative.
I succeeded Roy as the Director of RRCHNM, and from that point on, my day-to-day work on Zotero ebbed, as I handed off responsibilities to other, more capable people: Zotero’s talented lead developer Dan Stillman; Sean Takats, who has deftly overseen the massive growth of the project for much of the last two decades; many others in the orbit of RRCHNM and the non-profit Corporation for Digital Scholarship who handled outreach, support, and countless details; an international team of developers and volunteer contributors; and still more colleagues, only some of whom, regrettably, I’ve been able to highlight below. On this anniversary, you should visit Zotero’s Credits and Acknowledgments page to honor the full slate of productive collaborators. Sean Takats also has a great post on the continual improvement and influence of Zotero since its launch.
I’m proud to have been there at Zotero’s origin and early years, and to have helped bring it into existence. At the college graduation of one of my kids this past spring, an attendee who had heard that I had been involved with Zotero hugged me, unexpectedly and at length, in appreciation. No one has ever done that for my scholarly writing, or ever will.
Members of the Zotero team in late 2006. Roy Rosenzweig, months into chemotherapy, is kneeling at center; he had just gotten the Zotero license plate for his Prius to the great delight of all of us. Standing, L-R: Sean Takats, Trevor Owens, Josh Greenberg, me; Kneeling: Kari Kraus, Roy. Photo credit: Sharon Leon
* * *
Software development was not originally on the docket for the Center for History and New Media. With its now delightfully retro, turn-of-the-century name — new media rather than digital — it was largely in the business of developing sites for the web, which was just a few years old in 1994 at the center’s founding. I joined RRCHNM in January 2001, as a newly minted Ph.D., to work on a project on the history of science.
Alongside another new Ph.D. who had studied the history of technology, Jim Sparrow (now a history professor at the University of Chicago), we built a website called ECHO: Exploring and Collecting History Online, funded by the Alfred P. Sloan Foundation. “Collecting” was a new gerund for the center, and hinted at an expansion of our methods. Since science was growing exponentially but the number of historians of science was not growing at all, we thought we could use the web to help scientists self-document their work. For this, our site needed to be interactive, so we started to tinker with “tools” — small web apps — that could not only display history online, but also allow it to be uploaded, sorted, and archived.
Toward this end, I wrote several applications in PHP, then swiftly becoming the standard programming language for web applications, because it commingled well with HTML, the web’s lingua franca. One of these apps, Web Scrapbook, got a bit of traction, especially in classes, since it allowed students to capture images, links, and other resources from the web browser into a collection that could be shared with their classmates and instructor. Users clicked on a browser bookmark, which had a small bit of JavaScript embedded in it, to select the desired items and pass the data to the application. Web Scrapbook was not, shall we say, a rigorous piece of software — my initial version sent passwords, unencrypted, across the internet — and it paid only basic attention to metadata and other scholarly information that would be needed to assist in writing an article or book, and to form a proper footnote.
Be forewarned, ye who dare log into my web application c. 2002 (screenshot of the Web Scrapbook landing page)
Fortunately, at the same time, another colleague at the center, Elena Razlogova, who was the webmaster at RRCHNM while pursuing her Ph.D. (and is now a history professor at Concordia University), was working on a more scholarly application called Scribe, for taking notes and storing citations. Elena used a database application called FileMaker to build Scribe, which users downloaded and ran on their personal computers.
Elena Razlogova’s Scribe application, c. 2002
Scribe became popular among fellow historians as a good, free replacement for commercial software like EndNote. It ran locally on a Mac or PC, and had the organizational, search, metadata, and annotation capabilities that Zotero would later expand upon. Elena made several improvements to Scribe in the first few years of the century, as I worked in parallel on Web Scrapbook.
* * *
So by 2003, we knew how to write web apps and standalone apps, both helpful but both also naggingly insufficient. We were now starting most of our research on the web, and we thought that any robust research application of the future had to be connected to this environment where primary and secondary sources increasingly resided, either as metadata in library catalogs, or as full objects if they had been digitized. What we really needed was an application that had some aspects of Scribe and some aspects of Web Scrapbook: a full-fledged research tool that could exist independently of the web, but be fully cognizant of what was going on in the web browser, aware as the researcher surfed across digital collections and library catalogs.
On November 29, 2003, Roy asked Elena via email what she would like to do for a next version of Scribe. Roy, Tom Scheinfeldt, another recent history Ph.D. who had joined us to work on the September 11 Digital Archive (now a professor of digital humanities at the University of Connecticut), and I were working on a grant proposal for a new stage of ECHO, and we thought we would include some improved software tools as part of that. Elena wrote back on December 1, 2003:
Roy-- Here’s what I'd like to do:
setup to connect to online bibliographic databases more citation styles (at least APA and legal) setup to add custom citation styles setup to add more fields and types of references user/password setup to share and co-edit online bibliographies/notes setup to work locally then publish all or parts of bibliographies/notes online (like ical) foreign language support better manual online discussion list (using the software marty had installed)
best, elena
This was a good list. Tom wrote back that “it would be nice to update and ‘webify’ Scribe.” A good word: webify. Elena, Tom, Roy, and I agreed on this essential next step for Scribe: we needed to find a way to put it into the web browser, like Web Scrapbook, while retaining Scribe’s provenance as a detail-oriented research tool. As I later summarized our holy grail in a journal article:
We wanted the best of both worlds: the best parts of standalone applications and Web applications. We envisioned a tool that lived in the browser and that was very smart about what was going on in the browser, including the recognition of scholarly metadata and objects, and that could interact with elements both on the desktop, such as word processors or other programs, and via standards, services, and communication protocols to other tools and resources across the Web.
But how to merge these disparate worlds? That was not at all clear in 2003.
* * *
At RRCHNM, we had a large whiteboard with vague ideas, some of them serious and some of them silly, visible to everyone, just in case it gave someone a spark. Over group lunches, the faculty and staff at the center would often talk about new tech we had heard about, and muse about whether we could apply that tech to historical research. Our collective was growing quickly. In the summer of 2004, we were joined by, among others, Josh Greenberg, who had a Ph.D. in science and technology studies (and is now a program director at the Alfred P. Sloan Foundation); Sharon Leon, who had a Ph.D. in American history (and is now the Co-CEO at Digital Scholar); and Simon Kornblith, a young teenager (and son of a historian and friend of Roy’s) who was truly gifted at software development and many other things. (Simon would go on to get a Ph.D. in Brain and Cognitive Sciences at MIT, write important AI papers with future Nobelist Geoffrey Hinton, and work at Anthropic.)
One project that caught our eye was a spinoff of the open-source Mozilla web browser that came to be known as Firefox. That summer Firefox was in beta, and it would be officially released later that year. The new browser had much to recommend it. The original Mozilla browser, descended from Netscape and the early days of the web, had, over time, become slow and bloated. Firefox, on the other hand, was fast and lightweight. But much more exciting was how Firefox embraced and foregrounded XUL, which sounds like an alien god from a 1950s sci-fi movie, but stands for XML User Interface Language. It dawned on us that with XUL, you could — sweet mercy — customize and extend the web browser in any way you wanted.
The day Firefox was released to the world on November 9, 2004, Josh and I — always the earliest of adopters — downloaded it and started to tinker with its possibilities. That night I wrote an email to Josh, Elena, Tom, and Roy with the subject line “Firefox + XUL + AWS = OpenScribe/Online Scribe?” The AWS I was talking about was not the future cloud computing service from Amazon, but an API Amazon provided to retrieve information about books. I thought we could use a combination of a Firefox extension written in XUL and services like Amazon’s to “automatically load citation info into the fields.” Josh wrote back just before midnight:
Please pardon my language when I say "Holy crap, that's cool!"
The possibilities are astonishing - it looks like there are XPCOM wrappers for mySQL that would allow us to access a database (either remote or local), entirely replicating Scribe using a relational database (or, offering a custom "Scrapbook Browser.") Alternatively, we could chuck the database and use a Firefox interface to navigate Scribe XML data directly. Regardless, immediate cross-platform compatibility.
Either way, we could use a combination of Amazon and Proquest/ISI Citation Index to make it much easier to create bibliographic objects, and at the least create a Mac version that would use either Cocoa or more basic scripting to allow citation from within a word processor...
Neat-o!
With an open-source database instead of FileMaker to store bibliographic data and notes, and with a Firefox extension of the proper composition, all of the ingredients were within reach to create the research tool of our dreams.
That winter, with Roy as the principal investigator and Josh and me as co-directors, we applied for a grant from the Institute of Museum and Library Services. The abstract from that grant, which we titled “SmartFox: the Scholar’s Browser for Digital Collections,” displayed our new level of clarity, although we still envisioned what we wanted to make as a “set of tools” — such as the item capture tool from Web Scrapbook and Scribe's note-taking interface — rather than one app:
The web browser has become the primary means for accessing information, documents, and artifacts from libraries and museums around the country and the world, thanks in large part to the tremendous commitment these institutions have made to bringing their collections online (as either simple citations or complete text and images). Unfortunately for scholars, while tens of millions of dollars have been spent to create digital resources, far less funding and effort has been allocated for the development of tools to facilitate the use of these resources. The browser remains merely a passive window allowing one to view, but not easily collect, annotate, or manipulate these objects. Moreover, from the user’s perspective individual library and museum collections remain just that—separate websites with distinct designs and different ways of displaying their information, making traditional scholarly practices of bringing together and studying objects of interest from across these collections unnecessarily difficult.
SmartFox, a set of tools incorporated into popular, open, and free web software, will address these major problems by creating a web browser that is “smarter” in two key ways. First, one tool will enable the browser to intelligently sense when its user is viewing a digital library or museum object; this will allow the browser to capture information from the page automatically, such as the creator, title, date of creation, and copyright information. Second, another tool will store and organize this information, as well as full copies of items and web pages (not just their citation information) if so desired by the user and permitted by the institution’s site, allowing the user to sort, annotate, search, and manipulate these individualized collections created for scholarly purposes. Critically, all of this will occur within the web browser itself, not in a separate, standalone application; the web browser will be used not just to discover information, but also to collect, organize, and analyze scholarly materials.
(Zotero did indeed begin its life with a name, SmartFox, that was a riff on Firefox, and our potential trademark violation somehow got worse when we rebranded the app as Firefox Scholar in September 2005. More on that later.)
Even before IMLS funding had come through, Simon had started working with David Norton, another software developer, on a prototype of SmartFox. By July 2005, they had produced a very early “0.0.1” version, with the metadata panel on top, the notes field below, and the folders on the left; all of these would later be combined in a much better unified drawer.
SmartFox version 0.0.1 screenshot, July 27, 2005
Simon, who was paying attention not only to public releases of Firefox but also to the core code development itself, pointed out that summer that Mozilla planned to include “mozStorage” in Firefox, which could act as an interface to a database. (Firefox eventually included SQLite as its native, open-source database.)
Also in the productive summer of 2005, we hired Dan Stillman, a friend of an RRCHNM staff member, to help us with the center’s burgeoning set of servers and increasingly complex digital platforms. Dan fixed so many accumulated issues so well that by January 2006, Josh and I had asked him to join Simon and David on the dev team. (To this day, Dan is the lead developer of Zotero; what a run.) We also added Sean Takats, who had both a doctorate in French history and extensive technical experience, and became Zotero’s longstanding director in addition to his academic career in history and digital humanities; Kari Kraus, our technology evangelist, who became a professor in the College of Information Studies and the Department of English at the University of Maryland; and, later that year, Trevor Owens as our outreach coordinator. (Trevor is now the Chief Research Officer of the American Institute of Physics.)
Members of the Zotero team in late 2006. Standing, L-R: Simon Kornblith, Sean Takats, Trevor Owens, Dan Stillman, Matthew Eaton; Kneeling: me, Roy Rosenzweig, Josh Greenberg
With this team in place, development accelerated. In roughly six months, the developers produced a far more coherent design and much-improved functionality, as you can see in the June 2006 screenshot at the top of this piece. With summer always being a busy time at the center, as teaching responsibilities paused for some members and everyone focused on grant projects, the code base iterated rapidly and everything aligned for a fall 2006 release of the 1.0 version.
There was just the pesky issue of the name of the thing. I’m afraid we probably spent as much time in the summer of 2006 rethinking the name as we did fine-tuning the design. We had, by then, long abandoned “SmartFox,” and were calling the software “Firefox Scholar,” but we heard from a lawyer that any branding with “Firefox” in it would be problematic. We were also wondering whether this software was just for fancy “scholars” — it seemed to have broad applicability. We kicked around countless replacements, many of which are howlers in retrospect. Withholding who proposed each of these alternatives to protect everyone involved from embarrassment, we considered iScholar (these were the days of iPods), Scholr (these were the days of Flickr), Citopia, FreeCite, CiteHound, DynoCite, FireScribe, FireHoard, MetaFox, ClioFox, and every pun you could think of involving fires, foxes, and footnotes. The domain name also had to be available; it’s amazing how many of the bad names we came up with had somehow already been registered by other fools.
Late in the summer of 2006, as time was running out, we thought we would move beyond English, taking a cue from other new projects at the time, like Ubuntu, a Linux distribution that used a word borrowed from a family of African languages, and Wikipedia, whose name derived from wiki, the Hawaiian word for quick. Our colleague Mills Kelly, a historian of Eastern Europe, made the suggestion to consult an Albanian dictionary, which we thought was a good idea as it was unlikely that any other Americans had ever done so, for any reason, much less to find available domain names. We quickly selected the Albanian word for “to learn well or master,” zotero, which was memorable, sounded like it could be pronounced easily in many languages, and was still available with any domain ending we wanted (we registered .org/.com/.net).
On August 29, 2006, I sent Dan Stillman some Photoshop files with a Zotero logotype I created, set in Eurostile. I can neither confirm nor deny that this logotype may or may not have been inspired by Zpizza, a restaurant near where I was living in Silver Spring, Maryland, which also had a red Z at the front but used Futura Bold, a travesty that led to the restaurant closing soon thereafter. The Zotero font has changed slightly, but the logotype has survived to this day.
The first logotype for Zotero, August 29, 2006
* * *
Twenty years ago, I posted a one-sentence notification of the initial beta release of Zotero, asking people, a bit manically, to “try it out!” Try it out they did. We had 60,000 users of this new open-source research assistant within a month. The numbers escalated rapidly from there, and additional funding soon followed from the Mellon Foundation and the Alfred P. Sloan Foundation that allowed the app to sync collections across research teams and, eventually, to become independent of Firefox and work with other web browsers.
The library I now lead, like most research libraries, has workshops that teach students and faculty how to use Zotero. It’s part of the fabric of academia, and is widely used outside of the academy as well. Zotero is great, solid software. It continues to evolve in concert with the needs of its enormous user base. Its value, slow to emerge from the collective thinking of a friendly cohort of historians, has been incredibly durable. May it last another 20 years, or more.
Like the desktop version, Read Aloud on iOS reads PDFs, EPUBs, and webpage snapshots to you in high-quality, natural-sounding voices. It automatically skips in-text citations, headers and footers, and tables, and you can quickly skip ahead by sentence or paragraph or start reading from a given spot. While you’re listening, an Annotate Sentence button highlights or underlines the current sentence with a single tap, so you can make note of important points without selecting text by hand.
Read Aloud on iOS keeps reading if you lock your device or switch to another app, and you can use standard iOS methods (lock-screen controls, buttons on AirPods, Siri) to pause and resume or skip ahead by sentence. When you stop reading, Read Aloud remembers your exact position, so you can pick up later right where you left off.
The iOS version features the same voices as the desktop app. From the Zotero 9 announcement:
Read Aloud requires an internet connection and a Zotero account for high-quality voices, which we’re calling Zotero Voices. If you’d like to use Read Aloud offline, you can still use the text-to-speech voices available on your system, but the quality will be significantly degraded.
We offer two tiers of Zotero Voices: Standard and Premium. Standard voices are generated on Zotero servers, with unlimited minutes for Zotero Storage subscribers and 2 hours/month for free accounts. Premium voices are the highest-quality voices available, processed by external text-to-speech providers — they make fewer mistakes, sound more natural, support many more languages, and can handle multilingual text. Individual Zotero Storage subscribers will receive up to 2 hours of Premium usage (varying by voice) each month, and free and institutional accounts will also receive a small quota in order to try the voices out.
For now, you can request additional Premium minutes for free if you run out. We’ll provide more details on monthly allocations and options for adding additional minutes at a later date.
ts_server is a web server proposing a REST API to large language models.
They can be used for example for text completion, question answering,
classification, chat, translation, image generation, audio
transcription, speech synthesis, …
A group of French scientists sees OpenAI unjustly claiming it has solved
the mathematical Navier-Stokes problem and sounds the alarm. A handful
of tech moguls are establishing a private monopoly over cognition and
knowledge.
Software maintainers exercise considerable agency over the material in
their immediate care: code. However, in doing so they also find
themselves having to articulate dynamic, interdependent and hybrid
networks of relations which they are intimately entangled with, and
whose durability depends on the success of their ongoing, indeterminate
reconfiguration. Both the programmers and the software they maintain
must continuously navigate risks of breakage, burnout, bugs or falling
into obsolescence.
Eclectic, human-created radio programming is made possible not just
because of student devotion, but also because many campus stations house
irreplaceable collections of physical media. For example, Cleveland was
once considered the city with the second-largest Hungarian population
after Budapest, and WCSB was home to thousands of Hungarian records, a
culturally significant collection regularly played during the station’s
long-running “Hungarian Hour.”
How much LLMs memorize their training data is a central question, both
for copyright lawsuits and for AI research more broadly. But most
technical work that measures memorization doesn’t address the questions
that matter most for copyright. It tends to measure memorization over
random samples of all the training data to estimate average memorization
rates, yet copyright is about specific works. So the copyright-relevant
question is: how much of a particular book has a particular LLM
memorized?
Perhaps that’s what AI is exposing. We’ve spent years loading an
extraordinary number of responsibilities onto the humble code review:
quality gate, security check, architecture review, mentoring mechanism,
knowledge-sharing system, ownership model.
It worked, sort of, while humans could only produce code so quickly.
That constraint is disappearing. So perhaps the question isn’t how we
get the code reviewed faster. Perhaps it’s why we’re waiting until code
review to have all the important conversations in the first place.
Understand Anything is a Claude Code Plugin that analyzes your project
with a multi-agent pipeline, builds a knowledge graph of every file,
function, class, and dependency, then gives you an interactive dashboard
to explore it all visually. Stop reading code blind. Start seeing the
big picture.
This repository supports the article “Standardising Access to Digital
Platform Data: A Taxonomy for Social Research.” It provides guidelines,
platform-specific schemas, entity mappings aligned with the proposed
taxonomy, and example structures to help researchers access and use Data
Download Packages
While mass marketing is trying to instil a Fear of Missing Out hysteria,
the most rational and pragmatic approach is to strongly reject all
AI-generated contributions to your projects. For now.
Someday, we might realise that LLMs are doing good in the world, that
they are evolving toward ethical, reliable, sustainable solutions, and
that people who use them are happier (try to read that sentence again
without rolling your eyes). If that really happens, you could always
change your AI policy. It will cost you nothing.
God’s Waiting Room: a round-about selection of the cinematic, library
music, rock ‘n’ roll, psych, experimental, unclassifiable and
independent. From the mind of DJ, producer and creator of original
soundtracks, David Holmes.
The nuraghe, or nurhag, is the main type of ancient megalithic edifice
found in Sardinia, Italy, developed during the Nuragic Age between 1900
and 730 BC. Today it has come to be the symbol of Sardinia and its
distinctive culture known as the Nuragic civilization. More than 7,000
nuraghes have been found, though archeologists believe that originally
there were more than 10,000.
Automated feed with new posts from live music & unusual recording
repositories to accompany (LiveMusicArchiveUpdates?),
instigated by (bourgwick?), operated
via MastoFeed.
TL;DR: In this post, we show how an off-the-shelf LLM can make typed
decisions in a single forward pass. This approach makes it possible to
turn an LLM into a Jev-like decision model.
We evaluate the approach using GLM-5.3-Flash running on Privatemode.
Using a benchmark constructed from public data sets, we show that this
setup delivers results that are on par with TypeSafe’s Jev in terms of
decision accuracy/correctness and speed.
As a bonus, the setup with GLM-5.3-Flash enables typed decisions on
images, which is not possible with Jev
This is a long-form written guide for the fight AAC Heavyweight M1 (Savage) in the game Final Fantasy 14, where the challenger and their seven friends fight the Elezen succubus Vamp Fatale. You may call this the XeBin guide in party finder listings.
If you want to experience this fight, don't read this guide. This guide is based off of the Toxic Friends raidplan, the Hector Lectures strategies used by North America Party Finder raiding culture, and insights I have gained from running this fight in a static group. A lot of this advice is geared towards healers as I typically run as a healer (AST/SGE).
This fight assumes minimal raiding experience beyond what is required to beat the normal raids. All vocabulary and common mechanics terms will be explained inline.
If you or your static is looking for a healer that wants to prog through
ultimates, please get in touch. I've been trying to avoid using my clout to
help me through FFXIV raiding, but I've had such dogshit luck in Party Finder
(285 pulls to clear M10S) that I'll take whatever I can get at this point.
These strategies assume that every member of your group has an item level (IL) average of at least 780 (via Augmented crafted gear and an Enuo weapon). If you are under IL 780, you may need to mitigate (mit) more heavily than if you have higher item levels. Play it by ear and don't be afraid to break out Eukrasian Prognosis or Concitation if you are not sure what mits are required for each hit. This strategy guide will also tell you to drastically overmit beyond what is required in practice. This is intentional so that you know what fits where.
The theme of this fight is that every mechanic is blanketed in a coating of raidwide attacks (ones that hit everyone regardless of distance from the boss or other players). Every mechanic in this fight looks like a sandwich of the following:
Light mit raidwide
Mechanic
Another light mit raidwide or a heavy mit one for flavour
Raidwide attacks
In this fight, raidwide attacks come in three flavours:
Ignorable: Don't mit this attack, throw an AoE heal or two afterwards. Maybe fire off some shields so you don't have to care as much.
Light mit: This is enough damage to be annoying, but not enough to actually risk killing people. This is where your 10% mits, Addle, Feint, and Reprisal make a difference.
Heavy mit: If you are not careful, this is going to kill people. This is when you break out big guns like Spreadlo, Holos, Temperance, or Sun Sign. Remember: if people live with more than 1 HP, they didn't die.
Stack attacks
Stack attacks have two kinds to keep in mind:
Light mit: Mostly seen later in the fight when splitting into light parties for Undead Deathmatch or for the stack mechanic in Hell in a Cell. Use your judgement, make sure people don't die and get healed.
Akh Morn: Technically this fight doesn't have any hard casts of Akh Morn, but there are many multi-hit stack attacks that are functionally an Akh Morn. This is where you break out the big guns like Liturgy of the Bell, Panhaima, Macrocosmos, or Seraphism-boosted Spreadlo.
Tankbusters
This fight has particularly meaty tankbusters. These tankbusters really do need to be mitted for consistency. Every healer (save Scholar) has a 10% single target mit that makes a huge difference and can be the reason tanks live or die. A well-placed Haima can also make a huge difference during Hell in a Cell as every tank hit in that mechanic is a tankbuster hit.
Mit sheeting
Healer mits:
Mit class
White Mage
Astrologian
Sage
Scholar
Ignorable
Maybe Swiftcast -> Medica 3 if you feel generous
AoE heal over time
Swiftcast -> Prognosis after
Selene heals
Light mit
Plenary Indulgence
Heal over time
Kerachole
Sacred Soil / Fey Illumination
Heavy mit
Temperance -> Divine Caress
Neutral Sect -> Sun Sign -> Shields
Holos -> Physis
Seraphism -> Spreadlo
Note: healers WILL run out of mits, especially you have a Sage and they die at any point. Do not feel afraid to burn heavy mits in light mit mechanics in order to prog.
DPS / Tank mits:
Role
Light mit
Heavy mit
Akh Morn
Melee DPS
Ignore
Feint (trade with other melee)
Feint + Job gimmick
Reaper
Ignore
Feint
Arcane Crest
Tank
Ignore
Rampart or Reprisal
Reprisal + 40% mit
Caster
Ignore
Addle
Addle + Job gimmick
Pictomancer
Ignore
Addle
Addle + Tempera Coat -> Tempera Grassa
Ranged DPS
Ignore
Ignore
Job gimmick
Dancer
Ignore
Ignore
Shield Samba
Bard
Ignore
Troubadour
Troubadour if up
Fight-specific mechanics
This fight has a few mechanics that get repeated multiple times across the encounter. For brevity's sake, each of them will be explained here with backreferences so you can refer to them later.
Satisfied
Any time anyone fails a mechanic they get a 10% Damage Down debuff and Vamp Fatale gets a stack of Satisfied. If too many players get Damage Downs, you will not be able to clear the fight.
Every 4 stacks of Satisfied Vamp Fatale's raidwides hurt more, AoEs will get bigger, and her Half Moon cleaves get wider. If you get more than 16 stacks of Satisfied, you're probably going to get obliterated by the raidwides. Try to get as few stacks of Satisfied as possible.
Every so often Vamp Fatale will cast Crowd Kill which will make her gain 4 stacks of Satisfied regardless of player actions.
Half Moon
Half Moon is similar to Omega's Starboard/Larboard Wave Cannon. When she casts Half Moon, she will cleave half the arena and then immediately cleave the other half. This is a core verb in the fight and will often be woven into other mechanics. At eight (8) stacks of satisfied, Half Moon gets a lot larger and cleaves from her hitbox out instead of from halfway out.
These diagrams are based on some kinda janky simulation logic I've been
cooking up. They are simultaneously more and less strict than the actual game
logic and mostly intended to illustrate mechanics patterns rather than being
guides on how you should literally move.
For consistency: always assume that she has eight stacks of Satisfied and go way farther than you need to in order to avoid the cleaves.
You can tell which direction the cleave is going based on the position of the ball of aether Vamp Fatale is holding. If you're on the opposite side as the ball of Aether, you're good. If not, you're giving her a stack of Satisfied. If you want to practice this mechanic in particular, the normal version of this fight uses it pretty heavily.
Hardcore
Hardcore is a pair of meaty AoE tankbuster hits that target the top two entries in the aggro list. As Vamp Fatale gets more stacks of Satisfied, the tankbusters will hurt a lot more and the AoEs get bigger until it's unsolvable.
If she has more than 12 stacks of Satisfied, either this attack will kill you or one of the next ones will. If you find yourself in this situation, consider alternate strategies.
Vamp Stomp
Vamp Stomp is one of the easiest mechanics of the fight that will also cause you to wipe the most. Before I explain the mechanic, here's a high-level diagram that explains all the parts:
When you transition into this mechanic, you want to have everyone in modified clock spots with ranged on intercardinals and melee on cardinals, kinda like this:
When this mechanic starts, everyone gets a Curse of the Bombpyre debuff. Vamp Fatale will shoot out an expanding shockwave ring. When the ring hits you, it cleanses your debuff and explodes a point-blank AoE that hits you for about half your health and gives you a very short Magic Vulnerability (magic vuln) debuff.
If you get hit by any AoEs while you have a magic vuln, you will instantly die. This pierces tank invulnerability skills. Try to not get venn diagrammed.
Vamp Fatale spawns three rings of bats:
An inner ring with two bats
A middle ring with three bats
An outer ring with five bats
When bats are hit by the ring, they also explode into a point-blank AoE that hits for about half your health and gives a magic vuln.
The trick for this mechanic is to make sure everyone cleanses their debuffs without killing everyone else in the process. The best order to resolve this is tank/melee -> ranged -> healer. Keep in mind that the bats are randomly placed and will randomly go either clockwise or counter-clockwise. This will take some practice for you to know how to read your party to know when it is safe to cleanse your debuff.
Here's a few tricks I've learned to make things somewhat consistent:
If the inner ring of bats is positioned on the tanks, have melee go first.
If the inner ring of bats is positioned on the melees, have tanks go first.
If it's diagonally offset, agree who should go first with your group.
If anyone gets a magic vuln from the bats or dies from getting venn diagrammed, Vamp Fatale will gain a stack of Satisfied.
Practice this mechanic a bit until you can do it without thinking.
Basic Raidwides
There are a few other raidwide casts in this fight that are mostly the same light mit raidwide, but a few of the variants have some implications you should keep in mind:
Killer Voice: The basic light mit raidwide. Sometimes you can get away with ignoring this and dumping out heals after the hit. If you and your group are close to best in slot, you can get away with ignoring it for the most part.
Crowd Kill: Another light mit raidwide but it makes Vamp Fatale gain four stacks of Satisfied. This will be cast at key points in the fight and gradually increases the potency of all her attacks as the fight goes on. The last time she casts this, she will hard enrage and just kill the party.
Sadistic Screech: Another light mit raidwide that edits the arena. If the arena is a 4x4 grid, it turns into a 4x2 grid or vice versa.
Finale Fatale: Yet another light mit raidwide that makes the arena into a circle instead of a square. If you walk out of the circle, you get killed. Try not to do that.
Brutal Rain
Functionally, this is an Akh Morn but empowered by her stacks of Satisfied. The spread for Satisfied stacks vs Brutal Rain hits roughly translates to this:
Stacks of Satisfied
Brutal Rain hits
0-3
3
4-7
4
8-11
5
12+
6
When it gets to 5+ hits, you need to burn as many mits as possible because if you do not you WILL die. Some of my prog parties had us burn healer limit break 3 just to keep people alive during the worst of this. If you die right before this mechanic, cheese your rez immune to help soak the damage so your allies can live.
Suggestions for getting started
The normal version of this fight is very faithful to the savage version. It's actually a pretty brilliantly designed tutorial and can help you learn the savage mechanics in a low-risk environment. Most of the mechanics are basically the same as their savage counterparts with the exception of the removal of Hell in a Cell, Undead Deathmatch, the fun parts of Aetherletting, and most of the mechanics around stacks of Satisfied.
Opener
Vamp Fatale will start the fight out with a cast of Killer Voice. One healer should throw out a raidwide mit and top everyone up after the hit. In your first pull this will probably kill at least one of you. This is normal, and is a sign that you should over-mit until you get comfortable with the amount of damage you need to account for.
She will then cast Hardcore with the smallest tank AoEs in the fight. Everyone can maintain damage uptime during this mechanic. Just don't venn diagram eachother and you'll be fine.
After the Vamp Stomp, she casts Brutal Rain. Throw out an Akh Morn mit and make sure everyone is topped up for the raidwide that leads you into the next act: fighting the Coffinmaker.
Coffinmaker
Vamp Fatale changes the arena into a 4x2 grid and spawns a Coffinmaker enemy at the top of the grid. If you touch the area covered by the Coffinmaker, you instantly die. If you get hit by any Coffinmaker attacks, you get a Damage Down and Vamp Fatale gains a stack of Satisfied. Vamp Fatale doesn't throw out any auto-attacks or fire off any raidwides during this section of the fight, so healers can throw out a heal over time and focus on mechanics.
The Coffinmaker will have two saws glow blue and cast Coffinfiller. Those will become line AoEs when the cast finishes.. Vamp Fatale will also cast a Half Moon, meaning that at any point only 1/8th of the arena is safe ground.
In order to solve this:
Notice if the orb is on the north or south side of the arena. If it is on the north side, call out that south is safe and vice versa.
After the line AoEs fire, go diagonally opposite of where you were just standing because Vamp Fatale will fire off the opposite Half Moon as the Coffinmaker fires off another Coffinfiller with the saws that were just safe.
After the Coffinmaker casts Coffinfiller twice, it will cast Dead Wake, which will make it advance one square down the arena. If you don't kill the Coffinmaker before the end of the arena, it will enrage by destroying the floor, pushing everyone off, and wiping the party.
Suggested callouts:
Forward: closest to the Coffinmaker
Back: away from the Coffinmaker
One step forward: move to the next row to avoid Dead Wake crushing you
This is the only major DPS check in this fight and the first few times will murder you. You can also tolerate at least two Damage Downs before it becomes a problem.
When the Coffinmaker gets to the end of the arena, it spawns line AoEs as normal but Vamp Fatale's Half Moons go horizontal instead of vertical.
Suggested callouts:
Far left: far left column
Close left: close left column
Close right: close right column
Far right: far right column
If you kill the Coffinmaker before Vamp Fatale's last Half Moon cast, continue respecting the Half Moons. Once the Coffinmaker is dead, Vamp Fatale becomes targetable and casts Sadistic Screech to return the arena to normal and whallop everyone with damage for good measure.
Aetherletting
Vamp Fatale will cast Crowd Kill and gain four stacks of Satisfied no matter what. She will then spawn a bunch of randomly placed AoEs over the arena. Avoid them. She'll cast another Finale Fatale raidwide to turn the arena into a circle and begins casting Aetherletting.
Aetherletting has two main parts to it:
Vamp Fatale will spawn conal AoEs either clockwise or counterclockwise around the centre of the arena that inflicts massive damage, a damage down, a magic vuln, and gives her a stack of Satisfied.
Two players will be targeted with a hunt marker that leaves a cardinal/intercardinal bomb that gives you a damage down and her a stack of Satisfied if you get hit by its line AoEs.
The most common strategy for this is a technique called Static Aetherletting. With Static Aetherletting your goal is to place all the hunts so that they explode in a way that avoids the line AoEs crossing the centre of the arena. Everyone stacks dead centre under the boss. I cannot stress enough, you must be DIRECTLY UNDER THE BOSS.
If you place your markers on the safe sides of the pie slices, they will all explode and you take no damage.
Once she is done with that she will cast Hardcore. The AoEs will be bigger than last time. If you aren't sure how big they are going to be, sacrifice your uptime and run away to be safe. Use a gap closer to get back to her when you're done.
She will follow up with another Vamp Stomp into a Half Moon. Assume her Half Moon AoEs are much bigger than they are to be safe.
If anyone died during the Vamp Stomp, fire off the rez right at the end of the Half Moon because she's about to cast a Brutal Rain and the more bodies you have in the stack the less damage everyone takes.
She will cast Insatiable Thirst, then Sadistic Screech, and that brings you into the adds phase.
Adds phase
In the adds phase, the arena gets shrunk to the smaller arena size from Coffinmaker but with four additional hazards added in: buzzsaws. Buzzsaws are the most terrifying part of this mechanic. If you get a bleed from the saw, you will take about 40% damage per tick. If your healers don't realize it, you will die. Make sure to avoid the saws and let your healers know instantly if you get hit by them.
Either way in this phase she spawns three additional targets: two Fatal Flails that come out after tanks soak tank towers and a Deadly Doornail. Each flail has an enrage cast of Barbed Burst. If you don't kill the flail before it finishes its cast, it will either kill people or chunk out massive parts of people's HP and give a Damage Down debuff. If both flails pop, just throw yourself off the side because you're about to die anyways.
The Deadly Doornail will spawn a purple puddle that constantly gets bigger. This puddle gives you a small damage over time debuff and is just considerably annoying.
The main tank should take the towers on the north side of the arena and the off tank should take the towers on the south side of the arena. Tanks should pop their two minute burst on at least one of the flails.
Your kill order:
Tanks: burst the hell out of your flail as much as you can as fast as you can.
Melee: use your standard rotation on the nail until the purple circle gets bigger than max melee range, then go help your light party tank with their flail.
Ranged/Caster: focus the nail, then focus the flails.
Healers: Vamp Fatale is gonna blanket this in raidwides, so mit the raidwides.
Until you get confident with this you will need to have a caster Limit Break to kill the last flail and nail pair.
Once you kill the third set of flails/nails, she casts Sadistic Screech to restore the arena.
Hell in a Cell
This is your pot window, use your gemdraughts right now and burst the hell out of the boss with your party synergy buffs. If you didn't spend it in adds phase, pop your melee Limit Break 3.
Call out that people should align their camera to north such that the A waymark is behind the boss.
Vamp Fatale will now cast Hell in a Cell. This mechanic will require you to split up into light parties and have each light party take towers in clockwise priority relative to north: tank -> healer -> melee -> ranged. Everyone else should stack in the big gap and wait for the next cast from the boss.
Vamp Fatale will cast either Ultrasonic Amp (stack) or Ultrasonic Spread (spread). If you get Amp, stack in the big gap, mit, and heal after the hit. If you get Spread, do a light role spread with the tank in the big gap, both DPS in one smaller gap, and the healer in the other smaller gap.
After both hits, the remaining cells will explode and kill people. Try to make sure all the cells are dead before the second hit finishes.
Once that's done, the other light party gets their cells. Resolve it the same way.
Undead Deathmatch / Sanguine Scratch 1
If you are doing this fight with item level 780 and up and 12% echo, this will be one of the last mechanics you see. I've seen some parties kill during this mechanic.
Vamp Fatale spawns two party towers either north/south or east/west. Split up into light parties and soak the tower. One person not being in a tower is survivable, two is iffy, more than three is a wipe. Try to have everyone in the towers or mit like your life depends on it.
When the towers explode, they will spawn two untargetable bats that apply chains to everyone. Do not stretch the chains out or you will die. Follow the bats and avoid the proteans by moving from the safe spot to the place she just attacked. Stay close to the inside so you can move as little as possible.
When I cleared this fight, I still considered this mechanic my prog point. If
you are unsure, stop doing damage and focus on mechanics. You can have
multiple people do zero damage during this part and still kill the boss
before the enrage.
Once the bats get to 180 degrees away from where they started, they will either have a donut or circle marker above them. If they have a donut, stand under the bat. If they have a circle, stand near the boss.
If anyone dies in this mechanic, wait until the end to resurrect them. People
resurrect in the place where the caster finished the cast, and this will
make them be assigned to whatever bat the server wants, which will end up
killing them again.
After that, she casts another Brutal Rain. Mit and heal through this.
Act 7: Victory Lap
She will cast her final Vamp Stomp. You have the time to sort yourself out, so feel free to burn sprint and get to your spots.
She will also cast another Hardcore for the road. Respect this. This will be very meaty and the AoEs will be VERY LARGE. Tanks should go as far away as possible and heavily mitigate or pop their invulnerability skills.
There's one last cast of Sanguine Scratch but without the tether bats. Slide between the safe spots and keep doing damage.
Then there's one last Crowd Kill which makes her gain the buff Completely Satisfied. This will make her cast her enrage: Finale Fatale. If she finishes the cast, you die. Just kill the boss until it dies.
Win free books from the October 2026 batch of Early Reviewer titles! We’ve got 303 books this month, and a grand total of 3,321 copies to give out. Which books are you hoping to snag this month? Come tell us on Talk.
The deadline to request a copy is Sunday, October 25th at 6PM EDT.
Eligibility: Publishers do things country-by-country. This month we have publishers who can send books to the US, the UK, Canada, Australia, France, Belgium, Luxembourg, Denmark, Spain, Germany and more. Make sure to check the message on each book to see if it can be sent to your country.
Thanks to all the publishers participating this month!
The computing industry is extremely resource intensive. In terms of
materials. In terms of manufacturing. In terms of energy usage. In terms
of the knowledge and expertise required to build and design computer
hardware. In terms of needing computers to build the next
computers.
The list could go on, and it all was the case before “AI” or
LLMs. AI has just thrown tons of cash and political gasoline on the
already burning fire, that was already way out of control. Take a look
at this permacomputing
thread about how computing technology (the hardware) can continue to
exist after economic collapse, and you will start to see this picture
come into focus too.
I guess someone could say this position is a thinly disguised form of
whataboutism.
But I’m not really trying to redirect an argument against AI, so much as
I’m trying to sharpen the critique of AI as a form of centralization, by
zooming out and looking at the broader context of computing.
I say this because one of my main concerns to
date about the turn to AI in software development has been
centralization. The degree to which LLMs, and their attendant
infrastructure, centralize software development is literally ỉ̫̙̿͌̂̌n͍͔̑̇ͯ͞s̬̠̰͈̗͎̑̿ͣ̓ȃ̝̬̙͇̗̗̻͜n̲͈͉̙̽ͩ͌͐e̻̓̀͗ͭ̂ͮ̔⃝. But
this centralization is partly what these massive corporations want, or
even need, us to believe–that the model usage must be
centralized in order for it to work, and that they must
be the gate keepers entrusted to let us use it.
Whether it is by design, distillation, or really by whatever means
necessary, we are starting to see models being run in other places,
sometimes on laptops, or on mobile devices, even in cooperatively governed spaces. It
seems that maybe, just maybe, for software development, it might not be
necessary for the model to run by $EVIL_CORP. That it might be possible
to use LLM technology to make software development easier, and more fun,
more responsive to users’ needs, without running every prompt through
one (or a handful) of APIs. And without ingesting the entire Internet of
content.
The more this decentralization (d14n) happens, the more computing
returns to its always already
massively centralized condition. Computing wasn’t perfect before in
terms of d14n, and pretending like it was isn’t a sufficient critique of
the use of LLMs in software development. However, I think it is a
sufficient critique of software development more generally, and the idea
of Automating All the Things just because you (think you) can … but that
is a topic for another post.
One of the eye opening things I’ve learned in my own (albeit limited)
use of LLMs in coding has been the startling (and admittedly unwelcome)
realization of how much of the work I was doing as a software developer
was not creative at all. Transplanting patterns from one domain to
another (conceptual copy/paste), or mundane restructuring of code, or
finding bugs with unit tests. It turns out a lot of this work was
deeply repetitive, and didn’t really require much thought–or at
least not as much thought as I thought it did…
But it also helped me see what work was creative. The biggest
area for me has been being able focus on and understand what problems
are worth working on. What are the approaches I could bring to an
implementation? How can I understand my team members ideas and needs
better? How can I understand what the people using the software need
better? What areas of the codebase need attention to performance? What
areas need simplification? These were always deeply satisfying things to
work on before using LLMs in software development, but I didn’t
have as much time to devote to them because I was too busy indenting
some code, finding a bug, moving things to use some new framework, or
keeping some documentation in sync with its implementation, etc, etc.
Anyway, I guess I’ve found myself having to admit that LLMs are actually
useful for software development, not because they make drudge work go
away, but because they let you focus on the things that actually do
matter.
Let me return to where I started: that computing is always, already
centralized to a great degree. We should, nay we must, always
be looking for ways to decentralize and democratize access to computing.
This isn’t going to happen magically. It is hard work, and requires deep
levels of expertise and access to resources. But we also shouldn’t
deceive ourselves by not seeing how centralized computing as a
technology already is, and already was, before we started prompting LLMs
instead of searching Google to find the answer to a coding problem.
So how’s that for whataboutism? If you see a flaw in this argument, or
think it’s wrong headed, please drop me a line? For what it is worth, I
can’t shake the feeling that this is in fact the language of addiction.
This online event marked the conclusion of the Traceable AI Answers for Public Data project under the AI Learning Labs initiative, showcasing AI integration with open data portals. You can now watch the full event recording and explore all presentation materials.
It’s October, and the 2026 Virtual DLF Forum is almost here! On the 14th and15th, we’ll come together online for two days of learning, collaboration, and shaping what’s next for the field. In the coming days, we’ll share final Forum updates, ways to prepare, and opportunities to connect across the DLF community. We can’t wait to see you online! Be sure to register by tonight if you haven’t yet.
I’ll be in Spokane, Washington, for the Joint Council of Librarians of Color (JCLC) conference as an exhibitor with Sharon M. Burney, Program Officer and member of the CLIR grants team, from October 7-11, and I’d love the chance to meet up for coffee with DLF’ers while I’m there. If you’re nearby and available, send me an email.
With excitement,
-Shaneé
This month’s news
Last Chance to Register: Registration for the 2026 Virtual DLF Forum closes today, October 1. Register now to join us online! Can’t join us live? The keynote will be streamed live and available afterward on YouTube, so you can tune in or catch up later.
New Website: CLIR affiliate The Shared Print Partnership (SPP) has a new website! SPP is a federation of monograph and serial shared print programs across the United States and Canada. Explore the new site for information about available programs and resources.
Office Closure: The CLIR offices will be closed on Monday, October 12, to observe Indigenous Peoples’ Day.
This month’s open DLF group meetings:
For the most up-to-date schedule of DLF group meetings and events (plus conferences and more), bookmark the DLF Community Calendar. Meeting dates are subject to change. Can’t find the meeting call-in information? Email us at info@diglib.org. Reminder: Team DLF working days are Monday through Thursday.
Born-Digital Access Working Group (BDAWG): Tuesday, 10/6, 2pm ET / 11am PT.
Digital Accessibility Working Group (DAWG): Tuesday, 10/6, 2pm ET / 11am PT.
AIG Cultural Assessment Working Group: Monday, 10/12, 1pm ET / 10am PT.
AIG User Experience Working Group: Friday, 10/16, 11am ET / 8am PT.
AIG Metadata Assessment Group: Friday,10/16, 2pm ET / 11am PT.
DAWG IT & Development: Monday, 10/26, 1pm ET / 10am PT.
Digitization Interest Group: Monday, 10/26, 2pm ET / 11am PT.
Committee for Equity & Inclusion: Monday 10/26, 3pm ET / 12pm PT.
Climate Justice Working Group: Tuesday, 10/27, 3pm ET / 12pm PT.
Open Source Capacity Resources Group: Wednesday, 10/28, 1pm ET / 10am PT.
DAWG Policy & Workflows: Friday, 10/30, 1pm ET / 10am PT.
Yesterday Kevin Ford and I gave a presentation
(slides)
at the LD4 conference about the use of BIBFRAME “bounded descriptions” at the
Library of Congress (LC) and in the Blue Core project.
BIBFRAME data is RDF that gets serialized in many different ways. One of
the challenges of building tooling around BIBFRAME is that you don’t
necessarily know if you have to follow
your nose in the data (resolve URLs) to get some of the data, or if
it will be present in the data you have in hand.
This may seem like a small thing, but when you are building BIBFRAME
editors, or doing a large bulk loading operations (as LC and the Blue
Core project are doing) it is helpful to have it on hand. The
bounded description is a convention for making certain
information available in the BIBFRAME data.
The initial idea for the talk was to describe some interoperability
problems that we
experienced when loading data that had been designed for XML processing
in a RDF processing centered workflow. But rather than focus on those
specific problems it shifted instead to talk about the role of bounded
description for interoperability more generally.
The bounded description aside, one thing that fell out of this work for
me was the importance of having a JSON schema for
BIBFRAME. You might wonder why on earth you would want a JSON Schema
for BIBFRAME RDF data – wouldn’t that be overly prescriptive, since RDF
can be represented using XML, JSON, Turtle, N3, etc? The answer is yes,
and that is completely the point.
Following along with the principles of Linked Open Usable Data (LOUD), it
is extremely useful for tool building purposes to have a clear idea of
how the JSON-LD is structured. Even within JSON-LD there are many ways
of representing the same exact RDF graph. Working with this variety of
serialization formats requires tools to use an RDF parser. For some in
the Linked Data community this isn’t a big deal.
But most people working with data on the Web are used to loading some
JSON with their programming language’s JSON support, and interacting
with it like a data structure: hash maps, lists, strings, integers, etc.
If they can’t reliably do that it is a major interoperability problem.
If BIBFRAME is really going to replace MARC I think it’s important to
let the ontology and RDF serializations exist, while providing a strict
and easy to use representation for the many people who aren’t engaging
at that level.
The bibframe-json site is brand new. We are storing JSON-LD in the Blue
Core data store, and making it available via an API. But we aren’t
actively using this schema (or any other) for validating the data on the
way into the data store…at least not yet. The schema was developed
rapidly with Claude Code, mostly as a provocation for the LD4 talk. It’s
my (probably unpopular) belief that if we can’t represent BIBFRAME data
using JSON Schema then BIBFRAME really isn’t going to see widespread
adoption, ever. I’m sure that there ways for this initial version to be
improved. If you have ideas or suggestions please consider contributing
to the work, or get in touch with me directly.
I saw some recent
news going around about the Australian Internet
Observatory. Bergis
Jules, my collabrator on the (now ended) DocNow project alerted me to it, and I saw
it getting shared around in the fediverse. Bergis reminded me that we
had been talking about the importance of the “archive downloads” that
the GDPR required web platforms to make available to users, in order to
abide by the GDPR. At the
time we had been mostly focused on Twitter, but others like the National
Library of New Zealand were starting to collect
Facebook archives (Wayback link since it appears they may have
stopped doing this?).
The AIO seem to have recognized this too, and built a platform to
connect researchers and data donors to contribute their archives which
they have called Data
Download Packages or DDPs.
While the announcement is recent, it appears that they have been doing
a lot of work analyzing the DDPs from a host of platforms
(AirBNB, Facebook, Google, Instagram, Netflix, Spotify, TikTok, Uber,
and X). They’ve developed a taxonomy
and schema for the data to help researchers use the data they need.
One thing that is not clear to me is to what degree these schema are
used to provide access to the underlying DDPs. But either way this work
will be very helpful not just for their platform but for the research
community in general.
VDMC and Q.E.D. Systems teams at the Q.E.D. Center for Training and Development.
How do we really know when someone has learned a skilled trade? Is completing a task enough, or should we also understand how the person performed it, where they struggled, what demanded their attention, and how their performance changed as they gained experience?
These questions came to life during a recent hands-on maritime workforce visit to Q.E.D. Systems, Inc., by members of the Virginia Digital Maritime Center (VDMC) Workforce Innovation team. For the day, we stepped away from the roles we normally hold as researchers, developers, and workforce professionals and became novice maritime trainees. Instead of only discussing workforce training and performance from the outside, we experienced parts of the learning process firsthand.
The experience gave us an opportunity to learn several maritime skilled trades firsthand while also observing how skills are introduced, practiced, coached, and ultimately assessed. Combining hands-on participation with observation made the visit especially valuable for understanding both training and performance.
From the Learning Lab to the Maritime Olympics
VDMC Team 1 and Team 2 relay teams during the Maritime Olympics Challenge.
The day began in the Learning Lab, where we rotated through hands-on activities involving marine electrical work, rigging, welding, and fall protection. We quickly realized that learning a skilled trade is very different from simply watching someone perform it. We listened carefully to instructions, observed demonstrations, handled unfamiliar tools, attempted new tasks, made mistakes, received feedback, adjusted techniques, and tried again.
One important takeaway was realizing how much is involved in performing even a seemingly simple task well. We had to pay close attention to instructions, coordinate our movements, remember each step, make judgments as we worked, and continuously check whether the task was being performed correctly.
Later in the day, the Learning Lab became an Assessment Lab through Q.E.D.'s Maritime Olympics. The skills introduced earlier were no longer just something we had learned or practiced; we now had to put them to the test through a series of performance challenges and demonstrate what we could do.
Experiencing the Challenge as Learners
VDMC team member Lawrence Obiuwevwi is practicing welding during the hands-on training
Another activity that stood out to us was welding. It is easy to observe an experienced welder and focus only on the finished result. As we took on the role of learners, we began noticing everything that happens before that final result: where attention is directed, how the hands are positioned, whether the correct movement is maintained, how quickly an error is recognized, and how performance changes after feedback.
That experience raised an important question for us: How do we move someone from being introduced to a skill to being able to perform that skill reliably in a real working environment? And perhaps more importantly, how do we know when that transition has actually happened?
Looking Beyond Task Completion
Traditional workforce assessment can focus on whether the trainee completed the task, how long it took, how many errors occurred, and whether the final product met the required standard. Those measures are important, but participating in the activities reinforced that they tell only part of the story.
Two people may successfully complete the same task while going through very different processes. One person may move confidently and efficiently, while another may hesitate, repeatedly correct mistakes, require more guidance, or experience much greater cognitive demand. If we look only at the final product, much of that difference can remain invisible.
Within VDMC's Maritime Human Performance Science effort, led by Dr. Jessica M. Johnson at the Neuroergonomics and Simulation lab, we are interested in how multimodal technologies can help explain what happens during skilled performance, not simply what the final outcome looks like.
As we moved through the activities at Q.E.D., we began thinking about the kinds of questions that could be investigated using eye tracking,physiological sensing, neurophysiological measurement, behavioral observations, and performance measures. Where does a trainee look during a critical step? When does a task begin demanding more attention? Where do hesitation and uncertainty appear? How does a learner respond after an instructor provides feedback? And what begins to change as the learner becomes more proficient?
Those are the kinds of questions that move us beyond simply asking, “Did the trainee complete the task?” toward understanding how skill is actually acquired. This is where the field-trip experience connected directly with the research being pursued at VDMC.
Toward a Process-Level Understanding of Skilled Work
From a research perspective, skilled work is especially interesting because many different kinds of information are generated at the same time. Eye tracking can tell us something about visual attention. Physiological sensing can help us examine changes associated with workload and arousal. Neurophysiological measures can add another perspective on cognitive processes, while task-performance data provide the context needed to understand what was happening at a particular moment.
The challenge is not simply collecting all of these data. The important question is how we synchronize, organize, analyze, and model them in a way that produces information instructors and trainees can actually use.
The long-term goal is not to add more sensors simply because the technology exists, but to use these measurements to better understand how trainees learn, where difficulties emerge, and how proficiency develops over time.
One example is our ongoing pipefitting research in the lab, where we use a holographic training environment to study skilled-task performance. By combining task-performance measures with multimodal sensing, including eye tracking, physiological signals, and neurophysiological measures, we can examine how trainees respond to instructions, move through different task steps, encounter and correct errors, and adapt with practice. These insights can help inform better simulations, training approaches, and instructor feedback.
Sample trainee interaction with the Proto holographic pipefitting training environment.
Learning Maritime Skills Firsthand
The rope-knotting activity was another good example of why being physically present matters. Watching an instructor demonstrate a knot and actually trying to reproduce it are two completely different experiences. Reproducing the knot requires translating what was just observed into coordinated hand movements, while instructor feedback becomes part of the learning process. Experiences like this reminded us that it is difficult to fully understand skilled work from a laboratory alone. By taking the position of novice trainees—receiving instructions, using unfamiliar tools, making mistakes, correcting them, and later being assessed, we gained a different perspective on the complexity behind skilled performance.
VDMC team learning maritime rope knotting at the Q.E.D. Center for Training and Development.
Connecting Research, Training, and Workforce Readiness
One of the biggest takeaways from the visit was how closely workforce practice and workforce research can inform one another. We begin by identifying real workforce challenges, translating them into relevant research questions, collecting evidence, and using what we learn to improve training, assessment, and ultimately workforce readiness. For us at the Virginia Digital Maritime Center (VDMC), this is especially important because modern training environments can generate large and diverse streams of human-performance data. Our research increasingly focuses on understanding the process of skill acquisition, not only whether a trainee succeeds or fails at a task, but also how cognitive, physiological, behavioral, and performance patterns change as that trainee progresses toward proficiency.
This leads to an important research question: How can we move beyond measuring task completion and better understand the processes through which skilled competency develops? Answering that question requires collaboration across computer science, human-performance research, training, simulation, workforce development, and industry, with skilled workers and trainees at the center of the research.
We are grateful to Q.E.D. Systems, Inc. for opening its doors to us and allowing us to experience maritime training from the perspective of learners. We also appreciate the support of the Submarine Industrial Base Program Office and the broader maritime community working to strengthen workforce development and innovation.
We arrived at Q.E.D. as a workforce innovation team interested in training and human performance, but for part of the day we also became trainees. That shift in perspective was valuable because some research questions become much clearer when we step outside the laboratory, put on the safety equipment, pick up the tools, make mistakes, receive feedback, and experience the learning process firsthand.
Special Thanks and Acknowledgments
I would like to express sincere gratitude to Dr. Jessica M. Johnson, Director of the NeuroErgonomics & Simulation Lab and a leader of VDMC's Workforce Innovation efforts, for continued mentorship, guidance, and opportunities to participate in meaningful research at the intersection of human performance, workforce development, and innovation. We are equally grateful to Q.E.D. Systems, Inc. for sharing the expertise of its workforce and instructors and giving us the opportunity to learn by doing.
maritime human-performance research. We are grateful as well to the Virginia Digital Maritime Center, the Web Science and Digital Libraries Research Group, NIRDS Lab, and the broader Old Dominion University research community for fostering the interdisciplinary environment that makes this work possible.
Lawrence Obiuwevwi Graduate Research Assistant | Virginia Digital Maritime Center Department of Computer Science | Old Dominion University Norfolk, VA 23529 Email: lobiu001@odu.edu
Like most software developers, I devote some of my attention these days to what to make of the whole LLM thing. But because I’m a tech lead, I think about it a lot in terms of how it will impact my team. I, for example, value code review. I think it’s helpful for surfacing defects but particularly helpful for building shared domain knowledge and software practices across a team. But what happens to code review when a single dev can generate so many thousands of lines of code in a day? Current practices absolutely collapse under their own weight.
I’ve seen a ton of social media & blog posts about individual users’ techniques for navigating LLM coding, right? But they all read as “this is how I choose a model” “this is my fave harness these days” “here’s my top prompt hacks”, etc. — all about how single inviduals churn out code; fundamentally solipsistic. So I went looking1 for discussions on whether & how LLM coding can be metabolized by teams and didn’t find any.2 I guess that leaves me to write this post, for which, at the moment, I have more questions than answers. And insofar as I even have answers, someone is wrong on the internet and perhaps it is me, and thus I can harvest corrections!
And thus, some areas I’m wrestling with and possible solutions I’m mulling over. Stick around long enough and I’ll make a threat about legacy code.
Code review
A reviewer can’t usefully handle more than 200-400 lines of code in a review. One of my devs, experimenting, prototyped a 2,000-line feature in half an hour.
Do we, uh, have 5-10 code reviews? Who is doing those code reviews? Does a team need to be spending the actual majority of their time doing code review now? If so, does everybody hate their life?
This does not seem feasible.
Can you solve, or at least mitigate, the LLM-caused problem with different LLMs? I’ve been experimenting with a Claude skill to reorganize commit histories for big code dumps like this into reviewable units and it’s pretty okay, actually — but turning one 2,000 line PR into 10 200-line PRs merely converts the mostly-impossible to the possible without actually addressing the fact that everyone is now swamped.
I guess there’s automated code review tools. I’ve experimented only a little with them, mostly because when I tried, it sucked so much I immediately decided to spend my time on something else. But maybe there’s better stuff in the space I haven’t found.
I do think it immediately becomes important to beef up CI with all manner of automated (deterministic!) checks. I mean. Hopefully there was already CI checking stuff. But maybe you didn’t have a code complexity checker, and maybe you really really need one now. Test coverage checkers get super mandatory (on which more later). We can avoid even taking up reviewer time before a bunch of automated gates get passed. But it still seems like too much reviewer time. So….
Tests
Maybe code review just turns into reading the tests?? Particularly as the underlying code is disposable. Maybe it’s a waste of time to read said code (in detail or at all), as long as the tests pass.
But. Buuuuuuut.
Far and away the tests it’s going to be easiest to check are the integration tests, because we wrote the code at the level of integration testing: prompts which describe the overall behavior of a feature. We’re cognitively equipped to verify that the integration tests match the overall desired behavior in a way we’re not cognitively equipped to make sure the unit tests make sense.
But.
Integration tests kinda suck? I mean, they’re important, they’re the right level for testing certain things, sometimes you really do have to check that everything works together. But they’re slow and they’re brittle and they’re often smoke tests, rather than solid assertions that the underlying logic is consistently performed correctly. I have an important system in my life that has some integration tests and no unit tests3 and an enormously large fraction of my, and my team’s, time is spent dealing with this system. Building a world that rests on integration tests really sounds to me like building a world which will, sometime within the next 18 months, be a cannon pointed directly at my face.=
Also. Also?
LLMs suck at writing tests. Seriously, y’all. Perhaps you have One Weird Trick for avoiding this4 but in my experience they are simultaneously too verbose (in the way that LLMs are) and not verbose enough (missing important cases). Tests are where the stochastic-parrot-ness of it all show up the most, because to write well-chosen tests you really need to understand the nature and also the purpose of the system.
I could live with only reading the tests in code review, but they’d have to be good tests that really covered the bases. And it might be hard for me to tell what the bases were if I’d only grappled with the prompt and not the code. And if I left them up to the LLM, I wouldn’t trust them to be adequate.
Also, does anyone actually like reading the tests? Like isn’t that the part of code review people are most tempted to speedrun?
Observability
Cool cool, so maybe you don’t have unit tests and you have no idea how your code works.
Seems that it just got extra important to have pervasive and actionable monitoring. Not like that wasn’t already important. But are you going to know if your code, say, merrily opens up every available Postgres connection and never closes them? I mean. Not until you deploy it. Of course you could have missed that sort of thing in artisanal code too, but it’s a lot more likely you or your reviewer would have noticed.
Architecture and code quality
It’s good for a software team to understand 1. the architecture and key components of the systems it maintains and 2. the domain models within which it operates. Code review is a big part of how this gets spread! But that works less well and maybe not at all in an LLM world, per the above.
OK so. What tools are available here?
I think just talking about it gets even more important. I mean. It was always great to hang out and jam with coworkers about design ideas, data modeling, all that good stuff: what architectural decisions we’re going to make and why. But it gets more important if we no longer really have code review and we’re less often (or never) super deep in the code. Like maybe everyone’s schedule needs to intentionally have more meetings. I hear a lot of people groaning as they read. Sorry. Hold that thought because I’m about to make it worse: it might also get a lot more important to whiteboard stuff. And I’ve never actually found a tool for doing that which is anywhere near as good as a whiteboard…which means being in the same office together. (Groaning intensifies! I told you so.)
I think also it gets more important to collaboratively develop skills (like actual Claude skills, Copilot instructions, etc.). A good team will have more-or-less shared notions of things like…
what high-quality code looks like in their language(s) and framework(s) of choice
key ideas in the domain
architecture within and across codebases
important reusable components
favorite tools, dependencies, and/or patterns
….but they may or may not have documented these ideas. If LLMs are writing a lot of the code, I think it becomes much more important for the team to discuss and maintain (like, under actual version control) a set of skills which formalize all this. And then to point new devs at these skills during onboarding and have a clear expectation that everyone is using them. I think these git histories, conversations, et cetera get to be a really important mechanism not only for developing shared understandings but also for working them into the code.
Cognitive debt
Congrats, you’ve stuck around until I say that in an LLM-codegen-heavy world, all code is legacy code.
I can’t use the definition of “code without tests” here, because I just stipulated that we’re writing those, albeit in a super fraught way. I can’t even say “code we’re scared to change”, because LLMs have no such fear, and will churn out all manner of change! So I’m thinking in the sense of “code we do not understand”.
And yeah, I think code we don’t understand remains a problem in a world where we can write code super fast. Because code is not merely a set of tickets to be implemented, but also a model of a domain and a system for handling processes and all sorts of other things that actually require some sort of overarching Fred-Brooksian architectural understanding, something to provide conceptual unity and clarity across a broad scope.5
Because you can only go so far with implementing tickets or even features, right? Eventually you run into problems like “the fundamental architecture of this system collapses under our current load, and oh noes we want 10x more” or “I know I need a thing that is generally shaped like X but I don’t even know what things are shaped that way; I have to distill my understanding of the domain into something describable”6. And when you have problems like that, you actually need to sit down and think about things. What does my domain look like — what are its key nouns and verbs? What are the major components of this system, and how do they fit together, and are there seams where we can pull them apart and put them back together in a new way? If there aren’t seams, where and how do we make them?
I’ve had some luck telling LLMs to put particular kinds of seams in particular kinds of places, but I had to figure out what I wanted and how to phrase it first.
As with some of the paragraphs above, I think there’s some room for “use LLM tools to fix the problems created by LLM codegen” — huge shout-out to diffity-tour for helping me understand entire codebases in a radically accelerated way. But…
Meat
We’ve got these tools made of extremely eager linear algebra spamming the world with freshly laundered Stack Overflow, but our brains are still made of meat.7 And we made it faster to write code, but we didn’t make it faster to socialize ideas, or to build good mental models of new domains, or to hash out a design with a coworker, or quite frankly to manage the executive-function avalanche of identifying and allocating the number of tasks the robots can now do.8
I don’t actually think the tools we need for managing meat look like “how do be MORE productive! do the work of six devs by yourself! This is winning!”
I do think the tools we need for managing meat change when people’s jobs change, though. When we are suddenly needing to use different skills and tools. When we’re doing so in an atmosphere of hyper-precarity, both from geopolitics and economics writ large9 and from the rather obvious incentive to fire the other five devs who used to be doing that six-devs job, except maybe you’re one of the five, not the one.
Claude gets faster but meat does not.
By which I mean, I asked on bluesky, lol. This is not comprehensive research. ︎
I guess I could’ve also asked Mastodon and I’m sure I would have gotten more responses, but I’m also sure too many of them would have had me asking “why do I hate myself enough to have posted this on Mastodon”. ︎
There are good and interesting reasons for this, by which I mean “it seemed like a good idea at the time” and “may you live in interesting times”. But look, we know these things happen sometimes, much as we wish they would not. We are people of action; lies do not become us. ︎
The Brooks architect comes up in a context that’s entirely too into waterfall software dev, but it’s not like they had an alternative when he wrote it. And despite how much I dislike waterfall, I think he was right about the importance of conceptual clarity and unity in a codebase. Plus which if we’re writing a bunch of LLM code we are sort of secretly living in a waterfall world again, now aren’t we? Because we have just defined all the requirements up front and then sent our minions off to execute. So if we don’t know how to do that with the insight and elegance of Brooks’ architect, we are probably bad at things and should remedy our ways. ︎
I did this last year in a way that involved just kinda thinking about stuff for a month and turning that into a microservice that turned out to be pretty transformative for how the product works and what we can even do with it. The stuff I was thinking about included “graph theory” and “twelfth century Byzantine manuscripts”. Claude could never. ︎
It’s not just me, right? I’m not the only one who looks at LLMs and sees “oh, cool. So we have replaced a world of understanding and crafting things with a world of managing tasks, a world which has fundamentally higher and escalating executive function demands, but none of us are out here suddenly getting better at executive function and also software development is notable for specifically attracting people with executivefunctionchallenges and also in the US at least we’re run ragged from living under mad-king fascism, and so maybe a machine for increasing executive function demands is not actually our best move”? Anyone? Anyone? Bueller? ︎
Anyone who has interacted with me the last 3.5 months is likely to have heard that job searching is hard right now. Some describe it as a nightmare, saying that it’s harder than ever. There are so many stories of people who have been out of work for months and years. This post covers my … Continue reading "Job searching is hard right now"
Status quo costs are the direct costs, risks, and missed opportunities associated with allowing existing arrangements to persist. Several recent OCLC Research studies spanning a range of library-related topics have highlighted status quo costs as a key aspect of library decision-making. The takeaway from these studies is that status quo costs shouldn’t be neglected; recognizing them helps overcome inertia and makes the true cost of inaction visible in decision-making.
In this post, I’ll introduce a simple framework for thinking about status quo costs in library decision-making. Accounting for status quo costs helps libraries make more informed choices by bringing the often-hidden costs of continuity into view. By explicitly addressing the costs of maintaining current arrangements, library leaders can weigh the pros and cons of change versus continuity more realistically.
Status quo is a state of choice
The status quo is rarely an inviolable state of nature, or “the way things are.” Instead, the status quo should be understood as the accumulated result of past decisions based on conditions prevailing at the time: priorities, resourcing, technological capacities, and so on. Choices that were made under those conditions may have been optimal then, but are they still fit for purpose?
In the OCLC Research report Library Collaboration as a Strategic Choice, we frame the enduring influence of past choices using the concept of path dependency: the tendency for the status quo to continue because change is difficult to initiate. Path dependency is especially powerful when the choices involved in the creation of the current status quo gradually disappear from view. Over time, decisions lead to new routines, new routines become established practices, and established practices become unquestioned assumptions about how things should be. This makes the status quo seem inevitable rather than the result of earlier choices.
An anecdote from one of our earlier reports, Sustaining Art Research Collections: Case Studies in Collaboration, provides a compelling illustration of how path dependency, and the persistence of the status quo, can set in. A collaborative partnership between an art museum library and a nearby university library had, over the course of more than a decade, become routinized, and regular contact between stakeholders at the partnering institutions diminished. Routine, however, led to a lack of active management of the partnership, including regular evaluations of its progress or opportunities for adjustment or expansion.
This suggests an important lesson: decisions rarely are once and for all but instead should be revisited regularly as conditions and priorities evolve. This insight inspired one of the report’s recommendations: “One aspect of change management is having structures in place to regularly review the status of the collaboration and consider opportunities for expansion, course corrections, and other adjustments to the status quo. . . . Collaboration is an ongoing choice, not a one-time decision. Partners should continually and intentionally choose whether and how to collaborate.”
Whether the focus is library collaborations or something else, it is important to recognize that continuing the status quo is a choice that needs to be periodically reviewed and evaluated. This is a key first step toward loosening the grip of path dependency.
Surface the costs of the status quo
Once the status quo is visible, the next step is to make the costs of preserving it explicit.
All choices have costs. When the choice is change, the costs of adopting a new strategy are called switching costs. They may include necessary investments, steep learning curves, workflow disruption, and of course, the possibility of failure. These costs are very real, and library decision-makers rightfully take them seriously. In uncertain environments, switching costs often seem very apparent, even stark. In some cases, these costs are perceived as prohibitively high, producing a state of “lock-in”: the persistence of existing arrangements even when better alternatives are available.
Daunting as the costs of change may seem, the costs of preserving the status quo may be even higher. Maintaining the status quo can mean accepting ongoing inefficiencies, missed opportunities, and exposure to future threats. The challenge is to find a pathway through these competing considerations, a task which is often more art than science, shaped both by the magnitude of the costs involved and the decision-maker’s risk tolerance.
Accounting for status quo costs is vital for all types of library decision-making, including how to position an academic library within its parent institution. OCLC Research recently published the report The Library Beyond the Library, in which we make the case that academic libraries should engage more deeply with the broader institutional environment in support of research and learning. Increasingly, a library’s ability to fulfill its mission, retain influence, and demonstrate impact and value depends on that engagement.
For many libraries, this kind of engagement requires significant change in operational approaches. During an OCLC Research Library Partnership roundtable discussion on cross-campus partnerships in research support, many participants questioned whether a partnership-forward approach – an important pillar of the Library Beyond the Library framework – could work on their campus. They cited difficulties like the entrenched decentralization of the university environment and the pressure for individual units to demonstrate a distinct value proposition to campus leadership. These are genuine barriers to change, and overcoming them can involve substantial switching costs.
But the discussion also surfaced potential costs of continuing the status quo of providing research support services in isolation: diminished campus visibility; less impact; and ultimately, fewer resources. One participant provided a concrete example: on their campus, training workshops conducted in partnership between the library and another campus unit attracted far more attendees than workshops offered by the library alone.
By highlighting potential consequences of preserving the status quo, library decision-makers can see that maintaining continuity, while avoiding switching costs, nevertheless involves incurring other costs. And those costs can sometimes be higher than the costs of change.
Make the status quo a choice
Finally, make preserving the status quo an explicit choice. Place the status quo and its costs alongside alternatives (and their costs) to frame an intentional choice between continuity and change.
Once its costs are surfaced, the status quo is no longer a neutral state, but one option among others, with its own cost/benefit profile. This places the status quo on equal footing with alternative pathways—both need to be justified on the basis of their potential benefits and costs. It also counters the perceived inevitability of the status quo in the face of what might appear to be considerable switching costs for alternative options. The switching costs may indeed be high, but what matters is the comparison to the costs of maintaining the status quo. Evaluating these relative costs may reveal that continuity is the appropriate choice, but in this case, the status quo is intentionally chosen as the best option.
Making the status quo a choice could take the form of a periodic review, a set of established criteria triggering a change/don’t change decision, scenario comparisons, and so on. Consider this example from research data management. Once a data set is deposited into a repository, the tendency is often to let it remain indefinitely. But storage space is not infinite, stewardship consumes resources, and so the status quo of retention may not be optimal. The Illinois Data Bank counters this inertia by setting up an explicit choice about whether to continue retention. The Data Bank established a policy stipulating that retention of a data set is reviewed five years after deposit according to a defined set of criteria. Continuing the status quo—i.e., continued retention—therefore becomes an intentional choice within a clearly structured decision framework.
Another illustration is found in OCLC Research’s recent case study of the Portage Network, a collaboration among Canadian libraries to develop shared infrastructure and expertise for research data management. This included the creation of the Borealis Canadian Dataverse Repository, a shared repository that would supplant a number of existing local Dataverse repositories. In these cases, institutions made an intentional decision to move away from the status quo in favor of an alternative method of sourcing RDM capacity. The case study concludes that the status quo costs of maintaining a local Dataverse instance exceeded the switching costs of moving to shared infrastructure. What were the status quo costs? The cost of allocating staff to maintain local infrastructure was certainly a factor, but other costs were the levels of quality, efficiency, and speed foregone by retaining local repository implementations.
Making the status quo a choice is not a way to favor change over continuity. It is instead a way to avoid the pitfalls of path dependency, or a “this is how we’ve always done it” organizational ethos that may obscure better alternatives.
Counting the costs of continuity and change
The landscape of resourcing, staffing, technology, stakeholder needs, and much more is constantly evolving around libraries. In a dynamic and uncertain environment, library decision-makers need to recognize and evaluate both the costs of continuity and the costs of change.
What was once a choice can become entrenched as the status quo through time and familiarity. Making this visible, surfacing its costs, and deliberately reconsidering it alongside other options restores the status quo to what it should be: a decision.
In challenging environments, with limited or no internet connectivity or
in the face of repression, Tella makes it easier and safer to collect,
protect and hide sensitive data.
AI_CONTEXT is a lightweight, vendor-neutral pattern for preserving the
engineering context that normally gets trapped inside one AI chat, one
coding agent, or one context window.
The idea is simple:
Put the important project memory in the repository, then let every AI
agent read and maintain the same source of truth.
Instead of depending on a particular model’s conversation history,
AI_CONTEXT stores concise, durable project knowledge alongside the code:
today, I want to share a concept that’s helped me tremendously in this
shift. I call it micro-practices — 3 minutes, 7 minutes, 12 minutes — as
tiny, almost negligible containers of time that allows me a way into my
creative practices while sideskirting the resistance coming from my
brain that might say… no no no, but we’re too busy/tired/distracted/in
the wrong headspace for this right now…
To be a technology worker in a library, especially a publicly-funded
one, is to be subject to somewhat different pressures than those who are
employed by traditional technology companies. The financial pressure is
not worth lingering on – we know that we are paid less than those who
work for major corporations, but accept it as an inevitable result of
choosing to sell our labor to an organization that is not (at least
officially) oriented toward profit above all else. The mission of a
library is instead – in theory – the creation, preservation, and
transmission of knowledge, and those of us who appreciate that mission
may find ourselves drawn to this work
What does it take to help shape one of the most consequential
revolutions in tech history? Join Boris Cherny, creator and head of
Claude Code at Anthropic, for a wide-ranging conversation about the
past, present, and future of artificial intelligence. From the
challenges of building models like Claude to the possibilities ahead,
Cherny will offer an insider’s perspective on a technology rapidly
transforming our world.
Every request’s energy is measured directly from GPU hardware, not
estimated. Every carbon number uses the live carbon intensity of the
grid running your workload. No annual averages. No projections. We
report the physical carbon intensity of the grid our servers actually
run on, not the carbon we could claim by buying renewable energy
certificates elsewhere. The atmosphere doesn’t distinguish between a
certificate and a smokestack. Neither do we.
This led to the natural question: can gzip do language modeling?1 No
neural network, no learned parameters, nothing. Just the compressor that
ships with your operating system. You prime it with a corpus, give it a
normal text prompt, and it continues that prompt by searching for the
byte sequences that compress best.
Stalker (‹See TfM›Russian: Сталкер, IPA: [ˈstaɫkʲɪr]) is a 1979 Soviet
science fantasy film directed by Andrei Tarkovsky with a screenplay
written by Arkady and Boris Strugatsky, loosely based on their 1972
novel Roadside Picnic. The film tells the story of an expedition led by
a figure known as the “Stalker” (Alexander Kaidanovsky), who guides his
two clients—a melancholic writer (Anatoly Solonitsyn) and a professor
(Nikolai Grinko)—through a hazardous wasteland to a mysterious
restricted site known simply as the “Zone”, where supposedly exists a
room which grants a person’s innermost desires. The film combines
elements of science fiction and fantasy with dramatic, philosophical,
and psychological themes.
SearXNG is a free internet metasearch engine which aggregates results
from various search services and databases. Users are neither tracked
nor profiled.
For one, AI is not trained on what it means for code to be maintainable.
For instance, any reinforcement learning done needs a reward signal that
can be measured immediately, not in months or years. The AI learns rules
from rulebooks meant for beginners. The AI notices patterns from code in
the wild and let’s be honest, most code in the wild is pretty bad. There
is no fitness function you can define for maintainable code, at least
not one that we can discern, otherwise it would’ve been baked into our
linters.
Spain’s Second Section of the Intellectual Property Commission, a part
of the country’s Ministry of Culture, has ordered the blocking of
several domains of the Archive.today service, a web archiving service.
It means that most Spanish internet users who try to access the site are
redirected to a government page telling them that they are trying to
access an “illegal” website.
The last document from the Snowden archive was published on 29 May 2019.
The Guardian stopped publishing documents in February 2014, Der Spiegel
in January 2015, and The New York Times and ProPublica in August 2015.
After that, only The Intercept was still publishing documents, with only
a few exceptions, until it closed its archive in March 2019. Eleven
weeks later, on 29 May 2019, it released what would become the final
batch of documents from the archive. Since then, no news outlet,
journalist, or institution anywhere has published a single document from
the Snowden archive.
A Core Preservation Process is a specific action that every Trustworthy
Digital Archive should undertake adequately - either directly or through
its associated parties or services, in order to fulfill its digital
preservation missions as evidenced in its preservation policy.
Why, and how, are any of us capable of self-reflective thought? Of
experiencing emotion, ratiocination, preference, cold, whim, melancholy,
joy? Anything at all?
Userscripts run in a users web browser and make on-the-fly local changes
to specific web pages. In MusicBrainz they are generally used to change
the display of pages, facilitating editing.
Earlier this month, I added a post here about Five Thank Yous...this year's summer vacation project.
This weekend I was working on last summer's vacation project and realized I hadn't posted about it here.
That project is This Was News, a website and social media bot that encourages people to reflect on the news that was.
It seems (to me, at least) that the news cycle is winding faster and faster, which makes it easy to forget what the most important story of the day when it happened.
One reasonable reaction to this is to tune out the news entirely.
Another reaction would be to get periodic reminders of those stories, then be able to think about their impact after time has passed.
Although this site first debuted in 2025, its origins go back to February 2024 when a collision at DC's National Airport between a military helicopter and a commercial jet was the biggest news.
This was, of course, a big deal, but I wondered how soon it would fall off the front page and eventually be forgotten.
(Do you remember the around-the-clock reporting on that tragedy?)
It wasn't long after that before I wrote the "gatherer" stage of the project: an automated process to save lead article titles and URLs from the home pages of the Associated Press wire service, NPR, and the New York Times.
It ran for several months while I occasionally worked on other parts of the project.
(Eventually, I wrote a "back-fill" script that pulled the headlines from the Washington Examiner and Center Square websites to round out the political biases.)
The "Gather" script runs twice daily: 6:30am and 6:30pm Eastern U.S. Time.
That felt like the right time to capture the important news of the day: early morning East Coast time to get the important news from the previous day, then half a day later to see what happened in the first part of the current day.
The Gather script marks the "top" story from each site, then saves the next two to four stories in the order in which they appear in the HTML code of the page.
I'm using the order of appearance as a proxy for how important each news organization thinks the stories are.
The Editorial Policy page describes how I pick the news organizations and which stories are used from each site.
The way I envisioned using this was to get periodic posts in my social media feed, then replying to the post saying "remind me in 30 days".
Then a month later the bot would send me a personal message reminding me about the event.
As it turns out, that "remind me" function isn't as useful as I thought (and people haven't been using it) but it is still there.
A good number of people are subscribed to the Bluesky and Mastodon accounts, a few have subscribed to the daily email, and an unknown number use the RSS feeds.
(All of these methods are described on the "Get Reminded" page.)
The site's BlueSky and Mastodon bots will also record comments, likes, and reposts on each day's page to create a running commentary on the news.
Last summer's vacation project was a push to get all of the pieces functioning as a system.
I tried a "quiet launch" for ThisWas.News...I was curious to see how far it would go on its own without me pushing it with my existing (meager) social media presence.
For the first six months or so, it got a couple dozen people following the site's bots and reposting its content, which I thought was an interesting organic spread.
Since then I've posted about it a few times with my Bluesky and Mastodon accounts, and reshared some of its content, so the project is quite firmly tied to me now.
Fall 2026 Update
The site has been gathering news for about two and a half years and has been publicly available for about a year.
Over that time, I found that one of the news sources wasn't very useful, so I dropped Center Square and added the Wall Street Journal's news page.
(I had initially rejected the WSJ as mainly a finance news source, but reconsidered this year.)
I also added The Guardian from the U.K. and National Post from Canada as news sources with a bit of an outside-U.S. perspective.
The difficulty has come with the anti-webscraper features that news sites are putting in place now.
I'm not going any deeper than the front page of each news site to get the title and link to the article, but even that was too much for the New York Times and Associated Press.
I reluctantly switched to parsing the New York Times' RSS feed.
This isn't great because the RSS feed doesn't accurately reflect the order of articles in the page's HTML, so it feels like a derivative source instead of what the NYT editors think is really important.
There isn't an alternative for the Associated Press at the moment — they don't have a public RSS feed, and it seems like they are pushing people toward their developer API.
Needless to say, I'm not willing to pay for a developer API key for a solo, spare-time project like this, so I might drop the Associated Press.
(If anyone has another way to get the lead stories from the Associated Press, please let me know!)
A few months ago, I also added "tags" to the data model for each story, and those get output as linkable tags in the social media feeds.
That has helped with discovery of the posts in the social media feeds quite a bit.
The core idea still holds: the news moves fast, and it's worth having something that slows it back down.
Two and a half years in, the earliest stories the Gather script saved are now old enough to be genuinely surprising to me to look back on — which is really the whole point.
If you'd like a periodic nudge of your own, the Get Reminded page has the Bluesky and Mastodon bots, the daily email, and the RSS feeds — pick whichever fits how you already read.
If you have ideas on how the concepts of this site could be extended, let me know!
...those ideas might work their way into next year's summer vacation project.
Throughout the week, I attended keynotes and technical sessions, spent too much time walking around the exhibit hall, and had the chance to meet people working across quantum hardware, software, algorithms, HPC, and AI.
My parents and brother also tagged along for the trip, so outside the conference we had some time to explore Toronto together before eventually making our way to Niagara Falls after QCE.
Monday, September 14: Quantinuum and QUOPS
The conference started very strong.
The keynote that stood out the most to me during the week was given by Rajeeb Hazra, President and CEO of Quantinuum.
His talk focused on the next era of quantum error correction and how we should measure progress as the new systems move toward useful fault-tolerant quantum computing.
Quantinuum introducing QUOPS
One of the major announcements was the Quantum Universal Operation Performance System (QUOPS). QUOPS is a benchmark that does not depend on architecture, designed to measure how much useful quantum computation a system can perform and how quickly it can perform it. It was developed by Sandia National Laboratory, with input from Quantinuum and NVIDIA.
There was also an interesting connection that I had no idea about until the keynote. Dr. Hazra received both his master's and PhD in Computer Science from the College of William & Mary, which is very close to Old Dominion University.
Seeing someone who studied computer science basically down the road from ODU now runs one of the major quantum computing companies in the world was pretty inspiring.
The exhibit hall also opened on Monday night along with the posters. QCE had companies working on essentially every part of the quantum computing stack, from hardware and control systems to transpilers, software, and hybrid computing platforms.
Tuesday, September 15: IBM and the Road to Fault Tolerance
Tuesday had another keynote that I found interesting, although this one got technical very quickly.
Ali Javadi-Abhari, Head of Qiskit at IBM Research, talked about IBM's path from the noisy quantum computers we have today toward full fault-tolerant quantum computing.
One way he framed it that really stuck with me was as a complexity tradeoff between time (samples) and space (qubits).
Complexity tradeoffs behind quantum error mitigation and correction
On one end of the spectrum, error mitigation can work with fewer qubits, but you pay for it by repeatedly executing circuits and collecting many more samples. As you move toward full quantum error correction (QEC), you can reduce that sampling overhead, but now you pay in physical qubits and increasingly complex hardware.
I really liked this framing because fault tolerance is sometimes presented as a single jump from today's noisy machines to fully error-corrected quantum computers. In reality, it looks much more like a spectrum where different techniques make different trade-offs depending on what the hardware can support.
The other part that caught my attention was IBM's hardware roadmap.
IBM is building Quantum Starling at a new quantum data center in Poughkeepsie, New York, with the system planned for 2029. The architecture is highly modular, with multiple quantum systems working together instead of relying on one enormous (monster) processor.
The idea of connecting multiple systems also tied nicely into something that kept appearing throughout QCE: the future of quantum computing is probably going to be much more distributed and heterogeneous than a single QPU sitting by itself.
Wednesday, September 16: NVIDIA and the Quantum Supercomputer
Keynote by NVIDIA’s Krysta Svore
Wednesday continued with another keynote that really stood out to me; this time it was from Krysta Svore, Vice President of Applied Research for Quantum Computing at NVIDIA.
Her keynote focused on NVIDIA's vision for an accelerated quantum supercomputer, where CPUs, GPUs, and QPUs work together rather than as completely separate systems.
She also discussed CUDA-Q Logical, which is NVIDIA's freshly announced infrastructure for developing and testing fault-tolerant quantum applications, along with support for the QUOPS benchmark introduced earlier in the week.
I also had the chance to ask her for advice as someone coming from a computer science background.
I really liked seeing that connection between keynotes. On Monday, QUOPS was introduced as a way to think about useful quantum performance. On Tuesday, IBM went deep into how we might actually reach fault-tolerance. Then on Wednesday, NVIDIA was showing how fault-tolerant quantum computers could fit into a much larger software and accelerated computing stack.
The AI and quantum connection can roughly go in two directions. The first is AI for quantum, where LLMs are used to improve things like circuit generation, calibration, transpilation, error correction, or optimization. The other is quantum for AI, where quantum computers are used to accelerate or change how we perform machine learning itself. Right now, the first direction seems much more mature, but I think there is a lot of potential in both.
This whole AI-HPC-Quantum integration space is probably one of the research directions I am most excited about right now.
Thursday, September 17: AI Meets Distributed Quantum Optimization
Thursday had one of the talks that was most directly connected to my own research interests.
The work combines distributed QAOA with a generative model that produces candidate circuits for smaller optimization problems.
What made this interesting to me was that it combined almost every theme I had been hearing throughout the week: distributed quantum computing, GPUs, AI, optimization, and HPC.
It was also really cool seeing work from ORNL at QCE while I am currently doing an internship there.
By Thursday, it was apparent that hardware, AI, HPC, quantum algorithms, and software systems are no longer being discussed as completely separate areas. They are increasingly becoming parts of the same computing stack.
Friday, September 18: Microsoft and Presenting D-QEO
The last day of the conference started with the final keynote before it was finally my turn to present.
From scalar computing to quantum
I attended Matthias Troyer's morning keynote from Microsoft, and one idea from his talk really stood out to me: that quantum computing is simply the next computing paradigm. Over the last several decades, computing has moved from scalar to vector processing, massively parallel systems, GPUs, and now potentially to QPUs.
He also discussed Microsoft's Majorana 2 approach using topological qubits with the long-term goal of scaling the system to one million qubits on a single chip using fast, digitally controlled hardware.
Then, after spending several days watching everyone else present, it was finally my turn, and I was definitely nervous.
After sitting through a week of talks from people across the largest quantum companies, national laboratories, and universities, getting up to present our own research at QCE felt like a pretty big moment for me.
Presenting our D-QEO work
The main idea behind our work is different from the usual hybrid optimization approaches because we do not ask the quantum computer to directly find the final answer to a continuous optimization problem. Instead, we use it as a topographical preconditioner.
The quantum processor explores the discretized landscape and produces a probability distribution around the most promising regions of the search space before handing it back to the classical optimizer running on a GPU.
Rather than replacing the classical optimizer, the QPU helps it decide where it should search. For the separable functions studied in this work, we also decompose the problem into smaller independent subcircuits that can be evaluated separately. This allowed us to explore larger search spaces without requiring the entire problem to fit onto one very large quantum circuit.
What was funny was how well our work fit into the themes I had been hearing through QCE all week.
Again and again, people were talking about QPUs as accelerators inside larger classical workflows of CPUs and GPUs. Our approach follows a similar trend because we use the quantum processor for the part where it may provide something useful, and let the classical hardware handle what it already does well.
Presenting the work to a quantum-focused audience was also quite a bit different from presenting it to a more general computer science audience. The questions went immediately into things like circuit transpilation, hardware execution, decomposition, and scaling. I also had a chance to connect with other people afterwards for a potential collaboration.
Toronto and Niagara Falls
Niagara Falls from the Canada side
I obviously did not spend the entire week inside the convention center.
My parents from Hungary came to Toronto with me, which made this trip a little different from most of my conference travels so far.
Whenever I had some free time, we walked around Toronto and explored the city a little bit.
Since the conference was right downtown, it was easy to leave QCE and immediately be in the middle of the city.
After the conference ended, we also drove to Niagara Falls together for a day.
I had obviously seen Niagara Falls in pictures countless times before, but seeing it in person up close is completely different. It was a great way to finish the trip after spending an entire week thinking about quantum computing.
Looking Back
The CN Tower at night just outside of the conference venue
Overall, QCE 2026 was a really inspiring experience.
The original reason for going to Toronto was to present our paper, but just like with my trip to KDD, the conference ended up being much more than the presentation itself.
I heard from people building some of the most advanced quantum systems in the world, saw where companies are currently investing their efforts, learned about research that overlaps with my own work, and met a lot of interesting people from both industry and academia.
If there was one technical theme I took away from the week, it was definitely the growing convergence of quantum computing, AI, and HPC. I do not think these technologies are going to develop independently from one another, but rather in ways that support each other.
QCE made it feel increasingly likely that useful quantum computing will become one component of a much larger computing ecosystem. That also happens to be the part of quantum computing that I find the most exciting. Add in presenting our own work at a conference I was honestly nervous to present at, meeting people from across the field, exploring Toronto with my parents and brother, and finishing the trip at Niagara Falls, and it was a pretty memorable week.
I would also like to thank Dr. Nikos Chrisochoides and IEEE Computer Society for making this trip possible.
Yesterday Donald Trump ordered the Interior Department to change the name of Lake Ontario to “Lake America” in its Geographic Names System (GNIS). If your first thought on hearing the news was “I wonder if library subject headings will change?” you’re probably a librarian. Or at least someone who shares some of my uncommon interests.
Mind you, just because a government changes its name for something doesn’t mean that anyone else outside of government will go along with the change. But historically, many organizations have relied on the GNIS for names to use in maps, location designations, and place references in general. The Library of Congress is one such organization, and when in 2025 Trump made similar orders for the Interior Department to rename the Gulf of Mexico and Denali, bypassing the usual professional and stakeholder review for such changes, the Library soon made correspondingchanges to its name and subject headings. Other American libraries went along with it, adopting the new names as well (often by default, as they automatically imported catalog records from shared systems and commercial providers).
So it’s quite possible that we’ll soon start seeing headings like “America, Lake of (N.Y. and Ont.)” in catalog records at the Library of Congress and elsewhere. At this point, though, I don’t plan to make such changes in the Online Books Page catalog. To understand why not, it’s worth reviewing why libraries choose the headings we use in our catalogs. As I see it, there are three main concerns our heading choices should satisfy:
Findability: The main purpose of headings in a library catalog is to help people find the information resources they need. To do that, the headings need to employ language our readers are likely to use in their searches, and that they recognize and follow in their browsing. That’s a strong argument for using the same terminology that people use in their everyday speech and writing, even if that differs from official terminology.
Respect: It’s also important that the terminology we use in our catalog respects both our users and the people associated with whatever we’re naming. For instance, if someone announces a change in their own name, or the name of an organization they run, we typically respect their autonomy by reflecting that change in our catalog, even before the name becomes commonly used by others. Likewise, if a group of people associated with a thing or a name say that the name we use is insulting, or would be better changed to something else, we listen to them, and then often change the name if there aren’t compelling reasons not to.
Standardization: This is an often under-appreciated reason libraries choose the headings they do: Not only does it help users find things if we use the same names and headings consistently across our library systems, but it’s also often much easier for us to maintain our own catalogs if we use the same names and headings as our peer libraries do. That lets us easily exchange records, share cataloging work, and conduct searches across multiple libraries. Few libraries have the resources to create and maintain their own system of headings, but since the Library of Congress does, and is a major source of catalog records for libraries, most American libraries adopt their headings rather than maintain their own terminologies. The advantages of standardization typically outweigh concerns about less-than-ideal heading choices, or the differences in the needs of the Library of Congress’s primary user constituency (which ultimately is the US Congress) and the needs of local library communities.
Given these concerns, changing Lake Ontario’s name in my catalog would not be in the interests of the library I run, or of its users. Changing its name to Lake America would make books I list about the lake less findable, because few people will normally use that name. I’m not aware of anyone besides Trump using or calling for that name in the past, and the lake’s been continuously called “Ontario” since before either the United States or Canada were countries. (“Lake America” might get picked up by this president’s enthusiastic fans, his sycophants, or people and agencies he can order around, but those represent only a minority of my users.) The change is also blatantly disrespectful, not only because it ignores the usage and preferences of the people who live around the lake, but also because Trump announced the name change, for a body of water shared by the United States and Canada, explicitly to retaliate against Canada in a trade war that remains ongoing.
That only leaves standardization as a reason to go along with the change, should the Library of Congress adopt it (which I should note is not yet certain). But with the development of linked data and related technologies in libraries, and with software I’ve developed for The Online Books Page, it’s no longer as important as it once was that my library use all the same headings as other libraries. Simply holding back certain revisions to a standard terminology is easier to sustain than other deviations I might make from that terminology. I’m already doing some automated revision of headings in records I import to our extended shelves, and it’s not a big change to make those automated revisions use older headings for certain names and subjects. I’ve also had software support for automatic conversions between local library headings and standard Library of Congress headings for years. I can use that to seamlessly support links between searches in my catalog and searches in catalogs that use Library of Congress headings. (If I know other libraries are also making similar changes, I can also maintain data to support seamless mappings to their catalogs.)
So whatever the Library of Congress or other libraries do, Lake Ontario will stay Lake Ontario in the Online Books catalog. Or at least it will as long as that remains the predominant everyday name for the lake, and there hasn’t been a properly consultative process to change it. I will be making some changes in my systems and policies to accommodate this decision. I can elaborate more on how I’m doing that in a followup post if readers are interested.
The same day I posted Downgrades Peter Oppenheimer et al from Goldman Sachs (GS) weighed in on the same topic with Competition for Capital:
"There are two themes dominating our investor conversations: the impact of AI and the rise in interest rates. The two issues are linked as demand for capital from both the private and public sectors increase. In the private sector a surge in capex spending to fund AI infrastructure has eaten into free cash flow and forced companies to raise more in debt and equity markets. Meanwhile, government borrowing needs have increased as priorities shift towards upgrading critical infrastructure, energy security and defense at a time when cyclical inflationary pressures driven by higher energy prices are also resulting in higher policy rates. The combination has pushed up the cost of capital. As recently as 2022, for example, 30-year bond yields in Germany and Japan were close to zero ... A rise in yields, together with more uncertainty (over geopolitics and the future impact of AI) have, in combination, pushed up the cost of capital."
As one would expect from the professionals there is a lot to digest in their report. Below the fold I discuss the details.
GS are interested in capital allocation, and the the comparison between stocks and bonds. So they focus on the Equity Risk Premium (ERP), the extra you are paid for investing in stocks over bonds because the risk is greater:
This rise in yields comes after a near-record period of equity outperformance relative to bonds over 10-year holding periods
As equities have continued to outperform bonds, equity risk premia have fallen back to levels last seen in the late 1990s, leaving equity markets more vulnerable to further increases in bond yields. That said, in the US and Japan, the ERP has bounced off recent lows
Remember what was happening in the stock market in the late 90s?
Just as it did then, everything in the stock market just fine:
But despite the rise in yields, nominal GDP and profit growth remain strong and, until recently, the impact of rising bond yields has been offset by the strength in corporate profits. Earnings growth has been the main driver of equity returns over the past 18 months in all regions (Exhibit 5). This strength has meant that PE multiples have either been flat (Japan and Europe) or have fallen (the US, Asia and EM).
In the US, in particular, the forward PE multiple for the S&P has come down from 22x at the start of the year to 19x, in line with its long-run average, despite the market being close to its all-time high.
Note that the standard explanation for the valuation of a stock is that is the Net Present Value (NPV) of the company's future earnings (or more strictly its future free cash flows). An increase in interest rates decreases the NPV of these earnings, and thus decreases the stock's PE.
GS flags this:
The combination of higher cost of capital and greater capital intensity in the tech sector has reduced the value of their future cash flows and triggered a de-rating in the biggest companies. The dominant capex hyperscalers in the US now have a forward PE close to the average of the rest of the market.
A very large chunk of the S&P is the hyperscalers, whose future free cash flow has vanished.
GS describes the growth of hyperscaler borrowing:
While technology profit growth has remained strong, the surge in capex spending among the hyperscalers has increasingly eaten through their free cash flow (Exhibit 8) prompting companies to look for alternative sources of funding, in the credit and equity markets. The capex spending growth for AA-rated issuers has been substantial: the 65% year-over-year growth in Q2 marks the 10th consecutive quarter that aggregate AA capex growth exceeds 35%. They have also turned to the convertible bond market where volume has also increased year to date reaching $135 billion in the US, with AI-related borrowers driving 44% of total issuance, on our estimates.
Supportive profit growth has been accompanied by positive earnings revisions, with 2026 and 2027 estimates moving higher across major regions. There are four broad areas which have been driving much of this profit growth. First, technology earnings continue to be very robust. Second, rising energy prices have pushed up profits in the commodity sector. Third, banks have generally enjoyed strong earnings backed by positive nominal GDP growth, steep yield curves and strong private sector balance sheets. Finally, sectors such as Industrials have benefited from the surge in AI capex spending which has spilled over into improved revenues for the ‘pick and shovels’ of AI infrastructure. The breadth of the earnings' growth has also increased the opportunity for investors to diversify across sectors as well as countries.
Exhibit 11
The thing that worries GS is whether these current earnings, especially those of the hyperscalers are sustainable:
Bubbles in earnings, rather than valuations, have occurred in previous periods. The Bank sector, for example, briefly became the biggest sector in the S&P 500 in the run up to the financial crisis of 2008/09 (Exhibit 11). Unlike the technology bubble of the late 1990s, or the Japanese bubble of the late 1980s, Bank stocks did not experience a major valuation bubble at the time, but the surge in earnings that drove the sector's outperformance proved to be unsustainable.
The Paris-based Organisation for Economic Co-operation and Development (OECD), the International Monetary Fund (IMF) and the International Institute of Finance (IIF), the voice of global banking, on Wednesday highlighted the dangers of soaring interest rates on $365tn (£275tn) in global borrowing.
In its quarterly debt monitor, the IIF predicted a “structurally debt-intensive future” as governments and companies scramble to invest in new technologies and bear the costs of ageing societies.
“The buildup in global debt is set to accelerate as governments and corporates compete to boost growth and secure their positions in an economy reshaped by structural changes,” it said.
On September 10, 2026, Jacqueline J. Dearborn, Library Innovation Lab Fellow, gave a virtual talk about the Public Data Project’s International Data Infrastructure Research and Development effort. The transcript below has been lightly edited for clarity.
Hello everyone, before I get started, I want to give a shout-out to Molly Hardy and Jack Cushman at the Harvard Library Innovation Lab for giving me this opportunity to reflect on some of the most pressing challenges of our time.
With Jennifer Chapman as my counterpart, we’ve spearheaded the Public Data Infrastructure Research and Development project and through this work, we’ve gotten the chance to zoom way, way, way out and take a look at the global public data landscape.
Obviously this is a very huge, thorny, and complex topic. Much larger than any one talk could possibly resolve. So today, I share some preliminary insights that I hope will help us think together about public data infrastructure, long-term stewardship, and collective responsibility. In this spirit, I hope this talk is less of an ending and more of a beginning for all of us that spurs many future conversations to come.
Thriving. Struggling. Gone. Reborn?
Since I know some of you but not all of you, I thought it might be useful to tell you a little bit about myself. So, I began my library career right here at Harvard in 2011. I worked as a grad student at the Museum of Comparative Zoology’s Ernst Mayr Library to process and digitize collections for the Biodiversity Heritage Library. Since then, I’ve spent most of my time working on the infrastructures that hold public data.
As you can see by my career timeline in logos, the institutions and platforms have changed, but the mission to open up knowledge for all really hasn’t. I’ve moved through government, academia, libraries, museums, nonprofits, and research organizations. I’ve followed my purpose and, perhaps less romantically, I have also followed a lot of three-to-five-year funding cycles.
So while many of you may know me best as a member of the BHL Secretariat and Data Manager, I’ve also worked on and contributed to a wide range of other infrastructures, including Internet Archive, Digital Commonwealth, ADS, Boston Open Data (now known as Analyze Boston), Resource Watch, Dataverse, MassDocs, and a myriad of Wikimedia projects.
I suppose the charitable word to describe my career trajectory would be tenacity — or maybe just a stubborn refusal to abandon the work that I believe in. And all this moving around has given me an unusual vantage point. I’ve spearheaded new initiatives and I’ve been around projects that are thriving. I’ve watched others struggle, and heartbreakingly, I’ve watched some go extinct altogether. I’ve also seen some reborn and flourish again.
And when you’re working on the inside, you tend to measure success by fairly standardized metrics: Did we digitize the collection? How many pages? Did we build the API? Did we transcribe it? Did researchers use it? How many? All of these metrics definitely matter. But lately, I’ve become interested in different measures of success because data generation and stewardship is so incredibly labor-intensive.
Measuring success beyond funding cycles
Lately, I’ve begun to ask: Will the data still exist 20 years from now? What about 50? Or 100? And I don’t simply mean, will the files still be somewhere. I mean: Will anyone still understand what the data is, why it exists, and how to use it? Will there still be a community that cares for it? Will there still be enough funding and expertise to repair and maintain it?
These are very different metrics and they entail much more than preserving just the bits and bytes. And honestly, this is part of why I had to take the last year off. For me, this has been a critical moment to reflect. I spent a long time being very close to the machinery, especially in my last role as a data manager at the Smithsonian. And now I’m reflecting on the larger system that this machinery sits inside of.
And since I’m still figuring things out, I’m not going to give you some grand theory of the universe today. But I will give some personal observations I’ve made over the years. The things I’ve seen. The things I keep coming back to. As well as some uncomfortable truths that I think are becoming harder for us to ignore.
Straw, wood, or brick?
So before we get too serious, I want to begin this talk with a little story called “The Three Little Pigs.” This may not be where you expected things to go, but please bear with me. So, we’ve got 3 little pigs. One builds a house made of straw. One builds a house made of sticks. And one — the boring, sensible pig — builds a house made of brick. And then, of course, the big bad wolf comes along, huffing and puffing. And, well, we all know how this story ends.
I’ve been thinking about that story lately as a metaphor, and how it maps rather uncomfortably well onto the infrastructure that holds public data. Because we have built an awful lot of houses with data in them. Some are made of very strong materials. And some are built with straw.
And some infrastructure is beautiful, expensive, and sophisticated. But, nevertheless, it depends on one or two people holding the keys. And for a very long time, we as data home builders have tended to ask questions like: Is the data safe? Did we back it up? Is it online? Can people find it? Can the machines read it?
All of those are great questions. But I think we’re increasingly having to ask the harder ones: Is the house itself structurally sound? Have we built on shaky foundations? What happens if the wolf comes around? Are we ready? Because the big bad wolf isn’t really one thing, is it? Sometimes it’s a change in institutional priorities. Sometimes it’s a funding cut. Sometimes it’s an aging server well past its warranty. Sometimes it’s a person leaving after twenty-five years. And sometimes … it’s simply just time.
And that brings me to the loaded word I’ve been dancing around since the beginning of this talk: infrastructure.
The houses that public data live in
So far we’ve basically defined infrastructure as “some houses with data in them.” This word “infrastructure” may conjure thoughts of code, servers, platforms, networks, storage, software. But I think we’ve inherited a rather narrow technical definition of infrastructure. Because if the goal is to preserve and transmit knowledge across generations, then infrastructure is actually the entire system that allows data to be created, interpreted, trusted, preserved, and shared in the first place.
So let’s map it out. What are some of the core components of infrastructure? Well, there is obviously technology and data but we also have people, expertise, funding. And increasingly, I’ve been thinking about governance and the law as the wrapper around the entire house where human roles, responsibilities, decision rights, and accountability mechanisms all define how those other pieces actually work together.
Sure, there are twenty-seven ways to make this mental model more academically rigorous, but for now, I’m finding it really useful. Today I want to discuss the parts of the house that I know best which are the technical and deeply human dimensions. Because the various components in the house are not independent. They’re interdependent: Technology without expertise doesn’t get us very far. Expertise without funding disappears. Funding without governance becomes chaos. And so on.
So we shouldn’t think of infrastructure as a handful of separate pieces rather we should think of it as the house they build together. Like the electrical, plumbing, HVAC, and so on. All of these things have to work together if you want the house to function and keep its contents safe. And once you look at infrastructure this way, some of the things happening right now start to look less like isolated crises … and more like maybe we have a lot of straw houses and in them is our incredibly valuable public data.
When the house becomes brittle
Katherine Skinner, Director of Programs at Invest in Open Infrastructure, cuts straight to the issue for us:
Data is disappearing because the infrastructure in which it is nested was already brittle.
And in real time, we are watching that brittleness become increasingly visible. I’m not trying to catastrophize here, because this isn’t one single big-bad-wolf moment isolated to the U.S. Everywhere, infrastructure providers are facing some combination of AI bot traffic, leadership and funding shifts, aging technology, and the loss of key personnel.
Infrastructure survival can depend on a remarkably particular set of circumstances: a grant; a director; a willing host institution; a community champion; sometimes just a handful of people who really cared. And one of the most persistent “wolves” I have noticed has been chronic underfunding.
There is evidence for this. Invest in Open Infrastructure examined more than $550 million in grant funding across the public data ecosystem. Of the funding that flowed directly to infrastructure, nearly two-thirds supported research, development, and innovation. But only about 20 percent supported core operations and maintenance. So we are very good at funding shiny new things, but we are not so good at funding them to endure.
And when operational funding begins to evaporate, the effects compound. People leave; institutional knowledge leaves with them; the system stops evolving; users lose confidence; and the community shrinks. And from the outside, it might look as though the data is still there. And sometimes the files are technically still there. But the community-driven ecosystem that made new data generation possible begins to fall away.
When infrastructure stops evolving, in a sense, it stops living. And that is a different kind of loss. For those of us who have been close to the machinery, fighting to keep these systems alive, that loss can feel a lot like intense grief. Which brings me to an important distinction. When public data is endangered, rescuing the bits and bytes may be absolutely necessary. But rescue is only the first leg of a much longer intervention.
Rescue preserves the data. But stewardship preserves the conditions that allow that data to remain usable, meaningful, connected, and dynamically alive. If we copy the files somewhere safe but separate them from the communities, provenance, expertise, and infrastructure that gave them meaning, have we really secured them for the long horizon? Perhaps rescue is the emergency intervention. Stewardship is what has to come next.
Data, information, knowledge
So bear with me while I take you back to library school for a moment: A geo-coordinate can be recorded. A page image can be preserved. A database record can be copied. And we have the LOCKSS principles to safeguard this data. We can make multiple copies. We can checksum things. We can put them in geographically distributed storage. And we can keep them in highly durable formats.
But saving those data points doesn’t really mean we’ve preserved how to interpret and use them. And this difference really matters. Data can be recorded and replicated; information gives it context; knowledge tells us what it means and what to do with it — and much of that knowledge does not live in the files themselves. It lives in people. And that creates a major preservation problem. Because we are very good at preserving data. But people? You can’t LOCKSS a person … at least not yet.
You cannot LOCKSS a person
Since antiquity, human beings have always transmitted knowledge. We spoke it aloud; carved it into stone; wrote it on papyrus and parchment; printed it into books; and now we replicate it as digital data across the planet. The medium keeps changing, but throughout history, knowledge has survived through a continuous chain of human stewardship.
So despite all our new-fangled technology, I’m here to tell you something: the oral tradition is alive and well, folks. It never actually went away. Because ultimately, all of this knowledge is for human beings. We created this data. We are the core users of the public data ecosystem. And I think it’s time we start prioritizing the human use case — not just the AI use case.
AI can dramatically accelerate our ability to discover, connect, and use shared knowledge. But it does not eliminate the need for human stewardship. Its greatest promise is to expand humanity’s capacity to learn, understand, and act. And of everything in our infrastructure house — technology, funding, community, governance, law, expertise — the part I think we understand least well is the deeply embedded human one.
The people. Us.
Because we have a lot of good ideas about preserving data, but you can’t LOCKSS a person. You cannot make a second copy of somebody’s twenty-five years of experience and put it in another geographic location. You can’t checksum institutional memory. This kind of knowledge exists everywhere. We often call it tacit knowledge.
Sometimes that just means knowledge that has never been written down because, quite literally, its only storage medium is a human being. I’ve watched this happen. And I’ve participated in it too. I’m guilty: I’ve been the person who knew where something was; I’ve inherited systems from someone who later retired; and I’ve watched people leave organizations carrying extraordinary amounts of infrastructure out the door with them.
That’s changed how I think about resilience. Because sometimes the first failure isn’t a funding cut or a downed server. It’s a person. Somebody walks out the door, and the cascade begins. So when we talk about investing in infrastructure, we need to get much more comfortable saying something that sounds obvious, but apparently is not:
People are infrastructure. And if we don’t invest in the people who maintain, interpret, govern, and evolve our public data infrastructures, then I can’t really see how public data survives this decade. And that brings us back to the house. Because the same problem that happens when we put too much knowledge in one person’s head also happens when we put too much of our technology in one bespoke system.
We don’t need a gajillion Ferraris
So, the year is 2026 and we have built a remarkable number of beautiful, one-of-a-kind data platforms. Some of them are genuinely brilliant. They solve very specialized problems. They have really cool features and slick interactive interfaces that are optimized for a particular collection or community. And that’s not necessarily a bad thing.
But we’ve also developed a habit of building things that only one or two people in the world know how to maintain. And all that sparkle may be masking some very shaky foundations. And at some point recently I started thinking of data platforms as custom Ferraris. They’re beautiful, impressive, rare; but if something goes wrong, you really gotta hope the mechanic is answering their phone and won’t ask you for a hefty ransom to fix it.
We need a shared FOSS ecosystem
You know what I think? I think we need more Honda Civics. Not because Hondas are better cars, but because there is an enormous ecosystem around them. There are parts, mechanics, manuals, standards. There are many people who can look under the hood and have some idea what they’re looking at. That’s a really important property for public data infrastructure to have.
Because the question shouldn’t just be: can we build it? The question should also be: can more than one human maintain it? Can someone else easily contribute to its code base? Can another institution host it? Can a new person learn it? Can the feature that is genuinely unique sit on top of something shared? And I think this is where open source infrastructure becomes much more compelling than simply “free software.” The real value is not just that the code is available. The real value is that we can distribute the capacity to understand and maintain it.
We don’t need every institution to build its own bridge to some siloed data island. We need to build common infrastructure across datasets and collections using free and open source software, and then we do our distinctive, special-snowflake things on top. Because we need more houses built with brick. And in the virtual world, I think one of the key ingredients for a strong house is Free and Open Source Software (FOSS).
When convenience costs us sovereignty
Unfortunately, there is a rather enormous emerging issue with everything I just said. The FOSS ecosystem does not maintain itself. People maintain it. And many of the institutions that we have come to depend upon are losing the internal capacity to build, deploy, operate, and maintain open infrastructure.
As universities and research organizations outsource more of their technical capacity amidst constrained budgets, the immediate convenience is obvious: someone else manages the servers; someone else handles the updates; someone else keeps the lights blinking. And over time, institutions can stop employing the people who know how to build and operate infrastructure independently.
And this strategy might save us a short buck — but does it get us to the long horizon? One of my mentors, Professor Peter Cornwell, put this much more directly:
FOSS system platforms have advanced functionality; they share vigilance and overcome emerging security hazards via an expert community; they don’t attract license fees and operate indefinitely.
Many institutions have lost these skills: they’ve become support organizations for proprietary products. We must urgently set about restoring these capabilities.
And I think that last point is the one I want to underscore. We may eventually still have the FOSS code. But will we still have enough people who know how to deploy it? That is why there is some urgency here. If we want to preserve the option of building a global data commons on shared, open, interoperable foundations, we have to invest in the people capable of doing that — and we need to do it now.
Otherwise, we may discover that we still possess the blueprints … but have lost access to all of the builders.
What does resilience actually look like?
So I’ve just painted an elaborate picture using the Three Little Pigs and custom Ferraris to illustrate what human and technical fragility look like from the inside. Which brings us to the central question: What does resilience actually look like?
And for this, I want to borrow from biology — because I’ve spent most of my career in biodiversity and I can’t resist. Healthy ecosystems aren’t resilient because every component is invulnerable. They’re resilient because they are diverse, distributed, redundant, interconnected, adaptive, and regenerative. Instead of monocultures, we have biodiversity. Mother Nature really is the best designer, isn’t she?
And I think public knowledge infrastructure needs to start thinking in these terms. It’s not: How do we make one organization, brand, or institution the end-all, be-all house for the data? It’s: How do we build a system capable of surviving the failure of any one part?
That principle has to apply across the entire infrastructure house: Shared technology. Distributed expertise. Inclusive communities. Diversified funding. Transferable legal arrangements. Adaptive governance. Not one institution holding all the cards. Because if an organization has six months of runway left, that’s not resilience. That’s a countdown.
Resilience doesn’t mean making every component perfect. It means creating a system that is less dependent on perfection. We’re not trying to build one house that the wolf can never blow down. We’re trying to build an ecosystem where losing one house is a recoverable event — where the wider community has the resources, rights, knowledge, and capacity to carry the work forward.
We aren’t starting from scratch
Now, I realize I have spent quite a bit of time telling you what is going wrong. But I want to be very clear about something. We are not starting from scratch. In fact, many of the ideas we need already exist: We have LOCKSS and the wonderful idea that Lots of Copies Keep Stuff Safe. We have trusted repositories. We have open-source platforms. We have persistent identifiers. We have interoperable standards. We have a global community of experts operating all over the world.
And we already have enormous networks of libraries, archives, museums, universities, research organizations, governments, nonprofits, community groups, developers, collectives, and individual experts who care very deeply about keeping public data both sovereign and alive.
So the problem is not that humanity has no idea how to do this. Because we have many of the pieces. What we have not yet done is connect those pieces into a sufficiently resilient global system. LOCKSS taught us something incredibly important about data: Don’t depend on one copy. I think the challenge before us now is to take our learnings from LOCKSS one step further and apply them to not just the data but all of the dimensions of infrastructure:
Don’t depend on one institution. Don’t depend on one funder. Don’t depend on one platform. Don’t depend on one country. Don’t depend on one technical team. And certainly don’t depend on one person.
A global mycelium network
And there is something that makes a distributed system like this possible. Standards are the agreements systems make with one another. Law and governance are the agreements people and institutions make with one another. Together, these form a global mycelium network. Persistent identifiers. Shared protocols. Common data structures. Open-source stacks. And open governance processes that determine how we work best together.
Yes, these things feel painfully boring. Nobody is going to give a keynote about the metadata field they viciously argued about internally about for five years and finally standardized in 2017. But this is what makes interoperability and continuity possible. Shared agreements are what allow data and resources to move between systems. Because ultimately, standards and governance are relationships and social agreements expressed technically, organizationally, and legally.
And that’s why the next paradigm won’t be a collection of isolated successful data projects. It will be a unified community with shared foundations to steward the commons. And from my technical point of view, a shared FOSS ecosystem isn’t a nice-to-have. It’s the missing requirement for long-term public data stewardship. Which brings me to my next thought.
Towards a global data commonwealth
What would it look like if we stopped thinking about public data infrastructure as something that individual institutions steward and are ultimately responsible for? What if we thought about public data and infrastructure as a kind of shared inheritance? Not owned by one library or one institution or one country or even one generation. Something we steward collectively because the consequences of losing it are collective.
I don’t have a fully formed answer for what a global governance model looks like for that. But I’m hoping some of my friends here at Harvard Law Library and the people in the audience can lend a helping hand. And I actually think it’s useful to acknowledge that I don’t personally have all the answers. Because none of us do. And it is only through our own humility and our ability to cooperate and collaborate that any of this actually works.
I don’t think the answer lies in some bold visionary coming onto the scene to save the day. Or creating another organization, consortium, or intergovernmental interface. We already have all those things. Instead we need to answer the harder questions here: How do we connect what already exists? How do we change how resources flow? How do we protect and distribute expertise? How do we prevent single points of failure across all dimensions?
So as we near the end of this talk, I think we’ve evolved the narrow technical definition of “infrastructure.” To me, it’s beginning to sound a lot less like a bunch of data platforms and a whole lot more like a commons. Or perhaps a global commonwealth? Imagine a truly global network with different roles, different skills, and different interests — bound by the shared mission to steward public data and carry it forward for the collective benefit of all and future generations to come.
The future of the commons
A good friend of mine, Ben Vershbow, is the founder of As We May Think, which is a new lab dedicated to strengthening the digital commons. When I asked him what the future might look like, he had some interesting insights. He said:
A commons concentrated in a few prestigious and powerful institutions is a fragile commons. The future lies in a federation of diverse communities — attuned to their contexts, connected through shared standards and infrastructures — capable of stewarding knowledge together.
And I think the important word here is diverse. Not diversity as decoration or inclusion as a slogan. Diversity as resilience. A monoculture is fragile. A resilient commons needs many kinds of stewards: people, collectives, communities, organizations, and institutions — across geographies, disciplines, cultures, and various levels of power.
And if we genuinely want distributed stewardship, then we need to distribute resources more effectively and efficiently: money; authority; expertise; opportunity. Because we cannot decentralize stewardship if we are centralizing all the resources.
What now?
I’ve spoken to you today from the parts of the infrastructure house that I know best — the technical and intimately human side. But I’ve only lightly touched on the legal and international governance dimensions that will determine what a resilient global data commons might look like. That’s where my counterpart Jenn and the brilliant folks at the Library Innovation Lab, and many of you here, come in.
Jenn frames this moment as both a crisis and an opportunity to redesign and rethink the current paradigm. In her forthcoming paper, “Responding to the Global Access to Information Crisis through Sustainable Information Access Goals,” she writes:
In the face of mounting challenges to global information access, it is imperative that librarians, information professionals, and other stakeholders develop resilient, connected collections that counter current disruptions and withstand future attacks.
And I think that brings us back to the central point. None of us will solve this alone. The challenge is too big for any one institution, any one discipline, and probably too big for any one country. We may not know exactly what that future looks like yet. But I think we’ve reached the point where we need to start designing for it.
So who will steward it?
So, back to our original question: Who will steward humanity’s public data? I suppose I should admit something. I don’t actually know. But perhaps the answer starts with the people in this room. Perhaps many of us came here today because something about this question stirs something in your gut that transcends your career, your home institution, grant obligations, or personal loyalties to any particular brand, project, or platform.
And if we’re really serious about saving public data, then I think that asks that we all reflect a little bit on some very difficult questions:
Have we built systems that depend too much on our own expertise, authority, institution, or vision?
Are we making it easier for others to carry the work forward — or harder?
And perhaps most uncomfortably: are there moments when stewardship means holding on, and others when it means letting go?
As we know, the public data landscape is shifting and we are experiencing a lot of data losses. Now more than ever, our community needs to come together, globally. To do this work, it will require that we think way beyond ourselves and commit multiple acts of humility, generosity, and service. Because ultimately stewardship means building for a future that does not depend on us.
Thank you all so very much for letting me have this soapbox moment. I’m incredibly grateful to be in a room filled with brilliant people willing to think about the future of public data — and I hope to collaborate with some of you to get busy building brick houses for all the data we’ve been rescuing as of late.
The device to the right is the external CD RW backup, which I purchased
from Ebay, since I (foolishly) discarded John’s when we were clearing
out his apartment. I need this player in order to be able to be able to
read the CD backups of his music that he left, since the TASCAM 788
wrote the backups in a proprietary format.
Many years ago, I wrote a tips on job applications post, because I had been seeing a lot of poorly formatted and written applications. While good formatting and tailoring to job postings are important, the content is obviously important as well. Again, there is so much literature out there, but what I’ve seen recently has … Continue reading "Yet another resume (cv) writing guide"
The $28.5T forecast compares to U.S. Q1 2026 nominal GDP of nearly $32T, with the estimate for the market of AI enterprise applications of $22.7T about 70% of total U.S. economic output.
Sam Altman and Dario Amodei just aren't this good, but their projections of their Total Available Market (TAM) are still turning out to be vastly optimistic. In AI's Affordability Crisis I showed evidence that the AI platforms could no longer afford the massive subsidies they were using to artifically inflate demand for their product, and that reducing the subsidies had made their enterprise customers reconsider their enthusiasm for deploying them. This is leading to investors belatedly realizing that AI platforms' projections of their TAM and thus their valuations are totally implausible.
This re-calibration is just one of the many signs that the AI bubble is about to deflate. Below the fold I present a necessarily incomplete list of them, which I will try to update as more appear.
AI now accounts for nearly half of all IG issuance, 87% of VC funding and a growing share of HY, underscoring how deeply the AI investment cycle has penetrated every corner of finance.
Here’s the title page of this month’s Panmure Liberum market update from strategists Joachim Klement and Francisca Reis. Our emphasis in bold below:
In 1929, the cyclically-adjusted P/E-ratio (CAPE) of the S&P 500 reached 32.6x according to Prof. Robert Shiller’s data. This was 1.8 standard deviations above trend at the time. In 2000, the CAPE reached 44.2x, or 3.3 standard deviations above trend – a clear sign of a bubble. However, as our chart below shows, earnings in both instances were within normal range, less than one standard deviation above trend.
Today, the CAPE is at 41.0x, or 2.9 standard deviations above trend. Once again, we are clearly in bubble territory for stock market valuations. However, unlike in previous bubbles, we are having extremely high CAPE at a time when earnings themselves are 1.8 standard deviations above trend. In other words, we are in a valuation bubble at a time when earnings are in a bubble themselves.
If we correct for the earnings bubble, the current CAPE would be 67.6x or 4.6 standard deviations above trend, a bubble that surpasses anything ever seen in US history by an extreme margin. If valuations followed a normal distribution (which they don’t, so don’t take this literally), this would happen in 0.00019% of months or once every 43,432 years.
A massive chunk of this quarter’s blockbuster “growth” didn’t come from selling more software, shipping more microchips, or delivering more packages. Instead, it came from an accounting rule that forced massive, illiquid “paper gains” onto the income statements of tech giants.
In Q1 2026 alone, just three companies—Alphabet, Amazon, and Nvidia—reported a staggering $69.2 billion in non-operating windfall under their Other Income and Expenses (OI&E) lines. When you run the macro numbers, this single accounting phenomenon artificially inflated the entire S&P 500’s quarterly earnings by about 12%.
That 12% takes the factor from 67.6 to 75.7. Wang notes that correcting for the 12%, "the Q1 2026 earnings growth rate will not be very different from the 5-year average of 16%." In other words, the bubble is feeding upon itself — increased stock prices causes increased earnings causes increased stock price ... But suppose, for example, that OpenAI were to suffer a down round. Then Alphabet, Amazon, Nvidia and others who included paper gains in the "other income" on the way up would have to include paper losses in their income on the way down, amplifying the crash. OpenAI's last round valued the company around $750B and they were planning an IPO for at least $1T, but had to postpone it.
The foundation model era — roughly 2020 to 2025 — is over. The forces that defined it have inverted. Open source models have reached frontier performance while inference costs approach zero, exposing what was always structurally true: pre-training large language models at scale is not a durable competitive moat. The US government's formal designation of Anthropic as a supply chain risk in February 2026 accelerated a transition already underway — but did not cause it. The paper argues that the AI industry is restructuring simultaneously along four axes: economic, as the circular financing structure that inflated foundation model valuations collapses; technical, as the pre-training scaling paradigm gives way to post-training optimization, test-time compute, and agentic composition; commercial, as application-layer integrators displace the foundation model companies whose commodity they now consume; and political, as the government asserts its historic role as gatekeeper of strategic technology. These are not separate disruptions. They are one structural shift, arriving together.
The most consequential and least-discussed dimension is the permanent divergence between commercial AI and a classified national security AI track — built on different data, governed by different rules, and developing capabilities the public ecosystem cannot see, measure, or govern. Like every dual-use technology that has altered the calculus of state power, AI is being brought under government authority not by design but by the structural logic of what it is. The paper further argues that open-weight models are the counterintuitive instrument of sovereign control: a government that holds the weights commands the capability on its own terms, without dependence on vendor policy, financial continuity, or personnel clearance. The apparent openness of distributed model weights is, from a deploying government's sovereignty standpoint, the most governable architecture — because what cannot be withdrawn by a vendor's API policy cannot be taken away.
Grogran implicitly assumes that AI is useful but that the current margins and thus the valuations aren't sustainable.
DeepSeek-V4-Pro is priced through its API at $1.74 USD per 1 million input tokens on a cache miss and $3.48 per million output tokens.
That puts a simple one-million-input, one-million-output comparison at $5.22. With cached input, the input price drops to $0.145 per million tokens, bringing that same blended comparison down to $3.625.
That is dramatically cheaper than the current premium pricing from OpenAI and Anthropic. GPT-5.5 is priced at $5.00 per million input tokens and $30.00 per million output tokens, for a combined $35.00 in the same simple comparison.
Claude Opus 4.7 is priced at $5.00 input and $25.00 output, for a combined $30.00.
Six times cheaper for an equivalent product is likely to cause pricing pressure on the incumbents, who need to raise not reduce prices. DeepSeek is likely also subsidizing usage, but they do have real advantages. First, they have fewer resources so are forced to to be inventive. Second, the 40% of infrastructure capex that isn't the racks is much cheaper in China. Third, the power component of opex is much cheaper and more available in China. The result is:
In practical terms, DeepSeek does not need to win every leaderboard row to matter. If it can deliver near-frontier performance on many enterprise-relevant agent and reasoning tasks at roughly one-sixth to one-seventh the standard API cost of GPT-5.5 or Claude Opus 4.7, it still forces a major rethink of the economics of advanced AI deployment.
DeepSeek-V4-Pro-Max is clearly the strongest open-weight model in the field right now, and it is unusually close to frontier closed systems on several practical benchmarks.
While GPT-5.5 and Claude Opus 4.7 still retain the lead in most direct head-to-head comparisons across the company's benchmark charts, DeepSeek V4 Pro gets close while being dramatically cheaper and openly available.
Dan Davies agrees that the margins aren't sustainable but differs as to why in tokenalysis and john henry:
Which then brings to mind another issue – how confident are we in the pricing power that underpins that 60% gross margin in the first place? In the last paragraph I was talking about the R&D equivalent of a price war, but the normal kind is also possible. The combination of price-sensitive B2B customers, big fixed costs and rewards going to the dominant player doesn’t suggest to me that pricing power is going to be sustainable indefinitely.
But, I think there’s a danger of missing the big picture here. Which is that, when large companies are telling their employees to be sensible and use AI tokens wisely, then the game is up. The race is over and John Henry won against the steam hammer. If you need a human being in the loop to decide on the allocation of AI tokens, then all those predictions of mass redundancy are gone.
But what if the payoff takes longer than consensus assumes? That question is particularly pressing given that token prices continue to decline and Chinese models are gaining ground, both in their share of the world's most-used models and in token usage, where they now lead their US counterparts among the top 20 models,
Slok provides two charts, the first tracks market share by country of origin monthly since January 2025 among the top 50 models. It is bad for the US, showing a steady erosion of US market share until, eyeballing it, as of May 2026 it is roughly a 60/40 US/Chinese split.
Clearly, the market is voting with its feet that the prices charged by US AI companies are unsustainable.
The second compares monthly token use among the top 20 models by country of origin between May and June this year. It is much worse for the US.In May the Chinese had 80% of the US usage. In June they had 185% of the US usage. If this rate of market erosion were to continue for a few months it would be impossible for investors to continue to imagine the golden future awaiting OpenAI, Anthropic, xAI, Meta, Oracle and the neoclouds.
Slok's analysis of the fallout of the bubble deflating is worth reading. This is an issue I plan to return to in a future post.
Chinese AI providers are making more money; Zhipu’s revenues went up 60 times in Q1 compared to last year; Alibaba is the company behind Qwen, and their revenues are up 15 times just since the beginning of this year. The lion’s share of the profits, though, are still being realized by the hardware side; the chipmakers, at least for now. The model companies’ revenues are rising, and steeply. But so is their cost of compute.
That dynamic is causing some Chinese labs to raise prices; the cost to use Tencent Cloud increased by over five times in March, and Alibaba, Zhipu, and ByteDance also hiked prices.
DeepSeek, however, went the other way. Their latest version is priced at just a fourth of their introductory product, and they rolled out dynamic pricing that is aimed at corporate users, who use their models during the workday.
But even with these price increases, Chinese models just cost far less than those on offer from Silicon Valley, and explains those big jumps in the exports, we can say, of Chinese AI tokens. The LLM’s out of China are “90% as good at 10% of the cost”, and American firms are buying more tokens from Chinese companies. In early 2025, token demand from Chinese LLM’s was about zero. Even the release of DeepSeek didn’t move the needle much, but by the end of the year the secret was out, and it’s been a choppy but steady ride up to 46%, today.
Why would you invest in a company whose competitors were “90% as good at 10% of the cost” and which was rapidly losing market share?
If the margins, and thus the rational valuations, of AI companies are unsustainable, how long can the market remain irrational? In The Second Derivative: Why No One Understands the AI Boom Groundbreaker starts by examining the 2008 crash:
The implicit underwriting assumption, shared by originator and borrower alike, was that the loan would never reach its reset: rising home values would manufacture equity, the borrower would refinance into a fresh teaser and the clock would start again. The structure was a treadmill, and the treadmill was powered by appreciation. It worked spectacularly while it worked. Nearly four in five subprime hybrid ARMs originated in 2003 had been refinanced away by the end of 2006.
Now watch the timing. National home-price appreciation did not crash in 2006. It decelerated. The year-over-year rate of gain, which had run in the mid-to-high teens through 2004 and into early 2005, began bleeding off - still positive, still printing green, but slowing. Prices were higher than they had ever been. And yet, with prices at their peak and still rising, subprime delinquencies inflected upward.
...
The deceleration was endogenous to the structure; the structure required ever-accelerating prices to keep refinancing its way out of its own reset schedule, and no series accelerates forever.
The second derivative was always going to roll over. When it did, the first derivative followed it down through zero, negative equity spread from the margin inward, and the defaults the market insisted were caused by “falling prices” had in fact begun a year earlier, when prices were still rising but had stopped rising faster.
The market is pricing AI as a technology cycle when its actual anatomy is that of a credit-driven real estate cycle - which is precisely why the 2008 mechanics apply - and the two break for entirely different reasons.
...
Walk down the AI build-out and every feature is a property development in disguise: a data center on entitled land, financed with debt against the structure and leased to tenants on take-or-pay terms. This is not a software business that happens to own servers. It is a real estate business that happens to compute.
When a hyperscaler or a neocloud reports a record capex figure, the financial press reads it as confidence, as proof of demand. Read it instead as origination volume. Each gigawatt of committed build is a loan extended to whichever tenant has signed the take-or-pay beneath it, and the credit quality of that loan is precisely the credit quality of the tenant. The market is celebrating loan growth and calling it revenue growth.
Loans in this market take the form of Remaining Performance Obligations (RPOs). The biggest borrower in this market is OpenAI:
OpenAI has committed to pay for compute on a scale without precedent in corporate history: multi-year, take-or-pay capacity contracts whose aggregate obligations run to the hundreds of billions of dollars. Against them sits an operating business that does not yet earn a profit - revenue real, large, and growing quickly, but short of covering the company’s own cash burn and nowhere near covering the contracted payments.
Those payments are therefore not serviced out of earnings. They are serviced out of financing, and financing, for a borrower in this position, is available on a single condition: that each new round price above the last.
Measured as a level, OpenAI’s valuation is the most remarkable appreciation in the history of private markets - roughly $86 billion in early 2024, then about $157 billion, $300 billion, $500 billion, and approximately $852 billion by the spring of 2026. Measured as a rate of change, the same series inverts: the round-over-round step-up ran 1.83×, 1.91×, 1.67×, 1.70×, and falls to roughly 1.23× implied by the reported public-offering target. Private marks are inherently lumpy - negotiated, episodic, set by a handful of insiders - so no single step is decisive. But the trend is unmistakable: it bends down, and it bends hardest at the one mark set by the deepest, most unforgiving pool of capital - the public market. The implied IPO step-up is both the lowest in the sequence and the hardest to negotiate, and it is the one the structure must actually clear. This arithmetic is also the most probable explanation for OpenAI’s recent IPO delay.
In the same way that the private stocks generating unrealized gains are not liquid assets, neither are the Remaining Perfoemance Obligations:
An RPO is not a liquid asset; it is a forward contractual commitment - a promise of future payment in exchange for future compute. And a multi-year commitment is worth exactly the creditworthiness of the entity on the other end of it. When that entity is investment-grade and cash-generative, the backlog is what it claims to be: high-quality visibility, merely deferred. When that entity is a pre-profit company that loses tens of billions a year and can pay only by continuously refinancing its own equity valuation, the backlog is something else entirely. It is a subprime commitment, used to justify massive, un-depreciated capital expenditure, reported to shareholders as structural strength.
Now price the credit quality of that book. Of roughly $2.1 trillion in aggregate contracted backlog across the four big platforms, about half - on the order of $1.05 trillion - is owed by OpenAI and Anthropic. Microsoft’s book is about 49% these two names; Oracle’s is 54%, with roughly $300 billion owed by OpenAI alone; Google’s is 43%; Amazon’s is 51%.
The hyperscaler has, in economic substance, extended a concentrated, unsecured loan to cash-burning tenants. The RPO that Wall Street values as forward revenue is, in reality, a credit exposure to borrowers with no operating income.
Major financial institutions are similarly concerned. In their 2026 Annual Economic Report, the Bank for International Settlements writes:
In the near term, the ongoing AI investment boom raises questions about the sustainability of the current economic expansion. The five largest hyperscalers are set to spend over a trillion US dollars on AI-related capital expenditure from 2025 through 2026. These commitments are outpacing earnings and the free cash flow of these firms, leading some to issue debt to raise additional financing (Graph 11.A). This investment race may be partly driven by the perception that only a small number of players with superior technology will ultimately dominate the market shares. The intense competition raises the risk of firms over-committing resources to investment projects with still uncertain returns, leaving all firms vulnerable to disappointments in AI payoffs. Model analysis based on such contest motives highlights the downside risk of current AI exuberance. As competitive pressure drives capex higher, the net economic surplus – the total payoff less investment costs – declines for the sector as a whole and could turn negative in adverse scenarios (Graph 11.B). Disappointment in returns could trigger a sudden pullback in financing and turn the capex boom into a protracted investment bust, with potential knock-on effects on financial conditions (see below).
Another risk is that the AI boom runs into a supply side roadblock. The AI build-out has recently been facing growing bottlenecks in electricity, advanced semiconductors and grid equipment. Fast-growing demand for computing power is already pressuring electricity prices and input costs, with potential spillovers to inflation. Looking ahead, these temporary shortages may also amplify over-investment, as firms attempt to lock in future capacity through long-dated contracts that further expose them to any disappointments in demand.
Historical episodes of investment booms offer instructive parallels (Graph 11.C). The canal mania of the 1830s, the British railway mania in the 1840s, the electrification exuberance of the late 1920s (roaring 20s) and the dotcom boom of the late 90s all shared one common trait: a genuine technological breakthrough that attracted capital in excess of what commercial returns could ultimately justify. These episodes ended with an eventual reversal in investment, inducing economy-wide recessions. The scale and pace of the current AI investment boom accompanied by expectations of large productivity payoffs bear resemblance to these precedents, highlighting potential downside risks in the near term.
A draft report inside the Treasury Department is set to warn of the risks posed by the artificial intelligence market, likening key aspects of it to the dotcom bubble that upended the U.S. economy when it burst in the early 2000s.
The document, the existence and contents of which have not been previously reported but was obtained by NOTUS, is a significant departure from the Trump administration’s public tone, which has focused on encouraging unrelenting investment to unlock exponential growth.
Career Treasury analysts found that AI firms are more deeply entrenched in the U.S. economy than their dotcom predecessors and pose significant risk to the entire system if financial conditions change, productivity goals are missed or various choke points stymie growth.
Vanderbilt University's Asad Ramzanali has a detailed look at the range of impacts from the burst bubble, and an set of optimistic suggestions for policy responses in After the AI Crash. He frames the problem thus:
Companies are investing trillions of dollars based on tens of billions of dollars in revenues. Analysts at J.P. Morgan anticipate $5 trillion of AI infrastructure investment in the next five years. They estimate that the industry will need to generate annual revenues of $650 billion to justify this level of investment, while consultants at Bain & Co. estimate $2 trillion in needed annual revenues. Yet, OpenAI and Anthropic earned $13 billion and $4 billion, respectively, in 2025 revenues. OpenAI’s own financial expectations suggest negative cash flow until 2030, and Anthropic expects a small profit no sooner than 2029. Alphabet, Meta, Amazon, and others may experience increased marginal revenue from integrating AI into existing products, but that is far from certain.
Note that over the next 5 years around $3T (60% of $5T) of the investment in AI infrastructure goes in to buying the hardware which should be fully depreciated over much less than 5 years. So the gap between the investments and the revenue is much bigger than it appears.
Blackstone's QTS said on Thursday it had terminated its planned Digital Gateway data center project in Virginia and withdrawn the associated filings after years of planning and regulatory review.
The data center operator has faced years of local opposition and litigation over the project, despite it being approved by the Prince William Board of County Supervisors.
Among the hyperscalers, Oracle is the most exposed because, as Ed Zitron noted:
And Oracle ... is a company that, even before the AI bubble, was massively indebted. It just so happens that, as a result of its tryst with OpenAI, Larry Ellison saw fit to twist the debt knob to eleven.
Six firms alone — including Oracle, Microsoft and Meta — have committed $850 billion for data centers leases that haven’t begun yet. Oracle holds the largest share of these commitments owing to its $300 billion Stargate contract with OpenAI.
When Oracle mentions the risk of nonpayment, the unnamed elephant in the room is OpenAI. As part of the Stargate deal with the AI company, Oracle is developing massive data centers across the country to provide cloud computing power. For this plan to work, OpenAI needs to pay its Oracle Cloud Infrastructure bills.
“Some of our customers may be highly leveraged and subject to their own operating and regulatory risks and, even if our credit review and analysis mechanisms work properly, we may experience risks of non-payment and non-performance in our dealings with such parties,” Oracle said in the filing.
SoftBank Group Corp.’s talks with potential creditors to raise at least $6 billion from a margin loan backed by its OpenAI stake have stalled, people familiar with the matter said, just weeks after the Japanese conglomerate cut its initial target from $10 billion.
SoftBank Group has reopened talks with a consortium of lenders for a $10 billion loan backed by its stake in OpenAI, after earlier attempts to secure a loan stalled over concerns about the difficulty of valuing private companies, two people familiar with the matter said.
To make lenders more comfortable, the Japanese technology investor is offering to guarantee repayment of the loan, giving banks recourse to SoftBank if the OpenAI shares pledged as collateral lose value, the people said.
This all seems to indicate that potential lenders, such as banks, are highly skeptical of the value of OpenAI stock.
Despite Grok being so bad that employees use Claude, Musk is touting SpaceX as an AI company. This resulted in the most overvalued IPO in history, which failed to raise enough money to avoid the immediate need to borrow $25B. Nir Kaissar's SpaceX Is Junk. That’s What the Bond Market Says reports on the bond market's reaction:
Ratings companies and the bond market have very different views about how things are going. SpaceX’s bonds have an average rating of BBB across the three majors, Moody’s Ratings, S&P Global Ratings and Fitch Ratings, according to credit scores compiled by Bloomberg. In the alphabet soup of bond ratings, it’s the lowest grade still considered quality before falling into junk territory.
The bond market has other ideas. There, quality is judged by a bond’s credit spread or the additional yield it offers above Treasuries with similar maturity. The wider the spread, the lower the quality. Corporate bonds with a BBB rating are trading at an average credit spread of 0.92 percentage point. SpaceX’s bonds, by contrast, trade at a significantly greater average spread of 1.62 percentage points across maturities, higher than BB rated junk bonds’ average spread of 1.55 percentage points.
The bond market seems to agree with Softbank's lenders about the AI bubble.
OpenAI and Anthropic are competing for the next trillion-dollar IPO. Both would need to distract investors from their massive losses by focusing on growth. Recently, Anthorpic has been growing faster than OpenAI, so Keach Hagey and Berber Jin report that OpenAI Considers Drastic Price Cuts, Anticipating War for Users With Anthropic:
OpenAI is considering drastically lowering the prices it charges users as it seeks to win customers from its rival Anthropic.
The company is weighing significant cuts to what it charges for tokens, the unit of measurement artificial-intelligence firms use to bill for their products, according to people familiar with the matter. The move would be in anticipation of similar cuts the company expects at Anthropic, the people said.
Meta will also introduce a new Meta Model API system, which will be used to collect fees from developers. Its API pricing is roughly 25% of the cost advertised by other top models from OpenAI and Anthropic PBC. Developers will be able to use Meta’s model for free, but only up to a point; they’ll be required to pay for access after reaching a certain token threshold, Zuckerberg said.
“The pricing from some of the other labs is very extreme and has very high margins,” Zuckerberg said, underscoring that his strategy is to get Meta’s technology in front of as many people as possible. “We think that there’s a real ability to be able to offer frontier or very high-level intelligence at a much more affordable cost.”
Aggressive pricing means Meta Model API will still be 50% more expensive than DeepSeek but not 50% better. Planning to reduce current income in the lead-up to an IPO is an unusual move, but it is a response to AI's Affordability Crisis.
Factory electricity bills are generally rising faster than those for other business customers or residential customers, according to a Reuters analysis. It highlighted the example of the Belden Brick Company, a 141-year-old brick manufacturer in Ohio, whose electricity bills have soared from $1,600 to $12,000 per month due to a higher monthly capacity charge in the 13-state region served by the grid operator PJM Interconnection.
...
The Ohio-based steelmaker Metallus described its electricity costs as having jumped by 70 percent since 2024, leading the company to pay an extra $15 million in energy costs annually.
The higher electricity costs for manufacturers coincide with the attraction of large AI data center projects with substantial electricity needs to many states in PJM territory. That data center growth has driven up PJM’s capacity prices—paid to power generators according to supply-and-demand forecasts—from $28.92 per megawatt-day in 2024 to $329.17 per megawatt-day in 2026, according to Reuters’ reporting.
Source
As I see it, there are three separate markets for LLMs. First, there is an embedded market that runs on low-cost, low-power hardware and open-weights models. Small AI Models Gain Traction Around the World by David Berreby provides examples:
For example, a drone-based system developed by Bala Murugan and colleagues at the Vellore Institute of Technology, in India, takes photos of cashew plants and quickly identifies those with splotches that indicate disease. All the processing takes place on the drone itself, so there’s no need for a computer on-site, nor for a connection to a central server.
It isn't just that small hardware, such as the Raspberry Pi 5, is getting more powerful but also:
the shrinking footprint of language models. Both Google DeepMind’s Gemma 4 (released in April) and Alibaba’s Qwen 3.5 are “fantastic” for small AI, Rovai says. Both models are “open weight,” meaning users can adjust the connections between parameters to suit their needs. This makes it easy, for example, “to take a lot of data from, say, the milk industry and retrain the model specifically on that,” Rovai says.
The hyperscalers and AI platforms like OpenAI and Anthropic will garner no income from this market, because:
“I think the future of AI is not like one giant model, at a center. I think it’s millions of small, precise models deployed at the edge, each one solving like a specific problem, a specific context,” Alonge says. This is partly because much of humanity—including people in parts of rich countries as well as the developing world—lives without access to cutting-edge frontier models. But, he says, it’s also because those models are not sustainable.
“If someone is not subsidizing it, most people will not be able to afford those models. So those of us who are said to be small-AI developers are the ones who will have to build for the majority of the world,” Alonge says.
At Computex 2026 in Taipei on June 1st, CEO Jensen Huang announced the RTX Spark superchip — a single piece of silicon that combines a 20-core Arm CPU, a Blackwell GPU with 6,144 CUDA cores, and 128 gigabytes of unified memory, connected by NVIDIA’s NVLink chip-to-chip interconnect. The whole package delivers up to one petaflop of AI compute in a laptop form factor.
The number that matters: RTX Spark can run a 120-billion-parameter language model entirely locally, with a context window of one million tokens, without a single byte leaving your machine.
To put that in perspective: GPT-3 had 175 billion parameters and required clusters of A100 GPUs to run. The model that stunned the world when it launched in 2020 is now approximately the size of what fits in a consumer laptop chip announced this week. The capability that required a data centre in 2020 is coming to a device you carry in a bag in 2026.
In 2025, slightly more than a third of all smartphones shipped worldwide were capable of running generative AI, and that figure will reach 45 percent by the end of this year, according to the technology research firm Counterpoint. By the end of next year, slightly more than half of all smartphones will be able to run a small AI model.
If Nvidia can already put a 120B-parameter in a laptop, it will only be a few years until phones can run a GPT-3-class model, good enough for almost all consumer needs. Apple and Google own that channel. Owning the channel is better than owning the technology. They will dominate consumer AI, and the other players will garner no revenue from this market.
Third, there is an enterprise market; all that is left to generate the revenue to service the debts fuelling the AI bubble, lets say $2T by 2030. There are a number of problems that make this unlikely.
First, there are very few documented cases of LLM deployment that resulted in enough productivity improvement to cover its unsubsidized costs.
Second, the Trump administration just demonstrated that deploying mission critical systems on AI platforms such as OpenAI or Anthropic means your company can be disabled at 90 minutes notice with no recourse.
Third, this means that companies will have to run mission-critical LLMs on open-weight models on in-house hardware if they are not to be vulnerable to the whims of the US president.
Fourth, systems such as RTX Spark show that good enough in-house hardware is likely to become relatively cheap compared to the unsubsidized cost of the AI platforms. In-house systems need much less over-provisioning for demand spikes, and because they aren't shared they don't need to be as fast.
Fifth, companies need to balance the productivity benefits (if any) of mission-critical LLMs against the productivity costs they bring. These include a vastly greater attack surface, technical debt from reduced developer understanding of the software, and so on.
Thus it seems likely that the hyperscalers and AI plaforms will generate far less revenue than they expect, because they will be restricted to non-mission-critical applications with lower productivity gains, and thus lower pricing power. They will thus be unable to cover the debts they are incurring to build massive data centers predicated on centralized systems dominating (an inflated estimate of) the entire enterprise market.
Karp had been softly floating his critique for some time, but the CNBC event looked like a proper coming out. Just one day earlier Palantir had published a kind of manifesto devoted to what it described as the all-important principle of “A.I. sovereignty.” The central argument: Companies should seek to build their own A.I. tools, not just customize those on offer from the frontier labs. This might mean relying on open-source L.L.M.s rather than the proprietary ones on which the A.I. boom has mostly been built in America, but it would amount to a liberating declaration of independence from Big A.I., which in Karp’s estimation was sucking up much more value than it was generating.
Locking in a price with a multi-year neocloud contract insures a buyer against compute getting more expensive but not against it getting much cheaper. So as businesses around the world spend ever more on compute, they want to be able to hedge against price volatility just as they insure against changes in energy tariffs, interest rates or foreign-exchange movements—ideally in deep and liquid derivatives markets.
Two startups want to help companies do this, by turning nascent indices tracking compute costs into a futures market. Silicon Data, founded in 2024 and backed by DRW, a trading firm, has paired up with CME Group, which operates large derivatives exchanges. Ornn, created by recent graduates of the Massachusetts Institute of Technology and run from a flat rather than an office just a few months ago, has paired up with Intercontinental Exchange, the parent company of the New York Stock Exchange, to do the same. Both aim to launch compute futures later this year, to be traded on their partner exchanges.
One characteristic of bubbles is overbuilding the infrastructure. Signs of overbuilding of data centers include that both SpaceX and Meta are now in the business of rentling their GPUs to the competition.
As morale is hitting rock-bottom, his company is heavily relying on its competitors' AI models to build out its own in-house tools. And despite the many billions of dollars the company has spent in its flailing efforts to keep up in the AI race, even Zuckerberg himself is now acknowledging that progress is nowhere near where he wanted it to be.
As Reuters reports, Zuckerberg admitted during a town hall last week that AI agents in particular aren't progressing as fast as he anticipated, a devastating revelation following enormous layoffs that wiped out thousands of roles at the company.
The "trajectory of the agentic development over at least the last four months hasn't really accelerated in the way that we expected," he said according to a recording obtained by Reuters.
Despite a derisory ~3% share of the enterprise LLM market, SpaceX's IPO was marketed as an enterprise AI company. The IPO has to rate as the most manipulated of all time featuring, in addition to ludicrous financial projections, a tiny float, bent rules for index inclusion, and massive conflicts of interest at the banks and the analysts. Immediately afterwards, SpaceX issued $25B in bondsa. A month later we can see how the markets respond to the first trillion-dollar "AI company".
We’re old enough to remember when the market cap of the lossmaking telecom SpaceX was bigger than Amazon’s. Heck, for a few precious moments it was bigger than Microsoft’s. Maybe one day it will be again, but for now the stock is down 38 per cent from its peak post-IPO valuation.
Punters lucky enough to have been awarded a stock allocation at the outset are still sitting on a tasty [checks notes] 0.8 per cent paper profit at pixel time.
And this is before the lockups start expiring and the initial tiny float greatly expands.
SpaceX bod spread
Second, more interesting as being much less subject to manipulation, are the bonds. They are what Toby Nangle focused on in SpaceX bond yields rocket towards junk:
the full $25bn of benchmark bonds — issued across the curve — had a rocky first couple of days of trading. Checking back today, it turns out that the inauspicious beginning was just a prelude to the train wreck that has since unfolded.
...
If you’d been allocated $100mn of the SpaceX 2056 bonds, you’ve turned $100mn into $90.7mn in less than a month. Sure, long-dated US Treasury bonds have fallen in value, and this general sell-off at the long end has done some of the work. But the spread on SpaceX 2056 — the additional yield you’re paid to compensate you for the risk that you don’t get repaid (among other things) has now widened from the initial +175bps to a whopping +231bps doing more than two-thirds of the work.
Top 10 worst BBB
The same conflicts of interest that pumped the stock caused rating agencies to give SpaceX bonds a BBB rating, one notch above junk and crucially the lowest that many major institutions are allowed to hold. But Nangle notes that:
Looking only at the nine days since the bonds were included in ICE BofA indices at the end of June, this spread-widening has made SpaceX 2056 the single worst-performing US dollar triple-B benchmark bond:
Note that the top two worst bonds are SpaceX, but the rest of the top 10 are all Oracle, another financial disater area.
when we overlay the average spread for double-B US dollar corporate bonds across different maturities (the pink line), it looks a lot like the type of risk that the market has assigned to both SpaceX and Oracle bonds is junk risk.
SpaceX, OpenAI, Anthropic, Meta all need to raise vast amounts of debt to fund their plans for AI. With SpaceX's bonds trading as BB despite a BBB rating, this is going to be hard.
Founded in early 2025 by former OpenAI CTO Mira Murati, Thinking Machines' first model is a big one. Weighing in at 975 billion parameters, the model requires more than two terabytes of GPU memory — a quantity present in around eight of Nvidia's B300 accelerators, or sixteen H200s — to run at its native 16-bit precision. If that's asking too much of your hardware, Thinking Machines has also released a NVFP4 quantized version of the model capable of running on half the GPUs.
This makes it the largest American open weights model to date, and comparable to Chinese models like DeepSeek V4, GLM 5.2, and Kimi K2.6 in terms of size and capabilities. Take these claims with a grain of salt — gaming AI benchmarks isn't exactly difficult – but Thinking Machines says Inkling is competitive with these models in a variety of workloads, although its benchmark charts also show it trailing proprietary models like Anthropic's Claude and OpenAI's GPT.
If you don't like Chinese open-weight models, the US ones are getting better:
The model developer claims to have tuned the model to use these thinking tokens more efficiently and that Inkling therefore matches Nvidia's Nemotron 3 Ultra, up to now the largest and most capable American open weights model out there at 550 billion parameters, on Terminal Bench 2.1 using roughly a third the tokens.
The AI revolution is supposed to be in its early innings. Demand for GPU computing is exploding. Microsoft has relied on CoreWeave for massive amounts of AI compute. OpenAI has signed tens of billions of dollars in long-term contracts. NVIDIA isn’t just supplying the chips, it has also invested in CoreWeave and entered into agreements that support parts of its financing and capacity.
So why are CoreWeave insiders continuing to sell stock? Management says many of the sales were made under pre-arranged Rule 10b5-1 trading plans. But those plans explain how the shares were sold—not necessarily why executives continue converting stock into cash while investors are being told AI infrastructure demand has never been stronger.
New York has become the first U.S. state to stop construction of large new data centers, imposing a one-year moratorium due to growing concerns over power costs, water supplies and the burden on local communities.
"As data center development threatens to hike up utility bills, deplete our natural resources, and create uncertainty for New Yorkers, it's my responsibility to take action and lead," said New York Governor Kathy Hochul.
She added that she would also pursue legislation to repeal sales tax exemptions for large data centers.
Phurichai Rungcharoenkitkul continues the Bank for International Settlements's pessimism in The AI Investment Race:
The AI build-out ranks among the largest technology-driven investment booms in US history. Its scale, reliance on debt and circular equity ties raise questions about the boom’s sustainability and financial stability. We study a dynamic contest in which firms competing for a few dominant positions over-commit resources. The over-investment leaves the sector exposed to revenue disappointment that could turn boom into bust. The larger the boom, the deeper the eventual bust. The race to commit early through debt and circular financing also makes a bust more likely. Calibrated to balance sheet and deal data, the model points to over-investment of around 1.5 times the efficient level, rising to around three times where demand is less elastic. A network analysis shows that stress in one firm could cascade to others through chains of financial exposures.
As recently as this week, one executive at Anthropic PBC, who spoke on condition of anonymity, mused that the Claude maker’s technology was roughly six to 12 months ahead of Chinese rivals.
On Friday, Moonshot AI Inc. upended those assumptions. The Chinese AI lab released Kimi K3, a more advanced open-weight model that it said outperforms all rivals except for Anthropic’s Claude Fable 5 and OpenAI’s GPT-5.6 on overall capability. The implication is that Moonshot, and by extension China, could be closing the gap faster than expected.
...
Moonshot’s release could also undercut OpenAI, Anthropic and others on price at a moment when they’re confronting customers who are becoming more conscious of their surging AI spending. Some developers have turned to so-called model routing services that can seamlessly direct users to cheaper options, including from China, for specific tasks to maximize cost efficiency.
Kimi K3 is an open-weight model.
If a 120B parameter model is too small for you the RTX Spark won't cut it. But Michal Malewicz explains in NVIDIA just killed big AI and… You’re the winner? that one of Nvidia's OEMs will sell you a $94K box that will run a 1T model, the Exxact Valence Nvidia DGX Station.
"[O]ur bottom line is that AI adoption appears gradual and steady rather than rapid and transformative, with most households and businesses still reporting limited exposure to the technology. At the same time, evidence of a structural pickup in productivity growth remains surprisingly fragile: Aggregate productivity growth has improved during the post-pandemic period, but there is a strong rationale to attribute much of this improvement to cyclical variation in utilization rather than a sustained acceleration in productive capacity. We also find little compelling evidence in the available industry-level data that industries adopting AI more rapidly are already experiencing stronger productivity growth."
"productivity trends across all three levels have been relatively consistent over time, suggestive of micro-level productivity gains not adding up in aggregate"
Another instance of Betteridge's Law is Will OpenAI ever be profitable? from Leap Finance Academy. It is a very long and detailed examination of all the things that would have to happen for OpenAI to turn a profit by 2030. In summary, with my comments, the list is:
Reduce cost of inference which isn't going to happen enough because each successive frontier model uses more tokens to generate the same output.
Increase pricing which isn't going to happen because they already need to reduce pricing to stem the loss of customers to Anthropic, let alone to the open-weight models.
Increasing sources of revenue that are not inference dependent in other words the fantasy of $100B/year in advertising income, which isn't going to happen.
Securing and retaining human talent which requires an IPO to make their stock options worth something, which isn't going to happen it time to prevent bankruptcy.
Securing the funding they need to build Stargate and for working capital which isn't going to happen because even an IPO wouldn't come close to generating enough and the lenders are saying "enough".
Even though I'm skeptical of the wishful thinking of the conclusion, this is an impressive piece of work and well worth reading.
China’s open-source approach has spawned a crowded field of innovative and intensely competitive start-ups all offering systems at low cost.
So attempts to increase revenue by charging more for access to certain models can scare away customers. Price-conscious Chinese consumers — businesses and individuals alike — are quick to hop across platforms in search of inexpensive A.I. tools.
While China’s A.I. companies are searching for sustainable business models, spending is high and revenue low, said Richard Lin, a vice president at the Silicon Valley company Datastrato.
“In two or three years, we will still be trying to figure out how large models can earn money,” he said.
Offering low prices has helped the Chinese firms gain users, including in Silicon Valley, where many companies depend on the more affordable systems. Yet Chinese companies have struggled to translate huge numbers of global users into profits.
Bain & Company management consultant Jue Wang said many of the big businesses her firm advises have been taking a closer look at returns on their AI investments.
“The token cost for them has been doubling, almost every other month,” she said. “Let’s say $200 per developer per month. Multiply that by 20,000 developers, which is often what we’re dealing with at these companies, and that quickly gets you to a number that is not a line item that any general manager has planned for.”
Sometimes that just means not using the AI equivalent of a sledgehammer to crack a nut.
A huge wave of supply of debt intended to fund the data center build-out for AI means that the price of this debt has dropped, and thus that the interest rate buyers are demanding goes up. This reduces the net pressent value of the (hypothetical) future cash flows the data centers will generate, and thus their ability to pay the interest and principal. This increases the interest rate the buyers of future debt will demand, causing a feedback loop.
Both Ed and Torsten point to the increasing spreads above Treasuries that the AI debt is trading at, and the increased cost of insuring against these companies defaulting.
In a blink, [Leopold Aschenbrenner's] wildly successful hedge fund, Situational Awareness, was forced to sell billions of dollars of technology investments that had rapidly lost value, as nervous banks began to demand more and more collateral for his trades. Then came billionaire Ken Griffin.
In less than 24 hours — which included a conversation between Griffin and Aschenbrenner — Griffin’s Citadel hedge fund reached out to Situational Awareness and snapped up the investments at a discount, according to a person familiar with the matter who asked not to be identified citing private information.
It was a startling reversal for Aschenbrenner, a former researcher at OpenAI who — before starting his hedge fund roughly two years ago — had no previous investment experience. His fledging firm has watched its assets plunge from $45 billion at the start of July to about $10 billion.
Aschenbrenner was all-in on AI.
On the Pof. G. Markets podcast legendary short-seller Jim Chanos focused on the accounting inequality inherent in the AI (and earlier dot-com) bubble. The spending of the hyperscalers and neoclouds is investment, expensed over say 5 years. But for the chip makers, construction companies and so on that spending is this year's revenue. So, overall, it looks like the ROI is better than it really i
s.
US GAAP was built on the assumption that neither route would materially move profits, because which company would ever choose to grant ‘enough’ RSUs to employees that it could move the share price right?
Meta has granted ~US$70bn in stocks. And Microsoft ~US$42bn. The sheer size of these numbers IS enough to move the needle materially. RSUs, granted in large quantities every year, compound into visible profit erosion and free cash flow consumed by “obligatory” share buybacks, as employees must receive their promised shares annually regardless of market conditions.
As of FY25, total unvested RSU’s book value was ~US$60bn and corresponding market value was ~US$80bn. That $20bn gap is money Meta will have to find, one way or another, the moment these shares vest, either by diluting shareholders with fresh stock, or by spending real cash buying shares back to hand over instead.
...
Using the extensive information provided by Meta’s financial notes on the RSU programme, we can back-solve the profit erosion that is being kept ‘off-the-Income-Statement’:
Since December 2024, as the share price of Meta increased and the AI talent war started, ~10 points of EBIT margin per year have been given away as additional, uncaptured employee compensation.
Ed Zitron is back with another scathing analysis in The AI Demand Bubble. As usual, it is long and detailed. He concludes:
To put things really simply, Anthropic and OpenAI are a way that hyperscalers can feed their revenue to themselves by spending money on capex, backstopping compute contracts, or doing direct equity investments.
Their continued existence allows the AI bubble to continue inflating, but this can only continue as long as venture capital and hyperscalers are capable or willing to invest. There is simply not the demand — not from open source, not from other AI labs, not from self-hosting, not from anywhere — to justify the capex or the massive data center buildout.
And for those arguing that there would be a dot-com bubble recovery story, I must be clear that if there isn’t demand today, it won’t magically appear tomorrow. AI GPUs will cost just as much to run in five years as they do today, as will unfinished data centers cost just as much to finish, as will electricity remain expensive, and all this will be happening after it’s easy to raise venture capital to actually buy the compute.
In a recent survey of 300 business executives, 68% reported overspending their AI budget over the past year. It’s little wonder so many companies are now cracking down. Uber Technologies Inc. recently capped employees’ AI spending at $1,500 a month. Tesla Inc. reportedly set a limit of $200 a week. The swift reversal has left some workers complaining of whiplash, with Reddit forums filled with stories of engineers suddenly stymied by usage caps.
Bloomberg in partnership with researchers at Vals AI, an independent AI evaluation and benchmarking platform, tested seven models from frontier Chinese and US companies to see how they performed in a real-world task. They were asked to create a fictional coffee e-commerce site called Brewberg using the same prompts. Most of the models scored 100% functional accuracy despite occasional design misses, but with very different price tags. The experiment employed the top performing models in July from Anthropic and all the Chinese firms, as well as more affordable models from OpenAI and Google.
They all did reasonably well, but the two best were Claude Fable 5 at $48.99 and Kimi K3 at $11.99. Chinese models charging much less for almost the same performance are grabbing market share:
the use of Chinese models overtook US platforms globally for the first time in June, and accounted for more than 60% of market share last month, on OpenRouter, a tech platform that offers software developers access to hundreds of AI models. It is a widely watched gauge of model usage despite tracking just a fraction of global AI consumption. The US, parts of Europe and Asia now favor Chinese labs, according to the same data.
On Hugging Face, Chinese AI models account for 41.4% of generative model downloads among developers, 5 percentage points higher than US models.
In OpenAI’s unraveling has begun Gary Marcus notes that leaked Q2 numbers show OpenAI's quarterly revenue grew from $5.7B to $6.7B but its quarterly losses grew from $9.3B to $12.3B.
Perplexity is launching Portable Computer today, a version of its agentic "Computer" platform that runs entirely on hardware users already own — starting with Nvidia's DGX Spark desktop supercomputer and Linux machines equipped with Nvidia RTX GPUs.
The launch, developed in close partnership with Nvidia, is one of the most aggressive attempts yet to move serious AI agent workloads off the cloud and onto local devices. The model, the user's files, and the work itself can all stay on the machine. Work completed locally consumes no billing credits, and the company says every task starts on the device by default — with the system asking permission before sending any individual step to a more powerful frontier model in the cloud.
Apple announced new iterations of both desktops, along with two new chips: the M6, the first 2nm chip in Apple’s M-series lineup for Macs, and the M5 Ultra, now the most powerful chip in the lineup for most things—especially AI workloads.
There aren’t any major new features for either machine. This is just a specs bump. But based on how Apple is presenting these refreshes, they’re leaning hard into those use cases, which weren’t even a thought when earlier iterations were first engineered.
The devices’ popularity for production inference took off after macOS 26.2 shipped last December. According to Apple’s release notes, 26.2 enabled “low-latency communication between Thunderbolt 5 hosts for use cases including distributed AI inference using MLX.” Thunderbolt 5 is a very fast wired data connection, and MLX is an open source array framework designed to help machine learning workflows take full advantage of the M-series chips’ unified memory.
Since then, both hobbyists and professional developers and researchers have been essentially daisy-chaining Mac minis or Mac Studios to run inference on local large language models that are much bigger than anything that could run a single mass-market device—providing an alternative to ultra-beefy specialized hardware featuring specialized Nvidia GPUs.
IBM has rolled out the newest models in its family of open-weight large language models designed to be downloaded and self-hosted. The newly launched Granite 4.2 comes in 3B, 8B, and 30B parameter variants.
Like previous versions, IBM is taking a decoder-only approach here. These new releases offer a 128,000-token context window natively. The 8B and 30B variants (not the 3B one) also go through an agentic reinforcement-learning block; they were trained for expanded capabilities like using the terminal, searching the web, or using external tools. The 3B model supports tools too, but without the same level of specialized training.
Beyond those tweaks, this release is particularly notable because, as IBM itself writes, “Granite 4.2 is the reasoning-focused release of the Granite language-model family.
Morgan Stanley analysts today revised their model for how much power would be needed to switch on the chips it projects will be sold. They now estimate, based on a total data-centre power requirement of 257 GW by 2028, a gap of between 30 and 40 per cent between US capacity and sales projections. Or to put it in layperson’s terms:
New York City demands 5.5-6 GW of baseload power: we are short power by six New York Cities.
And that’s using their mid-estimate for innovative “time-to-power” solutions. Absent such mitigation measures, the estimate rises to 10 NYCs:
The innovative solutions include a massive expansion of Bloom Energy fuel cells, data centers alongside nuclear power plants, and other equally optimistic concepts.
But Morgan Stanley are equally optinmistic about Nvidia:
Fenyman, Nvidia’s next next chip architecture, is scheduled for launch in the second half of 2028. It’ll be another step change, Morgan Stanley says: the same dynamics will increase US power demand year-on-year by a further 40 per cent in 2029, Byrd and team estimate. They’re then looking at a cumulative power shortfall of 107GW (or approximately 18 NYCs).
“However,” the team add in bold, “all of these advancements converge on one central issue: if we cannot close the power shortfall, the expected pace of improvement and adoption may not fully materialise in line with market expectations.”
They probably don’t need to spell out what happens to all those big tech off-balance-sheet guarantees in that scenario. Obviously, the faster-rising thing will eventually overwhelm the slower-rising thing,
I think "market expectations" are out over their skis.
I was going to write and publish this post earlier, but the last couple of weeks have been trying to enjoy (the rest of) my vacation and submitting job applications. It’s also been a bit weird. I’ve never worked anywhere else longer than 2 years (mostly because of the jobs being contracts), so it’s sad … Continue reading "Reflection: The end of 8 years at GitLab"
I begin by addressing how we might define public data. It is, in short, data whose value is not determined by the marketplace, by its commodification, or by what Marx would call its “use value.” Public data may function in all of these ways, but its value is defined by something more abstract and less quantifiable. It serves civil society, and without it, an intrinsic public good—the health and safety of a polity, the protection of an environment, the advancement of creative and critical thinking—would be lost.
As stewards of the past and curators of the present who work in service of the future, we in libraries, archives, and museums have long turned our attention to cultural heritage—both in its material and digital forms—for their value as public goods. In the United States, we have, since the first Census in 1790, more or less “Render[ed] therefore unto Caesar” the production and dissemination of information produced by the government for the governed.
Since 1895, civil society in the form of libraries have made this information accessible through the Federal Depository Library Program, which self-describes as the “nation’s link between the American public and its government.” In the digital world, with uneven success, this distribution network has been replicated online through the aggregator Data.gov.
As James Jacobs and Jim Jacobs point out in their recent comprehensive study Preserving Government Information: Past, Present, and Future, “Although there is a general consensus that the government has an obligation to preserve its own information … the laws that affect preservation of born-digital Public Information are outdated and inadequate.” The last year and a half have done more to reveal the vulnerability — the precarity even — of public data as vital infrastructure than any other time in our country’s history.
We are among the collective effort to change the cultural climate around government data so that it is understood as a resource held in the public trust.
In response to this moment, Harvard Law School Library’s Public Data Project began with copying 311,000 datasets from Data.gov between November 2024 and January 2025. We now build cutting-edge tools and interfaces that enable access, discovery, and monitoring of large public datasets. We are among the collective effort to change the cultural climate around government data so that it is understood as a resource held in the public trust. We are grateful for the support of the MacArthur Foundation and the Rockefeller Brothers Fund, without whom we could not do this work. The opinions and views shared here do not necessarily state or reflect those of our contributors.
As part of that collective effort, in August of 2025, we copied 710 TB of public domain data from the Smithsonian Institution — the complete open access portion of the Smithsonian’s collections. We have updated it weekly since then, and now have approximately 828 terabytes (9.1 million files) sourced from more than 20 libraries, museums, and research centers across the Smithsonian. It is this work that I will focus on today.
We have chosen Source Cooperative as the ideal repository for a number of reasons, not all of which I will get into right now. I will just say that because it is built on cloud object storage, the repository supports direct publication of massive datasets, making it easy for us to add, update, and share data. Source also provides a handy user interface for browsing and previewing, along with an API for fast programmatic access.
At present, this repository mirrors the directory structure used by the Smithsonian, but navigating it has proven a challenge. More than 17 million metadata records, constituting over 47 GB, are included, but associating them with the images in this collection, let alone querying them for a particular subject, is difficult. We have therefore generated lightweight Parquet search catalogs for this collection, which can be found under the search directory.
And it’s being used! Early analytics suggest a considerable interest in this largely unmediated data as data. But for us, these 17 million metadata records and their accompanying images are the tip of the iceberg — for two reasons.
First, we position this work among the larger movement to call attention to the importance of federal data as vital public infrastructure, for which allies such as America’s Essential Data advocate by highlighting “how everyday Americans are benefiting from specific federal datasets.” As Fran Berman recently wrote in The Conversation, “society … need[s] to change both its expectations about digital critical infrastructure and its actions — the way the public and private sectors create and regulate these services and systems and the way the public uses them — to better protect the people who depend on them.”
Not only do we position the Public Data Project among such efforts, but — and this gets to the second reason for our work — we position the Smithsonian’s public data as an excellent ambassador for this mission. Want to get people to care about public data? Lure them in with the cool stuff … like this ceremonial Chinese wine vessel from the Zhou period at the National Museum of Asian Art, which can be viewed as a 3D model here.
Zhou period ceremonial wine container (fangyi) from the National Museum of Asian Art. Source: Smithsonian Open Access Archive.
To enhance discovery and monitoring of large government datasets through the building of interfaces and tools, we need collaborators. We could not be happier to have begun, this summer, an intensive collaboration with Kyle Deeds, Ben Lee, and their graduate students Akshay Mehta and Ying-Hsiang Huang.
As a collaborative, we maintain that the heterogeneous and interconnected nature of government data requires new models of search, and this group’s work with vision language and frontier models is leading the way to meet these needs. Together, we are building not just tools, but also a community of experts who will lead the pedagogical and technological interventions necessary to make a real difference in the cultural and political landscape.
As a team of engineers, librarians, and scholars nested in Harvard Law School Library’s Library Innovation Lab, the Public Data Project, in concert with our collaborators, seek — as our mission states — “to grow knowledge and community by bringing library principles to technological frontiers.” In this instance, we see our work with the Smithsonian public domain data as an opportunity to think differently about library, archive, and museum (LAM) catalogs as we have known them.
The word “catalog” is Greek, combining the word kata, which means “completely,” and legein — “to say, count, or gather.” We can understand LAM catalogs as counting or gathering completely, or at least aspiring to gather completely. Objects in collections — paintings, incunabula, newspapers — are the organizing principles by which catalogs are composed. A record is created for each item and then the records are “gathered” or aggregated into a list, an index drawer, an OPAC. And in the world we have known, the institution that holds the object, or its digital instantiation, creates that record and maintains that collective catalog.
Catbriar and huckleberry in the author’s backyard. Photograph by Nick Anderson.
But this story is changing. In our networked and media-saturated world, commentary, both expert and amateur, happens beyond the walls of the brick and mortar institutions that house the objects. I am thinking here of the work of Michelle Caswell and the idea of “community archiving” because, I want to posit, such “bottom up” descriptions offer us a challenge — and an opportunity — to reimagine catalogs.
We want to engage social media as sites of interpretation and cultural mediation.
Object-based records could still rule the day, but the sources for the accompanying metadata might become varied, so that institutionally created metadata sits alongside socially created metadata, what George Oates simply refers to as “social metadata.” We want to engage social media (and other sources created beyond the confines of the institution) as sites of interpretation and cultural mediation.
And in so doing, we want to enhance our understanding of what constitutes a collection, so that we are layering on two large datasets that pertain to the Smithsonian Institution collection and are created by the public.
“Social metadata” becomes part of how we describe and interoperate a collection. We begin with the Smithsonian Transcription Center, which hosts crowdsourced transcriptions of digitized Smithsonian materials. Established in 2013, the center supports volunteer transcription projects across the Smithsonian’s museums, libraries, archives, and research centers. Between July 7 and August 10, 2026, the Public Data Project collected the source files, transcription files, PDFs, metadata, and project manifests associated with 14,167 transcription projects — 854 GB of data in total.
The public cannot rely on industry platforms to see and study social media. The lack of ability to capture social media is one that permeates the digital archiving world.
In addition, we currently have two sources for social media data. First, the Smithsonian Flickr Data Lifeboat, which includes 3,308 photographs; 7.8 GB of data; 5,768 comments; 10,954 tags; and contributions from 4,710 users. We are also an early beta tester for Common Data, a service that will provide an easy way to see and study social media in real time. This platform and data trove are necessary because the public cannot rely on social media industry platforms to help study it.
The lack of ability to capture social media is one that permeates the digital archiving world. If Common Data is transforming that, we believe that this Smithsonian data project is well-positioned to showcase an early adoption of social media archives to enhance government data collections. Common Data is currently developing a system for collecting and preserving social media data from platforms including Bluesky, Telegram, TikTok, and YouTube as well as podcasts, and soon it will include data from Instagram. In the coming months, we will be calling for testers of this new interface and the fine-tuned LLMs and vision-language models underlying it to make connections.
The tech industry likes to talk about how AI enables us to know or capture all or total knowledge, and we might question that assertion as a receding horizon at best, a shibboleth at worst. But, when it comes to catalogs, I’d posit that it behooves us to strive for a deeper record, to “list completely” as the etymology of the word itself demands.
By supporting multi-modal search across different collections, institutions, and types of data — visual, temporal, geospatial, social — we hope to enable search across paradigms as well, different understandings of our past and present, different populations and experiences, and to surface sublimated and hidden connections.
This post was written by members of the DLF Assessment Interest Group’s (AIG) Metadata Working Group (MWG). Learn more about theAssessment Interest Group.
It was authored by: Xiaoli Ma (University of Florida), xiaolima@ufl.edu, Stasha Gardasevic (University of Hawaii at Manoa), gardase@hawaii.edu, Anna Goslen (University of North Carolina at Chapel Hill), goslen@ email.unc.edu and edited by: Anna Goslen
Electronic theses and dissertations (ETDs) often have specialized workflows and metadata management requirements due to factors such as varied ingest methods and inconsistent source metadata. This blog post features descriptions of ETD metadata management from three institutions in the United States, outlining the systems, tools, and workflows used to process and provide access to ETDs at each institution. This post is intended to help other metadata professionals working on ETD metadata management by providing brief descriptions of how the work is handled at other institutions. A list of additional resources on ETD metadata management is provided at the end of the post.
Name: Stasha Gardasevic
Organization: University of Hawaii at Manoa
Once a semester, we get a ProQuest dataset of ETDs to ingest into the Institutional Repository.
Our graduate division uses ProQuest ETD Administrator, where the students log in and upload their submissions. These submissions are then verified/approved by the graduate division.
Three months later, ProQuest sends us a batch of zip files that includes metadata in XML format together with all the files — one zip file per submission. They upload the batch to our file server, with our permission. Our repository admin converts their zipped files and metadata into a format we can upload to our DSpace instance (called ScholarSpace).
There are several things to look out for when importing, as minor corrections might be necessary. For example, student-supplied indicators might be wrong, e.g., D.P.H. (Doctor of Public Health) when it’s actually a Ph.D., or M.A. instead of an M.S. The DSpace collections are determined by those fields, so they need to be corrected; the error will appear when our script can’t find the collection. Submitters can apply an embargo of 6, 12, or 24 months. Those are noted in the metadata and applied when the file is uploaded. Some degrees opt for a permanent suppression, in which case we get only the opening page and the intro.
Submitters can also opt out of “3rd party searches” on the ProQuest form. Since all of our files are crawled by Google, we handle this by putting them behind authentication.
Prior to ingestion, the metadata librarian checks files to make sure the:
Titles are in sentence case.
Personal Names are in preferred format (Last name, First).
Supervisors’ name entries per batch are normalized; supervisor names may occur in strange shapes, and occasionally ETDs authors will use their name for supervisor as well.
Data in other fields appears as expected.
Recently, we have been testing an automated way to make these modifications using Claude Pro. We are working on integrating about 8,000 ETD records that are currently not in our database but are available via ProQuest and were downloaded via TDM Studio (trial). Finally, we are developing an ETDs Trends Exploration dashboard that works on the enriched metadata for previewing trends in topics, supervision, and other stats.
Name: Xiaoli Ma
Organization: University of Florida
At the University of Florida (UF), Electronic Theses and Dissertations (ETDs) are integrated into the Institutional Repository, one collection of the UF Digital Collections. The workflow begins with the Graduate School sending raw XML dumps to Libraries IT, where they are transformed into MARC XML format and uploaded into Alma. Following this, staff in Metadata for Digital Initiatives extract the information from Alma into Excel using the Alma lookup tool, structuring and preparing the data for ingestion into the UF Digital Collections and subsequent distribution to ProQuest. This is the general picture.
Challenges lie in this general picture. First, the locally developed tool that converts Graduate School XML files into MARC XML has problems with field mapping, data formatting, and boilerplate text. Secondly, we receive additional batches of theses, dissertations, project outcomes, and student write-ups outside of the Graduate School pipeline. Arriving throughout the year, these varied submissions require separate preparation workflows for ingestion. Although a single unified metadata standard would be ideal given the similarity of the content, the different intake routes result in varied metadata standards across data batches in practice.
While understanding the benefits of using a single unified metadata standard, we have a hard time moving in that direction.
Our outdated digital platform presents major obstacles to implementing it:
Lack of batch update capabilities:The system lacks an effective, built-in batch update function. Adopting a unified standard would require updating all legacy content, making this an overwhelming task without sufficient support from the system.
Technical and testing hurdles: Even if the new standard were applied solely to incoming content, Libraries’ IT would need to modify code and conduct multiple rounds of testing to ensure the digital platform takes in the XML files.
Insufficient documentation and unresolved errors: This process is further complicated by a lack of documentation regarding the platform’s validation rules, leaving us without clear guidance on what triggers errors. Consequently, many validation errors encountered over the years remain unresolved.
UF plans to transition to a new digital collections platform next year, which we hope will enable us to implement unified metadata standards across all ETD batches.
Name: Anna Goslen
Organization: University of North Carolina at Chapel Hill
Like many institutions, the University Library at UNC Chapel Hill receives a batch of ETDs from ProQuest once a semester which are uploaded to an FTP server. Our institutional repository, the Carolina Digital Repository (CDR), uses the Samvera Hyrax system. When we began using Hyrax in 2019, ProQuest metadata was mapped to our local metadata profile for ETDs and a script was written to transform metadata to the required format. We added additional functionality to our administrative dashboard that allows repository managers to ingest packages with an ‘Ingest from FTP’ button. Part of the metadata transformation process involves mapping student affiliations to our local controlled vocabulary for university departments. Periodically, we are required to adjust the mappings or controlled vocabulary to account for new or renamed departments at the university. After the repository manager ingests new batches of ETDs, the metadata librarian is notified and performs quality assurance to check for any issues with the source metadata or transformation process. The access settings selected by the student are also automatically applied during the ingest process. Our collection of ETDs is dated from approximately 2006 to present; occasionally, pre-2006 ETDs are digitized and made available if proper permission and approvals are secured. ETDs are additionally discoverable via our catalog through an ETL process which transforms CDR metadata to the format used by our discovery layer. Recently, we have been in discussion with the Graduate School at UNC about the approaching ADA Title II accessibility deadline in order to implement accessibility policies and workflows into the ETD submission process.
Additional resources:
Flynn, E. A., & Ahrberg, J. H. (2020). Electronic Theses and Dissertations (ETDs) Metadata Policies, Workflows, and Practices: A Survey of the ETD Metadata Lifecycle at United States Academic Institutions. Journal of Library Metadata, 20(2–3), 91–110. https://doi.org/10.1080/19386389.2020.1780689
Hollingsworth, C. (2025). Creating New Access Points for Print Theses and Dissertations. Repository@TWU. https://hdl.handle.net/11274/17287
Lake, S., & Nicholson, J. (2023). Remediation by Degrees: Enhancing ETD Metadata to Improve Discoverability. North Carolina Libraries, 81(1), 13. https://doi.org/10.3776/ncl.v81i1.5426
Thompson, S., Liu, X., Duran, A., & Washington, A. M. (2019). A Case Study of ETD Metadata Remediation at the University of Houston Libraries. Library Resources & Technical Services, 63(1). https://journals.ala.org/index.php/lrts/article/view/6764/9320
tl;dr btrix is a
bring-your-own-model chat interface for web archiving, which I demo
below with my shiny new inference.coop account
I’ve been experimenting
recently with a chat interface to the browsertrix-crawler
software. If you haven’t seen it before, browsertrix-crawler is an
excellent tool for creating high-fidelity archival snapshots of web
pages and websites. It has many, many options, and the browsertrix service provides a full
user interface to managing one or many browsertrix-crawler jobs.
So why would you ever want to operate browsertrix-crawler with a chat
interface?
I guess the main reason is that you may feel like its overkill to bring
up and run a browsertrix instance for the type of job you are
doing. Another is that perhaps you would like to explore the options for
archiving a website, including developing a custom
behavior for clicking on parts of the pages. Its possible that the
chat interface provides a more iterative way to discover the many
affordances that browsertrix-crawler makes available for web archiving,
in the context of actual web archiving work.
Ok, and truth be told, it was an opportunity to experiment with how well
this approach could work. I think the answer is mostly, yes.
The seed for this idea was a set of simple shell scripts, dubbed browsertricky, that I’d
created a few years ago for running browsertrix-crawler jobs. It helped
me keep around the YAML configuration files in a single location for
reuse, and took some of the guesswork out of remembering how to
configure browsertrix-crawler, and monitor its progress. This sort of
interaction on the command line was something I was interested in seeing
in the chat style interface.
I initially developed what became btrix as a Claude plugin to help
explore how to control browsertrix-crawler with Claude Code. But after
getting it mostly “working” I wasn’t really happy with the broad
permissions it needed to be granted to operate and introspect on
browsertrix-crawler. I found that Claude improvised too much, sometimes
reaching to write and run its own programs to work with
browsertrix-crawler, instead of using the skills and scripts I had
explicitly built for it to use. I also didn’t really like the lock-in to
Anthropic, or any single LLM model.
So, long story short, I ended up developing btrix using the pi framework, which lets you connect to a
large number of backend models. In the demo below I use
deepseek-v4-1-flash via Inference Cooperative, which is Nathan Schneider’s latest joint
(more about this in the future). pi let me write tools for the
framework to call out to. Tools have known programmatic interfaces that
can be tested, and pi can lock down other types of interaction.
So how does it work?
Below is a short-ish demonstration of using btrix to archive a small
part of the news.stanford.edu website. It shows how btrix is able to
write a custom behavior based on my description. btrix writes the
JavaScript behavior, the YAML configuration, and starts the crawl. It
provides diagnostic information about the crawl as it progresses. It
also shows a quick look at the artifacts it leaves on the file-system,
including the WACZ archive, which it dutifully plays back.
As you will see, the session wasn’t without its hiccups because rather
than using a big corporate LLM I chose to use my new Inference
Cooperative account, which is really brand new, and seemed to encounter
a brief outtage after the crawl completed. But it recovered!
In case you missed it, here’s what btrix wrote at the end:
The archive is being served now — open this in your browser:
http://127.0.0.1:8087/?source=stanford-news.wacz
No notes came back with it, so there's nothing to warn about; it's served from
the same origin as the archive, so replay needs no network access.
What you should see: the listing page, and the paging buttons should still work
— clicking 2 and 3 replays the archived API responses rather than hitting the
live site. The 30 stories are all in the WACZ as their own pages, so full-text
search in the replay UI should find them.
For reference, the behavior did what it was supposed to, per the crawl log:
- page 1 — 10 queued
- page 2 — 20 queued
- page 3 — 30 queued
All 31 pages returned HTTP 200, median page text ~11.9k characters. Worth
a look while you're in there: the numbered buttons go up to 5, and I only
clicked through to page 3 as you asked. If you'd like pages 4 and 5 too, it's
a one-line change to the behavior and a re-run.
One of the handy things that btrix does is embed its own replay server
which uses replayweb.page to let you inspect the WACZ file you created
with ReplayWeb.page. If you get a
chance to give it a try please let me
know how it works for you.
Lately I've been playing classic World of Warcraft to get a feel for it and see if I like it before I try and go head-first into WoW Forever fairly blind with some friends. People have told me that World of Warcraft is good, but I haven't personally played it (or really any MMO outside of Final Fantasy 14). I've wanted to see for myself, so I dug in.
I've been leveling an undead warlock and recently I've gotten my way to The Barrens. My goal was to kill some planestriders and collect their beaks as proof of my kills so that the population can get back under control or whatever. I wandered around and found another player also on that quest and waved to them. He was a gnome warlock named Borton and waved at me when I walked over. I waved back and invited him to a party. We walked around the area and started spreading our aggro to farm the planestriders and grab their drops. After a few kills he stopped for a second and said "this is why we play".
I was taken a bit aback by this. It felt out of nowhere but also felt like one of the more human encounters I've had with people in a video game in a long time. One of the great parts about MMOs is that you end up having a lot of these itinerant bits of community like this. You both get a quest from an NPC asking you to go and farm some drops and then this sends you all out into the world to naturally do it together. These conversations and liminal bits of community end up adding up into the greater experiences and really giving you the flavour of the game's vibe. You do it together for the fun of the game.
After a while we finished our farming, got all the planestrider beaks, commiserated about the drop tables, and decided to just sit down on a nearby hill and take a screenshot because it just felt right.
Two players sitting in golden fields with a backdrop of mountains and a blue sky enjoying the moment together wordlessly.
This has really gotten me thinking about what I really want to get out of games and why I play them. I think I've been doing it wrong.
The endless treadmill of “dead content”
At some level the “high end” content of online games like Final Fantasy 14 or Retail World of Warcraft looks endless. There’s intricate dances of choreography as you and your friends puzzle your way through each boss mechanic towards victory. Hell, games like Final Fantasy 14 have their fights defined by the little call->response pairs as the boss telegraphs things and you react to it in concert. Not to mention the feeling of utter serotonin that comes from figuring it out and winning.
One of the main problems with this is that the “high end” raids in this game tend to be a mile wide in theory but practically an inch deep. Don’t get me wrong, even as far back as a decade ago there’s still encounters that are actually hard (eg: XIV’s ultimate raids, I'm personally at Quickmarch Trio in UCOB if you have the right brainworms to know what that means), but as the fights get older there’s a sharp dropoff in the number of people that actually want to run them. This ends up with entire subsystems of the game (Deep Dungeons, Bozja before Shadowbringers went into the free trial, etc) becoming “dead content” because if you queue for them, nobody shows up. If you put up an entry for them in Party Finder, nobody will join you unless you set aside time to wait for hours. You end up with modes where you're encouraged to do it in a group being bereft of people so you're "forced" to figure it out on your own.
As a result everyone is funneled into the high end and trying to min-max it as much as possible to squeeze all the game they can out as fast as they can before “everyone else” gets what they want out of it and moves on. The world becomes a theme park where only the newest rides have the most people there and getting a group for anything else becomes a slog in party finder or needing you to seek out people that focus on that thing and nothing else.
This also means that if you’re “slow” or have to take a break during the small sliver when content is “current”, you’re going to be stuck "behind" on it for a long time without significant effort to find a way around it. It really makes you wonder what the point of it all is if you can’t find anyone to do it. Worse, when you're trying to find a group to do it regularly (a "static" group), they look at how you did on previous raid tiers as a way to vet if you're a "shitter" or not.
The records of your ultimate achievements of an entire raid tier, or the relic weapons for an entire expansion just end up becoming things you can do if you want that give you a shiny skin for your weapons to coordinate with an outfit that looks better. Maybe in edge cases those stats will be better for complicated maths reasons. All that effort you put into them is just ultimately "wasted" when the next ride comes out, all the stats get reset, and everyone moves on to wait in line for the next thing. All of your gear you spent weeks grinding for is reduced to sticks that increase the statistics correlated to aspects of your character that make the funny numbers go up so you can kill the boss faster to move on to the next thing.
Maybe this is just something endemic to the theme park design philosophy where you’re given a goal by the developers and a series of actions that really boil down to TODO items that are the barrier from you and that thing you actually want (do these quests, fight these normal raid bosses, beat the savage versions of those bosses, beat the ultimate boss, get the shiny weapons, and finally look cool afk-ing in Limsa Lominsa). Maybe this is why people end up seeing things like the overworld as the barrier to the rides instead of an integral part of the experience. Maybe this is the pressure that makes people want to have flying mounts or other ways to "skip the grind".
A meditation on Death Stranding
I haven't really touched Death Stranding since I did my review on it way back in 2019. It's always had a special place in my heart though, it's kind of a touching experience even though it's incredibly isolating at the same time. This game has two main characters: Sam "Porter" Bridges and the game world. The overworld's system of traversal is the protagonist as much as the actual protagonist is.
Sam "Porter" Bridges looking out past the hill he just climbed and seeing an upside-down rainbow.
God sent the rainbow as a promise to mankind that he wouldn't try to destroy them again, and this rainbow is upside down after mankind's near destruction.
Honestly I think one of the best features of Death Stranding is the camera mode. At any point you can hit a button (on the Steam Controller it's one of the back buttons by default) and the action stops so you can position the camera and just soak in the scenery. If you've never played Death Stranding before, I highly suggest you stop reading this post right now and buy it on whatever platform you buy games on. Worst case you can experience it on the iPhone as Hideo Kojima surely intended.
If you actually do this on an iPhone, please do it with a controller. The
touch controls are kinda awful.
As you progress your way delivering packages to save America, every so often the camera will pull back, the game actions will become muted, and the normally contemplative and silent experience will suddenly be augmented by soft music.
The protagonist of Death Stranding riding on a tricycle down to a port city with the Death Stranding logo overlaid.
It turns this process of progression into a serene bit of negative space in a way that makes you really stop and think about the experience, the dramatis personae, and the kind of story that's trying to be conveyed. The world itself turns into this sense of wanderlust that makes you mentally pause and just process things.
I don't really know if I'm doing this justice, in the moment it's an almost spiritual kind of connection between you and the overworld mediated by the soft sounds of Silent Poets' Asylum For The Feeling or Low Roar's Don't Be So Serious. The game makes you feel so small yet so connected to the world around you and there's just this overwhelming sense of equanimity.
Just take a moment to watch the Don't Be So Serious video with Death Stranding gameplay underlaid. That's a snapshot of how the game literally plays out. It's these liminal bits of wanderlust combined with a story that only Kojima or Yoko Taro could come up with.
I miss the feeling of exploring
This kind of overwhelming emotion is the kind of thing that made me fall in love with gaming back in the days when the most complicated games I'd play would be Banjo Kazooie, Super Mario 64, Super Mario Sunshine, Pikmin 2, Sonic Adventure 2, and other games of that vintage. At some level I kind of long for not knowing what's beyond the next hill and being forced to figure it out by just going there to find out.
Hell, I also get this sense of romance in Breath of the Wild or Tears of the Kingdom when I just turn off all the UI elements and wander around wherever the wind takes me.
Link in Breath of the Wild looking towards a sunset bathing a lake in twilight as the light refracts on the smooth stone surface he's standing on.
Breath of the Wild is one of those games where you can just wander around the overworld for ages and still find new things. Combine it with flying and you just feel things as you go from hither to thither. Even with fast travel you still have to go there yourself. You have to climb every mountain. You have to cross the vast expanse of the desert. You have to figure out how to get a boat to the island with some awesome gear. And even months or years after you capture screenshots/video of the adventure, looking at them takes you back there and you just feel it again.
You have the power and you have to pick what you do with it. There is no wrong way to do it. Want to fly over Gerudo Town after sunset knowing well and good that the moment you land you're gonna get kicked out of it for being a guy? Hell yeah, you can just DO IT.
Link in Breath of the Wild gliding above Gerudo Town at sunset, just soaking in the vibes.
This is making me think, fuck, am I actually getting what I want out of my time? In my race to min-max my way through "current" things so I don't get "behind" have I just lost that sense of wanderlust from the experience? I miss that feeling of just exploring the world for what it is and letting myself get there the long way.
Have I been skipping over the real "good part" in my effort to be efficient in getting to what I thought the "good part" really was?
Have I just been playing games the wrong way?
This is why we play
I think I've been playing the games I'm immersing myself in incorrectly and I want to change that. I want to lose myself in the fantasy worlds. I want to let myself see the sights, hear the sounds, feel the breeze on my back, go fishing at what feels like the little corner of civilization just at the edge of the universe, and just let that sense of overwhelming wanderlust come back into my life.
This is why we play, we play to let the fantasies of these fantasy worlds take charge and show you something that you didn't think you needed in your life yet you're all the richer for it. As World of Warcraft Forever comes out in November and Final Fantasy 14's Evercold expansion comes out next year, I'm going to just let those worlds enrapture me in their designs and lose myself to the wanderlust. I'll let The Fourth take me where it does and that's okay.
It's absurd that the root of this revelation was a bot in a single-player MMO on my homelab picking a random chat message out of a list of dozens randomly triggered by a random overworld mob dying.
Yeah, I forgot to mention, this all was caused by an experiment in playing with bots to teach me what it means to have a community. I was gonna lead this post with that fact but I think it all lands better if I just spill it all out at the end while I'm having the revelations like this. Given the utter absurdities of this modern era where we live in Clown World, I think it only makes sense that it would boil down to something like that.
Honestly I think it's even more absurd that it took a proc of a proc of a bot saying things for me to figure out what was wrong with myself.
I feel like I've just torn out a chunk of my soul and rended it unto the canvas for this. I don't really know why this helped me, but I hope this helps you too. Stay fresh out there, and if you're going to also venture forth to that millennial retirement home of a launch, maybe I'll see you in Azeroth in early November for World of Warcraft Forever. I'm gonna go in blind, you should too. We'll figure it all out together, one terrible loot table at a time.
The guest editors for this special issue showcasing examples and ideas for integrating GenAI tools into galleries, libraries, archives, and museums generative AI summarize the articles included in it.
“Prompting Critical Thinkers” follows one librarian’s evolution of a library lesson on generative AI tools into an information literacy lesson grounded in the ACRL Framework for Information Literacy for Higher Education. The revised lesson focused on generative AI as an information source and blended definitions of AI literacy from academia and industry. This case study demonstrates how growing interest in AI skills can give librarians new opportunities to get into the classroom and teach students enduring information literacy skills.
Large-scale historical corpora present persistent challenges for research, particularly in relation to the navigation, interpretation, and extraction of meaningful information across extensive and heterogeneous collections. The Legislación Mexicana corpus, a 42-volume compilation of legal dispositions spanning more than two centuries, exemplifies these challenges due to its size, structural complexity, and evolving terminology. Traditional approaches to working with such corpora have relied on close reading and the use of indexes. While effective within defined limits, these methods constrain the scope of inquiry and require significant time and expertise to produce meaningful results. This article presents the development of LegMexIA, an AI-enabled research assistant designed to extend interaction with the corpus beyond conventional retrieval methods. The system integrates traditional text retrieval and retrieval-augmented generation via a coordinated architecture that distinguishes between different types of user queries. Structured queries are processed using Elasticsearch and BM25 ranking, while exploratory and interpretive queries are routed through a retrieval-augmented generation workflow, where relevant text fragments are retrieved using vector similarity and assembled into contextual inputs for a language model. A key aspect of the system is the agentic decision layer that analyzes user intent and dynamically selects the appropriate processing strategy, in alignment with established reference practices in academic libraries and in a way that maintains the user’s role in verification. The article also addresses key considerations related to prompt design, technological sustainability, institutional constraints, and the implications of public deployment.
This paper reviews Project Gutenberg’s (PG) use of artificial intelligence (AI) with the free books in its digital library. The paper covers three main projects where PG has integrated AI into its workflow. The three areas discussed are disparate in nature and include audiobook, book summary, and browsing category generation. The paper reviews the AI models and processes used, as well as how volunteers and users have responded to the organization’s use of AI. PG’s dataset is extremely large and includes more than 75,000 books. Large datasets can be challenging and costly for AI models to process. The main goal of this paper is to share how PG has incorporated AI processes into the workflow of a very large digital library while maintaining human oversight.
Many heritage institutions hold extensive collections of show programs. Too numerous to be cataloged individually, they remain difficult to access and are still largely undigitized. This paper presents a workflow for transforming such documents into structured, interoperable data by combining vision-language models (VLMs), a custom extension of the Linked Art ontology for the performing arts, and different approaches for automatic semantic data annotation. Using the Festival d’Avignon programs (1947–present), preserved at the Bibliothèque nationale de France, as a case study, we demonstrate how VLMs achieve more than 98% text transcription accuracy from heterogeneous heritage document images and how constraining large language model generation through a knowledge graph significantly reduces hallucinations in semantic annotation. We argue that this workflow opens new possibilities for large-scale, interoperable performing arts historiography, including the creation of catalogues raisonnés for performing artists, a form of scholarship common in visual arts but absent in performing arts studies.
This paper explores the emergence of artificial research intelligence (ARI), autonomous AI systems designed to conduct end-to-end scientific research, from ideation to manuscript production. Unlike simple AI enhancements, ARI utilizes ensembles of agentic models to automate the scientific method, potentially creating a “fifth scientific research paradigm.” While ARI offers rapid, low-cost discovery, it faces significant challenges including “epistemic capture,” where knowledge production shifts from public universities to private AI corporations. Technically, these systems currently struggle with hallucinations, a lack of deep domain expertise, and a tendency to overclaim results. However, the growing number of ARI frameworks points to an emerging research methodology of transformational possibilities. For research libraries, ARI necessitates a profound shift from managing static collections to stewarding dynamic, AI-driven knowledge systems. Libraries can lead by reimagining processes and services supporting scholarly communications, enhancing AI competencies, and conducting applied research through library-based AI labs. The combination of advanced compute and the guiding framework of the scientific method makes ARI a human–AI co-creation model where AI performs the "reckoning" (computation) and humans provide the "judgment" (ethical and intellectual oversight).
AI Benchmarking Exercise home page.
Agreement, cost, and time graphs for the models’ runs.
Descriptive statistics about the models’ runs.
Screenshot of the first alpha of what we initially called SmartFox and then Firefox Scholar, before we consulted 1) a lawyer, and 2) an Albanian dictionary for other options, June 2006
Members of the Zotero team in late 2006. Roy Rosenzweig, months into chemotherapy, is kneeling at center; he had just gotten the Zotero license plate for his Prius to the great delight of all of us. Standing, L-R: Sean Takats, Trevor Owens, Josh Greenberg, me; Kneeling: Kari Kraus, Roy. Photo credit: Sharon Leon
Be forewarned, ye who dare log into my web application c. 2002 (screenshot of the Web Scrapbook landing page)
Elena Razlogova’s Scribe application, c. 2002
SmartFox version 0.0.1 screenshot, July 27, 2005
Members of the Zotero team in late 2006. Standing, L-R: Simon Kornblith, Sean Takats, Trevor Owens, Dan Stillman, Matthew Eaton; Kneeling: me, Roy Rosenzweig, Josh Greenberg
The first logotype for Zotero, August 29, 2006
Zotero 10.0, released on August 17, 2026
︎
The catalog of yesteryear. Source: 
Usage statistics for the 
Catbriar and huckleberry in the author’s backyard. Photograph by Nick Anderson.
![[object Object]](https://files.xeiaso.net/blog/2026/this-is-why-we-play/gods-promise-broken.jpg)
![[object Object]](https://files.xeiaso.net/blog/2026/this-is-why-we-play/low-roar.jpg)
![[object Object]](https://files.xeiaso.net/blog/2026/this-is-why-we-play/lanayru.jpg)
![[object Object]](https://files.xeiaso.net/blog/2026/this-is-why-we-play/gerudo-town.jpg)
