Win free books from the April 2025 batch of Early Reviewer titles! We’ve got 187 books this month, and a grand total of 3,309 copies to give out. Which books are you hoping to snag this month? Come tell us on Talk.

If you haven’t already, sign up for Early Reviewers. If you’ve already signed up, please check your mailing/email address and make sure they’re correct.

» Request books here!

The deadline to request a copy is Friday, April 25th at 6PM EDT.

Eligibility: Publishers do things country-by-country. This month we have publishers who can send books to Canada, the US, the UK, Australia, New Zealand, Spain, Peru, Netherlands, Ireland, Germany and more. Make sure to check the message on each book to see if it can be sent to your country.

Thanks to all the publishers participating this month!

This text, part of the #ODDStories series, tells a story of Open Data Day‘s grassroots impact directly from the community’s voices.

Together with the local community, bring the ODD spirit through this activity

Pari Island, a gem in the Thousand Islands Regency, lies 232 kilometers from Jakarta. Renowned for its pristine beaches, vibrant coral reefs, and lush mangrove forests, Pari Island boasts incredible coastal and marine potential. However, beneath this natural beauty lies a heartbreaking reality: corporate interests, cloaked under the guise of tourism privatization, threaten the island’s delicate ecosystems.

Beach reclamation on Tengah Island for private tourism development. Credit: Yunsan.E

Unmasking Environmental Injustice

Large-scale corporations have continually encroached on Pari Island, engaging in land reclamation and sea dredging around Tengah Island and Kudus Island, two islands within the Pari Island cluster. These activities have resulted in the destruction of mangroves planted by local communities and the decimation of coral reef zones. The consequences of these actions are dire risking ecological disasters and threatening marine biodiversity.

In response, the local community of Pari Island has risen to demand climate justice, holding corporations accountable and advocating for the cessation of these environmentally destructive practices. Leading this grassroots movement are several local communities Forum Pulau Pari and Perempuan Pulau Pari (Women Group). Their fight for justice is a testament to the islanders’ resilience and deep connection to their environment.

Women Group of Pari Island

Citizen Science for Coral Conservation

In celebration of Open Data Day, we joined the local community in investigating the impacts of reclamation on coral reefs using publicly accessible remote sensing data. Through a hands-on demonstration of open tools rooted in citizen science, we employed Coral Watch method that simplifies coral health monitoring. Our activities extended beyond data collection. We brought environmental education to the island’s youth through storytelling and coloring competitions, instilling a sense of environmental stewardship in the next generation.

Preparing the tools that will be used for the Coral survey

Alarming Findings: Sediment, Bleaching, and Urchins

The Coral Watch observations revealed troubling results. Coral reefs near the reclamation sites were smothered by sediment, a consequence of dredging and land expansion. In some areas, corals showed signs of bleaching, particularly along the reef’s outer edges. However, not all findings were grim other stations exhibited healthy coral populations, offering a glimmer of hope.

One peculiar discovery emerged at a site favored by local fishers for snorkeling: a high concentration of sea urchins (Echinoidea). While sea urchins are natural reef inhabitants, excessive populations can hinder coral growth and disrupt reef balance. Such ecological shifts warrant further investigation to understand the long-term impacts.

1 – Two people take reef samples using snorkeling method and bring coral watch chart. One person stays on the boat to record coordinates and secure data sheet.
2 – This coral plate is in a state of concern because it is being hit by reclamation sedimentation.
3 – It is not uncommon to find coral that has shown signs of bleaching.
4 – Healthy corals have strong color pigments.

Empowering Communities with Data

For Pari Island’s residents, these findings are more than data points; they serve as crucial evidence in their fight for climate justice. The observations gathered through citizen science empower the community, providing a scientific basis to advocate for the protection of their natural heritage.

Storytelling about marine life with children

The battle for Pari Island is far from over. Yet, through collective action and open data initiatives, local voices grow louder, demanding accountability and sustainable practices. This island, with its rich marine biodiversity and resilient community, stands as a symbol of hope and determination in the face of environmental injustice.

Images of Coralwatch Chart and tools used for survey

Image of community-led coral monitoring using Coral Watch, capturing local participation.

About Open Data Day

Open Data Day (ODD) is an annual celebration of open data all over the world. Groups from many countries create local events on the day where they will use open data in their communities. ODD is led by the Open Knowledge Foundation (OKFN) and the Open Knowledge Network.

As a way to increase the representation of different cultures, since 2023 we offer the opportunity for organisations to host an Open Data Day event on the best date over one week. In 2025, a total of 189 events happened all over the world between March 1st and 7th, in 57 countries using 15+ different languages. All outputs are open for everyone to use and re-use.

For more information, you can reach out to the Open Knowledge Foundation team by emailing opendataday@okfn.org. You can also join the Open Data Day Google Group or join the Open Data Day Slack channel to ask for advice, share tips and get connected with others.

The following post is one in a regular series on issues of Inclusion, Diversity, Equity, and Accessibility, compiled by a team of OCLC contributors.

^{Photo by Faris Mohammed on Unsplash}

Building spaces of solidarity and connection

The free virtual summit open to all that is being sponsored by ALA’s Social Responsibilities Round Table (SSRT) “Building Spaces of Solidarity and Connection: A Historical, Theoretical, and Practical Lens on Libraries’ Commitment to Inclusion, Diversity, Equity, and Justice” will offer five sessions over the course of Monday, 7 April 2025. The sessions are entitled as follows: “Cultural Partnerships and Programming to Support Inclusive Communities;” “Toward Equity and Inclusion in Library Services and Professions;” “How Can Libraries Support Immigrant and Refugee Communities?;” “Institutional Disregard, Denial, and Disfranchisement: Treatment and Experiences of African Americans in LIS Professions and Practices;” and “Issues in School Librarianship, Children’s Literature, and Youth Services.” One must register separately for each individual session and each session will be recorded.

Speakers from across the United States will share how they and their institutions (including public libraries, academic libraries, and library-adjacent entities) have dealt compassionately with communities who have been marginalized and otherwise made vulnerable. Programming suggestions, tools to help provide more equitable services, and plans to develop inclusive spaces will be on offer. Full details on the schedule and each speaker can be found at https://www.ala.org/srrt/buildingspaces. Contributed by Jay Weitz.

Recommendations for gender identity in archival descriptions

The Yale Reparative Archival Description (RAD) Working Group has released their recommendations for addressing gender identity in archival description. This is the most recent addition to a set of guidelines that have recommendations for identifying how reparative descriptive practices have been applied, documentation for a project to identify full names for women previously referred to only by their husband’s names, and guidelines for addressing materials related to Japanese American incarceration during World War II.

I appreciate the work that the RAD Working Group has done over time and the guidelines for gender identity in archival descriptions are a great addition. The guidelines pertain not only to those engaging in description of collections but also those working in acquisitions, collection development and even donor relations. For collections acquired in the past, it can be difficult to know with certainty how a given person identified but for collections that are acquired now and in the future can be respectful of preferred identity, which may be nuanced. The work of the RAD Working Group, which has been ongoing since 2019, reflects how this work can take time and focused attention. Contributed by Merrilee Proffitt. 

Creating a psychologically safe environment with inclusivity

Academic librarians Amanda Clay Powers and Dustin Fife discuss how to create a psychologically safe team in the library workplace in the March 2025 issue of College & Research Libraries News. The article “Psychological Safety in Libraries: It’s a Team Sport” captures a conversation in which the librarians share their insights about what works. Powers explains the importance of inclusivity saying, “Part of this is the hard work of making diverse teams safe. For me, that has meant all library retreats around microaggressions, anti-racism, LGBTQIA+ inclusion, accessibility in libraries, and intersectional identities.” Fife recommends considering co-workers with curiosity rather than criticism, writing, “How we do things is rarely as important as why we do things together and curiosity will bolster that connectivity.”

This article is an important reminder for me of why institutions undertake DEI initiatives. Employees feel safer when they are accepted and understood by their peers and supervisors, creating a more positive and productive workplace culture. When people feel unsafe, there is an automatic physiological reaction called the “fight or flight response” that in the short term will cause people to be unable to focus on their work and in the long term can cause burnout and quitting. Creating a psychologically safe workspace is good for individuals and organizations. Contributed by Kate James.

The post Advancing IDEAs: Inclusion, Diversity, Equity, Accessibility, 1 April 2025 appeared first on Hanging Together.

A monthly round-up of news, upcoming working group meetings and events, and CLIR program updates from the Digital Library Federation. See all past Digests here. 

Hello and happy April, DLF community! It’s officially spring, and our community is staying quite busy with Working Group meetings and events. We’re particularly excited to share that the DLF Forum Call for Proposals is now open through the middle of April. Read on to learn how to be a part of all the great things happening in our world this month. We’ll see you around soon!

— Team DLF

This month’s news:

• Submit: The Call for Proposals is now open for the DLF Forum and Learn@DLF, to be held in Denver, Colorado, November 16-19. Learn more and submit by April 14.
• Program Available: The preliminary program is now available for the 2025 IIIF Annual Conference in Leeds, June 2-6. View the program and register for the event.
• Attend: Association of College and Research Libraries (ACRL) Conference, April 2-5, 2025 in Minneapolis. Learn more about it here.
• Participate: DLF’s Born Digital Access Working Group (BDAWG) invites participation in their AI for Access survey, which investigates how archivists are currently using or not using artificial intelligence to facilitate access to born-digital archival material. Learn more and participate by April 14.
• Attend: The Big Data and the Preservation Problem Webinar, part of CLIR’s Climate Resiliency Action Series, is happening Wednesday, April 9, 12pm ET/9am PT. Read on for more information about the series and this workshop.
• Discuss: After attending the webinar, join Climate Circle Discussion #6: Big Data and the Preservation Problem, on Friday, April 18, 1pm ET/ 10am PT
• Catch Up: Past Climate Action Webinars are available on the CLIR/DLF YouTube channel or the project website’s Resource page. 

This month’s open DLF meetings:

For the most up-to-date schedule of DLF group meetings and events (plus NDSA meetings, conferences, and more), bookmark the DLF Community Calendar. Meeting dates are subject to change. Can’t find meeting call-in information? Email us at info@diglib.org. Reminder: Team DLF’s working days are Monday through Thursday.

• DLF Born-Digital Access Working Group (BDAWG): Tuesday, 4/1, 2pm ET / 11am PT
• DLF Digital Accessibility Working Group (DAWG): Wednesday, 4/2, 2pm ET / 11am PT
• DLF AIG User Experience Working Group: Friday, 4/18, 11am ET / 8am PT
• DLF AIG Metadata Assessment Working Group: Thursday, 4/24, 1:15pm ET / 10:15am PT
• DLF Digital Accessibility Policy & Workflows Subgroup: Friday, 4/25, 1pm ET / 10am PT
• DLF Digital Accessibility IT & Development Group: Monday, 4/28, 1:15pm ET / 10:15am PT
• DLF Digitization Interest Group: Monday, 4/28, 2pm ET / 11am PT
• DLF Committee for Equity & Inclusion:  Monday, 4/28, 3pm ET / 12pm PT

DLF groups are open to ALL, regardless of whether or not you’re affiliated with a DLF member organization. Learn more about our working groups on our website. Interested in scheduling an upcoming working group call or reviving a past group? Check out the DLF Organizer’s Toolkit. As always, feel free to get in touch at info@diglib.org. 

Get Involved / Connect with Us

Below are some ways to stay connected with us and the digital library community: 

• Subscribe to the DLF Forum newsletter.
• Join, start, or revive a working group and browse their work on the DLF Wiki.
• Subscribe to our community listserv, DLF-Announce.
• Bookmark our Community Calendar.
• Learn more about becoming a DLF member organization. 
• Follow us on Instagram, Facebook, LinkedIn, YouTube, and X. 
• Contact us at info@diglib.org.

The post DLF Digest: April 2025 appeared first on DLF.

This post is part of a growing series on the Library Beyond the Library. 

^{Photo by Quino Al on Unsplash}

Research institutions like those in the OCLC Research Library Partnership (RLP) are striving to meet growing demands for research support services that assist scholars and researchers throughout the research lifecycle. These services are distributed across a complex web of campus units at most institutions, creating an invisible barrier to access for researchers.

Montana State University (MSU) has addressed this challenge by establishing the Research Alliance: a centralized hub that co-locates several research support units within the main library, improving awareness, convenience, and usage by researchers. Additionally, co-location fosters improved knowledge sharing and collaboration among participating units.

The Research Alliance exemplifies how libraries are adapting to new operational structures and communicating an evolving value proposition. By hosting the Research Alliance, the MSU library is strategically positioned as a central research support hub for the entire campus, both physically and operationally. This initiative provides a robust framework for ongoing cross-unit engagement and partnership, increasing visibility and engagement with other units, and ensuring closer alignment with institutional priorities.

Operationalizing a cross-unit partnership is a multi-phase project

Creating this centralized research support hub required years of persistent and patient effort to socialize the idea, identify partners, and eventually secure institutional support. But making the institutional decision to establish the Research Alliance was just the first phase, which I roughly equate to the initiating or design phase in project management lingo. According to the Project Management Body of Knowledge (PMBOK)^[1] project management is described in five discrete phases:

• Initiating—defining a new project and obtaining authorization to begin
• Planning—processes to define objectives, scope, and next steps
• Executing—performing the work to implement the project
• Monitoring and controlling—tracking, reviewing, and initiating changes as needed
• Closing—processes to close out a project, if applicable

As any project manager knows, the work really begins with project approval, and this was certainly true for the Research Alliance.

I recently spoke with Doralyn Rossmann, Dean of the Library, and Jason Clark, Head of Research Analytics, Optimization, and Data Services (ROADS) at MSU. They expanded our understanding of the MSU Research Alliance story and offered a candid inside view of the significant challenges and frequently invisible library leadership presented by the project’s planning and execution stages.

Challenges

Turnover in executive leadership. The leaders of all of the original units – the library, the center for faculty excellence, and the vice president of research all left their positions after the agreement to form a partnership was made, but before any physical movement or logistics regarding shared spaces took place. And, while information technology’s research cyberinfrastructure was envisioned to have a role in the alliance, the chief information officer had been minimally involved in initial planning. In turn, subsequent leaders had to move a partially realized vision into being a reality.

Ill-defined scope and partners. When the Research Alliance concept was initially approved by the leaders of the involved units, it was still a loosely defined vision – even the name “Research Alliance” was not established. Details of the agreement were not captured in a Memorandum of Understanding (MOU) at this phase of the project, resulting in significant uncertainty for subsequent leaders. The office for undergraduate research entered the alliance at the eleventh hour as the reality of the alliance became more visible, creating needs for scoping expectations for subsequent growth.

Poorly defined leadership roles. Working primarily on a handshake agreement, there was no clear articulation of goals, roles, or responsibilities. No one was assigned responsibility to lead the effort, and the project advanced largely due to the adoption of new work responsibilities by library leadership. The library was the implied leader in large part because the Research Alliance was to be housed there —- this involved taking responsibility for architectural planning, remodeling, formulating an MOU, and convening ongoing committees of both executive sponsors and Alliance partners.

Internal pushback. Not everyone was excited about the idea of centralizing research support services in the library, and there was significant pushback from both students and library employees. Some students who protested the loss of study space even left hostile messages on a library whiteboard, visible to Research Alliance workers. Some library staff members who had little awareness about the strategic reasons for the partnership opposed the focus on a research support unit housing non-library personnel in what had been quiet library study spaces. Library leaders had to manage resistance, explain library and campus priorities, and provide sustained leadership on a project unpopular with some.

Benefits

Enhanced and visible library value proposition. As I’ve described in an earlier blog post, the Research Alliance positions the library as the physical and strategic center for research support on campus. Just as physical collections have long demonstrated the library’s contribution to supporting scholarship, today the Research Alliance offers a tangible, compelling story about how the library is evolving to support research and campus priorities in new ways. It demonstrates the library’s evolving role in support for research that may not be recognized by other campus stakeholders particularly those who have a role in decisions about library funding and autonomy. 

Increased interaction with other campus leaders. Implementing the multi-tenant Research Alliance has required increased interaction between the library dean and other campus executives. While the work of sustaining this group has fallen to the library dean, it offers a significant opportunity for her to demonstrate how the work of the library aligns closely with institutional priorities. Busy campus leaders may still adhere to a collections-focused view of the library, failing to understand the important roles that research libraries now play in research support and student success. Ongoing engagement with these leaders—on a project that supports high priority campus research productivity—provides an opportunity for the university librarian to help other stakeholders better understand the library’s value. Additionally, the Research Alliance has brought numerous faculty and administrators into the library who had not regularly entered the building in recent years. This has had the added benefit of these people seeing how incredibly busy the library is and has created increased interactions between library employees and people coming into use Research Alliance services.

Improved services for researchers and students. And, of course, there is the intended benefit of the Research Alliance—to reduce friction for researchers navigating campus services. More researchers come together in this shared environment and this results in improved referrals and collaborations between units. 

Recommendations to other libraries

Doralyn and Jason have gained valuable insights from their experience implementing the Research Alliance at MSU, including these recommendations that can help other libraries undertaking similar collaborative efforts:

Create decision support structures ASAP

Assemble executive sponsors and convene regularly. Executive sponsors play a crucial role in decision-making and removing project obstacles. They should meet regularly—at least quarterly—to receive updates, offer advice, and make joint decisions. Establishing their oversight role early in the project planning can significantly reduce project friction. Ideally, a member of this group would meet regularly with members of the alliance to communicate between the executives and members to understand needs and communicate priorities.

Establish an MOU. Drafting an MOU requires a thorough examination of roles, responsibilities, leadership, decision making authority, and partner expectations. It also involves defining the collaboration’s objectives, scope, and efforts, serving as an important project planning document. Finally, it should delineate equitable financial responsibilities; given the situation of the Research Alliance in the library, it was, de facto, responsible for space and furniture costs. This MOU should be revisited annually to accommodate needed adjustments.

Create goals and an assessment plan. Part of planning the partnership is also determining how team members will monitor and track progress. Ideally, the partnership has clearly established goals and all members of the partnership have annual goals that set expectations to participate in meeting the goals of the partnership. Assessment planning can be integrated into the MOU process and provide resources for measuring success, resourcing, and evaluating whether the partnership should continue.

Communicate library value

Prepare for internal pushback. Resistance is almost a certainty, particularly in university environments characterized by decentralized, independent agents who often work at cross-purposes and have the freedom to openly oppose institutional initiatives. Universities are “complex adaptive systems” that make collaboration and change particularly challenging.

Relentlessly communicate the project (and library) value proposition. Establishing a new offering like the Research Alliance provides a visible, ongoing way to sell the library’s value to a wide range of stakeholders. Leverage this in storytelling, personal narratives, and metrics that demonstrate the central role the library plays in supporting campus strategic priorities.

Ensure the library receives financial recognition. While the library reaps significant benefits by hosting a research support hub, it also incurs costs, including the reallocation of valuable campus space. Libraries should seek remuneration by leveraging funding from participating research partners and through facilities and administration (F&A) monies. This isn’t just about the budget—it’s also about other units recognizing and rewarding the library’s value.

Extending the library beyond the library

The MSU Research Alliance represents a shift in library operational structures, as libraries forge new partnerships with other campus units to support research and institutional priorities. OCLC Research describes this shift as extending “the library beyond the library.” A current research project examines increasing library interactions with the broader campus, and how library expertise and capacities are combined with those of other campus units. By documenting the transformation of collaborative operational structures and value propositions, this project will provide valuable insights and recommendations for library leaders navigating the expanding landscape of cross-campus partnerships.

Writing this blog post was a collaborative effort. Many thanks to Doralyn Rossmann and Jason Clark for their input and for inviting us to continue the conversation. Special thanks also to Merrilee Proffitt.

^[1] The PMBOK is maintained by the US-based Project Management Institute (PMI) professional organization which offers the Project Management Profession (PMP) certification.

The post Lessons learned from implementing a research support hub in the library appeared first on Hanging Together.

I don't know how many there were at the protest today in front of the Easton Tesla store, but for the first time we covered all four corners of the intersection.

I think there were at least 600 people...maybe more. Some observations:

1. The weather was good—windy, but warm—and the families with young children did come out again. But there were just MORE people there overall.
2. This week I recognized more cars making a circuit around the block. More people honking with thumbs up, turning around, then coming back again. I don't remember seeing that on past Saturdays.
3. There were more Tesla sedans driving by that I remember seeing in the past. Quite possibly, they were just making the circuit around the block, too.
4. I'm starting to recognize familiar faces at each protest.

There was no live music this time, but that was okay because there was definitely more noise from the sidewalks and more energy in the air. The Proud Boys made noise about coming in counter-protest, but I didn't see them. One of the event marshals said they were there early, but the police effectively separated them. As the panorama picture shows, though, we had all four corners covered and we were raucous.

This week's protest sign

This week's #TeslaTakedown protest sign.

I went off-script this week with a sign about the political nonsense we have at the moment. It is a play on the phrase "The call is coming from inside the house!" — a play on a famous movie trope where the police tell the person in a home that they have traced the antagonist's call to that home. In this case, the danger to democracy is coming from inside the Whitehouse! Or, at least, that is what I was aiming for. This sign probably only get's one week's use; let's hope by next week, one or more people on this sign are fired because of the released of what sure looks like classified information on a Signal group chat.

If you want to use this 26" by 16" sign for yourself, I've made it available for download. Ping me on Mastodon or Bluesky if you use it, and include a picture if you'd like!

My protest sign at the #TeslaTakedown.

Two years ago this month, I wrote a DLTJ Thursday Threads article on the copyright implications of foundational AI models. A lot has happened in those 24 months. This issue mostly focuses on lawsuits, plus an announcement of a service offering image generation from licensed content. These articles highlight the growing tension between content creators and technology companies as AI technologies increasingly rely on large datasets that include licensed and, in some instances, pirated content.

• From late 2023, the New York Times sues OpenAI and Microsoft for alleged copyright infringement in AI training (with late-breaking update).
• U.S. judge partially favors OpenAI while permitting unfair competition claim in authors' copyright lawsuit in this ruling from early 2024.
• Last month Thomson Reuters wins landmark U.S. AI copyright case, potentially establishing a legal precedent.
• Microsoft guarantees legal protection for Copilot users from copyright lawsuits.
• Meta's training of its AI with pirated LibGen books sparks legal and ethical debate.
• Nvidia denies copyright infringement in the use of shadow libraries for AI training.
• Getty Images launched an AI image generator using its licensed library in 2023.
• This Week I Learned: "But where is everybody?!?" — the origins of Fermi's Paradox
• This week's cat

Also on DLTJ this past week:

• In OCLC v Anna's Archive, New/Novel Issues Sent to State Court: The case of OCLC against Anna's Archive, accused of “data scraping” from OCLC's WorldCat, takes a turn as the U.S. District Court for the Southern District of Ohio decides to certify several “novel and unsettled” legal questions to the Supreme Court of Ohio.
• My protest signage improved at this week's #TeslaTakedown: My improved sign said "Our GOVERNMENT was fine. Now it is MUSKed UP! FIRE ELON!" Read the post for instructions on printing your own copy of this protest sign.

Feel free to send this newsletter to others you think might be interested in the topics. If you are not already subscribed to DLTJ's Thursday Threads, visit the sign-up page. If you would like a more raw and immediate version of these types of stories, follow me on Mastodon where I post the bookmarks I save. Comments and tips, as always, are welcome.

New York Times sues OpenAI and Microsoft for alleged copyright infringement in AI training

We're starting with the lawsuits, and this is one of the bigger ones. At the time the lawsuit was filed, OpenAI announced deals with content providers to use their backfiles of content, but the New York Times was a holdout. The lawsuit claims that OpenAI and Microsoft used millions of Times articles, which directly competes with the newspaper's services. While OpenAI maintained that it respects content creators' rights and believes its practices fall under fair use, the lawsuit cites instances of AI reproducing Times articles verbatim. This case has had many twists and turns, including a report last year that OpenAI intentionally trashed the research of the Times' lawyers. You can follow along with the court case in the Southern District of New York.

LATE BREAKING NEWS: As I was finishing the edits on this issue, I saw the judge issued a brief ruling on the defendant's motion to dismiss. In short, the lawsuit continues, but the portions on "common law unfair competition by misappropriation claims" are dismissed. The full version of the opinion hasn't been released yet, but should be coming soon!

US judge favors OpenAI, permits unfair competition claim in authors' copyright lawsuit

A US judge has largely sided with OpenAI in a lawsuit brought by authors alleging that ChatGPT was trained using pirated copies of their books. The judge dismissed most claims except for direct copyright infringement. While authors failed to show ChatGPT outputs were substantially similar to their works, one unfair competition claim was allowed to proceed based on OpenAI allegedly using copyrighted works without permission. This case has been quiet for a while because I think the remaining claims were consolidated into the Tremblay v. OpenAI, Inc. case being overseen by the same judge.

Thomson Reuters wins landmark U.S. AI copyright case, potentially establishing legal precedent

Last month, Thomson Reuters achieved a significant legal victory by winning the first major AI copyright case in the United States. Notably, the court rejected the notion that using content to train a foundational language model was not fair use. This case sets a precedent in the ongoing discussions surrounding copyright laws and artificial intelligence, and its outcome may influence how AI-generated content is treated under copyright law.

Microsoft guarantees legal protection for Copilot users from copyright lawsuits

In mid-2023, Microsoft announced a "Copilot Copyright Commitment" to address customer concerns regarding potential copyright infringement when using its AI-powered tools. The commitment includes indemnity in cases where customers are sued for copyright infringement, provided they have implemented necessary guardrails and content filters. The company acknowledges the need to respect authors' rights and aims to balance innovation with protecting creative works. This either says something about how Microsoft trained its foundational models with all copyright-free and licensed content, or that Microsoft believes its lawyers are better than everyone else's.

Meta's training of its AI with pirated LibGen books sparks legal and ethical debate

The article discusses the ethical and legal implications of Meta's use of pirated books from Library Genesis (LibGen) to train its AI model, called Llama 3. Faced with high costs and slow licensing processes for acquiring legal texts, Meta employees opted to access LibGen, which contains over 7.5 million books and 81 million research papers. Internal communications revealed that Meta acknowledged the medium-high legal risks of this strategy and discussed methods to mask their activities, including avoiding the citation of copyrighted materials. The communications were part of a motion for partial summary judgement in a lawsuit against Meta. And Meta is not going quietly—in response to that motion, it has filed dozens of documents on the court docket.

Nvidia denies copyright infringement in use of shadow libraries for AI training

Nvidia is the company making the news for creating the GPUs that are so popular with companies training foundational models. In creating their own model, they say that using "shadow libraries" like Z-Library and Library Genesis does not necessarily violate copyright law, and that its AI training process is a "highly transformative" fair use of the content. On the other hand, authors have argued that the AI models are derived from the protected expression in the training dataset without their consent or compensation. Nvidia's position seems pretty gutsy...admit that you are using copyrighted content, and arguing that such use is okay. A ruling against them would take a bit bite out of their sky-high stock market valuation. The case is currently in the discovery phase.

Getty images launches AI image generator using its licensed library

In 2023, Getty Images launched a AI image generation tool that uses its vast library of licensed images. The company says users of its output have full copyright indemnification for commercial use. Developed in partnership with Nvidia (yes—the same Nvidia mentioned in the article above) and leveraging the Edify model, this tool allows users to create images while being protected legally. Getty plans to compensate creators whose images are used to train the AI model and will share revenues generated from the tool. Unlike traditional stock images, AI-generated photos will not be included in Getty’s existing content libraries.

This Week I Learned: "But where is everybody?!?" — the origins of Fermi's Paradox

This retelling of the Fermi Paradox coms from this story about why, despite the vastness of the universe, we have yet to encounter evidence of extraterrestrial civilizations. Enrico Fermi famously posed the question, "Where is everybody?" suggesting a disconnect between the expectation of abundant intelligent life and the lack of observable evidence. The concept of the Great Filter is introduced, proposing that there may be significant barriers preventing intelligent life from becoming spacefaring. The article goes on to speculate where we are relative to the "Great Filter" — are we past it, or is it yet in front of us? In other words, have we survived the filter or is our biggest challenge ahead of us?

What did you learn this week? Let me know on Mastodon or Bluesky.

It is hard to write with a cat on your lap

n 2023 North Korean hackers made away with a total of $661m, according to Chainalysis, a crypto-investigations firm; they doubled the sum in 2024, racking up $1.34bn across 47 separate heists, an amount equivalent to more than 60% of the global total of stolen crypto. The ByBit operation indicates a growing degree of skill and ambition: in a single hack, North Korea swiped the equivalent of $1.5bn from the exchange, the largest-ever heist in the history of cryptocurrency.

Source

Crypto-thievery is a more efficient way to earn hard currency than traditional sources, such as overseas labourers or illegal drugs. The United Nations Panel of Experts (UNPE), a monitoring body, reported in 2023 that cyber-theft accounted for half of North Korea’s foreign-currency revenue. North Korea’s digital plunder last year was worth more than three times the value of its exports to China, its main trade partner. “You take what took millions of labourers, and you can replicate that with the work of a few dozen people,” says Mr Carlsen.

Attackers were able to compromise the devices of at least these three core contributors through a sophisticated malware injection. These compromised devices were then used to sign malicious transactions.

Although three compromised devices have been confirmed, it is likely that more were targeted — the means by which they were compromised remains unknown and under investigation. The devices were compromised in such a way that the front-end of Safe{Wallet} (f.k.a. Gnosis Safe) displayed legitimate transaction data while malicious transactions were signed and executed in the background.

Each transaction was simulated for accuracy on Tenderly and individually reviewed by multiple developers at each signature stage. Front-end checks in both Tenderly and Safe showed no anomalies during these reviews.

To underscore the significance of this point, the compromise was completely undetectable during the manual review of the Gnosis Safe UI and Tenderly simulation stages of the routine transaction.

Source

The UI for a Safe is clean and intuitive. You can browse the assets in the safe easily, as well as check and manage the participating addresses.

Front-end verification of all three multi-signature transactions showed no signs of compromise, aside from Safe App transaction resubmissions due to failures. It is important to highlight that resubmitting Safe transactions due to failures is a common and expected occurrence. Transactions submitted on the Safe front-end can fail due to gas price fluctuations, nonce mismatch, network congestion, insufficient gas limit, smart contract execution errors, token insufficiency, pending transactions, front-end synchronization issues, timeouts, or permission/signature errors in multi-signature setups. As a result, this behavior did not raise immediate suspicion. The malicious actors exploited this normalcy, using the process to collect multiple compromised signatures over several attempts, all while mimicking the appearance of routine transaction failures.

Take raw transaction data out of your wallet provider when a signature is prompted (e.g., Metamask, Rabby) and plug it into https://etherscan.io/inputdatadecoder. Confirm that the function you are calling, and the ToAddress all match up with the intended behavior. If a device were to be compromised, like in the case above, it would either not decode at all, call a different function than the one you thought you were calling, or it would result in a different owner address than the one you intended.

1. Compromised a developer machine at Safe
2. Injected malicious JavaScript into a development container
3. Specifically targeted Bybit exchange to stay undetected longer
4. Manipulated what Bybit signers saw in the Safe interface

...
This represents a sophisticated supply chain attack rather than a direct compromise of end-user devices.

This attack followed an almost identical pattern to recent attacks on WazirX Exchange and Radiant Capital. This suggests that the same threat actors are repeatedly using this technique successfully.

It's now believed that the Safe UI was the compromise point in all these cases rather than end-user machines, which explains the similar attack patterns.

By examining the machines of three Signers from Bybit, malicious JavaScript payload from app.safe.global was discovered in the Google Chrome cache files.

There are two javascript files that were modified: _app-52c9031bfa03da47.js and 6514.b556851795a4cbaa.js.
...
From the Wayback Archive (https://web.archive.org/), we also identified an instance of this malicious JavaScript file dating back to Feb 19, 2025 17:29:05

At 19:09 UTC today, @zachxbt submitted definitive proof that this attack on Bybit was performed by the LAZARUS GROUP.

His submission included a detailed analysis of test transactions and connected wallets used ahead of the exploit, as well as multiple forensics graphs and timing analyses.

The submission has been shared with the Bybit team in support of their investigation. We wish them all the best.

Bybit’s team trusted what they saw on their screens, and Safe’s engineering team trusted that their systems were secure.

“Don’t trust, verify” has become a blockchain mantra for a reason.

When the signers reviewed the transaction but did not verify the calldata on their physical hardware devices, everything appeared correct. This underscores the critical need for thorough transaction verification beyond what’s displayed on the screen.

To spell it out, the computers showed a spoofed transaction that tricked them, but their wallets showed the malicious transaction. They could have caught this on the hardware wallet, but as of today, calldata can be tricky to verify on a wallet.

The attack began on Sept. 11, when a Radiant Capital developer received a Telegram message from someone impersonating a trusted former contractor. According to the message, the contractor was looking for a new job opportunity in smart contract audits. It requested comments on the contractor’s work and provided a link to a compressed PDF detailing their next assignment. The hackers even mimicked the contractor’s legitimate website to add credibility.

The zip file contained a disguised executable named INLETDRIFT. Upon opening, it installed malware on the developer’s macOS device, granting attackers access to the developer’s system. The malware was designed to communicate with a hacker-controlled server.

Tackling the problem requires multilateral efforts across governments and the private sector, but such collaboration has been fraying. Russia used its UN veto to gut the UNPE last year. President Donald Trump’s cuts to American development aid have hit programmes aimed at building cyber-security capacity in vulnerable countries.

By contrast, the North Korean regime is throwing ever more resources at cybercrime. South Korea’s intelligence services reckon its cybercrime force grew from 6,800 people in 2022 to 8,400 last year. As the crypto-industry expands in countries with weaker regulatory oversight, North Korea has an increasingly “rich target environment”, says Abhishek Sharma of the Observer Research Foundation, an Indian think-tank. Last year, Mr Sharma notes, North Korea attacked exchanges based in India and Indonesia.

© Grant's Interest Rate Observer

Crypto prices are prone to crashes — one could say that crypto exchanges are designed to facilitate them. It would be strange if the immensity of the market caps involved did not introduce a new element of risk into a world that hardly needs more. As for America’s multitasking president, he boasted (through his lawyer) in 2019 that he could commit murder on Fifth Avenue in broad daylight and not be charged with a crime. A conflict of interest is no capital crime, but Trump continues to mix the business of state with the pleasures and temptations of money-making.

One can borrow up to 78.5% loan-to-value on Aave against the special tokens and make an interest spread of 29 basis points by staking the newly borrowed ether. Starting with 100 ETH, for instance, one could stake them (earning 3%), use the automatically generated liquid staking tokens to borrow 78 more ETH, stake those to earn a net 29 basis points, use the resulting 78 more liquid staking tokens to borrow 78% of 78, i.e., 61, more ether, stake them for another 29 basis points and so on. ... In fact, an enterprising decentralized financier will need to iterate the borrowing and staking process 12 times until he is levered 4.5 times, having borrowed 3.5 ether tokens for each originally laid out, before the resulting digital hoard will even break 4% in yield,

On May 19, 2021, a flash crash in multiple cryptocurrencies briefly knocked more than 25% off the price of bitcoin and 34% off the price of ether before the prices recovered somewhat, all inside of an hour.

the entire portfolio, not just in ether, but all the other positions, will get taken out by the exchange. The exchange is not supposed to sell these things straight away. They have things called “guarantee funds” or “insurance funds,” which are supposed to absorb the liquidated positions [and hold them]. But they don’t. They dump them into the market. I’ve got all the data to show that.

“With World Liberty Financial [Trump] stands to make billions. . . . They want to provide lending and borrowing services like Aave. Aave is the biggest shadow bank [in decentralized finance],” Alexander explains, representing a market for crypto-token deposits and loans worth $29.1 billion.

There's a war going on on the Internet. AI companies with billions to burn are hard at work destroying the websites of libraries, archives, non-profit organizations, and scholarly publishers, anyone who is working to make quality information universally available on the internet. And the technologists defending against this broad-based attack are doing everything they can to preserve their outlets while trying to remain true to the mission of providing the digital lifeblood of science and culture to the world.

Yes, many of these beloved institutions are under financial pressures in the current political environment, but politics swings back and forth. The AI armies are only growing more aggressive, more rapacious, more deceitful and ever more numerous.

I'm talking about the voracious hunger of AI companies for good data to train Large Language Models (LLMs). These are the trillion-parameter sets of statistical weights that power things like Claude, ChatGPT and hundreds of systems you've never heard of. Good training data has lots of text, lots of metadata, is reliable and unbiased. It's unsullied by Search Engine Optimization (SEO) practitioners. It doesn't constantly interrupt the narrative flow to try to get you to buy stuff. It's multilingual, subject specific, and written by experts. In other words, it's like a library.

At last week's Code4lib conference hosted by Princeton University Library, technologists from across the library world gathered to share information about library systems, how to make them better, how to manage them, and how to keep them running. The hot topic, the thing everyone wanted to talk about, was how to deal with bots from the dark side.

Bots on the internet are nothing new, but a sea change has occurred over the past year. For the past 25 years, anyone running a web server knew that the bulk of traffic was one sort of bot or another. There was googlebot, which was quite polite, and everyone learned to feed it - otherwise no one would ever find the delicious treats we were trying to give away. There were lots of search engine crawlers working to develop this or that service. You'd get "script kiddies" trying thousands of prepackaged exploits. A server secured and patched by a reasonably competent technologist would have no difficulty ignoring these.

The old style bots were rarely a problem. They respected robot exclusions and "nofollow" warnings. The warning helped bots avoid volatile resources and infinite parameter spaces. Even when they ignored exclusions they seemed to be careful about it. They declared their identity in "user-agent" headers. They limited the request rate and number of simultaneous requests to any particular server. Occasionally there would be a malicious bot like a card-tester or a registration spammer. You'd often have to block these based on IP address. It was part of the landscape, not the dominant feature.

The current generation of bots is mindless. They use as many connections as you have room for. If you add capacity, they just ramp up their requests. They use randomly generated user-agent strings. They come from large blocks of IP addresses. They get trapped in endless hallways. I observed one bot asking for 200,000 nofollow redirect links pointing at Onedrive, Google Drive and Dropbox. (which of course didn't work, but Onedrive decided to stop serving our Canadian human users). They use up server resources - one speaker at Code4lib described a bug where software they were running was using 32 bit integers for session identifiers, and it ran out!

The good guys are trying their best. They're sharing block lists and bot signatures. Many libraries are routinely blocking entire countries (nobody in china could possibly want books!) just to be able to serve a trickle of local requests. They are using commercial services such as Cloudflare to outsource their bot-blocking and captchas, without knowing for sure what these services are blocking, how they're doing it, or whether user privacy and accessibility is being flushed down the toilet. But nothing seems to offer anything but temporary relief. Not that there's anything bad about temporary relief, but we know the bots just intensify their attack on other content stores.

The view of MIT Press's Open-Access site from the Wayback Machine.

Thousands of developer hours are being spent on defense against the dark bots and those hours are lost to us forever. We'll never see the wonderful projects and features they would have come up with in that time.

The thing that gets me REALLY mad is how unnecessary this carnage is. Project Gutenberg makes all its content available with one click on a file in its feeds directory. OAPEN makes all its books available via an API. There's no need to make a million requests to get this stuff!! Who (or what) is programming these idiot scraping bots? Have they never heard of a sitemap??? Are they summer interns using ChatGPT to write all their code? Who gave them infinite memory, CPUs and bandwidth to run these monstrosities? (Don't answer.)

We are headed for a world in which all good information is locked up behind secure registration barriers and paywalls, and it won't be to make money, it will be for survival. Captchas will only be solvable by advanced AIs and only the wealthy will be able to use internet libraries.

Or maybe we can find ways to destroy the bad bots from within. I'm thinking a billion rickrolls?

Notes:

1. I've found that I can no longer offer more than 2 facets of faceted search. Another problematic feature is "did you mean" links. AI bots try to follow every link you offer even if there are a billion different ones.
2. Two projects, iocaine and nepenthes are enabling the construction of "tarpits" for bots. These are automated infinite mazes that bots get stuck in, perhaps keeping the bots occupied and not bothering anyone else. I'm skeptical.
3. Here is an implementation of the Cloudflare Turnstyle service (supposedly free) that was mentioned favorably at the conference.
4. It's not just open access, it's also Open Source.
5. Cloudflare has announced an "AI honeypot". Should be interesting.
6. One way for Open Access site to encourage good bot behavior is to provide carrots to good robots. For this reason, it would be good to add Common Crawl to greenlists: https://commoncrawl.org/ccbot
7. Ian Mulvaney (BMJ) concurs. 

Photo: Lucas Pretti/OKFN

As Chief Executive Officer of the Open Knowledge Foundation, I have been selected to join and work in the United Nations Commission on Science and Technology for Development (CSTD) Working Group on Data Governance at All Levels and the Working Group on Universal Safeguards for Digital Public Infrastructure (DPI).

We have led the open movement towards a rights-enhancing and inclusive digital society for over two decades. Our legal tools in the Open Data Commons have enabled unrestricted data sharing, empowering global communities to innovate and collaborate. The Open Definition sets a standard for openness, guiding countless individuals and institutions to share their data, knowledge and tools. The School of Data, one of our flagship projects, has provided thousands worldwide with skills to turn data into actionable insights. As members of the Digital Public Goods Alliance, we have actively advocated for open, interoperable, sustainable, and lasting technology that serves all. Through CKAN—a digital public good that was created by our technical teams —we have provided hundreds of governments and organisations with a platform for open data sharing, fostering economic opportunities, accountability and innovation all over the world. We are not only thinkers but doers, and it is such accumulated experience that I am bringing to the groups.

In changing times, when multilateral cooperation mechanisms are questioned, when the word openness is being abused to mask extractive businesses, and when technology and data are used to commit injustices at large scale, it is urgent to be present in the spaces where key actors convey to explore solutions to break such dynamics and build together feasible paths towards a different digital society, breaking down data monopolies and delivering unprecedented access to knowledge, for all —unlocking pathways for innovation and growth. That is why principles of open knowledge, open data, open collaboration and open standards should guide the data governance in every area, and our examples and experiences can lead to better, more effective data governance systems.

In the Working Groups, I will suggest the “open by design” principle, for datasets and systems to become the backbone of our future. Highly digitised societies—such as digital identities and payment platforms—, if designed with opacity, without meaningful participation and accountability, these risk entrenching exclusion, corruption and control; if they are discussed openly with the people they intend to serve and if they are developed open and kept open, they can enable accountability, interoperability and the ability to build upon, while also reducing the risks for the most vulnerable in society. Through my role in the Digital Public Goods Alliance and stewardship of CKAN, I have gained experience and a better understanding of the urgency of building Digital Public Infrastructure rooted in open standards, open-source software (ideally free software) and open data. Open technologies, standards, and practices remove barriers, ensuring that infrastructure enables participation and economic advancement rather than perpetuating disparities.

As the work progresses in these groups, I will share as many insights into its progress as I am permitted and seek advice and guidance from our valuable community and international network, which is closely in touch with the latest challenges and progress achieved in these areas.

We’ve recently added more than ten thousand freely readable recent books from academic publishers to the extended shelves of The Online Books Page. These online titles are provided by JSTOR’s Open Access Books collection, and represent the first large-scale automated inclusion of new monographs to our catalog. We’ll be updating the listings from this collection on a regular basis. (Most of our extended shelves imports, which also include collections like HathiTrust, Project Gutenberg, and the Directory of Open Access Journals, get refreshed about once a month.)

We’ll also continue to add individual recent academic books of interest, both from that collection and others, to our curated collection. (You’ll see individually-chosen additions in our new books listing.) Automated addition of JSTOR’s open access books will let us include many more new titles than we’d be able to do manually by ourselves. They’ll help our site’s readers find up to date information from experts on many subjects, complementing the historical sources and other older books also available in our catalog. (As always, I recommend that information seekers also consult their local libraries to get a fuller understanding of the subjects our collection covers, since local libraries have many copyrighted books we can’t offer for free reading on The Online Books Page. But we now also have a growing number of recent books you’re not likely to find, or obtain as easily, in your local library.)

A growing number of academic and peer-reviewed monographs and edited volumes are now available through open access. The Directory of Open Access Books now lists over 90,000 of them, and the related OAPEN site offers direct downloads of many such books as well, as do many individual publisher sites. We may expand our automated listings in the future to cover a wider array of open access academic titles. But I consider JSTOR’s collection to be a good starting point for automated imports, both because it includes over 140 high-quality publishers, and because it provides high-quality metadata. That makes it relatively easy to integrate their offerings with the other books in our catalog, using similar author and subject headings, and avoiding redundant listings. As always, our “extended shelves” imports are subject to change and not guaranteed to be as persistent as the “curated collection” listings we catalog ourselves. But you can always request that we add specific titles you’re interested in to our curated collection, whether from JSTOR’s offerings or elsewhere.

JSTOR has a growing collection of information resources that are freely and openly available to all readers (as well as resources they offer by subscription or by registered access). Along with JSTOR’s open access books, we also link to issues of many of the serials JSTOR offers, sometimes via JSTOR’s own Early Journal Content offerings, sometimes via other online providers. We’re also continuing to document the public domain status of serials from JSTOR and other providers in our Deep Backfile knowledge base, (You can use the “Contact us” links in our Deep Backfile tables to let us know about particular serials you’re interested in, or for which you have information that we could use.)

My thanks to JSTOR, and to a growing number of publishers and authors, for making current scholarship freely available and discoverable!

Part of our goal in developing the Open Data Editor (ODE) this year is to increase digital literacy among key communities through the application. We’re thrilled to introduce our latest learning resource: Quality and Consistent Data with the Open Data Editor – a free, hands-on course designed to help non-technical users improve their data skills with the Open Data Editor (ODE).

Developed in partnership with Open Knowledge Brazil and Escola de Dados, this course is part of our mission to increase digital literacy and empower communities with practical tools for working with data—no coding required.

What you’ll learn

The course follows a learn-by-doing approach, covering essential topics like:

Detecting and fixing errors in tables

Learn to work with tabular data

Don’t get lost when validating your spreadsheet

Clean up your spreadsheets to gain valuable insights

FAIR data principles

Understand how to guarantee the Findability, Accessibility, Interoperability, and Reuse of digital assets

Master all about metadata

Understand the Open Definition and the principles of open data

Learning-by-doing

Follow in the footsteps of an exclusive instructor

Get hands-on and learn from your mistakes and successes

Learn how to handle databases without writing code

Why ODE? More than just error detection

The Open Data Editor (ODE) isn’t just a tool for spotting mistakes in tables—it’s a gateway to open data literacy. Pilot organisations have shared that taking this course was key for their teams to advance their data knowledge, helping them not only clean datasets but also understand how to handle, share, and use open data effectively. Whether you’re new to data or looking to refine your skills, ODE makes the process intuitive and accessible.

Who is this for?

This course is specially designed for:
Non-technical users who work with data but lack advanced technical training
Journalists, researchers, and civil society members looking to handle data more effectively
Anyone interested in improving data consistency and reliability

Why take this course?

The Open Data Editor (ODE) simplifies data work, making it accessible to everyone. Whether you’re cleaning up a dataset, checking for errors, or preparing data for analysis, this course will give you the confidence to do it efficiently and and knowledge of the principles that govern good practices in open data management.

Enroll now and start mastering data quality with ODE!

The course will soon be translated into Portuguese and Spanish. If you would like to contribute to the translation into other languages, please contact us at info@okfn.org.

Join us in making data more open, usable, and reliable for all. Happy learning!

In collaboration with Patricio Del Boca

This post discusses Open Knowledge Foundation’s (OKFN) approach to integrating AI into the Open Data Editor (ODE), our new open source desktop application for data practitioners to detect errors in tables.

To ensure thoughtful AI integration, we are emphasising the importance of interdisciplinary collaboration, user feedback, and ethical considerations rather than just technical implementation. We are prioritising user needs, such as limiting data shared with OpenAI to metadata and making AI features optional, while acknowledging practical constraints like deadlines and resource limitations.

Learn more about the lessons learned during the process, including the value of asking critical questions, avoiding perfectionism, and balancing innovation with ethical and practical considerations.

Why has OKFN opted for a non-technical approach to reflect on AI? 

OKFN is convinced that AI integrations cannot be approached solely from the technical side. There are conversations and reflections that teams must have both internally and with the community to ensure that such integrations are targeted to help with specific problems, and at the same time ensure that they are communicated in a clear and transparent way. These conversations not only helped us define what kind of implementation we were going to do, but also allowed us to build internal capacity and new soft skills to be able to answer key questions or be aware of possible red flags when working with AI. 

Limitations and general comments on AI reflections

As in any project, our work had certain external and internal deadlines and constraints. On the one hand, we had a limited period of time for implementation. Initially, we defined a workflow of two months, four work calls and asynchronous revision of a document with guidelines and ideas. However, as we progressed with the reflections, we called Madelon Hulsebos (our AI Consultant) to resolve doubts and validate the integration angle. It should be noted that the first general conversation was not only between Madelon and Romina Colman (Product Owner). The meeting was also attended by part of the project’s technical team and our UX designer. Why? Not only to address the integration from all angles, but also to de-centre the idea that working with AI is something that is solved at the code level and avoid the current disconnect seen in the ecosystem where there are conversations between technical people on the one hand and academia, activists and NGOs on the other. 

This is what we learned:

• Create an interdisciplinary team to reflect on artificial intelligence integrations: While incorporating AI into products may be easy with a skilled technical person, it is important not to limit its implementation to the realm of code and to open up the space for conversations where multiple voices can be heard. For example, in some of our early team talks we talked about possible integrations where the conclusion was ‘But we don’t need AI to do that’.
• Ask yourselves many questions: it is important not to take anything for granted, to ask questions all the time throughout the process. For example, in the case of the ODE these were some of the questions we asked ourselves: ‘How do I explain to the user what is happening at this stage?‘, ‘How do I make sure they know what information we are sharing when they click on the button?‘, etc.
• Listen to your user needs and concerns: months before we started working with AI integration, in the first interviews with our potential users we asked them if they used AI for their work with data. At that time, journalists and people working in NGOs shared their concerns about its use and the importance of privacy in their work. These two elements were central to the decisions we made later, such as the idea of having a predefined prompt in the application or the idea of explaining what information ODE shares if the user decides to work with AI.
• Don’t strive for perfection: in every project there are deadlines, resource constraints and AI work is often one component of many. It is necessary to create a balance between all elements. For example: in our case, as mentioned above, the ODE team wants to start testing with open source LLMs. When we started talking about it, we realised that we couldn’t do it in the time we had and so we started asking ourselves how to improve and narrow down the integration (bearing in mind that we were using OpenAI) and then move on to other models.

The ODE uses OpenAI as a third party for the AI integration. Will we keep using it?

The current AI integration of ODE uses OpenAI. This decision was a practical one: it was easy to implement and it is the current state-of-the art when integrating LLMs to applications.

The quick integration of Open AI provided useful feedback:

• We were able to quickly test integrations and uses of LLMs into the ODE to start exploring capabilities and limits of the technology.
• User research that we conducted in March 2024 showed that journalists (potential users of the ODE) and other data practitioners expressed concerns on using the ODE if the app shared the full content of their files. 

Based on the feedback we made two decisions:

• Limit the data we share with Open AI to only the metadata of the file.
• Make the use of Open AI optional (nothing is shared by default)

In 2025, we are working on:

• Keep exploring other uses cases where LLMs can be useful
• Work towards the integration of local LLMs, this is, to make ODE work with LLMs running in the users machine (instead of a third party provider)

Introduction

Back in November, I posted about ManoWhisper, where I briefly introduced the work I was doing with the UWaterloo team for the Digital Feminist Network. Today, I’ll delve a bit more into that work and provide a closer look at some of the tools and their output in the context of a broader research project I’m working on. For that project, we have compiled an ever-growing dataset of podcast transcripts comprising over 11,000 episodes from 20 podcasts associated with the Intellectual Dark Web, conspiracy theories, QAnon, the Alt-Right, White Supremacist/Nationalist movements, and the Manosphere.

What’s the purpose of this thing and all it’s utilities with silly names? Let’s start here. Nobody – and I mean nobody – wants to sit down and closely read over 11,000 transcripts of material from these communities. It can get real traumatic real quick. So, what are some ways we can do macro and micro-level readings of the content? Or give us the ability to ask the data some questions, perform information retrieval, and allow us to do close reading to see things in full or fuller context? I’ll walk through a few of the ManoWhisper utilities below, and hopefully that’ll provide some insight into them. If you just want to see a bunch of the visualizations produced with the utilities, jump over here.

Show Statistics

Broad overviews are a good way to get some basic understanding of a given podcast in our dataset. They’ll help us get a good perspective on each host or team, and provide us with some comparators for the entire corpus. But also us to look at some apples and oranges together since not every podcast is the same format, style, or length. Still, they might share some underlying commonalities.

Red Pill Caliper takes a directory of transcripts (WebVTT files), and outputs six different histograms

Usage: red-pill-caliper.py [OPTIONS] TRANSCRIPTS

  Analyze a directory of podcast transcripts (WebVTT) and generate histograms.

  TRANSCRIPTS: Path to the directory of WebVTT files.

Options:
  -p, --podcast-name TEXT  Name of the podcast for graph titles.
  --help                   Show this message and exit.

As an example, we let’s take a look at “The Jordan B. Peterson Podcast” corpus.

Episode Lengths

Character Count per Episode

Words per Episode

Sentences per Episode

Vocabulary Size per Episode

Speaking Rate per Episode

Keyword trends

Next, one entry point for analysis is keyword searching the corpus. Since they’re all text files, grep or ag works well here, but you might also want to examine how many times a given keyword or phrase (these are “wildcardable” as well) appears in a given podcast compared to others. Red Pill Resonator generates a keyword frequency graph of a comma separated list of terms or phrase for our corpus of podcasts.

Additionally, because not all episode lengths are the same, we should normalize the data by looking at the rate at which a keyword or phrase appears per episode. For instance, an Andrew Tate episode might only be 8 minutes long, while a Nick Fuentes episode might be 2 hours. If we simply count the occurrences of a term, the results could be distorted due to differences in episode lengths. That’s why I chose the per-episode metric as the default. You’ll see below that there is also a mode flag to compute overall counts if needed.

Usage: red-pill-resonator.py [OPTIONS] OUTPUT_IMAGE

  Generate keyword frequency graphs across a corpus of WebVTT files.

  Arguments:
    OUTPUT_IMAGE  Filename of the graph.

Options:
  --keywords TEXT           Comma-delimited list of keywords.  [required]
  --mode [overall|episode]  Choose whether to display overall keyword counts
                            (--mode overall) or average counts per episode
                            (--mode episode).  [required]
  --width INTEGER           Width of the graph in pixels.  [default: 800]
  --height INTEGER          Height of the graph in pixels.  [default: 600]
  --title TEXT              Title of the graph.  [default: Keyword Frequency
                            Graph]
  --help                    Show this message and exit.

This example looks at the keywords “trump, vance, biden, harris, walz” in the entire corpus of materials at the time, which included “Candace Owens”, “Loomer Unleashed”, “The Culture War Podcast with Tim Pool”, “The Jordan B. Peterson Podcast”, “The StoneZONE with Roger Stone”, and “The Tucker Carlson Show”.

Emotions (episode)

The remainder of the functionality I’ll go over in this post uses three different text classification models to classify sentence segments and plot those classifications.

Emotional Rollercoaster breaks a given podcast episode transcript into sentence segments and feeds each sentence to j-hartmann/emotion-english-distilroberta-base to generate an emotion classification. The classifications are then plotted in a heatmap over time. I think this provides valuable insight into the vibe of a given episode and is well complemented by the Hate Speech and Misogynistic Speech classification utilities documented below.

Usage: emotional-roller-coaster.py [OPTIONS] INPUT_VTT_FILE OUTPUT_HTML_FILE

  Generate a heatmap of emotions for a given WebVTT transcript.

Options:
  -t, --title TEXT  Title of the heatmap

This example takes a look at an “America First with Nicholas J. Fuentes” episode where Nick Fuentes reviews the 2023 Barbie movie.

Misogyny (episode)

Wave of Misogyny breaks a given podcast episode transcript into sentence segments and feeds each sentence to MilaNLProc/bert-base-uncased-ear-misogyny to generate a misogyny classification. The classifications are then plotted in a dual-axis chart over time.

Usage: wave-of-misogyny.py [OPTIONS] INPUT_VTT_FILE OUTPUT_HTML_FILE

  Generate a dual-axis chart of misogyny scores for a given WebVTT transcript.

Options:
  -t, --title TEXT  Title of the chart
  --help            Show this message and exit.

This example takes a look at the same “America First with Nicholas J. Fuentes” episode mentioned above.

Hate (episode)

Dicks Hate the Police breaks a given podcast episode transcript into sentence segments and feeds each sentence to facebook/roberta-hate-speech-dynabench-r4-target to generate a hate speech classification. The classifications are then plotted in a dual-axis chart over time.

Usage: dicks-hate-the-police.py [OPTIONS] INPUT_VTT_FILE OUTPUT_HTML_FILE

  Generate a dual-axis chart of hate scores for a given WebVTT transcript.

Options:
  -t, --title TEXT  Title of the chart
  --help            Show this message and exit.

This example takes a look at a “Get Off My Lawn Podcast with Gavin McInnes” episode titled, “COMPOUND CENSORED 160 - TACS 1796: GOD IS A NATIONALIST”.

Misogyny (Podcast)

The final two examples take a look at the corpus of an entire podcast instead of just an episode, and generate a donut chart.

Donut Hate Women breaks a given podcast’s transcripts into sentence segments and feeds each sentence to MilaNLProc/bert-base-uncased-ear-misogyny to generate a misogyny classification for the entire corpus. The classifications are then plotted in a donut chart.

Usage: donut-hate-women.py [OPTIONS] INPUT_PATH OUTPUT_HTML_FILE

  Generate a pie chart of misogynist vs non misogynist classifications for
  WebVTT transcripts.

  Accepts either a directory of VTT files or a single VTT file.

Options:
  -t, --title TEXT  Title of the chart
  --help            Show this message and exit.

This example takes a look the “Fresh & Fit Podcast” corpus.

Hate (Podcast)

Finally, Donut Hate breaks a given podcast’s transcripts into sentence segments and feeds each sentence to facebook/roberta-hate-speech-dynabench-r4-target to generate a hate speech classification for the entire corpus. The classifications are then plotted in a donut chart.

Usage: donut-hate.py [OPTIONS] INPUT_PATH OUTPUT_HTML_FILE

  Generate a pie chart of hate vs not hate classifications for WebVTT
  transcripts.

  Accepts either a directory of VTT files or a single VTT file.

Options:
  -t, --title TEXT  Title of the chart
  --help            Show this message and exit.

This example takes a look “The Roseanne Barr Podcast” corpus.

If you’ve found useful the many mid-20th century serials that are now freely readable online through The Online Books Page, you can thank the Institute of Museum and Library Services. The IMLS (as it’s generally known) funded the completion in 2018 of a survey of copyrights I led for serials published before 1950. The completed survey made it possible for the first time to quickly ascertain the public domain status of decades of serials. It also laid the groundwork for our bigger Deep Backfile project that now documents rights and free online availability for well over 10,000 serials. I’d say that’s a pretty good return on a $25,000 IMLS investment.

The IMLS has made that kind of investment many times over, with libraries and museums across all 50 states. The IMLS makes a lot of mostly modest grants for projects and programs that make a big difference in the communities libraries and museums serve. You can read about a sampling of them in Devon Akmon’s recent Conversation article. Or you can ask your local librarian or museum curator. They may tell you how the IMLS supports them providing access to information online, promoting literacy, preserving unique parts of our country’s cultural heritage, and many other functions. They do it with a budget that requires less than $1 per American per year. It’s hard to imagine a more efficient use of government funds.

Despite its efficiency, the IMLS has been a recurrent target for elimination. The first Trump administration proposed eliminating the IMLS starting with its first budget request. But Congress listened to its constituents and continued to fund it. This time, however, Trump has tried to shut it down on his own, without involving Congress. He issued an executive order for IMLS’s activities to be “eliminated to the maximum extent consistent with applicable law”, aiming to “effectuate an expected termination” of the agency. He also appointed a new acting director who’s pledged to act “in lockstep with this Administration” to carry out the president’s wishes.

So far, Trump has been unable to completely shut down the organization, as he has managed with some other agencies. When IMLS remote staffers got word that the new acting director and some of Elon Musk’s DOGE workers were going to come to the IMLS office last Thursday, they reported to the office in person, many of them dressed in formal (or funereal) black. Faced with more people than DOGE expected, an anonymous staffer related that “instead of laying everybody off immediately they left the building, because they didn’t want to create a scene with us there. Otherwise, they would have locked the doors and taken over our systems and sent a mass notification out to everyone.”

As I write this, IMLS is still operating. But in the next week, the new acting director may try to cut off its funding to libraries and museums. Or, he could try to make libraries and museums receiving funds to change their programs and collection policies to conform to his preferences, or Trump’s. The acting director has already issued a press release expressing his intent to”restore focus on patriotism”. And the president has not been hesitant to cut funding to institutions to make them change their programs to his liking, as some have now agreed to.

But right now, there’s still time for us who value libraries and museums to make our voices heard. We can tell our lawmakers that the IMLS should continue to be fully supported, and should support all the libraries and museums whose diverse programs and collections help make our country great. If you want to participate, see the calls to action at EveryLibrary and the American Alliance of Museums.

My protest sign for the #TeslaTakedown today.

I'm a long way from a career change to graphic design or protest communications, but this week was a definite improvement. About a half dozen people asked for pictures of my sign. That's a good signal, so I'm including instructions below on printing and making one yourself.

It was another windy, gloomy day at the Easton Tesla store, but the number of people increased from last Saturday. One organizer said between 450 and 500 people, which seemed about right to me. It was just a little more than we had last week. The weather forecast for next week is about 15 degrees warmer, so it will be interesting to see if the families with young children come out again like on March 8th.

The entertainment definitely improved. Someone set up an amplified acoustic guitar and a microphone, and people took turns singing. We marched two laps around the block, and there were many more honks and cheers from the cars driving by. Promptly at 5:30, the organizers walked around and asked people to leave to be respectful of the Columbus police dialogue team that had been called out for what was advertised as a one hour protest. That seemed reasonable.

This week's protest sign

My protest sign at the #TeslaTakedown.

Back to basics, I thought. People are driving by quickly, so too much text won't be read. So this was the idea.

• Set the context: "Our GOVERNMENT was FINE."
• Deliver the punchline: "Now it is MUSKed UP!"
• Clear call-to-action: "FIRE ELON!" (in a flaming font, nonetheless)

And that seemed to work.

Refer to the March 8th blog post for instructions on creating the sign. If you want one too, I've uploaded the 6-page PDF of page tiles to make the sign. When you print them, line them up with three on top and three on the bottom. Then, trim the bottom and right edges of each page. For the two right-most pages, there will be a lot of extra, unused space to cut off, and there are crop marks you can use to trim at just the right spot. There are a few millimeters of overlap between pages, so your trimming doesn't have to be exact. Then line up the pages and tape them to a poster board (or, as in my case, a recycled campaign yard sign.) This is set up to make a 26" by 16" sign — the exact dimensions of a typical campaign yard sign! Ping me on Mastodon or Bluesky if you use it, and include a picture if you'd like!

So now that I've shown improvement week-by-week, I need to figure out how to step up my game for next Saturday...

This text, part of the #ODDStories series, tells a story of Open Data Day‘s grassroots impact directly from the community’s voices.

For Open Data Day 2025, Thailand is spotlighting disasters as a key theme under the title:
“Floods, Smog, Wildfires: Decoding Thailand’s Endless Disaster with Data.”

We recognize disasters as part of a polycrisis—a web of interconnected crises that cannot be solved in isolation. In Thailand, floods, smog, and wildfires are not isolated events; they are deeply linked to socio-economic vulnerabilities, political decisions, and environmental degradation. We believe that better data and effective data use are key to tackling these challenges.

This year, WeVis (Civic Tech), in collaboration with partners ThaiPBS, theactive_net, Policy Watch – Thai PBS, and 101_pub, organized an Open Data Day onsite event on 2 March 2025 at Siam Paragon as a half-day, open-access, free-to-attend gathering, livestreamed on Facebook and YouTube. The event was not only part of the global celebration of International Open Data Day but also a reaffirmation of our commitment to transparency and civic engagement through open data. With around 100 attendees, including both registered participants and walk-ins, we welcomed a diverse audience—ranging from those familiar with Open Data to newcomers who had never heard of it before but were drawn in by the event’s format and discussions.

Activities

The event was structured into three main segments:

1. Policy Forum: Using Data to Solve Thailand’s Never-Ending Disasters

This panel discussion tackled Thailand’s ongoing disaster management challenges through a data-driven lens. We invited leading experts in disaster management to analyze floods, smog, and wildfires, exploring how data could improve policy and response strategies.

The takeaway from this session:

• Thailand has plenty of disaster-related data, but it is scattered and disorganized, making it hard to use effectively.
• Lack of standardized, accessible formats prevents data from reaching the right people at the right time.
• Experts emphasized the need for greater data transparency, cross-agency data sharing, and community-level disaster preparedness.
• One of the biggest upcoming challenges? Climate change, which is making disasters more frequent and unpredictable.
• Additionally, certain personal data protection laws may pose hurdles in responding to disasters efficiently.

2. Mini Exhibition: Sonification – Listening to Disaster Data

This year, we introduced a new interactive approach to data interpretation: sonification—the process of converting data into sound, allowing attendees to listen to trends instead of reading graphs or statistics.

We showcased three datasets:

1. Flood damage areas across Thailand.
2. PM2.5 air pollution trends (monthly average data from 96 monitoring stations nationwide, spanning January–October 2024).
3. Cumulative wildfire burn areas in nine northern provinces (data spanning 2018–2024).

3. Project Showcase Talk: Behind the Data – Understanding Disasters Through Open Data

More than 10 projects from 9 organizations shared their experiences in using data-driven approaches to tackle Thailand’s disaster challenges. The discussion emphasized how data can be leveraged for disaster prevention, response, and long-term resilience.

At WeVis, our commitment to Open Data Day remains unwavering. We see this as an opportunity to advocate for open data policies and push Thailand closer to a future where transparency, accessibility, and data-driven decision-making are the norm. We hope to see Thailand fully embrace open data in the near future!

About Open Data Day

Open Data Day (ODD) is an annual celebration of open data all over the world. Groups from many countries create local events on the day where they will use open data in their communities. ODD is led by the Open Knowledge Foundation (OKFN) and the Open Knowledge Network.

As a way to increase the representation of different cultures, since 2023 we offer the opportunity for organisations to host an Open Data Day event on the best date over one week. In 2025, a total of 189 events happened all over the world between March 1st and 7th, in 57 countries using 15+ different languages. All outputs are open for everyone to use and re-use.

For more information, you can reach out to the Open Knowledge Foundation team by emailing opendataday@okfn.org. You can also join the Open Data Day Google Group or join the Open Data Day Slack channel to ask for advice, share tips and get connected with others.

The U.S. District Court for the Southern District of Ohio released an opinion in the case of OCLC v. Anna's Archive. As you may recall, the case stems from an accusation that Anna's Archive—a search engine for 'shadow libraries'—scraped the content of OCLC's WorldCat. Anna's Archive itself is an anonymous effort, and OCLC named one person in the lawsuit—Maria Matienzo—with a weak and dubious connection to Anna's Archive.

Here are bits of the court's order from its introduction (the start of the order) and conclusion (at the end):

This case is about data scraping. Plaintiff Online Computer Library Center, Inc. ("OCLC") is a non-profit organization that helps libraries organize and share resources. In collaboration with its member libraries, OCLC created and maintains WorldCat-the most comprehensive database of library collections worldwide. OCLC alleges that a "pirate library" named Anna's Archive along with Maria Matienzo, and other unknown individuals (collectively, "Defendants") scraped WorldCat's data. OCLC claims that, in doing so, Defendants violated Ohio law. Specifically, OCLC invokes causes of action arising under the Ohio common law of tort, contract, and property, as well as a provision of the Ohio criminal code.

But whether Ohio law prohibits the data scraping alleged here poses "novel and unsettled" issues. No Ohio court has ever applied its law as OCLC would have this Court do (as far as the Court is aware). Nor have courts uniformly applied analogous laws of other jurisdictions that way. So, to resolve this case, the Court would need to answer "novel and unsettled" questions about Ohio law.

When that is true-when a federal court faces "novel and unsettled" state-law issues-the federal court may certify those issues to the state's high court. Unwilling to sleepwalk into a drastic expansion of Ohio law, this Court thus resolves to certify the issues presented here.

[...]

The Court is sympathetic to OCLC's situation: a band of copyright scofflaws cloned WorldCat's hard-earned data, gave it away for free, and then ignored OCLC when it sued them in this Court. But mindful that bad facts sometimes make bad law, the Court requests that an Ohio court intervene before this Court makes any new state tort, contract, property, or criminal law.

The Court resolves to CERTIFY the novel Ohio-law issues identified above to the Supreme Court of Ohio. Plaintiff's counsel and Matienzo's counsel are ORDERED to propose an order containing all the information Ohio Supreme Court Practice Rule 9. 02 requires by April 11, 2025. The parties may file their proposed orders separately, or, if they so choose, they may file one joint proposed order. The Court will finalize a certification order afterward.

OCLC's motion for default judgment is DENIED without prejudice. See Lammert v. Auto-Owners (Mut. ) Ins., 286 F. Supp. 3d 919, 928-29 (M. D. Tenn. 2017) (adopting this same disposition). Because the answers to the certified questions may also determine Matienzo's motion to dismiss under Federal Rule of Civil Procedure 12(b)(6), ECF No. 21, the Court DENIES without prejudice that motion too. See id. The Court invites the parties to reraise their motions after the certification proceeding. See id.

The Court also grants OCLC leave to amend its Complaint to correct any of the above-identified pleading deficiencies.

OCLC has brought twelve claims against the defendants, including breach of contract, unjust enrichment, tortious interference, criminal violations under Ohio law, trespass to chattels (a fancy way of saying "breaking and entering", I think), and conversion of property to deny benefits to OCLC. Almost every claim raises novel legal questions that have not been definitively addressed by Ohio courts. As is the practice of federal courts in such cases, the judge decides to certify several "novel and unsettled" legal questions to the Supreme Court of Ohio, given the absence of clear precedent in state case law. Interestingly, this includes a question about the enforceability of "browserwrap" contracts—or terms of service that appear as links on the bottom of web pages. (This is contrasted with "clickwrap" contracts where the user must affirmatively click an "I agree" button or link.) Other questions include the definition of unjust enrichment in the context of data scraping and the interpretation of Ohio Revised Code § 2913.04 regarding unauthorized access to computer systems. If it were only that simple, though...the order also discusses potential preemption by federal copyright law, suggesting that OCLC’s claims may conflict with federal statutes, which complicates the legal landscape further.

There is a curious footnote at the end that makes me wonder if the judge is signalling to Matienzo's lawyers that there may be a way to get their client out of this sticky mess (legal citations removed):

As an aside, the Court also wonders whether the intracorporate conspiracy doctrine bars OCLC's conspiracy claims. Under that doctrine, an agreement between agents of the same legal entity is not an unlawful conspiracy. OCLC's conspiracy counts allege, in effect, that Matienzo is an agent of Anna's Archive who conspired with other agents of Anna's Archive to scrape WorldCat's data. If Anna's Archive is a legal entity, then OCLC may have alleged an intracorporate conspiracy. The Court pulls this thread no further (because it decides to certify). But, after the certification proceeding, the Court expects the parties will brief whether the intracorporate conspiracy doctrine applies here.

So it would seem that this case is not done, and the focus now shifts to Ohio's top state court. I don't know what the odds are that Ohio's court system will take up these questions or what happens if it declines to do so.

Mark your calendars! Next Friday, 28 March 2025, the Open Knowledge Foundation will host a new iteration of The Tech We Want initiative — our ambitious effort to reimagine how technology is built and used. This time we’ll focus on our decade-long Frictionless Data project, a progressive open-source framework for building data infrastructure – data management, data integration, data validation and data flows. The goal is to bring together maintainers, users, and contributors to discuss challenges, priorities, and governance for the project moving forward.

The Frictionless Data project aims to make it easier to work with data by reducing common data workflow issues (what we call friction). Its software and standards are the basis of many projects developed by the community all over the world, ranging from climate scientists, to humanities researchers, to government data centres. 

At the summit, we want to promote successful cases and projects, and collectively discuss how such a long-running project adapts to the current context and Open Knowledge’s current vision of technology.

What to Expect

We will start the event with short presentations of Frictionless use cases, in order to have some examples of how Frictionless is used today.

Open Knowledge’s Tech Lead, Patricio del Boca, will then briefly present The Tech We Want initiative, to share the current vision of technology that is driving all development efforts at OKFN today.

Following that, we will have a genuine community discussion on how to envision the future of the project with selected active members of the community leading the discussion.

Programme

15:00 CET – Frictionless Today (demos)

Here’s how we are using Frictionless today.

Confirmed speakers and projects:

• Pierre Camilleri, Johan Richer: Validata (government data)
• Romina Colman: Open Data Editor (data quality for non-technical audiences)
• Peter Desmet: Camtrap DP (bioinformatics)
• Nicholas Kellett: Deploy Solutions (climate/disaster response + citizen science)
• Phil Schumm: Data curation with Frictionless at NIH/University of Chicago (biomedical data)
• Adam Shepherd, Amber York: BCO-DMO (oceanography data)
• Ethan Welty: Global englacial temperature database (glaciar monitoring data)

16:30 CET – The Tech We Want Vision

A presentation about new practical ways to build software that is useful, simple, long-lasting and focused on solving people’s real problems.

• Patricio Del Boca – Open Knowledge Foundation

16:45 CET – Frictionless Tomorrow (community discussion)

What will Frictionless be like tomorrow and how will the current tech changes impact it?

Confirmed discussion leaders:

• Pierre Camilleri
• Peter Desmet
• Stephen Diggs
• Keith Hughitt
• Kyle Husmann
• Phil Schumm

Although it’s an event with and for the Frictionless community, the event is open to anyone interested, especially the open data community (technical and non-technical). If you use Frictionless Data or the Data Package in any project, please come along. If you want to learn more about how to optimise your data workflow and learn more about the future of the project, please come along too. 

For updates, follow the Open Knowledge Foundation on Mastodon, Bluesky, X and LinkedIn.

We can’t wait to see you there!

Average Fee

Mempool Count

• The number of pending transactions ramps up and peaks the day before the spike in fees.
• The number of pending transactions remains high for the days after the spike in fees.

Mempool Bytes

Transactions/Day

Bitcoin blocks now have a theoretical maximum size of 4 megabytes and a more realistic maximum size of 2 megabytes

Average $/Transaction

Median Wait

Average/Median Times

Professional bitcoin OTC traders FT Alphaville spoke with see this as an alarming development and one of the drivers of rival cryptocurrency ether’s growing popularity. The views of one trader:

It definitely tempts people into ether. This is the biggest problem with bitcoin, it’s not just that it’s expensive to transact, it’s uncertain to transact. It’s hard to know if you’ve put enough of a fee. So if you significantly over pay to get in, even then it’s not guaranteed. There are a lot of people who don’t know how to set their fees, and it takes hours to confirm transactions. It’s a bad system and no one has any solutions.

Transactions which fail to get the attention of miners sit in limbo until they drop out. But the suspended state leaves payers entirely helpless. They can’t risk resending the transaction, in case the original one does clear eventually. They can’t recall the original one either. Our source says he’s had a significant sized transaction waiting to be settled for two weeks.

The OCLC Research Library Partnership Metadata Managers Focus Group met in January 2025 to explore community cases for the use and re-use of linked data URIs found in MARC records. Members were invited to bring examples of work they are doing or have seen that activates linked data augmentations to MARC records in their discovery systems and integrations.

The round-robin of three separate discussions included 38 participants from 25 RLP institutions in 5 countries:

These discussions explored the technical and social challenges of transitioning to next-generation metadata formats. This blog post synthesizes the key topics that came up in the discussions, concerning URIs in MARC, the use of AI, and more general aspects of getting started with linked data in library metadata.

Background

Like many complex systems, our transition to the next generation of metadata has both technical and social challenges. Over the past decade, the metadata community has been on a journey to transform the way we work from “strings” of description in MARC records toward knowledge graph structures that identify different entities and their relationships. Because building a graph structure is a bit like building an arch—you need all the pieces in place for it to be self-supporting—we’ve also had to deploy scaffolding to temporarily support our transition. One piece of this scaffolding is “linky MARC,” that extends the MARC format to include linked data URIs. Emerging from the PCC Task Group on URIs in MARC, the inclusion of these URIs facilitates the move toward next-generation metadata formats such as BIBFRAME.

^{New London Bridge under construction. © The Trustees of the British Museum / CC BY-NC-SA 4.0}

As part of OCLC’s linked data strategy, we have added more than 500 million WorldCat Entities URIs to WorldCat records for five entity types: Persons, Organizations, Places, Events, and Works. Enriching WorldCat bibliographic records with WorldCat Entities URIs establishes a bridge between MARC data and linked data, providing a starting point for connecting data across local systems and workflows and for using linked data functionality, such as in local discovery systems. These URIs and associated WorldCat Entities data are free for anyone to retrieve and use, through our website or via API, under a Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC4.0) license. (Meridian subscribers also have creating/editing privileges).

While generative AI may be pulling at our attention, it’s important to remember that linked data also emerged from an earlier era of symbolic artificial intelligence systems—developers of the semantic web needed to present factual metadata statements and a mechanism for machines to interpret them in reasoning activities. In current conversations about AI, this has largely been replaced by the kinds of stochastic prediction methods offered by large-language models. However, the work the community has done on linked data over the last decade remains significant. A recent paper highlighted the strengths of knowledge graphs (aka linked data), search engines, and large-language models in addressing specific user needs. The authors suggest that integrating these technologies could lead to more effective solutions and emphasize the importance of considering the diverse information needs of users. They believe “that research on their combination and integration—with users in mind—will be particularly fruitful.” 

The January 2025 Metadata Managers Focus Group round-robin session focused on the following discussion prompts:

• What do you want to do with linked data URIs being added to MARC records?
• What are you doing that activates linked data augmentations to MARC records in your discovery systems and integrations?

The discussions revealed that members are navigating many issues:

• While linked data is of interest, regular work and Library Service Platform (LSP) migrations leave little room for experimentation or even adoption. Linked data is neither a driver for moving systems not supporting the move in any way.
• Metadata managers need tools that support seamless workflows, with a focus on data quality and trust in features and sources.
• While few attendees are yet using OCLC URIs, the need for linked data entities in repositories and cultural heritage collections is clear.
• Generative AI and machine learning present opportunities for creating and improving linked data. But metadata managers are also concerned about maintaining awareness of when AI has been used in metadata workflows.
• Participants wish to better understand what users want and need from linked data, especially as generative AI shakes up how researchers interact with our data.

Little time to think about linked data

While all participants were eager to learn about the benefits of URIs in MARC, most shared that there was little time to think about linked data. The immediate focus for many of them was the implementation of a new library service platform or the preparation for one. The latter would require prepping existing metadata for migration to the new platform in addition to keeping up with regular workflows, leaving little room for anything else.

This raises an important question about how the presence of linked data URIs in the metadata could make platform migrations smoother in the future. 

Participants also shared that, at this point in time, linked data is not yet a driver for switching LSPs.

URIs: Little experience, much curiosity

Few attendees had a clear vision for how they would use OCLC URIs in their systems and workflows, and how linked data features in their chosen LSP would take advantage of them.  Also, there was some confusion about why OCLC is adding URIs, and under which license terms these could be used.

However, participants noted the need to provide linked data entities for institutional repositories, data repositories, and cultural collections, and felt that WorldCat Entities could be useful in this area. Person and organization entities were of particular interest, less so subject or classification URIs.

Current linked data tools not yet convincing

A more modern LSP could be an opportunity to use MARC encoded URIs in new cataloging workflows or user-facing discovery features. However, several participants noted challenges of working with currently available linked data features. They are seeking seamless tools and finding disjointed environments that aren’t ready for production use.

For example, URIs could be used to expand information on a particular entity and expose that information to users in a knowledge card. This can be an easy configuration change in systems where this is a feature. According to our attendees who are exploring this, metadata quality is still a significant part of making such a feature accurate and useful. Anecdotally, they shared examples where Person cards pointed to the wrong entity or contained incorrect information. Since these may rely on external sources, such as Wikidata, it requires skill and knowledge to update the sources. Even then, it may take time for those changes to propagate to local environments and become visible to end users.

Organizations who have adopted open-source discovery layers may need to dedicate local resources to develop workflows and interfaces that take advantage of linked data URIs.

Even where linked data features are available, if the underlying metadata is not aligned, users may still need to navigate between silos to find what they are looking for. A common example is that it is not always possible to look across both monographs and article-level publications for the same person. Even when ORCIDs and ISNI persistent identifiers are available, they may not be integrated into a single entity. When expanding beyond just bibliographic materials to include art and special collections, similar challenges remain when entity information hasn’t been unified.

This lack of linked data integration with current systems, tools, and workflows is a barrier to adoption, as is the lack of trust in existing linked data features or sources.

Participants also desire to better understand user needs to inform their work. None of our participants actively pursued systematic studies of how their users take advantage of entity information or browse, and there was a consensus that user studies are needed in this area. In addition, much of what we’ve previously learned about users that inform current discovery experiences is being shaken up by access to interactive AI chatbots. We flagged this topic as something to explore in a future session.

Opportunities and concerns about AI-generated metadata

An ongoing concern among metadata managers is the role that AI will play in future cataloging and metadata workflows, especially as we move toward linked data approaches. During our discussion, metadata managers wanted to better understand how we can leverage AI’s advantages responsibly, in part by considering how best to inform others (catalogers, users, etc.) about the provenance of metadata records.

Balancing AI opportunities with quality

Participants with access to emerging AI cataloging assistants shared how these tools can help bootstrap brief records that are augmented in later human-centered workflows. Especially given that both general and special collections face significant backlogs (see our previous discussion about this in Keeping up with next-generation metadata for special collections and archives), having an AI assistant do the initial work of transcribing information from print would save time. Whether we can trust that these agents produce records of sufficient quality remains an area of study and interest.

Needs for consistent AI data provenance

A significant question that emerged in our conversation about the use of AI is how we are providing data provenance statements and at what level of granularity they are needed. For the agents mentioned above and in other early AI-driven MARC workflows, a consensus has developed around using a 588 Source of Description note to indicate that AI was used to create the description.

Because linky-MARC enables us to record a URI alongside textual labels, these URIs can also serve as a guide to data provenance. Participants noted that an LC/NACO Authority File record contains a great deal of information about how the authority was established and provides human-readable information that catalogers can use to make judgments about whether to trust it. This kind of trust can easily be extended to the linked data entities and URIs at id.loc.gov. How, then, can we express our confidence in different linked data sources in a way that replicates catalogers’ professional judgments, especially if those judgements were made by an AI agent? When presented with repeated entities that use different URIs, how do we choose among them? As one participant suggested, there may not be one single answer to this question. Rather, we may need to develop application profiles that express different levels of trust for different environments and use cases.

Final thoughts

This blog post started by talking about how we’re building a linked data ecosystem that requires some scaffolding to support emergent data structures and workflows. In addition to shepherding their organizations onto new platforms, LLM-powered AIs are disrupting how we think about discovery and meeting user needs. Just as Google search upturned our concept of the OPAC and ushered us into an era of modern discovery layers, AI is challenging us to reimagine what we are doing with linked data. An example of this is the Library of Congress’ “Modern MARC” approach, which will further extend “linky” MARC by including additional identifiers and embracing linked data modeling choices that are better suited for translation into knowledge graphs. Beyond displaying knowledge cards in user interfaces, knowledge graph structures can fulfill their original promise by minimizing LLM hallucinations through metadata trusted by the humans in the loop.     

Writing this blog post was a collaborative effort. Many thanks to all those who contributed, in particular my colleagues Rebecca Bryant, David Heimann, Erica Melko, Mercy Procaccini, Merrilee Proffitt and Chela Weber.

The post Activating URIs in linky MARC: an OCLC RLP discussion summary appeared first on Hanging Together.

During my sabbatical last year, one of the projects I worked on involved collecting images and videos from Instagram daily. I focused on US and Canadian political leaders, aiming to align this project with the 2024 US presidential election and the upcoming 2025 Canadian Federal Election.

My UWaterloo DigFemNet colleagues and I were looking to track misogyny and hate via hashtags associated with candidates and party leaders, so I created a small Python utility that I called digfemig. It grabbed the top 100 media for a given Instagram hashtag, and I ran it each day on a cron job. The utility is basically a wrapper around some core functionality in instagrapi.

It worked for a time – August to mid-December 2024 – until Instagram/Meta banned my collecting account and my dog’s account for a short period of time 😅. We’d hoped to collect at least through the inauguration in January 2025 (at which time I quit social media which is another story), and possibly through the 2025 Canadian Federal Election. But alas, after being banned multiple times, I decided to give up.

The awesome team of folks I’m working with find the juxtas I’ve created in the past really useful, so I pulled together the images I was able to grab, and updated juxta to work with the csv files I produced with digfemig. If you’re curious, you can check them all out on the “Insta-Politics: Visualizing Political Leaders via Collages” page I created for the project.

Hopefully we’ll have more to say about these images in the future!

Rest easy Toke. I miss you buddy 😭.

Social media saturates nearly every facet of our lives, and understanding its effects on society has never been more critical. This week's DLTJ Thursday Threads delves into recent studies and discussions of why misinformation is spread on platforms and ways to counteract it. As platforms continue to shape the way we communicate and process information, they also spark moral outrage and other intense emotions that can lead to the further spread of false content. Researchers are exploring how these dynamics unfold, as well as the roles of opportunists who exploit these platforms for personal or political gain.

As we navigate these challenges, there are things that individuals can do and things that we could expect platforms to do to reduce the impact of misinformation. While individuals can adopt practices to avoid contributing to misinformation, there is also a call for platforms to refine their moderation strategies, such as combining fact-checking with community-driven initiatives. Amidst these discussions, the potential impact of social media on adolescent wellbeing remains a concern, with experts debating its true role in rising mental health issues among young adults.

• Did you really read that article? Moral Outrage fuels the spread of misinformation online.
• Maybe that outrageous article wasn't pushed to you because of moral outrage. It could be opportunists exploiting online conspiracy theories for influence and profit.
• We can clean up social media from the ground up: strategies to avoid becoming a 'misinformation superspreader' on social media.
• For a more top-down approach, we could insist that platforms combine fact-checking and community notes for better social media content moderation.
• On the other hand, research showed that the community notes system fails to curb misinformation on social media.
• Exploring the complex impact of social media on teen mental health.
• This Week I Learned: most plastic in the ocean isn't from littering, and recycling will not save us.
• This week's cat

Also on DLTJ this past week:

• Another Saturday, another #TeslaTakedown

Feel free to send this newsletter to others you think might be interested in the topics. If you are not already subscribed to DLTJ's Thursday Threads, visit the sign-up page. If you would like a more raw and immediate version of these types of stories, follow me on Mastodon where I post the bookmarks I save. Comments and tips, as always, are welcome.

Moral Outrage Fuels Spread of Misinformation Online

The article discusses the phenomenon of misinformation spreading online, particularly when it evokes "moral outrage." It starts with a fabricated quote attributed to Rob Bauer, chair of a NATO military committee, suggesting that NATO should preemptively strike Russia, which garnered significant attention despite being untrue. The misinformation received nearly 250,000 views on social media, amplified by figures like Alex Jones. This research challenges the common assumption that misinformation is primarily shared by mistake; rather, it suggests that outrage drives people to share content without verifying its accuracy. Because of that, traditional solutions aimed at promoting accuracy in sharing have proven ineffective.

Opportunists Exploit Online Conspiracy Theories for Influence and Profit

The article explores the phenomenon of individuals who share conspiracy theories online without genuinely believing in them. The study identified various types of conspiracy-spreaders, including extremist groups that use conspiracies as a gateway for radicalization, governments that manipulate narratives for political gain, and others who do it for profit or to gain influence. Many everyday users even share conspiracies for attention or social validation, often without verifying the information. The article warns that these opportunists can eventually convince themselves of their own lies.

Strategies to Avoid Becoming a 'Misinformation Superspreader' on Social Media

Whether your social media feed is filled with content driven by moral outrage, attempts to influence you, or schemes to profit from you, there are ways that you can break the cycle of spreading this misinformation. The strategies are: educate yourself about common disinformation tactics, recognize your own vulnerabilities and biases, carefully evaluate the credibility of information sources, pause before sharing content, be aware of how emotions can influence the spread of misinformation, gently challenge the misinformation you see, and stand with others when you see someone else challenge it. The article emphasizes that while there is no perfect solution, these steps can help protect individuals and their social networks from the harmful effects of false and misleading information.

Combine Fact-Checking and Community Notes for Better Social Media Content Moderation

This is just a brief article from the Center for Information Technology Policy, but one to go to for links to other research and researchers. It was published around the same time as the next article that highlights deficiencies in the community-notes approach. Researchers from CITP suggest that both fact-checking and community notes can be complementary rather than mutually exclusive. Ultimately, the discussion reflects a need for a more nuanced understanding of these moderation strategies in combating disinformation effectively.

Community Notes System Fails to Curb Misinformation on Social Media

This article is marked as an opinion piece in Bloomberg, but it is well researched with lots of hyperlinks to source material. It discusses the limitations of the Community Notes system implemented by social media platforms like Ex-Twitter and Meta to combat misinformation. Community Notes allows users to provide context and evidence to posts, promoting a sense of democratized information sharing. However, the article argues that this approach falls short of addressing the deeper issues inherent in social media's structure. The authors suggest that without significant changes to how these platforms operate, even innovative systems like Community Notes cannot resolve the ongoing challenges of misinformation. Ultimately, the piece highlights the struggle between user-generated content and the need for reliable information in digital spaces.

Exploring the Complex Impact of Social Media on Teen Mental Health

This is a review of The Anxious Generation: How the Great Rewiring of Childhood is Causing an Epidemic of Mental Illness, which was published a year ago. The article suggests that while social media can foster connections, it often fails to provide genuine social interaction, potentially leading to feelings of isolation and mental distress. Various studies are cited to examine the relationship between social media usage and mental health outcomes, indicating that the effects are complex and multifaceted. The discussion also addresses the need for further research to understand the nuances of this relationship.

This Week I Learned: Most plastic in the ocean isn't from littering, and recycling will not save us

Contrary to popular belief, most plastic in the Great Pacific Garbage Patch stems from the fishing industry, with only a small fraction linked to consumer waste. The video highlights that mismanaged waste, rather than individual littering, is the primary contributor to plastic pollution, with 82% of macroplastic leakage resulting from this issue. It emphasizes the ineffectiveness of recycling as a solution, noting that less than 10% of plastics are currently recycled, and the industry has perpetuated the myth that recycling can resolve the plastic crisis. Microplastics, which are increasingly recognized as a major problem, originate from various sources, including tires and paint, with new data suggesting that paint is a significant contributor. The video stresses the need for systemic changes, including reducing plastic production and simplifying the chemicals used in plastics, as current efforts and pledges from corporations and governments have not effectively curbed plastic pollution.

What did you learn this week? Let me know on Mastodon or Bluesky.

Alan snoozing behind me

This text, part of the #ODDStories series, tells a story of Open Data Day‘s grassroots impact directly from the community’s voices.

In a groundbreaking event held in Owerri, Imo State, journalists from various media organizations gathered for the Open Data Day 2025 workshop, aimed at equipping them with the necessary skills to utilize open data in climate and agricultural reporting. The workshop, hosted by Harriet Ijeomah, Managing Director of Harsco Media & Communications, was supported by Open Knowledge Foundation (OKFN) and Dataopian.

The event began with Harriet Ijeomah introducing the key supporters and emphasizing the crucial role of open data in tackling climate change and agricultural challenges in Nigeria. In her opening speech, she highlighted how Imo State has been severely impacted by extreme weather events, including flooding, excessive rainfall, and soil erosion. She referenced data from 2024, which showed that flooding displaced 7,832 people and affected 60,761 others in the state, while 16% of erosion incidents in Owerri were directly linked to intense rainfall.

Speaking on the significance of Open Data Day 2025, Ijeomah underscored the importance of open data in investigative journalism, stating, “Access to accurate and well-structured data is essential for journalists to tell compelling and solution-driven stories that can influence policy changes and drive sustainable development.”

Deep Dive into Data Reporting

The first session was led by Udochukwu Chukwu, a research and data expert, who delivered an extensive lecture on data reporting for agriculture and climate issues. He introduced participants to various sources of open data, with a strong emphasis on the Open Data Editor, showcasing its functionality and practical applications for journalists. Chukwu explained how data-driven journalism can help uncover hidden trends in climate change and agricultural production, making reference to real-world examples.

To reinforce the learning process, he facilitated a group exercise where journalists analyzed open datasets and developed reports on climate-related challenges in Imo State. These reports were designed to serve as a foundation for further investigative stories, bridging the gap between raw data and impactful storytelling.

Practical Insights from Experienced Journalists

Renowned development journalist Chinedu Hardy Nwadike took the stage to share his expertise on open data’s role in solution journalism. He stressed the need for journalists to harness available data in holding policymakers accountable and addressing key societal challenges. Hardy illustrated practical strategies for integrating data visualization into reports, ensuring that complex information is easily understood by the public.

The final session featured an insightful presentation by Adanna Ononiwu, who spoke on “How Journalists Can Position Themselves for International Opportunities” and “Leveraging Development Reporting as a Community Journalist.” She highlighted how data-driven journalism not only enhances the credibility of a journalist but also opens doors to global collaborations and funding opportunities. Adanna encouraged journalists to explore grants, fellowships, and networking platforms that support investigative and development reporting.

Engaging Discussions and Networking

Beyond the technical sessions, the event provided a platform for networking and collaboration among media professionals. The interactive discussions allowed participants to share their experiences, ask questions, and exchange ideas on innovative approaches to data journalism.

As the workshop concluded, journalists expressed their appreciation for the training, emphasizing how it broadened their perspective on using data to drive impactful storytelling. A key takeaway from the event was the realization that open data is a vital tool in investigative journalism, and when properly utilized, it can influence policies, raise awareness, and drive sustainable solutions in climate and agriculture.

Next Steps and Future Engagements

The event ended with a call to action for journalists to actively incorporate open data in their reporting and explore further training opportunities. The organizers assured continued engagement with participants through follow-up sessions, access to online resources, and mentorship programs to ensure sustained impact beyond the workshop.

With Open Knowledge and Dataopian supporting the initiative, the event marked a significant step toward strengthening the capacity of journalists in Imo State to use data for meaningful storytelling, ultimately contributing to informed decision-making and policy advocacy in Nigeria’s climate and agricultural sectors.

About Open Data Day

Open Data Day (ODD) is an annual celebration of open data all over the world. Groups from many countries create local events on the day where they will use open data in their communities. ODD is led by the Open Knowledge Foundation (OKFN) and the Open Knowledge Network.

As a way to increase the representation of different cultures, since 2023 we offer the opportunity for organisations to host an Open Data Day event on the best date over one week. In 2025, a total of 189 events happened all over the world between March 1st and 7th, in 57 countries using 15+ different languages. All outputs are open for everyone to use and re-use.

For more information, you can reach out to the Open Knowledge Foundation team by emailing opendataday@okfn.org. You can also join the Open Data Day Google Group or join the Open Data Day Slack channel to ask for advice, share tips and get connected with others.

DLF is pleased to announce that we have opened the Call for Proposals for the in-person DLF Forum and Learn@DLF, happening in Denver, Colorado, November 16-19, 2025.

We encourage proposals from members and non-members; regulars and newcomers; digital library practitioners from all sectors (higher education, museums and cultural heritage, public libraries, archives, etc.) and those in adjacent fields such as institutional research and educational technology; and students, early- and mid-career professionals and senior staff alike. We especially welcome proposals from individuals who bring diverse professional and life experiences to the conference, including those from underrepresented or historically excluded racial, ethnic, or religious backgrounds, immigrants, veterans, those with disabilities, and people of all sexual orientations or gender identities. 

Learn more about our event and session formats, view the Call for Proposals, and submit.

The submission deadline for all proposals is Monday, April 14, at 11:59pm Mountain Time.

If you have any questions, please write to us at forum@diglib.org. We’re looking forward to seeing you in Denver this fall.

All best,

Team DLF

P.S. Want to stay updated on all things #DLFforum? Subscribe to our Forum newsletter.

The post Be a Part of the 2025 DLF Forum: Submit Your Proposal Today appeared first on DLF.

The following post is one in a regular series on issues of Inclusion, Diversity, Equity, and Accessibility, compiled by a team of OCLC contributors.

^{Photo by Delaram Bayat on Unsplash}

Women’s History Month 2025

In the United States, March has been commemorated as Women’s History Month since 1987. Shorter observances go back to at least 1911, when 8 March was first declared as International Women’s Day. Each March, Toward Inclusive Excellence (TIE), the blog from ALA’s Association of College and Research Libraries (ACRL), collects all kinds of resources to keep libraries — especially but not just those that serve higher education — informed about important ideas and initiatives. “Commemorating Women’s History Month with TIE and Choice Content” brings together recent book reviews, blogs, webinars, podcasts, interviews, and other resources to help librarians, researchers, and other interested readers to prepare, learn, and study.

This annual service provided by ACRL’s Choice publishing unit may be more consequential than ever. “Moving Forward Together! Women Educating and Inspiring Generations” is the 2025 theme of National Women’s History Month as declared by the National Women’s History Alliance. Under its former name, the National Women’s History Project, NWHA was instrumental in establishing the commemorative month in 1987. Then as now, the organization has been devoted to “writing women back into history.” As part of that effort, OCLC worked with NWHA in 2023 to create the ”Women’s History” topic page on WorldCat.org. Thanks to the dynamic nature of the WorldCat bibliographic database, the preprogrammed searches are as up-to-date as WorldCat itself. Contributed by Jay Weitz.

A focus on elders and elder justice in libraries

When I read “Because I am Old” in the title of an academic journal article, it caught my attention–as it was intended to do. I know that older adults makeup a sizeable portion of active public library users and catering programs and services for them makes sense. The article describes the specific motivation for seniors to attend digital literacy training programs, which are all the things you would expect like feeling connected to society and family members and feeling ill-equipped to proficiently use information technology. Giving older adults a safe space to experiment is key to their success and boosts confidence. Libraries being that safe space feels natural, and many are already committed to serving this population. ALA’s new Elder Justice Task Force (EJTF) brings together librarians and advocates to centralize resources and learning around the myriad issues facing older adults. The EJTF will support librarians to create safe spaces for digital literacy and health and wellness courses that are sensitive to societal and internalized age discrimination. As noted in the Public Libraries Online article about the EJTF, serving older adult populations is often skipped in library science education and with the expected growth in this demographic the task force wants to aid librarians as they transition to a service model that cares for older adults at higher rates than children, young adults or adults.

There is a part of me that always has patience for people who need extra help. Sometimes that is people with disabilities, sometimes that is children and older adults. It’s a desire to be inclusive and caring in ways that society often moves past and forgets about, especially in the development of information technology. The EJTF fills a major gap in our social and library instruction, one that will affect the library profession in major ways. My hope is that the legacy of EJTF is that the library profession becomes more caring and inclusive of older adults in library programming.  Contributed by Lesley A. Langa.

American Library Association / Gallup research on diversity in librarianship

A new report from the American Library Association and Gallup, Empowering Voices, Inspiring Change: Advancing Diversity Within Librarianship, spotlights one of the profession’s most persistent issues, diversity in the library workforce. Based on interview and survey data (and drawing on previous research as well as workforce data), the report presents a nuanced view of the profession as it stands and makes recommendations for improvement.

One of the things I appreciated most about this report is that it digs into the concept of diversity as a nuanced and complex issue. The library workforce is not only white, but also overwhelmingly female, aging and lacking socioeconomic diversity. The profession faces issues around retention, degree attainment for new workers, and also faces issues due to persistent stereotypes around who librarians are, but what library work entails. One interviewee commented, “There’s this misconception that librarians are agreeable, nice, they drink tea and like cats.” As a dog-loving, non-tea drinker this resonates with me; similarly, my work had fallen outside of what the public would imagine as library work. I also appreciated the report’s focus on the intersectional experiences and the need for representation from multiple facets including neurodiversity. Contributed by Merrilee Proffitt. 

ALA president shares experience with autism

American Library Association (ALA) president Cindy Hohl shared that that she is one of the millions of people with autism spectrum disorder (ASD) in her 3 March 2025 message Spectrum of Leadership. The World Health Organization estimates that one child in 100 is autistic. It does not provide estimates on how many autistic adults there are in the world, but it seems reasonable to assume a similar figure for adults. Autistic adults may not disclose their status because of the social stigmas. ALA’s Reference and User Services Association has a webpage about ASD designed to provide information about providing services to ASD. While the page uses “may” language as in “a person with ASD may be unusually sensitive to smells,” this description of ASD patrons does not mention a single trait that might be seen as positive. It has instructions about contacting security or police to remove a person with ASD “in extreme cases” and informing law enforcement “that this person has a developmental disability and is not a criminal.” The tone of this page is a sharp contrast to Hohl’s description of ASD. Describing how she struggled with school, she writes, “I was not prepared to see my place in this world since I was focused on living in my own space and time, far from the societal pressures to be perfect. Turns out, I’m okay, and so are you.”

Hohl’s message is an important reminder that autistic people can be successful in a variety of careers, may communicate effectively with individuals and large groups, and may have any of the personality strengths that a neurotypical person does. As Hohl notes, the adversity she has experienced has taught her to use empathy when interacting with others. Hohl’s message reminded me of this quotation from the character Raymond Holt in the television show Brooklyn Nine-Nine: “Every time someone steps up and says who they are, the world becomes a better, more interesting place.” Contributed by Kate James.

The post Advancing IDEAs: Inclusion, Diversity, Equity, Accessibility, 18 March 2025 appeared first on Hanging Together.

I work at a non-profit academic institution, on a site that manages, searches, and displays digitized historical materials: The Science History Institute Digital Collections.

Much of our stuff is public domain, and regardless we put this stuff on the web to be seen and used and shared. (Within the limits of copyright law and fair use; we are not the copyright holders of most of it). We have no general problem with people scraping our pages.

The problem is that, like many of us, our site is being overwhelmed with poorly behaved bots. Lately one of the biggest problems is with bots clicking on every possible combination of facet limits in our “faceted search” — this is not useful for them, and it overwhelms our site. “Search” pages are one of our most resource-constrained category of page in our present site, adding to the injury. Peers say even if we scaled up (auto or not) — the bots sometimes scale up to match anyway!

One option would be putting some kind of “Web Application Firewall” (WAF) in front of the whole app. Our particular combination of team and budget and platform (heroku) makes a lot of these options expensive for us in licensing, staff time to manage, or both. Another option is certainly putting the the whole thing behind (ostensibly free) CloudFlare CDN and using its built-in WAF, but we’d like to avoid giving our DNS over to CloudFlare, I’ve heard mixed reviews of CloudFlare free staying free, and generally am trying to avoid contributing to CloudFlare’s monopoly unaccountable control of the internet.`

Although ironically then, the solution we arrived at is still using CloudFlare, but Cloudflare’s Turnstile “captcha replacement”, one of those things that gives you the “check this box” or more often entirely interactive “checking if you are a bot” UXs.

[If you’re a tldr look at the code type, here’s the initial implementation PR in our open repo, there are some bug fixes since then
Update March 18 2025: There is now a gem implementation, bot_challenge_page. It is pre-1.0 and still evolving as we learn more about the problem space]

While this still might unfortunately lock people using unconventional browsers etc out (just the latest of many complaints on HackerNews), we can use this to only protect our search pages. Most of our traffic comes directly from Google to an individual item detail page, which we can now leave completely out of it. We have complete control of allow-listing traffic based on whatever characteristics, when to present the challenge, etc. And it turns out we had a peer at another institution who had taken this approach and found it successful, so that was encouraging.

How it works: Overview

While typical documented Turnstile usage involves protecting form submissions, we actually want to protect certain urls, even when accessed via GET. Would this actually work well? What’s the best way to implement it?

Fortunately, when asking around on a chat for my professional community of librarian and archivist software hackers, Joe Corall from Lehigh University said they had done the exact same thing (even in response to the same problem, bots combinatorially exploring every possible facet value), and had super usefully written it up, and it had been working well for them.

Joe’s article and the flowchart it contains is worth looking it. His implementation is as a Drupal plugin (and used in at least several Islandora instances); the VuFind library discovery layer recently implemented a similar approach. We have a Rails app, so needed to implement it ourselves — but with Joe paving the way (and patiently answering our questions, so we could start with the parameters that worked for him), it was pretty quick work, bouyed by the confidence this approach wasn’t just an experiment in the blue, but had worked for a similar peer.

• Meter the rate of access, either per IP address, or as Joe did, in buckets per sub-net of client IP address.
• Once client has crossed a rate limit boundary (in Joe’s case 20 requests per 24 hour period), redirect them to a page which displays the Turnstile challenge — and has the original destination in a query param in url —
• Once they have passed the Turnstile challenge, redirect them back to their original destination, which now lets them in because you’ve stored their challenge pass in some secure session state.
• In that session state record that they passed, and let them avoid a challenge again for a set period of time.

Joe allow-listed certain client domain names based on reverse IP lookup, but I’ve started without that, not wanting the performance hit on every request if I can avoid it. Joe also allow-listed their “on campus” IPs, but we are not a university and only have a few staff “on campus” and I always prefer to show the staff the same thing our users are seeing — if it’s inconvenient and intolerable, we want to feel the pain so we fix it, instead of never even seeing the pain and not knowing our users are getting it!

I’m going to explain and link to how we implemented this in a Rails app, and our choices of parameters for the various parameterized things. But also I’ll tell you we’ve written this in a way that paves the way to extracting to a gem — kept everything consolidated in a small number of files and very parameterized — so if there’s interest let me know. (Code4Lib-ers, our slack is a great place to get in touch, I’m jrochkind).

Ruby and Rails details, and our parameters

Here’s the implementing PR. It is written in such a way to keep the code conslidated for future gem extraction, all in the BotDetectController class, which means kind of weirdly there is some code to inject in class methods in the controller. While it does turnstile now, it’s written with variable/class names such that analagous products could be made available.

Rack-attack to meter

We were already using rack-attack to rate-limit. We added a “track” monitor with our code to decide when a client had passed a rate-limit gate to require a challenge. We start with allowing 10 requests per 12 hours (Joe at Lehigh did 20 per 24 hours), batched together in subnets. (Joe did subnets too, but we do smaller /24 (ie x.y.z.*) for ipv4 instead of Joe’s larger /16 (x.y.*.*)).

Note that rack-attack does not use sliding/rolling-windows for rate limits, but fixed windows that reset after window period. This makes a difference especially when you use such a long period as we are, but it’s not a problem with our very low count per period, and it does keep the RAM extremely effiicent (just an integer count per rate limit bucket).

When the rate limit is reached, the rack-attack block just sets a key/value in the rack_env to tell another component that a challenge is required. (setting in the session may have worked, but we want to be absolutely sure this will work even if client is not storing cookies, and this is really only meant as this-request state, so rack env seemed the good way to set state in rack-attack that could be seen in a rails controller)

Rails before_action filter to enforce challenge

There’s a Rails before_action filter that we just put on the application-wide ApplicationController, that looks for the “bot challenge key” required in the rack env — if present, and there isn’t anything in the session saying they have already passed a bot challenge, then we redirect to a “challenge” page, that will display/activate Turnstile.

We simply put the original/destination URL in a query param on that page. (And include logic to refuse to redirect to anything but a relative path on same host, to avoid any nefarious uses).

The challenge controller

One action in our BotDetectController just displays the turnstile challenge. The cloudflare turnstile callback gives us a token we need to verify server-side with turnstile API to verify challenge was really passed.

the front-end does a JS/xhr/fetch request to the second action in our BotDetectController. The back-end verify action makes the API call to turnstile, and if challenge passed, sets a value in Rails (encrypted and signed, secure) session with time of pass, so the before_action guard can give the user access.

if the JS in front gets a go-ahead from back-end, it uses JS document.replace to go to original destination. This conveniently removes the challenge page from the user’s browser history, as if it never happened, browser back button still working great.

In most cases the challenge page, if non-interactive, wont’ be displayed for more than a few seconds. (the language has been tweaked since these screenshots).

We currently have a ‘pass’ good for 24 hours — once you pass a turnstile challenge, if your cookies/session are intact, you won’t be given another one for 24 hours no matter how much traffic. All of this is easily configurable.

If the challenge DOES fail for some reason, the user may be looking at the Challenge page with one of two kinds of failures, and some additional explanatory text and contact info.

Limitations and omissions

This particular flow only works for GET requests. It could be expanded to work for POST requests (with an invisible JS created/submitted form?), but our initial use case didn’t require it, so for now the filter just logs a warning and fails for POST.

This flow also isn’t going to work for fetch/ajax requests, it’s set up for ordinary navigation, since it redirects to a challenge then redirects back. Our use case is only protecting our search pages — but the blacklight search in our app has a JS fetch for “facet more” behavior. Couldn’t figure out a good/easy way to make this work, so for now we added an exemption config, and just exempt requests to the #facet action that look like they’re coming from fetch. Not bothered that an “attacker” could escape our bot detection for this one action; our main use case is stopping crawlers crawling indiscriminately, and I don’t think it’ll be a problem.

To get through the bot challenge requires a user-agent to have both JS and cookies enabled. JS may have been required before anyway (not sure), but cookies were not. Oh well. Only search pages are protected by the bot challenge.

The Lehigh implementation does a reverse-lookup of the client IP, and allow-lists clients from IP’s that reverse lookup to desirable and well-behaved bots. We don’t do that, in part because I didn’t want the performance hit of the reverse-lookup. We have a Sitemap, and in general, I’m not sure we need bots crawling our search results pages at all… although I’m realizing as I write this that our “Collection” landing pages are included (as they show search results)… may want to exempt them, we’ll see how it goes.

We don’t have any client-based allow-listing… but would consider just exempting any client that has a user-agent admitting it’s a bot, all our problematic behavior has been from clients with user-agents appearing to be regular browsers (but obviously automated ones, if they are being honest).

Possible extensions and enhancements

We could possibly only enable the bot challenge when the site appears “under load”, whether that’s a certain number of overall requests per second, a certain machine load (but any auto-scaling can make that an issue), or size of heroku queue (possibly same).

We could use more sophisticated fingerprinting for rate limit buckets. Instead of IP-address-based, colleague David Cliff from Northeastern University has had success using HTTP user-agent, accept-encoding, and accept-language to fingerprint actors across distributed IPs, writing:

I know several others have had bot waves that have very deep IP address pools, and who fake their user agents, making it hard to ban.

We had been throttling based on the most common denominator (url pattern), but we were looking for something more effective that gave us more resource headroom.

On inspecting the requests in contrast to healthy user traffic we noticed that there were unifying patterns we could use, in the headers.

We made a fingerprint based on them, and after blocking based on that, I haven’t had to do a manual intervention since.

def fingerprint
result = “#{env[“HTTP_ACCEPT”]} | #{env[“HTTP_ACCEPT_ENCODING”]} | #{env[“HTTP_ACCEPT_LANGUAGE”]} | #{env[“HTTP_COOKIE”]}”
Base64.strict_encode64(result)
end

…the common rule we arrived at mixed positive/negative discrimination using the above

request.env["HTTP_ACCEPT"].blank? && request.env["HTTP_ACCEPT_LANGUAGE"].blank? && request.env["HTTP_COOKIE"].blank? && (request.user_agent.blank? || !request.user_agent.downcase.include?("bot".downcase))

so only a bot that left the fields blank and lied with a non-bot user agent would be affected

We could also base rate limit or “discriminators” for rate limit buckets on info we can look up from the client IP address, either a DNS or network lookup (performance worries), or perhaps a local lookup using the free MaxMind databases that also include geocoding and some organizational info.

Does it work?

Too early to say, we just deployed it!

I sometimes get annoyed when people blog like this, but being the writer, I realized that if I wait a month to see how well it’s working to blog — I’ll never blog! I have to write while it’s fresh and still interesting to me.

But encouraged that colleagues say very similar approaches have worked for them. Thanks again to Joe Corral for paving the way with a drupal implementation, blogging it, discussing it on chat, and answering questions! And all the other librarian and cultural heritage technologists sharing knowledge and collaboration on this and many other topics!

I can say that already it is being triggered a lot, by bots that don’t seem to get past it. This includes google bot and Meta-ExternalAgent (which I guess is AI-related; we have no particular use-based objections we are trying to enforce here, just trying to preserve our resources). While Google also has no reason to combinatorially explore every facet combination (and has a sitemap), I’m not sure if I should exempt known resource-considerate bots from the challenge (and whether to do so by trusting user-agent or not; our actual problems have all been with ordinary-browser-appearing user-agents).

Update 27 Jan 2025

Our original config — allowing 10 search results per IP subnet before turnstile challenge — was not enough to keep the bot traffic from overwhelming us. Too many botnets had enough IPs making apparently fewer than 10 requests each.

Lowering that to 2 requests was enough to reduce traffic enough. (Keep in mind that a user should only get one challenge per 24 hours unless IP address changes — although that makes me realize that people using Apple’s “private browsing” feature may get more, hmm).

Pretty obvious on these heroku dashboard graphs where our succesful turnstile config was deployed, right?

I think I would be fine going down to challenge on first search results, since a human user should still only get one per 24 hour period — but since the “success passed” mark in session is tied to IP address (to avoid session replay for bots to avoid the challenge), I am now worried about Apple “private browsing”! In today’s environment with so many similar tests, I wonder if private browsing is causing problems for users and bot protections?

You can see on the graph a huge number of 3xx responses — those are our redirects to challenge page! The redirect to and display of the challenge page seem to be cheap enough that they aren’t causing us a problem even in high volume — which was the intent, nice to see it confirmed at least with current traffic.

We are only protecting our search result page, not our item detail pages (which people often get to directly to google) — this also seems succesful. The real problem was the volume of hits from so many bots trying to combinatorially explore every possible facet limit, which we have now put a stop to.

When envisioning a high-tech makerspace, you might first think about younger customers; however, since opening in December 2021, Carroll County Public Library’s Exploration Commons at 50 East has created a place of learning and sharing that knows no age limit. From programs designed for seniors, resources for entrepreneurs, and opportunities for volunteering, Exploration Commons provides a center for intergenerational knowledge exchange where our community’s senior population thrives. 

The open movement at large is one of radical redistribution of power. However, as it currently stands, access advocacy often falls short when it comes to individuals within the prison industrial complex. The pervasive assumption that open means accessible inhibits scholarship, especially for justice-impacted scholars who do not have "regular" internet access. While this is especially difficult for students experiencing incarceration, a lack of equitable access is an issue for all library users. This is a reflection on open access, intellectual freedom, digital equity, and the duty of librarians to be mindful of access barriers in conversations surrounding these topics, as well as an overview of the technology and scholarship access within the prison industrial complex. It's informed based on my own experience with justice-impacted scholars in California State Prisons as well as current literature around education programs and digital (in)equity issues for those currently and recently incarcerated. It is a call for furthering our discussions and advocacy by acknowledging current limitations in order to appreciate the full potential of openness as an equalizer, a philosophy and practice that promotes information equity for all.

This paper presents a case study about opportunities for libraries to elevate reporting on their organization's financial status, transactions, and available funds using data visualization, specifically with Power BI, based on work at University of Oregon Libraries. The Power BI dashboard aggregates data from the university’s financial system and integrated library system to provide a consolidated view, and to tell the library's story in a more engaging way. The author proposes using this type of business intelligence technology to provide a transformative impact on financial operations of libraries.

My protest sign for the #TeslaTakedown today.

There was another #TeslaTakedown march today. It was a windy, gloomy day; despite that, lots of spirit at the Easton Tesla store. I was expecting the weather would keep people away, but there were about 400 people — 50 or so more than last Saturday.

There were three differences that I noticed. First, there weren't as many families with young children. I would chalk that up to the weather; if it were nicer, there probably would have been more. A week out, next Saturday's weather forecast is slightly cooler but sunny. We'll see what happens then.

The second difference was the increased police presence. They probably were expecting more people, and so had a bigger group there. Most wore pale blue "Columbus Police Dialogue Team" high-vis vests and walked behind us. The only interaction with them that I saw was when a young adult walked along the sidewalk shouting "Trump 2028!" with his phone up recording. As far as I could tell, the Dialogue Team was the only one to engage with this person.

The third difference was the support from the cars driving by...much more supportive with honking and thumbs up. I counted three middle fingers, which is one more than last weekend. Not a bad ratio for an hour-long protest.

I think the construction of my protest sign is improving. I noticed when I got there that I still had too many words in the sign to make it easy to read and understand at a distance. At least this one didn't require a blog post to fully explain.

No one knows how to use a point-and-shoot camera

My protest sign at the #TeslaTakedown.

I've had to ask people to take these pictures of me with the protest sign in front of the Tesla store. And everyone says they don't know how to use a camera anymore. Even people my age and older!

I brought an old point-and-shoot camera because it doesn't have any radios in it. One of the guidelines I've read for attending a protest is not to bring devices that can identify you, although I might be the only person following that guidance. If I were a technical Elon-ite, I'd recommend that the Tesla store's WiFi and Bluetooth hotspots capture metadata of every device they encounter. Although both Android and iOS use network address randomization, those are not foolproof for preventing devices from being tracked.

Now, admittedly, I'm not hiding from Elon Goons by posting this on my personal blog. Still, I'm also not going to make it easy for them to automate finding me either.

Introduction

Backups

• I run my own mail and Web server. It is on my DMZ network, exposed to the Internet. It is backed up to a Raspberry Pi, also on the DMZ network but not directly accessible from the Internet. Once a week there is a full backup, and daily an incremental backup. Every week the full and incremental backups for the week are written to two DVD-Rs.
• My desktop PC creates a full backup on an external hard drive nightly. The drive is one of a cycle of three.
• I back up my iPhone to my Mac Air laptop every day.
• I create a Time Machine backup of my Mac Air laptop, which includes the most recent iPhone backup, every day on one of a cycle of three external SSDs.

with no special storage precautions, generic low-cost media, and consumer drives, I'm getting good data from CD-Rs more than 20 years old, and from DVD-Rs nearly 18 years old.

Archival Data

By Pkirlin CC BY-SA 3.0 Source

Archival Media

Source

"drive the size of a teacup"

Market Size

Source

Archival-only media such as steel tape, silica DVDs, 5D quartz DVDs, and now DNA face some fundamental business model problems because they function only at the very bottom of the storage hierarchy. The usual diagram of the storage hierarchy, like this one from the Microsoft/UW team researching DNA storage, makes it look like the size of the market increases downwards. But that's very far from the case.

Source

Timescales

Customers

Source

The Cloud

Source

Threats

Source

Archival Storage Systems

IBM TS4300

Project Silica

• The media is very cheap and very dense, so the effect of Kryder's Law economics driving media replacement and thus its economic rather than technical lifetime is minimal.
• The media is quasi-immortal and survives benign neglect, so opex once written is minimal.
• The media is write-once, and the write and read heads are physically separate, so the data cannot be encrypted or erased by malware. The long read latency makes exfiltrating large amounts of data hard.
• The robotics are simple and highly redundant. Any of the shuttles can reach any of the platters. They should be much less troublesome than tape library robotics because, unlike tape, a robot failure only renders a small fraction of the library inaccessible and is easily repaired.
• All the technologies needed are in the market now, the only breakthroughs needed are economic, not technological.
• The team has worked on improving the write bandwidth which is a critical issue for archival storage at scale. They can currently write hundreds of megabytes a second.
• Like Facebook's archival storage technologies, Project Silica enjoys the synergies of data center scale without needing full data center environmental and power resources.
• Like Facebook's technologies, Project Silica has an in-house customer, Azure's archival storage, with a need for a product like this.

$48,880 femtosecond laser

Retrieval

Svalbard by Oona Räisänen CC-BY-SA4.0

The voyage

Archival Storage Services

• In all cases getting data out is much more expensive than putting it in.
• The lower cost of archival storage compared to the same service's near-line storage is purchased at the expense of a much stronger lock-in.
• Since the whole point of archival storage is keeping data for the long term, the service will earn much more in storage charges over the longer life of archival data than the shorter life of near-line data.

• Glacier is the only one of the three that is significantly cheaper in real terms than it was 6 years ago.
• Glacier can do this because Kryder's Law has made their storage about a factor of about 6 cheaper in real terms in six years, or about a 35% Kryder rate. This is somewhat faster than the rate of areal density increase of tape, and much faster than that of disk. The guess is that Glacier Deep Archive is on tape.
• Google's pricing indicates they aren't serious about the archival market.
• Archive services now have differentiated tiers of service. This table uses S3 Deep Archive, Google Archive and Microsoft Archive.
• Lock-in has increased from 13.8/12.0/8.1 to 50/175/20. It is also increased by additional charges for data lifetimes less than a threshold, 180/365/180 days. So my cost estimate for Google is too low, because the data would suffer these charges. But accounting for this would skew the comparison.
• Bandwidth charges are a big factor in lock-in. For Amazon they are 77%, for Google they are 38%, for Microsoft they are 32%. Amazon's marketing is smart, hoping you won't notice the outbound bandwidth charges.

LOCKSS

Original Logo

Today, we are delighted to announce the selected organisations for a new pilot programme of the Open Data Editor (ODE), Open Knowledge’s new open source desktop application for data practitioners to detect errors in tables. Since we opened the call for applications last January, we’ve received 71 applications in total, and we couldn’t be more inspired by the level of interest in testing it.

The applications came from 41 different countries and organisations with a wide range of data practices. The Open Data Editor has the potential to be a useful tool for working with different types of datasets, such as climate, gender, infrastructure, health, genomic, GLAM, digital literacy, public sector data and government accountability, among others.

Our goal this year is to enhance digital literacy through no-code tools. Using Open Data Editor helps to fill the literacy gap for people and teams in non-technological sectors. The selected organisations will work closely with our team from March to June 2025, integrating the application into their internal workflows and thus informing the next development steps through their use. And while doing so, they will gain lasting data skills across their institutions.

We are impressed by the vital work those organisations are doing in their fields. We hope that better data will enhance this work through no-code tools like the Open Data Editor. We’re looking forward to seeing ODE in action to help solve real-life problems.

Read more about the pilot organisations below, in alphabetical order:

• Bioinformatics Hub of Kenya Initiative (BHKi)
• City of Zagreb
• The Demography Project
• Observatoire des armements / CDRPC
• Open Knowledge Nepal

Bioinformatics Hub of Kenya Initiative (BHKi) will be working with genomic data and metadata

Country: Kenya
Area of Knowledge: Computational Biology
Targeted Datasets: Genomic data and metadata, and Community data

Staff and students at the BHKi campus outreach at the University of Nairobi. Photo: BHKi (source)

Pauline Karega, Coordinator

• What will you use the Open Data Editor for?

“We will explore the use of ODE to curate and harmonise genomic data and metadata, and community data. Additionally, we will gather information on bioinformatics/computational biology learners, who collect and use this data, and their experiences with the data. Our goal is to track their needs and interests across East Africa, to better inform outreach activities.”

• Why is a no-code tool useful for you?

“We have a big community of students and researchers doing fieldwork who are new to data collection and have minimal resources, but need to accurately collect, store and communicate data to others. This tool can be a gentle introduction to practice as they learn advanced coding skills to curate and work on the data.”

City of Zagreb will tackle the challenges of working with infrastructure data

Country: Croatia
Area of Knowledge: Public Administration
Targeted Datasets: Infrastructure data

ZG HACKL – Hackathon for Open Zagreb 2024 (source)

Kristian Ravic, Senior advisor

• What will you use the Open Data Editor for?

“ODE will be used for bridging interoperability challenges within our existing (open) data infrastructure, ensuring frictionless data exchange between various data platforms, data sources and data formats. We will use ODE to standardise open data processes and data management procedures by improving data transformation and data validation processes and publishing of open data on multiple data platforms.”

• Why is a no-code tool useful for you?

“Currently, we have a non-technical data team with low or no-coding skills. So, there is a skill-gap within our organization and a tool like ODE would help us be more efficient in connecting various data (re)sources and, thus publishing more high value datasets on our central open data platform.”

The Demography Project will focus on water, air quality and electoral data

Country: Kenya
Area of Knowledge: Environmental Justice
Targeted Datasets: Water and Air quality data, and Electoral data

Flagging off of the 2024 Ondiri Wetlands Conservation Run in Kikuyu Municipality. Photo: The Demography Project (source)

Richard Muraya, Executive Director

• What will you use the Open Data Editor for?

“ODE is a welcome opportunity for enhanced data quality of our environmental monitoring projects as well as our electoral/parliamentary monitoring project. We intend to deploy ODE to analyse large open government and citizen-generated datasets on atmospheric (air quality) and freshwater resources for enhanced personal and collective responsibility over rapidly degrading natural resources in Kenya.”

• Why is a no-code tool useful for you?

“We are a small team of determined volunteer citizen scientists, environmental journalists and innovators with inadequate technical expertise in programming or advanced computing techniques. A no-code tool will truly be a game-changer in how my team reviews and validates our tabulated environmental datasets to identify trends and errors and ultimately publish our outputs for collective climate action.”

Observatoire des armements / CDRPC will work with defence spending data

Country: France
Area of Knowledge: Weapon Monitoring
Targeted Datasets: Arms and defence expenditure databases

Act for nuclear disarmament in partnership with several organisations. Photo: Guy Dechesne (source)

Sayat Topuzogullari, Coordinator

• What will you use the Open Data Editor for?

“We are setting up a monitoring network of arms companies, the Weapon Watch Open Data Environment platform to monitor public and private spending on defence and armaments. It will be used by researchers, journalists, investigators, parliamentarians, whistleblowers and activists across Europe. In this context, we need a simple data entry tool, suitable for people who are not computer literate.”

• Why is a no-code tool useful for you?

“Our monitoring network aims to involve as many people as possible in defense and security issues, a taboo subject in France. For this reason, it’s essential to offer an easy tool for finding errors in databases and improving data quality.”

Open Knowledge Nepal will work together with local governments and their infrastructure data

Country: Nepal
Area of Knowledge: Government Data
Targeted Datasets: Infrastructure data (local governments)

Moment of the Data Hackdays 2024 in Kathmandu. Photo: OKNP (source)

Nikesh Balami, CEO

• What will you use the Open Data Editor for?

“We plan to integrate ODE into the IDMS project workflow, a system that allows local governments to store, update, and access ready-to-use data. We will use it to audit existing system datasets, identify errors, and enhance data quality. The goal is to ensure that citizens have access to high-quality datasets. Additionally, we will localise ODE user guides in the Nepali language to assist local government and non-technical users in learning how to use the tool effectively.”

• Why is a no-code tool useful for you?

“ODE will empower non-technical staff to identify, clean, and validate data without requiring coding skills. By allowing municipal staff to handle data errors directly, it will reduce the burden on technical teams and improve data efficiency. The current manual data cleaning process is time-consuming, and introducing a no-code tool like ODE will simplify and streamline workflows, saving time and resources while ensuring high-quality datasets.”

Features to simplify your work

Above you learned what these organisations will do with ODE. But you can also download it now and try it out for yourself. ODE is an open, free desktop application, available for Linux, Windows, and macOS.

Here are a few tasks that ODE can help you with:

If you have huge spreadsheets with data obtained through forms with the communities you serve, ODE helps you detect errors in this data to understand what needs to be fixed.

If you manage databases related to any social issue, ODE can quickly check if there are empty rows or missing cells in the data collected by different social workers and help you better allocate assistance.

If you monitor and review government spending or public budgets for a particular department, ODE helps you find errors in the table and make the data ready for analysis.

The Open Data Editor helps you find errors in your tables and spreadsheets

A big thanks again to all organisations that applied for the pilot programme! We will be launching another call for a second cohort in May 2025, and we strongly encourage those who were not selected to apply again. We know a lot of you spend much more time than you desire cleaning up the data before starting to do the work that is really of interest to you. ODE is here to help. 

If you have any questions or want any additional information about ODE, you can contact us at info@okfn.org.

Funding

All of Open Knowledge’s work with the Open Data Editor is made possible thanks to a charitable grant from the Patrick J. McGovern Foundation. Learn more about its funding programmes here.

This week's thread of articles looks at the ever-evolving landscape of digital security and privacy through end-to-end encryption. End-to-end encryption is a method of securing communication where only the people communicating can read the messages. In principle, it prevents potential eavesdroppers — including telecom providers, Internet providers, and even the provider of the communication service — from being able to access the cryptographic keys needed to decrypt the conversation. In practice, governments and others want to be able to put themselves in the middle of those conversations for both noble and dishonorable reasons. From unprecedented cyberattacks leading US officials to urge citizens to use encrypted messaging apps, to tech companies like Apple butting heads with the UK government over data privacy, the balance of power and privacy is under constant tension.

• As much as the U.S. government fights for ways to bypass commercial encryption, I thought I'd never see that same government urge the use of encrypted messaging apps in the wake of a major telecom breach.
• Apple takes on the UK government over data access demands that would break the end-to-end encryption of its most secure systems.
• Sweden's proposed backdoor law for encrypted messaging apps ignites global privacy concerns.
• Signal Foundation president warns of threat to privacy as an EU proposal to scan encrypted messages sparks cybersecurity concerns.
• The Signal Foundation and Apple announce revisions to their messaging systems to deal with a post-quantum encryption threat.
• Exploring the intersection of AI and end-to-end encryption. It isn't great when an AI agent wants to snoop on all of your conversations.
• This Week I Learned: Plants reproduce by spreading little plant-like things.
• This week's cat

Also on DLTJ this past week:

• My Economic Wake-Up Call Protest Sign: A #TeslaTakedown Story

Feel free to send this newsletter to others you think might be interested in the topics. If you are not already subscribed to DLTJ's Thursday Threads, visit the sign-up page. If you would like a more raw and immediate version of these types of stories, follow me on Mastodon where I post the bookmarks I save. Comments and tips, as always, are welcome.

U.S. government urges use of encrypted messaging apps in the wake of a major telecom breach

Late last year, the U.S. announced a significant attack against telecommunication companies. This hacking campaign, known as Salt Typhoon, is one of the largest intelligence breaches in U.S. history, with officials stating that the full extent of the compromise has not been resolved. The attackers accessed various types of sensitive information, including call metadata and live conversations of specific targets, notably around Washington, D.C. In light of that, the FBI and CISA recommended that Americans use messaging apps that feature end-to-end encryption. There is more than just a touch of irony here because federal law enforcement pushed for the passage of the Communications Assistance for Law Enforcement Act (CALEA) in the mid-1990s that put backdoors into telecommunications equipment for law enforcement. It was these backdoors that were used by the Salt Typhoon attackers. There is no such thing as an encryption backdoor that will only be used by authorized law enforcement.

Apple takes on the UK government over data access demands

In response to the UK order, Apple removed ADP from the UK market rather than create a "backdoor" for access. The UK Home Office maintains that privacy is only compromised in exceptional cases related to serious crimes. But, as the previous article points out, there is no such thing as a law-enforcement-only capability; if there is a weakness in an encryption system, it will eventually be exploited by someone with the time or talent to break it.

Sweden's proposed backdoor in encrypted messaging apps ignites global privacy concerns

A few paragraphs down in the article, the Swedish Armed Forces are mentioned as opposing the bill because they routinely use Signal, and a backdoor could introduce vulnerabilities that bad actors could exploit.

Signal Foundation president warns of threat to privacy

The open source Signal messaging app is considered the gold standard for end-to-end encrypted messaging. Meridith Whittaker is the president of the Signal Foundation, and she has strong words for lawmakers' efforts to weaken encryption algorithms. Ms Whittaker was also quoted in the previous article about Sweden's efforts. The European Commission originally proposed legislation to scan private messages for child sexual abuse material, but the European Parliament has rejected the approach. Experts like Whittaker argue this would create vulnerabilities that could be exploited by hackers and hostile states. The EU's data protection supervisor has also voiced concerns that the plan threatens democratic values.

Signal Foundation prepares for quantum threats with a revision to its end-to-end encryption protocol

I don't know if quantum computing will be what breaks the current generation of encryption protocols, but progress in faster hardware and more research into encryption means that the day will come at some point. The Signal protocol revision uses a "post-quantum cryptography algorithm" adopted by the U.S. National Institute of Standards and Technology (NIST). There are researchers on both sides of this divide: those working to advance encryption protocols and those seeking to break them.

Apple Launches Post-Quantum Encryption in iMessage

Apple follows Signal's lead in deploying its own quantum-safe encryption protocol for iMessage. Apple is using the same Kyber algorithm tha Signal adopted. Deploying post-quantum encryption now aims to limit the impact of "harvest now, decrypt later" attacks, where encrypted data is collected and held until quantum computers can break it.

Exploring the intersection of AI and end-to-end encryption

This blog post discusses the implications of AI technologies on the security and privacy of encrypted communications. The author emphasizes the importance of maintaining robust encryption standards in the face of evolving AI capabilities that could potentially undermine these protections. Take, for example, the need for AI agents to be snooping in on your conversations so it has the context to take actions on your behalf: "Agent, book a two-person reservation at the restaurant Dave just messaged me about." The author advocates for a collaborative approach between cryptographers and AI developers to ensure that AI advancements do not compromise encrypted data security.

This Week I Learned: Plants reproduce by spreading little plant-like things

What did you learn this week? Let me know on Mastodon or Bluesky.

Pickle and Mittens bask in a sunspot

Last Friday, we wrapped up another edition of Open Data Day with the feeling that despite recent backlashes and new structural and geopolitical challenges to open data and technologies, the open data community remains active and resilient in every corner of the planet.

In 2025, we saw powerful bottom-up energy across 189 events in 57 countries and more than 15 languages. We are small data teams within public institutions, enthusiast collectives working with communities, innovative academics, mappers, and makers from around the world. And together we make a difference!

Before starting the #ODDStories 2025 series, with stories of impact from events around the world, here’s a high-level report with the main figures and data on the 2025 edition.

We at the Open Knowledge Foundation would like to thank Datopian and the Communauté d’Afrique Francophone des Données Ouvertes (CAFDO) for their support in sponsoring the mini-grants. This financial support allows us to expand local open data networks in places that often don’t receive any form of funding (such as Francophone Africa, the focus of an exclusive stream in this year’s edition).

Let’s move on to 2026 with an even bigger, more diverse and impactful event!

About Open Data Day

Open Data Day (ODD) is an annual celebration of open data all over the world. Groups from many countries create local events on the day where they will use open data in their communities. ODD is led by the Open Knowledge Foundation (OKFN) and the Open Knowledge Network.

As a way to increase the representation of different cultures, since 2023 we offer the opportunity for organisations to host an Open Data Day event on the best date over one week. All outputs are open for everyone to use and re-use.

For more information, you can reach out to the Open Knowledge Foundation team by emailing opendataday@okfn.org. You can also join the Open Data Day Google Group or join the Open Data Day Slack channel to ask for advice, share tips and get connected with others.

The NDSA Levels Steering Group has been diligently keeping tabs on how the Levels get deployed “in the wild” at our monthly meetings. The task is a way to understand the needs of the digital preservation community, and to document the usefulness of the NDSA’s signature digital preservation tool. 

Levels sightings occur in a variety of formats, ranging from their use in program assessment, to specific applications for “niche” digital preservation adjacent activities, to vendors employing them as a yardstick to gauge their service offerings. They’ve been spotted in numerous published articles on digital preservation, including the oft-cited “How to Talk to IT about Digital Preservation”, and in preservation reports that range from presentations on cyber-security to a graduate student assessing the state of digital preservation for the College Park Maryland Aviation Museum. The NDSA Levels are not just used in the United States, but are an international presence as well. The Bibliothèque et Archives nationales de Québec has used the NDSA levels to help define their information model, as shared at iPres 2024; and the UK Archives Accreditation standards suggest using the NDSA Levels or DPC-RAM for self-assessment. 

The scholarly literature on digital preservation is fertile ground for sightings, too, with several articles invoking the Levels just in the past year:

• Silva, W. da. (2024). Why Digital Preservation Should Be Treated as Public Policy. Collections: A Journal for Museum and Archives Professionals, 20(4).
• Underdown, David H. “Case study: Using the DiAGRAM tool for digital preservation.” Journal of Digital Media Management 11.3 (2023): 258-270.

Have you seen an example of the NDSA Levels being used by colleagues or referenced in a presentation or even by a vendor? We’d love to hear about it! As always, we also encourage the whole community to provide feedback on the Levels – including Levels sightings! – at any time.

The post NDSA Levels of Digital Preservation ‘Sightings in the Wild’ appeared first on DLF.

My protest sign for the #TeslaTakedown today.

I made a sign for today's #TeslaTakedown, and I should have listened to my family. They suggested that the initial version, without the "How much do you have in common with Elon Musk?" at the bottom, was too confusing. Adding that sentence improved understanding, but now there was too much to read in a protest sign for cars whizzing past. My point was that me and the person driving by giving me a middle finger have far more in common than what either of us have with Elon Musk (and Donald Trump).

Ladder rungs of economic prosperity

My son is taking a gen-ed psychology class in his first year of college, and he was describing a class exercise demonstrating the difficulty people have with probabilities and proportions. That got me thinking about the "cosmic distance ladder". The cosmic distance laddercosmic distance ladder is a series of methods astronomers use to determine the distances to celestial objects, acting as a tool to map the universe. It's called a ladder because each step relies on the previous one, starting with measurements to nearby galaxies and progressing to farther objects.

Let's suppose the median net worth of an American—the point at which half the people in the country have more and half the people in the country have less—is $100,000. So, standing there in the middle of the protest, the 10 people around me have a total net worth of $1,000,000—a million dollars. And the 100 people or so between and the street corner? That is ten million dollars. And the 1,000 people walking and driving by the #TeslaTakedown protest? That's a hundred million dollars.

The capacity of a triple-A minor league baseball stadium is about 10,000 people; the median net worth of that crowd is a billion dollars. The capacity of Ohio Stadium, where the Ohio State University football team plays, is 100,000, and the net worth is ten billion dollars. It is only at this point that reach Donald Trump's net worth. The population of Franklin County, Ohio—the seat of the state capitol—is just over 1,000,000 people, and the total median net worth is $100,000,000. Elon Musk's net worth is about $350,000,000—so three Franklin-county's-worth of people.

So when I said that the person throwing the middle finger at me has more in common with me than Musk, that's what I meant.

The actual median net worth of an American household is just short of $200,000, so we are not that far off with this economic prosperity ladder (assuming two earners per household). And the really perverse part? Remember that the median is the point at which half the population is above that amount and half the population is below. Don't confuse that with the average...that'll be the sum of everyone's net worth divided by the number of people in the country. That number is just over $1,000,000 per household. The highest highs have skewed the average that much.

Now, at the risk of reducing a person's value to a dollar amount, that is what I was trying to say in my sign. Wealth is a tool, not an identity; a person's kindness, resilience, and hope for a better world hold real value...and I saw a lot of kindness, resilience, and hope at the protest today. Yet, despite the immense kindness and resilience I was in the middle of, it's disheartening how wealthy people are overriding the collective interests of the country.

And that is far too much to put on a sign.

About making the protest sign

I'm adding my notes here about creating the protest sign, because clearly I'll need to make a different one for next week. The base of the sign is an old campaign yard sign—about 26 inches by 16 inches. Using a graphics program, I made an image at those dimensions. My plan was to print it out as a set of tiles on letter-sized paper and then tape them together. Unfortunately, the MacOS printer driver doesn't do this (...anymore? I thought it did at one point). Fortunately, a free web service called Rasterbator will make a PDF of tile pages for me. I uploaded the sign, selected US-Letter paper in landscape orientation, then selected an output size of "2.45 sheets wide". That will output 6 sheets for a final size of 24.98" x 15.38"...pretty close! This is what it looked like in the end.

My protest sign at the #TeslaTakedown.

Source

In the earlier share price chart RELX was the lower of the three lines but given its higher dividend payouts its overall shareholder returns have paced with its peers, and it is one of the best-performing large-cap stocks in the UK. The following chart shows that it is now in the top five largest in the UK by stock market value and bigger than BP. Its price-to-earnings multiple of around 30x is more like a tech giant.

The second largest listed player is Thomson Reuters worth around $80 billion (and this trades on a much higher price-to-earnings multiple than RELX). Add in Wolters Kluwer, and the much smaller Springer Nature which was listed late last year, and the combined stock market value of the four firms is more than $215 billion.

Source

The following chart illustrates the revenues and operating profits of the four firms in 2024. RELX generated as much operating profits as its two largest peers combined.

Firstly, the customer base is captive and highly vulnerable to price gouging. A university library has to have access to the best journals, without which the members of the university can’t keep up with their field or do their own research. Secondly, although the publishers who bought the titles took over the responsibility for their administration and distribution, this is a small part of the effort involved in producing an academic journal, compared to the actual work of writing the articles and peer-reviewing them. This service is provided to the publishers by academics, for free or for a nominal payment (often paid in books or subscriptions to journals). So not only does the industry have both a captive customer base and a captive source of free labour, these two commercial assets are for the most part the same group of people.

In essence, the academics fight to help the publishers extract monopoly profits. The whole process depends on the value of having your articles appear or receive citations in the best journals. This becomes an unaccountability sink to which universities outsource their whole system of promotion and hiring. Davies compares this to the PageRank algorithm used by Google.

This market for academic publishing generates around $20 billion of revenue per year with half of the market controlled by the five largest firms: Elsevier (part of RELX), John Wiley & Sons, Taylor & Francis, Springer Nature, and SAGE.

Source

the chart below illustrates that RELX-owned Elsevier is not only the clear market leader, but it has also grown significantly in volume.

The first result concerns the total number of articles published, which follows very closely an exponential growth (+5.6% per year). Even taking into account the increase in the number of researchers over this period, we can deduce that the time spent on obtaining the results, validating and peer reviewing them has decreased significantly.

By computing the average annual number of articles published per journal, the authors observe that the growth (significant, but not overwhelming) of the traditional publishers is mainly due to the expansion of the number of journals in their catalogues, whereas the very strong growth of Frontiers and MDPI is the result of an explosive increase in the number of articles per journal. It should be noted that these two publishers, which appeared more recently, are thriving through publication fees paid by authors.

The triad (MDPI, Frontiers, Hindawi) increasingly use special issues to publish more and more articles, and this phenomenon is accompanied by a significant shortening of the time allotted to the peer-review process.

Scientists are increasingly overwhelmed by the volume of articles being published. Total articles indexed in Scopus and Web of Science have grown exponentially in recent years; in 2022 the article total was approximately ~47% higher than in 2016, which has outpaced the limited growth - if any - in the number of practising scientists. Thus, publication workload per scientist (writing, reviewing, editing) has increased dramatically. We define this problem as the strain on scientific publishing. To analyse this strain, we present five data-driven metrics showing publisher growth, processing times, and citation behaviours. We draw these data from web scrapes, requests for data from publishers, and material that is freely available through publisher websites. Our findings are based on millions of papers produced by leading academic publishers. We find specific groups have disproportionately grown in their articles published per year, contributing to this strain. Some publishers enabled this growth by adopting a strategy of hosting special issues, which publish articles with reduced turnaround times. Given pressures on researchers to publish or perish to be competitive for funding applications, this strain was likely amplified by these offers to publish more articles. We also observed widespread year-over-year inflation of journal impact factors coinciding with this strain, which risks confusing quality signals. Such exponential growth cannot be sustained. The metrics we define here should enable this evolving conversation to reach actionable solutions to address the strain on scientific publishing.

Source

• The triad of predatory publishers do far less reviewing, and the amount they do decreases over time.
• The less predatory publishers do far more reviewing and the amount they do increases over time.
• The predatory publishers' distribution of review times increasingly skews to the short end, where the other publishers' distribution is stable and skews slightly to the long end.

RELX Elsevier has over 3,000 journals including leading brands like the Lancet. These journals published more than 720,000 articles in 2024. This is almost one-fifth of all scientific articles. Elsevier’s online platform ScienceDirect has tens of millions of pieces of peer-reviewed content from tens of millions of researchers. Its citation database has content from tens of thousands of journals.

The scientific, technical, and medical information division is RELX’s second largest with revenues of around $4 billion of revenues, roughly equally split between articles/research content vs databases, tools, and electronic reference solutions. The business is largely subscription fee driven and 90% of its revenues are from electronic products rather than print publishing and face-to-face events.

Despite the growth in the number of articles, this is RELX’s slowest-growing division with 3-4% revenue growth and 4-5% profit growth in recent years. They are expanding in line with our faster than the industry. But it is a very healthy cash cow with $1.5 billion of operating profits in 2024. This circa 40% operating margin is the best in the RELX group and far superior to other professional and business information businesses. The number two player Springer Nature has lower operating margins of around 30% in their academic publishing business, but this is much higher than their other products.

Discover how your feedback shaped Commerce Studio, our new AI merchandising platform that combines powerful automation with complete human control.

The post Commerce Studio: A New Chapter in AI Merchandising, Built From Your Feedback appeared first on Lucidworks.

Last year, a dedicated group of digital stewardship professionals formed the Climate Watch Working Group. Our mission is to raise awareness of the threats that climate change presents to preserving cultural heritage materials, especially those in digital form.

We are excited to announce the publication of our Climate Watch Substack. 

Global warming and climate change are currently wreaking havoc on the world. As digital stewardship professionals, it is our responsibility to mitigate threats that impede our ability to manage digital materials through time. Climate change not only jeopardizes our data through more frequent and more severe weather disasters, but also through reductions in food supply, mass migrations, economic contraction, and political upheaval. 

Climate change reports and publications are published frequently and can be hard to keep on top of. Our objective is to curate the most relevant articles ensuring that we stay informed about the current state of the climate. We hope these resources will help make the case for appropriate long-term planning so that our field can begin proactive adaptation efforts.

We aim to publish quarterly updates reviewing climate change-related news, articles, reports, and publications. Each brief review will highlight key points from the publication and relate it back to why it is relevant to digital preservation specifically, or cultural heritage overall. Additionally, we will also maintain a list of Foundational Resources for digital stewardship colleagues that seek to become more familiar with climate scholarship, climate risks, and how these risks will impact our work, our workforce, and the collections we steward.

The post Announcing the NDSA Climate Watch publication appeared first on DLF.