Planet Code4Lib

Public Domain Day countdown on public social media networks / John Mark Ockerbloom

I’m starting my countdown today to Public Domain Day, when a year’s worth of copyrights in many countries expires on January 1. In the US, we’ll be getting works that were copyrighted in 1927, and whose copyrights were consistently maintained since then. I’ll be featuring 60 works, or sets of works, one per day in the days to come. You’re welcome to read and join in.

In recent years, the primary venue for my countdown has been Twitter. Recent events, though, make me less confident in the future of that site as a useful and enjoyable social media platform. When it comes down to it, I’m not all that optimistic about any large-scale discussion forum that’s controlled by the whims of one rich guy. So this year, I’m trying out a new primary venue that’s not subject to any one person or organization. I’ve dusted off an account I created a while back on mastodon.social, and will be posting my countdown first there. They’ll be readable not just on that site, but on any other site that exchanges messages with it using the ActivityPub protocol. That’s a large array of sites, collectively known as the “Fediverse” (for the way that sites federate with each other to do that message exchange), or as “Mastodon” (which, strictly speaking, refers to the open source software used by many, but not all, of those sites). On any federated site you can read and respond to my posts by following or searching the #PublicDomainDayCountdown hashtag (linked here via the mastodon.social website, which can be slow at times). Or follow my personal account, though that will also show you posts I make on other topics, and might not show posts others make using the hashtag.

You can join whatever Fediverse site you like, or join multiple ones. If you later want to move to another Fediverse site that you like better, you can take your follows with you and leave a forwarding address. (I might do that myself after I finish the countdown from my current site.) If you’d like to learn more, Ruth Kitchin Tillman, who co-administers one of those sites, has a useful introductory guide, written especially with library folks in mind. Newcomers, or those who would like refreshers, might also find the documentation on fedi.tips useful. You can also find a large directory of sites to consider, as well as various apps for using them if you don’t want to just use their websites, at joinmastodon.org.

I do plan to continue posting the countdown on Twitter as well, though it might appear in more abbreviated form there due to the smaller character limits for posts, and might appear later than it does in the Fediverse. (Or possibly earlier in some cases; I’m not quite sure myself how long it typically takes posts to propagate through the Fediverse.) And, as usual, I’ll also post and periodically update my countdown here on this site, where the only controlling whims are my own.

Reformatted for my blog, here is my countdown for 2023:

November 2: “Wait a minute- you ain’t heard nothing yet!” said Al Jolson onscreen in 1927, and the movies would never be the same again. Arun Starkey on The Jazz Singer, the first “talkie” feature film, joining the public domain in 60 days. #PublicDomainDayCountdown

November 3: The Hardy Boys was the first multi-generation breakout hit series from the Stratemeyer fiction factory. Leslie McFarlane, its original ghostwriter, writes about taking the job. His first 3 mysteries join the public domain in 59 days.

Many of us read syndicate-revised versions with updated settings and less overt racism, but arguably less interesting prose. In the public domain, we’re all free to both read the originals & adapt them as we like. #PublicDomainDayCountdown

November 4: “An almost mythic tale of a life simply lived in the American southwest” is how the Literary Ladies Guide describes Willa Cather’s Death Comes for the Archbishop, joining the public domain in 58 days. More on the book at the University of Nebraska’s Willa Cather site. You can find bibliographic information and scholarly articles there now; I look forward to them adding full text, as they have for earlier Cather novels. #PublicDomainDayCountdown

November 5: “I ask you very confidentially…”

Jack Yellen wrote the music, and Milton Ager wrote the lyrics, inspired by his daughter (a toddler at the time, later a regular on 60 Minutes).

“…Ain’t She Sweet?”

Published in 1927, it’s been covered by scores of artists, including the Beatles and the Muppets. I’ll sing it to my sweetie when I go out with her this weekend. It’ll be in the public domain in 57 days, and probably in your head right now. #PublicDomainDayCountdown

November 6: Sherlock Holmes fans have reason to rejoice in 8 weeks. Sir Arthur Conan Doyle’s last two Holmes stories, and The Case Book of Sherlock Holmes, join the US public domain then. Not only does this free the complete Holmes canon, but it also frees adapters from worry that Doyle’s estate might shake them down for payment or approval based on claims of copying aspects of Holmes allegedly only in the late stories. (See e.g. this story.) #PublicDomainDayCountdown

November 7: “Who do you think can hold God’s people / When the Lord God himself has said, / Let my people go?”

God’s Trombones consists of James Weldon Johnson’s poetic distillations of sermon themes and preaching styles he often heard in African American churches. It’s notable for Aaron Douglas’s art and design as well as Johnson’s poetry, as the Norman Rockwell Museum shows: https://rockwellcenter.org/essays-illustration/gods-trombones-judgment-day/

God’s Trombones goes into the public domain in 55 days. #PublicDomainDayCountdown

November 8: “There are those who foresee the decline of partisan politics in this country,” wrote Dartmouth professor Harold R. Bruce in 1927, “but such people deceive themselves…” His textbook American Parties and Politics, which includes this quote, went through multiple editions. If you’re in the US, you can vote today, and in 54 days when the book’s first edition joins the public domain, you can more easily read it and compare the states of US politics then and now. #PublicDomainDayCountdown

November 9: Virginia Woolf’s To the Lighthouse is on multiple lists of best novels of the 20th century, though not all readers easily get into it. Kate Flint has an introduction, with accompanying content from the British Library, to this “carefully structured, psychologically complex novel that ultimately asks the reader to reflect on their own ever-changing experience of being in the world.” Woolf’s novel joins the US public domain in 53 days. #PublicDomainDayCountdown

November 10: Walter R. Brooks wrote over two dozen children’s books about Freddy the Pig and other remarkably intelligent animals living on the Bean Farm in upstate New York. The series ended in 1958, the year Brooks died, but it continues to have dedicated fans, some of whom run this website: http://freddythepig.com/

Freddy the Pig first appeared in _To and Again_ in 1927. The original edition, illustrated by Adolfo Best-Maugard, joins the public domain in 52 days. #PublicDomainDayCountdown #Bookstodon

November 11: “For those who will climb it, there is a ladder leading from the depths to the heights – from the sewer to the stars – the ladder of Courage.” That title card opens 7th Heaven, a 1927 film set in Paris on the verge of the first World War. Aubrey Solomon writes about it here for the Library of Congress.

This silent film, which earned three of the first-ever Academy Awards, joins the public domain in 51 days. #PublicDomainDayCountdown

November 12: Béla Bartók wanted the percussive effect of his first piano concerto magnified by placing the piano directly in front of the tympani and other percussion instruments. While it’s not staged that way here, pianist Yuja Wang and the Swedish Radio Symphony Orchestra still bring out the energy that Bartók called for.

Bartók himself played the piano in the concerto’s premiere in Germany in 1927. It joins the public domain in the US in 50 days. #PublicDomainDayCountdown

November 13: Two memorable spinster detectives make their debuts in 1927. In Dorothy L. Sayers’ Unnatural Death (published in the US as The Dawson Pedigree), Miss Climpson goes undercover in an English village to investigate a murder for Lord Peter Wimsey. And Agatha Christie’s Miss Marple begins a decades-long crime-solving career in “The Tuesday Night Club”, a story published in the December 1927 Royal Magazine. Both join the US public domain in 49 days. #PublicDomainDayCountdown

November 14: Hermann Hesse’s Der Steppenwolf focuses on a man getting through severe mental and spiritual crisis. While it had mixed reviews at its 1927 release, it gained new popularity in the social crises of the 1960s, though Hesse then called it “violently misunderstood”.

Hesse’s original German novel joins the US public domain in 48 days. Prior English translations may stay copyrighted, but public domain status should foster new translations, audiences, and understandings. #PublicDomainDayCountdown

November 15: Charles Lindbergh won instant celebrity with his 1927 transatlantic flight. His memoir We, published weeks later, soared to the top of bestseller lists.

His fame turned darker later. His first child was murdered, he fled back over the Atlantic to escape the press, and later returned to advocate white supremacy and tolerance of Nazi aggression as ‘America First’ spokesman. But when We came out, he was still flying high. It lands in the public domain in 47 days. #PublicDomainDayCountdown

November 16: Fiction can be copyrighted, but facts are public domain. Lindbergh’s transatlantic flight inspired not just his own We, but hundreds of works by others. Among the first to take off was “Lucky Lindy”. L. Wolfe Gilbert and Abel Baer wrote and released the song within days of Lindbergh’s landing. It in turn joins the public domain in 46 days.

The flight may have also inspired the name of the “Lindy Hop”. Both name and dance are public domain. #PublicDomainDayCountdown

November 17: African American poet Countee Cullen had a productive year in 1927. He published two collections of his own poetry, The Ballad of the Brown Girl and Copper Sun. He also edited Caroling Dusk, one of the major poetry anthologies of the Harlem Renaissance, featuring works by 38 Black poets. Danielle Sigler writes about the anthology for the Ransom Center Magazine.

All three books join the public domain in 45 days. #PublicDomainDayCountdown #Bookstodon

November 18: “Like the seed, I would be able to die when the plant had developed, and I began to see that I had lived for its sake…”

Marcel Proust died on this day 100 years ago. Five years later, Le Temps Retrouvé, the final volume in his masterwork À la Recherche du Temps Perdu, was published. It reaches the US public domain in 44 days. The quote above, translated by Ian Patterson, is part of Proust’s summing up of his work. #PublicDomainDayCountdown #Bookstodon #Proust

November 19: Today I went #hiking through the Trailside Museums & Zoo in Bear Mountain, NY. It’s the only such exhibition I’ve seen designed for passing hikers. Founded in 1927, it features local wildlife, and was intended as a model for other outdoor education sites along the Appalachian Trail.

I wonder if it was inspired by Paul Griswold Howes, a nature writer who lived not far from there. His Backyard Exploration was published that year, and joins the public domain in 43 days. #PublicDomainDayCountdown

November 20: “On Friday noon, July the twentieth, 1714, the finest bridge in all Peru broke and precipitated five travelers into the gulf below.”

So begins The Bridge of San Luis Rey, Thornton Wilder’s Pulitzer Prize-winning novel relating the lives of the victims of the disaster, and of a witness who thought he could show why God had them die. Wilder later said “In my novel I have left this question unanswered.” It joins the public domain in 6 weeks. #PublicDomainDayCountdown

November 21: Felix Frankfurter’s The Case of Sacco and Vanzetti is one of the few contemporary writings on their politically charged trial that’s not yet public domain. He published the book criticizing the proceedings as a Harvard law professor in 1927, months before Sacco and Vanzetti were executed. He renewed its copyright as a Supreme Court justice in 1954. It expires in 41 days, as does the copyright of another 1927 book he coauthored, The Business of the Supreme Court. #PublicDomainDayCountdown

November 22: In 1927, Abraham Lincoln’s assassination was about as far in the past as John F. Kennedy’s is today. One of the former’s last surviving eyewitnesses, actor Joseph Hazelton, related what he saw to Campbell MacCulloch, in a story that ran in the February 1927 issue of Good Housekeeping. Allan Ellenberger has more.

Most 1927 magazines did not renew copyrights, but Hearst magazines like Good Housekeeping did. It joins the public domain in 40 days. #PublicDomainDayCountdown

November 23: “‘S Wonderful,
‘S Marvelous,
That you should care for me!”

“”S Wonderful” was written by George and Ira Gershwin for Smarty,Smarty. Little of that musical got to Broadway, but the song’s had a long, wonderful history since then. Rachel Fernandes tells some of it for the Gershwin Initiative.

The song joins the public domain in 39 days. But if there’s someone you care for, you don’t need to wait to tell them. I still sing it with my sweetheart, and we think ‘s wonderful. #PublicDomainDayCountdown

November 24: William Hazlett Upson’s work experience (including a stint as a tractor mechanic) inspired his humorous short stories about Alexander Botts, salesman for the Earthworm Tractor Company. They ran in the Saturday Evening Post for nearly 50 years. Octane Press has more on the character and his creator.

The first Botts story, “I’m a Natural-Born Salesman”, joins the public domain in 38 days, along with the rest of the 1927 Saturday Evening Post. #PublicDomainDayCountdown

November 25: The BBC writes on Clara Bow as Hollywood’s original “‘It’ Girl”. She got that nickname starring in It, a 1927 silent film based on a story by Elinor Glyn, which defined ‘It’ as “that quality possessed by some people which draws all others with its magnetic life force”. Both the movie and Glyn’s story join the public domain in 37 days. #PublicDomainDayCountdown

November 26: In John Buchan’s Witch Wood, a young Presbyterian minister moves to a rural Scottish village in 1644. Soon he discovers sinister goings-on in the nearby forest, implicating outwardly pious and powerful members of his parish. The novel was Buchan’s favorite, and many critics felt similarly, though its extensive Scots dialogue may challenge American readers. Phil Wade reviewed it earlier this year. It joins the US public domain in 36 days. #PublicDomainDayCountdown

November 27: In the Toronto Review of Books, Craig MacBride calls the Jalna series “A Canadian Downton Abbey, minus the pomp”. Mazo de la Roche wrote 16 novels of this intergenerational drama, which made her career and were bestsellers in Canada and elsewhere for decades. They’ve been in the Canadian public domain since 2012. The first novel, Jalna, was originally serialized in the US, in the Atlantic, and joins the US public domain in 5 weeks. #PublicDomainDayCountdown #Bookstodon

November 28: One of the more unlikely scholarly publishing hits of 1927 was Helen Waddell’s The Wandering Scholars. It’s about the Goliards, young European clergy who wrote irreverent Latin poetry such as collected in Carmina Burana. A contemporary critic called Waddell’s book “jazzing the Middle Ages”. Current readers might like an edition explaining its many allusions (as Harry Cochrane’s review implies.) The book joins the US public domain in 34 days. #PublicDomainDayCountdown

Fellow Reflection: Clarissa West-White / Digital Library Federation

Dr. Clarissa West-WhiteThis post was written by Dr. Clarissa West-White (@clarissawhite), who attended the 2022 DLF Forum as a Historically Black Colleges and Universities (HBCU) Fellow.

Dr. Clarissa West-White is the University Archivist, Assistant Professor at Bethune-Cookman University in Daytona Beach, Florida. A native of Quincy, Florida, she has degrees in Creative Writing, Curriculum & Instruction English Education and Information from Florida State University. Dr. West-White has experience as a middle and high school English teacher, program coordinator, adult literacy director, university English department chair, and assistant professor and adjunct at a number of public and private universities in the state of Florida and online. She has completed archival projects and served as an education consultant with the U.S. and Florida Department of Education. She has received fellowships and scholarships, including: Robert Frederick Smith Internship, National Museum of African American History & Culture, Smithsonian Institute; Teacher and Librarian Scholarship Recipient, Key West Literary Seminar; National Information Standard Organization (NISO) Plus Scholarship Recipient; Preserving digital Objects With Restricted Resources Institute Recipient; Fulbright-Hays, Turkey. When not at work, Clarissa enjoys spending time with her family, road trips, downtown strolls, traveling, photography and the beach.


Accession numbers. When I transitioned to the library as a Reference Librarian from serving as Department Chair and Assistant Professor of English in 2018, I recall briefly discussing assigning accession numbers to collections with the Curator and University Archivist. I listened. The information nestled in the far, far recesses of my mind as I moved into my new space absorbing data and information, engaging with students and faculty, and attending countless webinars. I also began an online Master of Information program. I cannot recall the subject of accession in any course, which is okay because why would I, a Reference Librarian, need to know anything about assigning accession numbers? Four years and four months later I found myself appointed as the University’s Archivist. There was so much to do that I did not readily need to know how to assign the numbers. After settling a dozen student workers and an English class of 32 volunteering freshmen, and working with faculty to craft research projects with the Archives as subject, it slowly became apparent when delegating work assignments why I needed to know how to assign accession numbers. For instance, how do you differentiate new additions to a collection? Like any librarian I searched and found a gamut of definitions, anecdotes and theoretical theses. My prediction that there was a scientific spreadsheet was quickly proven false. There was no science to this and examples demonstrated vast differences: some numbers were under 8 characters, others more than 20. Thus, I arrived in Baltimore with many questions to pose to Forum goers; however, I was reticent about asking just anyone. I needed a strategy. I figured I would ask the person who smiled the most or greeted me warmly. Well, that was just about everybody! This is no easy feat as conference attendees can attest. I hesitantly but enthusiastically raised my confusion in what proved to be a very safe space. I was able to breathe and enter deep-probing conversations with people I just happen to sit next to at lunch or in a session; the Forum began to feel like work – in a good way.  The metaphors used to describe the process was really what the literati in me needed. After much thought, what I heard was, “Think of it as a tag you place on items on a shelf. The number needs to tell you the story of the item.” I immediately thought of the items in my home. My grandmother’s living room set, my mother’s vase. I would ‘tag’ different numbers to the rocking chair I inherited from my grandmother compared to the one I ordered from Wayfair a couple of years ago. Thanks to conversations with a fellow Fellow (LOL), a project manager, and a recently minted collections librarian within reach, I realized that my angst was imagined, self-imposed.  I now have a system in place for assigning accession numbers that tell the story of each collection. Collections will live beyond my temporary placement as archivist. My role is to assign numbers that will assist those who follow in figuring out the collection’s how. The number will not say as much as the finding aid, but it should speak to the collection’s internal journey from some roughly stacked boxes or crates to a curated reimagined narrative. 

The post Fellow Reflection: Clarissa West-White appeared first on DLF.

Better together than alone: say hello again to the Open Knowledge Network / Open Knowledge Foundation

We have a very exciting announcement today. We are relaunching the Open Knowledge Network!

If you haven’t heard about it in the past, the Network connects those in the open movement across the globe. The members use advocacy, technology and training to unlock information, to create and share knowledge, with the goal of enabling people to take action to achieve local impact.

For the record, the Network started in 2010 with a bunch of passionate people from the open movement starting some informal community meetups in Germany and Greece. Since then, building on the global enthusiasm around open knowledge (and open data more specifically), the Network has expanded to 40 countries across the globe.

Today, we are in a different phase. Over the years, the communities composing the Network became mature. Now operating independently, they are setting their own priorities for projects relevant to each country or domain. It is very gratifying to realise that people in the Network have grown their expertise and have become established leaders of the open knowledge movement worldwide.

All this potential could not be wasted. We all really believe in the power of openness and in doing it together, not alone. For this reason, at the beginning of 2022, the Open Knowledge Foundation started convening Network representatives to understand what future direction we imagined for us all.


A Year-long Relaunch

In March 2022, we organised a session at Mozilla Festival where we gathered the Network and old friends from the open movement to brainstorm all together on what an open future with knowledge at its core would look like considering today’s contexts, and how we can build it together as a Network.

Screenshot from the MozFest session “An Open Future with Knowledge at its Core”

In April and May, we organised a series of online workshops that culminated in a face-to-face meeting of chapter representatives in Barcelona. Big thanks to Tarmo Toikkanen from Finland, Sonja Fischbauer and Kristina Klein from Germany, Charalampos Bratsas from Greece, Nikesh Balami from Nepal, Elenor Weijmar from Sweden, Florin Hasler and OIeg Lavrovsky from Switzerland for joining us for a couple of days of lively discussions, strategic thinking, and forward-looking ideas.

Chapter representatives enjoying a sunny lunch break during the strategic meeting in Barcelona.

Two online sessions for the rest of the Network were also held in parallel, to give a chance to everyone to provide their input and reconnect. Thanks to Fernanda Campagnucci from Brazil, Tomoaki Watanabe from Japan, Michal Kubáň from the Czech Republic, Maarja-Leena Saar from Estonia, Poncelet Ileleji from the Gambia, Oluseun Onigbinde from Nigeria, Andrea Borruso, Francesca De Chiara, and Marco Montanari from Italy, Ivan Begtin from Russia, Hussein Osman Abdullahi and Mohamud M. Hassan from Somalia, and T.H. Schee from Taiwan, for joining us online and giving their take on the future of the Network.

As a collective, we want open knowledge to be a design principle adopted broadly, shaping institutions, processes, infrastructures, organisational models and movement dynamics.


Hello, World!

So what is happening today? 

We are launching two projects to shine a light on the tremendous work the Network has been doing in the past decade. 

One is called the Open Knowledge Project Repository, and it is a searchable database of the most prominent projects from the Network. We’re kicking off with 40+ projects, but we’re sure there will be hundreds in a very short time. Our hope is that these initiatives can inspire one another and that the connections generated there will dramatically amplify our collective impact.

Go and have a look: https://network.okfn.org/project/.

The other one is the Open Knowledge Global Directory, a global and multilingual excellence cluster of leaders of the open movement, carefully selected by the Network, reflecting our will to become the place where the whole open movement is convened. For now, you can get in touch with people from 25+ countries speaking 15+ languages, and growing. 

You can check out the Directory here: https://network.okfn.org/specialist/.


How can I join?

Get in touch if you want to join the Open Knowledge Network! You can drop us an email at network@okfn.org. Everyone is welcome. We are always looking for friendly faces and open enthusiasts from anywhere in the world to make the Network stronger, better (and merrier!). No special skills or background are needed — just an interest in open knowledge. 

We have very exciting plans for 2023, and we can’t wait to let you all know about them. Stay tuned to know more.

With Mastodon on the Rise, Who Archives the Digital Public Square? / Peter Murray

DALL•E generated image DALL*E prompt: photorealistic waves of twitter logos and mastodon logos crashing onto a sandy beach

Much has been made about the differences between Twitter and Mastodon: the challenge of finding a home for your account (and the corresponding differences between your “local” timeline and your “global” timeline), the intentional antiviral design choices (no quote-tweets and a narrow search system), and the more-empowering block and mute features. A recent article in MIT Technology Review about the potential loss to history if Twitter goes away had me thinking of another one difference: a Mastodon-filled world changes expectations for archiving this kind of primary source material.

Think Bigger Than Mastodon

Let’s set some common ground. “Mastodon “ is being used here as a shortcut for the growing federation of servers that follow the ActivityPub protocol—the “fediverse”. Most people caught up in the migration away from Twitter are looking for a “Twitter-equivalent”, and the option that has caught the popular imagination is Mastodon. As we view the fediverse digital public square, we could just as well be talking about Mastodon forks like Hometown. We should also include in the genre-specific ActivityPub software like Pixelfed (for photographers, me there), Bookwyrm (for book groups and reader commentary, me there), Funkwhale (for music), and write.as (for long-form articles). Although Mastodon is getting the most traction right now, the question of archiving the digital public square is bigger than just Mastodon…just keep that in mind as you read below.

Twitter Archiving Challenges

As the MIT Technology Review article points out, there are challenges to archiving Twitter.

For eight years, the US Library of Congress took it upon itself to maintain a public record of all tweets, but it stopped in 2018, instead selecting only a small number of accounts’ posts to capture. “It never, ever worked,” says William Kilbride, executive director of the Digital Preservation Coalition. The data the library was expected to store was too vast, the volume coming out of the firehose too great. “Let me put that in context: it’s the Library of Congress. They had some of the best expertise on this topic. If the Library of Congress can’t do it, that tells you something quite important,” he says.

The challenges include that of scale:

[In January 2013] We now have an archive of approximately 170 billion tweets and growing. The volume of tweets the Library receives each day has grown from 140 million beginning in February 2011 to nearly half a billion tweets each day as of October 2012. - Update on the Twitter Archive at the Library of Congress, Library of Congress blog, January 2013.

And also of scope—the Library does not receive the multimedia parts of tweets. As the whitepaper attached to the Update on the Twitter Archive at the Library of Congress says:

The Library only receives text. It does not receive images, videos or linked content. Tweets now are often more visual than textual, limiting the value of text-only collecting.

Both points speak to the changing nature of Twitter from when its origins as an extension of text messaging geared towards a U.S. audience into a world-wide multimedia platform. Michael Zimmer wrote in great detail about these challenges and the issues of processing, privacy, and user consent for First Monday in 2015. The donor agreement between Twitter and the Library of Congress is silent on the matters of privacy and user consent as well.

On December 26, 2017, the Library of Congress announced that it was no longer collecting a comprehensive archive of tweets as of January 1, 2018. What is at the Library now has known limitations in its comprehensiveness, and we may not see open access to that archive in our lifetime because of privacy concerns.

The MIT Technology Review article talks about the loss to historians, human rights lawyers, and researchers using “open source intelligence” — that which is openly published in the digital public square. Given that we are facing this moment of reckoning as Twitter may be on the brink of disappearing and people are finding community on Mastodon, should we consider an explicit archiving role for the fediverse?

Mastodon Archiving Challenges

With Twitter’s recent upheaval and the migration to Mastodon, I’ve seen mentions of how Twitter was unique to its time. At Twitter’s public unveiling in March 2006, the only way to interact with Twitter was through text messages. Apple would introduce the iPhone the following year, and it was a year after that when an app for iPhone would launch. Twitter’s growth was jumpstarted by the influx of users at the 2007 South-by-Southwest (SXSW) conference as attendees publicly shared their experiences in real time in a way they could not have previously. The combination of an experience that straddled mobile and desktop devices and the ability of the company to scale to meet the demand made this Twitter’s moment. A moment that it ran with for the next 15 years.

Mastodon is different. Conceptually, there isn’t one “Mastodon” (like there is one “Twitter”); there are many little Mastodons that have a standard way of talking to each other. (Yes, this is where the “ActivityPub” standard becomes key.) And crucially, these many little Mastodons are run by individual users and organizations. We witnessed firsthand the difficulties these Mastodons had in scaling to meet the demand from the outflow of Twitter users. (Many of the larger Mastodon instances halted or greatly limited new user registrations in November 2022.)

Now consider what would be needed to construct a “Mastodon Digital Archive” similar in scope to how Twitter donated its timeline of tweets to the Library of Congress. At the very least, it would mean contacting each of these Mastodon instances to get copies of their databases and feeds of ongoing posts. And even if there was a mechanism to do that, internet users are more aware about rights to their digital content (or at least more savvy of their digital footprint); some sort of user consent would likely be needed.

Inherent in the structure of independent Mastodon instances is the fact that there isn’t a central point of aggregation, and that is seen by the broader community as a good thing. (The most common reason I’ve heard is that the lack of a search tool makes finding the discussion of controversial topics harder and decreases the likelihood of bad actors “dogpiling” into a conversation.) There have been attempts to aggregate content for a search engine, but Mastodon administrators quickly ban those kinds of ActivityPub peers. Creating an archive of Mastodon posts will likely run into the same issues.

Do We Want a Digital Public Square Archive?

Let’s step further back: should there be an archive of the digital public square? Physical public squares don’t have comprehensive archives. The fact that a digital public square is made up of ones and zeros in files and databases makes it at least conceivable. (Setting aside the technical challenges that the Library of Congress faced with the Twitter archive; with progress in technology and techniques, having such an archive will likely be technically possible at some point.) As the MIT Technology Review article points out, there are benefits to such an archive. Perhaps archivists and historians can help aim us toward ideas that make sense for this new public space.

Calls For Cryptocurrency Regulation / David Rosenthal

On 8th July 2022 Lael Brainard, Vice-Chair of the Federal Reserve governors gave a speech entitled Crypto-Assets and Decentralized Finance through a Financial Stability Lens in which she writes:
Distinguishing Responsible Innovation from Regulatory Evasion
New technology often holds the promise of increasing competition in the financial system, reducing transaction costs and settlement times, and channeling investment to productive new uses. But early on, new products and platforms are often fraught with risks, including fraud and manipulation, and it is important and sometimes difficult to distinguish between hype and value. If past innovation cycles are any guide, in order for distributed ledgers, smart contracts, programmability, and digital assets to fulfill their potential to bring competition, efficiency, and speed, it will be essential to address the basic risks that beset all forms of finance. These risks include runs, fire sales, deleveraging, interconnectedness, and contagion, along with fraud, manipulation, and evasion. In addition, it is important to be on the lookout for the possibility of new forms of risks, since many of the technological innovations underpinning the crypto ecosystem are relatively novel.
The G20's Financial Stability Board followed with FSB Statement on International Regulation and Supervision of Crypto-asset Activities making a similar pitch for regulation. As did the European Central Bank with Decentralised finance – a new unregulated non-bank system?. Paul Krugman asks the right question in Crypto Is Crashing. Where Were the Regulators?:
Traditional banking is regulated for a reason; crypto, in bypassing these regulations, [Lael Brainard] said, has created an environment subject to bank runs, not to mention “theft, hacks and ransom attacks” — plus “money laundering and financing of terrorism.”

Other than that, it’s all good.

The thing is, most of Brainard’s litany has been obvious for some time to independent observers. So why are we only now hearing serious calls for regulation?
Below the fold I argue that these calls are both very late, and are couched in self-defeating language.

These calls from the regulators are not that new. For example, on 1st November 2021 the US Treasury issued President’s Working Group on Financial Markets Releases Report and Recommendations on Stablecoins saying:
The potential for the increased use of stablecoins as a means of payments raises a range of concerns, related to the potential for destabilizing runs, disruptions in the payment system, and concentration of economic power. The PWG report highlights gaps in the authority of regulators to reduce these risks.

To address the risks of payment stablecoins, the agencies recommend that Congress act promptly to enact legislation to ensure that payment stablecoins and payment stablecoin arrangements are subject to a federal framework on a consistent and comprehensive basis.
According to the FT, Treasury claimed it would take direct action if congress failed to act. But congress was already failing to act. Nearly a year earlier, on 2nd December 2020 Representatives Tlaib, García and Lynch had introduced the Stablecoin Tethering and Bank Licensing Enforcement (STABLE) Act to subject stablecoin issuers to banking regulation. Did Treasury take direct action? Not so much.

When the bill was introduced USDT and USDC together had issued about $23B, when the Treasury called for regulation they had issued about $105B — about $7.5B/month. Combined issuance peaked at around $132B before the Terra/Luna collapse since when it has declined to $122B, still more than 5 times the amount when the bill was introduced. USDT has suffered a marked loss in market share to USDC, whose claims of backing are deemed more credible, which in itself suggests the need for regulation.

It may be that the regulators' calls for regulation are intended to deflect blame for inaction by pointing fingers at the legislators. But if not, regulators are making a set of fundamental mistakes:
  1. By saying the discussion, as Brainard does, is about "digital assets" or as others do, about "digital ledger technology" or "blockchains", the regulators are adopting the crypto-bros' frame. These terms conflate two completely different technologies; permissioned or centralized systems with an obvious locus of control to which regulations can be applied, and permissionless or decentralized systems which claim to lack a locus of control, and thereby to be immune from regulation.

    The major design goal of permissionless cryptocurrencies such as Bitcoin and Ethereum was to escape regulation; conflating them with permissioned systems allows the latter to claim immunity from regulation because they are "digital assets" just like Bitcoin. Regulators should focus the discussion on permissionless systems, and in passing mention that existing regulations apply to permissioned systems.

  2. In focusing on permissionless systems, regulators again accept the crypto-bros' frame by accepting the claim that they are "decentralized" and thus lack loci of control to which regulations can be applied. As has been obvious all along, but has recently received support from Trail of Bits and Prof. Angela Walch, these systems in practice do have loci of control, and use the claim of "decentralization":
    as a veil that covers over and prevents many from seeing, ... the actions of key actors within the system.
    Regulators should not use the term "decentralized" but should instead focus on the many ways in which permissionless systems are in practice centralized, targeting these loci of control for regulation. Notice how the "digital asset" framing prevents them doing so.

  3. Regulators typically start the way Brainard does, by praising the promised benefits of cryptocurrency "innovation" such as:
    increasing competition in the financial system, reducing transaction costs and settlement times, and channeling investment to productive new uses.
    Again, buying in to the crypto-bros' framing in this way is counter-productive for two reasons. First, permissioned blockchain technology is thirty years old, and permissionless blockchain technology is not that innovative either; Satoshi Nakamoto simply assembled a set of well-known techniques to implement a cryptocurrency. Second, the products built on these technologies are not at all innovative, they simply replicate existing financial products without all the pesky regulation that would prevent their promoters ripping off the suckers.

    Avg. $/Transaction
    Regulators should avoid any reference to pie-in-the-sky potential benefits and argue that permissionless systems are necessarily less efficient and slower than centralized systems because of the cost and time overhead of the underlying consensus mechanism. And that they are less secure because of the risk of 51% attacks and their increased attack surface.

    Median Confirmation Time
    They can support this argument by pointing out that over the last year (a) the average Bitcoin transaction cost has varied between $50 and $300, (b) the median time to six-block finality has varied between 24 and 80 minutes, and (c) the Bitcoin network is processing less than five "economically meaningful" transactions between individuals per minute. These are numbers that would not strain a centralized system running on a Raspberry Pi. Note again how the "digital assets" framing prevents them from making this argument.

Permissionless cryptocurrencies were designed to evade regulation, so fighting them on their own "decentralized" ground isn't a winning strategy. Permissioned cryptocurrencies arose because the permissionless systems were so slow, inefficient and unwieldy; lumping them together with permissionless systems obscures their vulnerability to regulation, and the permissionless systems' weak points.

The Exchange You Can Trust / David Rosenthal

One of the many ironies about "decentralized, trustless" cryptocurrencies is that they are neither decentralized nor trustless. Since in practice you can neither buy nor sell real goods using them, you need to trust an exchange to convert fiat to cryptocurrency and vice versa. Exchanges range from those you definitely shouldn't trust, such as Binance, through somewhat less sketchy ones such as Kraken (now being investigated for sanctions busting) to Coinbase, which presents itself as a properly regulated, US based exchange that is totally trustworthy.

But recently cracks have been appearing in their façade of respectability, big enough that even the New York Times has noticed. The Humbling of Coinbase by David Yaffe-Bellany and Mike Isaac summarizes the situation:
Coinbase rose to prominence as one of the first major crypto companies, a gateway to the chaotic world of digital assets for amateur investors. But as it has grown from plucky start-up to publicly traded company, its status as an industry leader has been threatened by a series of missteps and a steep decline in the crypto market over the last six months.
Below the fold I probe into some of these cracks.

Trying To Sell Unregistered Securities

Last September David Gerard described the Coinbase Lend fiasco:
Encouraged by BlockFi and now Celsuis’s complete lack of any issues with the authorities, popular crypto exchange Coinbase decided it would get into the crypto lending game too — with Coinbase Lend! Coinbase would make loans, and you could buy a share in the loans. [Coinbase, archive]

As it happens, this is a type of security called a “bond”, and it’s listed in the first page of the Securities Act of 1933.

The SEC sent Coinbase a Wells Notice — the letter they send before prosecuting. Coinbase haven’t put the letter up, but they did say it was the SEC threatening them with prosecution if they went ahead with Lend.

Coinbase’s CEO, Brian Armstrong, posted a Twitter thread about the SEC’s “really sketchy behavior,” and Chief Legal Officer Paul Grewal blogged about how “We don’t know why.” [Twitter; Medium]

The responses divided roughly into:
crypto people: darn that SEC and their vicious lack of regulatory clarity!
non-crypto people: it’s clearly a bond, Coinbase can’t possibly be this stupid in real life.
The SEC tweeted a helpful thirty-second video explaining what a bond was. [Twitter]

Doomberg wondered how the Wells Notice could be about Lend, because it hadn’t launched yet — “you don’t get a Wells Notice for something you are considering doing, you get one for deeds you’ve already done.” But Coinbase filed an 8-K about the Wells Notice, saying it was for Lend. [Substack; SEC]

Coinbase insiders dumped a pile of stock the day they received the Wells Notice. Coinbase pointed out that the sale was scheduled in the proper manner — but you’d think a Wells Notice was a sufficiently material event to cancel such a dump. [Twitter; Medium]

Eventually, Coinbase realised that a public company blatantly violating the Securities Act after an SEC warning is probably not a winning move — even if it gets you clout on crypto Twitter. Lend is no longer being offered. The company plans to still argue the point, though. [Coinbase, archive; Twitter]

Fiasco In India

The same week that Coinbase announced their launch in India:
Coinbase got some bad news. A government-backed group issued a statement suggesting that the company would be unable to use a crucial payments platform — a system that was supposed to allow Coinbase customers to convert their rupees into virtual currencies like Bitcoin and Ether. Not long after its grand opening, Coinbase halted much of its trading service in India.
...
Despite its early start, Coinbase has never had a strong hold over the international market, which is dominated by Binance. The company went into India despite widespread uncertainty about how the government would react, an approach that industry experts considered unwise.

Misleading Customers About Their Funds And Keys

In April David Gerard reported on panic among Coinbase's customers:
In April, the SEC put forward new accounting rules for holding cryptocurrencies on behalf of a customer: you should account these as liabilities. [SEC]

So Coinbase duly stated in their quarterly 10-Q SEC filing that: [SEC]
because custodially held crypto assets may be considered to be the property of a bankruptcy estate, in the event of a bankruptcy, the crypto assets we hold in custody on behalf of our customers could be subject to bankruptcy proceedings and such customers could be treated as our general unsecured creditors.
Not just cryptos on deposit for trading — but cryptographic private keys that customers gave Coinbase to look after, because keeping your keys safe turns out to be super-hard.

Proper banks cannot rifle through the safe deposit boxes to pay their bills — but Coinbase isn’t a bank, it just wants to pretend it’s as safe as one, when it absolutely isn’t.

Coinbase put on its “no risk of bankruptcy” shirt, which promptly raised a lot of questions answered by the shirt. [FT, paywalled; Twitter]
It was clearly news to many Coinbase customers that they were not protected by SIPC, like customers of a real exchange, or FDIC, like customers of a real bank.

"Losing Money Running A Casino"

In the same post Gerard added the wry note:
The 10-Q filing also revealed that Coinbase had somehow managed to lose buckets of money running a casino. Shares in Coinbase fell 23%. [Bloomberg]
Coinbase Sinks After Warning the Slide in Volume to Worsen by Olga Kharif and Yueqi Yang provided more datils:
Coinbase Global Inc. shares tumbled after first-quarter revenue missed estimates and the largest U.S. cryptocurrency exchange warned that total trading volume in the current quarter will be lower than in the first.

The company’s shares fell about 16% after the close of regular trading. Monthly transacting users fell to 9.2 million, below an estimate of 9.5 million. First-quarter revenue slumped to $1.17 billion, while analysts were expecting revenue of $1.48 billion, according to Bloomberg data.
...
Coinbase earns the bulk of its revenue from trading fees, and its shares have fallen to all-time lows -- down more than 70% from where they traded when the company went public a year ago.
Note that when the story came out, BTC was trading at $31.5K, a month before Terra/Luna would start the slide to below $20K.

NFT Marketplace Fiasco

Coinbase was late to the NFT bubble:
Coinbase hoped to unveil the marketplace in the first quarter of 2022, Mr. Saxena said in an interview. But it was delayed until late April. By that point, the broader NFT market had cratered: Sales were down more than 80 percent from the fall.

After its release, the marketplace got scathing reviews. In the last week of July, it generated about $24,000 a day in trades; its main competitor, OpenSea, which serves as a kind of eBay for NFTs, generated 600 times that amount.

Tolerating Insider Trading

The next issue arrived with Coinbase Insider Trading Arrest Highlights “Altcoin” Problem by Max Chafkin, who started by explaining Coinbase's notorious "pivot to shitcoins":
Shortly after taking his company public, Armstrong announced that Coinbase would expand the number of coins listed on its service to keep up with investor demand. ... Over the next year, the company would add more than 100 new tokens.
...
This made a certain amount of sense. Up to that point, Coinbase had been more cautious than competitors, listing only Bitcoin, Ether, and a handful of other well-established tokens. These restrictions had mostly kept its customers from getting wrapped up in the scams, hacks, rug-pulls, and pump-and-dump schemes that have dogged crypto for years. But Coinbase’s restraint created an opportunity for competitors—especially Binance, which offered hundreds of tokens—to gain market share.
How did the pivot work out?
At first, the pivot to shitcoin seemed to work out for Coinbase, which saw its stock price climb in the months that followed as digital asset speculation approached its peak. But in hindsight, one could be forgiven for seeing the move as reckless. Last week, federal prosecutors arrested a former Coinbase product manager, Ishan Wahi, accusing him of insider trading.
In "You Don't Own Web3": A Coinbase Curse and How VCs Sell Crypto to Retail, Fais Khan explained the opportunity for insider trading at Coinbase:
For years, being listed for trading on Coinbase has been the holy grail of crypto - the equivalent of an IPO on Wall Street. And like an IPO, that seems to come up with a “pop” - Messari, a crypto research firm, documented in a report that the average Coinbase listing leads to a 91% gain in 5 days, on average.
Chia Coin
If you knew ahead of time which coins were going to be listed, you could lock in an average 91% gain. The chart shows A16Z-funded Chia Coin, which launched at $669 and four days later hit $1346 for a 100% gain.

And in The Unstoppable Grift: How Coinbase and Binance Helped Turned Web3 into Venture3, Fais Khan explains how these price spikes led Coinbase to list shitcoins:
the average returns on Binance blew Coinbase out of the water - although the difference in the returns was staggering.

And the thing is, I think Coinbase knows that. Because if you look at the direction they took in 2021, they made a sharp turn to not only add a lot more assets, but being first to list coins, even some on the first day.

That seems risky to me. These are coins with tiny floats that may have only been founded as little as a year prior, and now they’re in the hands of tens of millions of retail investors - who the data shows are often less sophisticated than investors in stocks.

Why else does it matter? It’s also clearly the direction all of the industry is going in: exchanges creating huge venture funds and then aggressively listing coins (often that they’ve invested in) faster and faster. All while they shovel tens of billions of dollars into web3 startups to create an “ecosystem” of more coins that they can use to generate trading revenues.
Doubling in four days is quite the ROI, and thus quite the temptation. Not just for A16Z and the other VCs to run List And Dump Schemes, but also insiders:
Prosecutors said that starting last June, Wahi tipped off his brother and a friend about new listings, which allowed them to make about $1.5 million in profit by buying lightly traded cryptocurrencies ahead of new listings.
Chafkin points out that the insider trading wasn't exactly secret:
The incident hardly suggests rigor on Coinbase’s part, however. The alleged insider trading seems to have been first spotted by a crypto influencer, Jordan Fish, who tweets under the pseudonym Cobie (short for Crypto Cobain) and who said he’d been complaining publicly for months about insider trading on Coinbase. That Coinbase investigated his Twitter tip is to the company’s credit. On the other hand, as Cobie put it, “surely Coinbase should have found this before randoms on Twitter did?”
Coffeezilla interviewed Cobie, who pointed out that it was even worse (my transcript):
Cobie: Coinbase has had listings front-running issues for a while. Originally they only listed like Bitcoin and they had this policy like they were only going to list the best of the best. And then as time passed and they got to like the last couple of years, meme stocks became a thing and obviously they went public, they started losing market share. They sort of changed their policy and started listing like real, real, real garbage but at the same time they started having a lot of issues around people buying the garbage they were listing beforehand. Then they published this blog post about how they're going to change all their listing processes so that this stuff cannot happen in the future and they can get better controls around it. And the very next listing, every single coin got front-run once again.
How did Cobie know?:
Cobie: I'm in a little group of like crypto people who like look at interesting blockchain stuff and one sent this Etherscan link and said "hey, has anyone looked into this, I think these things are going to get listed on Coinbase because they all got bought again by a person who front-run successfully last time. Does anyone know what any of these are? Some of them seem like really, really garbage". I think one of them was called "The Food" or something. I looked at the link, I waited for the coins to get added, about 80% of the ones that were bought on this day were added. So then I tweeted the screenshot of everything he purchased and the blog post like a few weeks ago saying we're tightening up our controls just to say like Coinbase is clearly incompetent at these issues, they're unable to stop it. They've just done this whole song and dance about how much they're going to improve the ecosystem and how much they care about like getting these things right and the very next coin listing has been front-run.
Wahi was a Coinbase employee. He was part of the team responsible for listing new coins. He had been front-running for 18 months. Coinbase didn't notice this until Cobie tweeted about it. He was arrested at the airport as he tried to flee to India.

Actually Selling Unregistered Securities

The very same day Allyson Versprille and Lydia Beyoud reported that Coinbase Faces SEC Probe on Crypto Listings; Shares Tumble:
Coinbase Global Inc. is facing a US probe into whether it improperly let Americans trade digital assets that should have been registered as securities, according to three people familiar with the matter. The company’s shares dropped 21%.

The US Securities and Exchange Commission’s scrutiny of Coinbase has increased since the platform expanded the number of tokens in which it offers trading, said two of the people, who asked not to be named because the inquiry hasn’t been disclosed publicly. The probe by the SEC’s enforcement unit predates the agency’s investigation into an alleged insider trading scheme that led the regulator last week to sue a former Coinbase manager and two other people.
The "digital assets that should have been registered as securities" are the shitcoins to which Coinbase pivoted in an attempt to catch up to Binance.

Forcing Arbitration On Customers

Greg Stohr's Coinbase Asks Supreme Court to Halt Account-Holder Suits shows how Coinbase thinks unhappy customers' disputes should be handled:
Coinbase Global Inc. asked the US Supreme Court to halt two lawsuits by users of the cryptocurrency exchange platform while the company presses appeals that seek to send the cases to arbitration.

In one case, a man says Coinbase should compensate him for $31,000 he lost after he gave remote access to his account to a scammer. In the other, Coinbase is accused of violating California consumer law by holding a $1.2 million Dogecoin sweepstakes without adequately disclosing that entrants didn’t have to buy or sell the cryptocurrency. Both suits seek class action status.

Federal trial judges in both cases rejected Coinbase’s bid to send the disputes to arbitration, which the company says is required under its user agreements.
How likely is it that an arbitrator will rule for the little guy?

Staking Customer's Coins

Yueqi Yang's Coinbase Under SEC Scrutiny Over Its Crypto-Staking Programs reveals:
Coinbase Global Inc. said it’s being probed by the US Securities and Exchange Commission over its staking programs, which allow users to earn rewards for holding certain cryptocurrencies.

The company “has received investigative subpoenas and requests from the SEC for documents and information about certain customer programs, operations and existing and intended future products,” according to a quarterly regulatory filing. The requests relate to Coinbase’s staking programs, asset-listing process, classification of assets and stablecoin products, the company said.
...
At Coinbase, blockchain-rewards revenue, primarily from staking, accounted for 8.5% of net revenue in the second quarter. It fell 16% sequentially to $68.4 million during the quarter, less than the decline in trading revenue.

And The Result Is

Coinbase Falls After Second-Quarter Revenue Misses Estimates by Olga Kharif and Yueqi Yang reports on the outcome:
Shares of the company, which were first listed last April, dropped about 4% after the close of regular trading. Coinbase has slumped 65% so far this year
...
Revenue declined to $808.3 million, missing the $854.8 million estimate from analysts polled by Bloomberg. Monthly transacting users dropped to 9 million in the second quarter, a 2% decline from prior quarter.

Coinbase lost $1.1 billion in the three months ended June 30, including a $446 million non-cash impairment charges related to investments and ventures, making it the largest amount since it became a public company.

They're Not Alone

Another example of a supposedly trustworthy US exchange heavily into cryptocurrencies is Robinhood. Robinhood Crypto Unit Fined $30 Million by New York Regulator by Annie Massa reveals that:
Robinhood Markets Inc.’s cryptocurrency arm was fined $30 million by New York’s financial regulator after the brokerage was accused of violating anti-money-laundering and cybersecurity rules.

The unit must enlist an independent consultant to monitor compliance, according to an order filed Tuesday. The firm disclosed last year that it expected to pay the penalty.

The enforcement action by the New York State Department of Financial Services underscores the continued regulatory scrutiny Robinhood faces, even as it pushes a message to investors that it’s taking a “safety first” stance toward digital tokens.

Impossibilities / David Rosenthal

I'm starting to see a series of papers each showing that some assertion about the cryptocurrency ecosystem that crypto-bros make can't be true. I wrote about the first one I noticed in Ethereum Has Issues, but I have since seen several more. Below the fold I briefly review them, I'll update this post if I see more to maintain a chronological list of these research results.

The list so far is:
  1. Blockchain Economics by Joseph Abadi and Markus Brunnermeier (18th June 2018) introduced the Blockchain Trilemma:
    The ideal qualities of any recordkeeping system are (i) correctness, (ii) decentralization, and (iii) cost efficiency. We point out a Blockchain Trilemma: no ledger can satisfy all three properties simultaneously.
  2. Bitcoin: A Natural Oligopoly by Nick Arnosti and S. Matthew Weinberg (21 November 2018) formalizes my observation in 2014's Economies of Scale in Peer-to-Peer Networks that economies of scale drive centralization:
    (a) ... if miner j has costs that are (e.g.) 20% lower than those of miner i, then miner j must control at least 20% of the total mining power. (b) In the presence of economies of scale (α>1), every market participant has a market share of at least 1−1/α, implying that the market features at most α/(α−1) miners in total.
  3. Impossibility of Full Decentralization in Permissionless Blockchains by Yujin Kwon et al (1st September 2019) provides a different formalization of the idea that economies of scale drive centralization by introducing the concept of the "Sybil cost":
    the blockchain system should be able to assign a positive Sybil cost, where the Sybil cost is defined as the difference between the cost for one participant running multiple nodes and the total cost for multiple participants each running one node.
    ...
    Considering the current gap between the rich and poor, this result implies that it is almost impossible for a system without Sybil costs to achieve good decentralization. In addition, because it is yet unknown how to assign a Sybil cost without relying on a TTP [Trusted Third Party] in blockchains, it also represents that currently, a contradiction between achieving good decentralization in the consensus protocol and not relying on a TTP exists.
  4. SoK: Communication Across Distributed Ledgers by Alexei Zamyatin, Mustafa Al-Bassam, Dionysis Zindros, Eleftherios Kokoris-Kogias, Pedro Moreno-Sanchez, Aggelos Kiayias, and William J. Knottenbelt (2nd October 2019, updated 5th February 2021, also here) shows the impossibility of trustless cross-chain communication:
    Since the inception of Bitcoin, a plethora of distributed ledgers differing in design and purpose has been created. While by design, blockchains provide no means to securely communicate with external systems, numerous attempts towards trustless cross-chain communication have been proposed over the years. Today, cross-chain communication (CCC) plays a fundamental role in cryptocurrency exchanges, scalability efforts via sharding, extension of existing systems through sidechains, and bootstrapping of new blockchains. Unfortunately, existing proposals are designed ad-hoc for specific use-cases, making it hard to gain confidence in their correctness and composability. We provide the first systematic exposition of cross-chain communication protocols.

    We formalize the underlying research problem and show that CCC is impossible without a trusted third party, contrary to common beliefs in the blockchain community. With this result in mind, we develop a framework to design new and evaluate existing CCC protocols, focusing on the inherent trust assumptions thereof, and derive a classification covering the field of cross-chain communication to date. We conclude by discussing open challenges for CCC research and the implications of interoperability on the security and privacy of blockchains.
    They show the impossibility by reducing CCC to the Fair Exchange problem:
    On a high level, an exchange between two (or more) parties is considered fair if either both parties receive the item they expect, or neither do. Fair exchange can be considered a sub-problem of fair secure computation, and is known to be impossible without a trusted third party
    Their analysis is informed by their earlier work implementing Xclaim, described in Xclaim: Trustless, Interoperable, Cryptocurrency-Backed Assets by Alexei Zamyatin, Dominik Harz, Joshua Lind, Panayiotis Panayiotou, Arthur Gervais and William Knottenbelt (6th July 2018):
    We present Xclaim: the first generic framework for achieving trustless and efficient cross-chain exchanges using cryptocurrency- backed assets (CBAs). Xclaim offers protocols for issuing, transferring, swapping and redeeming CBAs securely in a non-interactive manner on existing blockchains. We instantiate Xclaim between Bitcoin and Ethereum and evaluate our implementation; it costs less than USD 0.50 to issue an arbitrary amount of Bitcoin-backed tokens on Ethereum. We show Xclaim is not only faster, but also significantly cheaper than atomic cross-chain swaps. Finally, Xclaim is compatible with the majority of existing blockchains without modification, and enables several novel cryptocurrency applications, such as cross- chain payment channels and efficient multi-party swaps.
  5. High-frequency trading on decentralized on-chain exchanges by L. Zhou, K. Qin, C. F. Torres, D. V. Le, and A. Gervais (29th September 2020) points out a problem facing "decentralized exchanges" (DEX):
    Our work, sheds light on a dilemma facing DEXs: if the default slippage is set too low, the DEX is not scalable (i.e. only supports few trades per block), if the default slippage is too high, adversaries can profit.
  6. Responsible Vulnerability Disclosure in Cryptocurrencies by Rainer Böhme, Lisa Eckey, Tyler Moore, Neha Narula, Tim Ruffing & Aviv Zohar (October 2020) see also Rethinking Responsible Disclosure for Cryptocurrency Security by Stewart Baker (8th September 2022). Both essentially argue that it is impossible for a trustless, decentralized system to prevent the inevitable vulnerabilities being exploited before they can be fixed. My restatement of their argument is:
    • Cryptocurrencies are supposed to be decentralized and trustless.
    • Their implementations will, like all software, have vulnerabilities.
    • There will be a delay between discovery of a vulnerability and the deployment of a fix to the majority of the network nodes.
    • If, during this delay, a bad actor finds out about the vulnerability, it will be exploited.
    • Thus if the vulnerability is not to be exploited its knowledge must be restricted to trusted developers who are able to upgrade vulnerable software without revealing their true purpose (i.e. the vulnerability). This violates the goals of trustlessness and decentralization.
    Both Böhme et al and Baker provide examples of the problem in practice.
  7. Irrationality, Extortion, or Trusted Third-parties: Why it is Impossible to Buy and Sell Physical Goods Securely on the Blockchain by Amir Kafshdar Goharshady (19th October 2021) reveals a major flaw in the use of cryptocurrencies as currencies rather than as gambling chips:
    Suppose that Alice plans to buy a physical good from Bob over a programmable Blockchain. Alice does not trust Bob, so she is not willing to pay before the good is delivered off-chain. Similarly, Bob does not trust Alice, so he is not willing to deliver the good before getting paid on-chain. Moreover, they are not inclined to use the services of a trusted third-party. Traditionally, such scenarios are handled by game-theoretic escrow smart contracts, such as BitHalo. In this work, we first show that the common method for this problem suffers from a major flaw which can be exploited by Bob in order to extort Alice. We also show that, unlike the case of auctions, this flaw cannot be addressed by a commitment-scheme-based approach. We then provide a much more general result: assuming that the two sides are rational actors and the smart contract language is Turing-complete, there is no escrow smart contract that can facilitate this exchange without either relying on third parties or enabling at least one side to extort the other.
  8. The Consequences of Scalable Blockchains on Datafinnovation's blog (1st April 2022) shows that implementing an Ethereum-like system whose performance in all cases is guaranteed to be faster than any single node in the network is equivalent to solving the great unsolved problem in the theory of computation, nicknamed P vs. NP. And thus that if it were implemented, the same technique could break all current cryptography, including that underlying Ethereum:
    What we are going to do here is pretty simple:
    1. Describe some thing a scalable blockchain could do.
    2. Prove that thing is NP-Complete.
    3. Show how, if you have such a blockchain, you can right now break hash functions and public-key cryptography and constructively prove P=NP.
    If you build this thing you can break nearly all the major protocols out there — blockchains, banks, SSL, RSA, nearly everything — right now.
    NB: it appears that the first application of computer science impossibility results to cryptocurrencies was in Ethereum's DAO Wars Soft Fork is a Potential DoS Vector by Tjaden Hess, River Keefer, and Emin Gün Sirer (28th June 2016), which applied the 'halting problem" to "smart contracts" when analyzing possible defenses against DOS attacks on a "soft fork" of Ethereum proposed in response to "The DAO".
  9. Sharding Is Also NP-Complete by Datafinnovation (2nd April2022) uses the same proof technique to show that sharding is subject to the same worst-case problem as scaling a single blockchain:
    The point of this post is not that sharding is useless. Sharding certainly helps sometimes. It might even help “on average.” But this is a hard problem. This leaves us with two choices:
    1. Scalable solutions which are prone to accidents
    2. Truly reliable scalability but P=NP etc
    What do I mean by accidents? Systems that fall over when they are overloaded. Whether that is exploding block times or proper crashing or whatever else is a software engineering question rather than a math one. But something bad. Mitigation is a requirement if you want a robust system because you can’t engineer around this challenge and still have the cryptography.
  10. Positive Risk-Free Interest Rates in Decentralized Finance by Ben Charoenwong, Robert M. Kirby, and Jonathan Reiter (14th April 2022) is summarized in Impossibility of DeFi Risk-Free Rates:
    This paper explores the idea of risk-free rates in trustless DeFi systems. The main result is that it is impossible, under a clearly stated set of conditions, to generate conventional risk-free rates.
    The paper uses a model:
    representing a large class of existing decentralized consensus algorithms [to show] that a positive risk-free rate is not possible. This places strong bounds on what decentralized financial products can be built and constrains the shape of future developments in DeFi. Among other limitations, our results reveal that markets in DeFi are incomplete.
    The paper was updated on 28th August 2022.
  11. Blockchain scalability and the fragmentation of crypto by Frederic Boissay et al (7th June 2022) formalizes and extends the argument I made in Fixed Supply, Variable Demand (3rd May 2022):
    To maintain a system of decentralised consensus on a blockchain, self-interested validators need to be rewarded for recording transactions. Achieving sufficiently high rewards requires the maximum number of transactions per block to be limited. As transactions near this limit, congestion increases the cost of transactions exponentially. While congestion and the associated high fees are needed to incentivise validators, users are induced to seek out alternative chains. This leads to a system of parallel blockchains that cannot harness network effects, raising concerns about the governance and safety of the entire system.
  12. Decentralized Stablecoin Design by Ben Charoenwong, Robert M. Kirby, and Jonathan Reiter (28th August 2022) uses the halting problem and Goharshady (2021) to investigate the stability of metastablecoins that are not fully backed by fiat held by a trusted party:
    Our methodology is as follows. First, we present a product with definitions taken from an economics context. This product may or may not have some desirable property. The question of whether it has the property is then formulated as a decision problem. Then, we use the theory of computation to reduce the problem using isomorphism and show that the reduced decision problem is undecidable based on the large extant literature on computer science. Finally, we then conclude that due to the undecidability, constructing such a product which provably satisfies the desirable property is impossible.
    They are careful not to over-sell their results:
    a caveat of our results is that the theoretical results pertain to the computational feasibility and provability of the property. It does not imply that a given design will not work for some (possibly very long) period of time. It simply means that we cannot know for sure that it will.
  13. The Compliance-Innovation Trade-off (October 31st 2022) by Datafinnovation describes a generalization of the impossibility proof technique they have used in their previous results. They start from Gödel's incompleteness theorems via the halting problem and Kleene's Recursive predicates and quantifiers which showed that:
    all systems that can express arithmetic contain the halting problem and offer the degree of computational power we call Turing-complete today.
    to Rice's theorem which shows:
    you cannot write a program that checks if another program has any particular non-trivial property.
    They observe that:
    the most powerful sort of computer we can have that doesn’t experience the halting problem and therefore Rice’s extension is known as a linear bounded automaton (LBA).
    They argue thus:
    1. We want to mechanize finance
    2. Finance needs arithmetic
    3. Godel tells us if we can do arithmetic we need to choose if our system is inconsistent or incomplete
    4. We need certainty over whether a given payment was made and around account balances, so we cannot pick inconsistency and therefore choose incompleteness
    5. Kleene tells us our system has the halting problem and all that comes with it
    6. We can then copy Savage’s proof for Rice’s theorem and adopt that result
    Thus the properties of the finance system cannot be checked automatically. The code can be reviewed, but if it calls external functions nothing can be proven. Even if it doesn't, unless either it is immutable, or if any changes are implemented using an LBA, nothing can be proven.

    They conclude:
    Permissionless systems that offer changeability >LBA cannot credibly comply with any regulations.

    Permissioned systems can — to the extent we trust whoever set up the system and does the permissioning going forward.

Non-Fungible Token Bubble Lasted 10 Months / David Rosenthal

Although the first Non-Fungible Token was minted in 2014, it wasn't until Cryptokitties bought the Ethereum blockchain to its knees in December 2017 that NFTs attracted attention. But then they were swiftly hailed as the revolutionary technology that would usher in Web 3, the Holy Grail of VCs, speculators and the major content industries because it would be a completely financialized Web. Approaching 5 years later, it is time to ask "how's it going?"

Below the fold I look at the details, but the TL;DR is "not so great"; NFTs as the basis for a financialized Web have six main problems:
  1. Technical: the technology doesn't actually do what people think it does.
  2. Legal: there is no legal basis for the "rights" NFTs claim to represent.
  3. Regulatory: much of the business of creating and selling NFTs appears to violate securities law.
  4. Marketing: the ordinary consumers who would pay for a financialized Web absolutely hate the idea.
  5. Financial: like cryptocurrencies, the fundamental attraction of NFTs is "number go up". And much of the trading in NTFs was Making Sure "Number Go Up". But, alas "number go down", at least partly because of problem #4.
  6. Criminal: vulnerabilities in the NFT ecosystem provide a bonanza for thieves.

Source
P Nielsen Hayden (@pnh) tweeted "1942. Flann O'Brien invents the NFT". O'Brien wrote:
You get nothing you can see or feel, not even a receipt. But you do yourself the honour of participating in one of the most far-reaching experiments ever carried out in my literary work-shop.
Alas, Flann O'Brien was ahead of 1942's technology. For the real story of the early history of NFTs you should consult David Gerard and Amy Castor. Here I am just going to discuss each of the problems that emerged once people started using the technology.

Technical

Moxie's NFT in a wallet
The technical problem is that the connection between an NFT and the resource it purports to represent is tenuous in the extreme, as Moxie Marlinspike brilliantly demonstrated (both links point to the same NFT) in My first impressions of web3. I explain the problem in detail in NFTs and Web Archiving but the brief version is:
the purchaser of an NFT is buying a supposedly immutable, non-fungible object that points to a URI pointing to another URI. In practice both are typically URLs. The token provides no assurance that either of these links resolves to content, or that the content they resolve to at any later time is what the purchaser believed at the time of purchase.
Ownership of a non-fungible real-world object, say a house, confers the right to control access to it. So it was natural that NFT owners would expect the same right. They were disappointed, which led to loud complaints about "right-clicker mentality among the uninitiated:
what is the “right-clicker mentality”? Quite literally, it is referring to one’s ability to right-click on any image they see online to bring up a menu and select the “save” option in order to save a copy of the image to their device. In this term we have a microcosm of the entire philosophical debate surrounding NFTs.
Another thing Marlinspike documented is that almost all NFT marketplaces, wallets and associated services don't use the underlying blockchain directly, but only via a couple of API services (Infura and Alchemy). Thus there will always be a centralized service between you and whatever your NFT is supposed to do. An example of this fragility of the connection between an NFT and the resource it purports to represent is Molly White's Unstoppable Domains disables .coin extensions, illustrating an issue with the idea that "you'll always own your NFT":
Unstoppable Domains is in the business of selling "domains" — at least that's what they call them, but they're not the kind of domain that you can plug into your web browser. Instead, they are more like the ENS domains that you have may have seen (the ones ending in .eth), and they typically map to a crypto wallet address. The organization just discovered that they were not the first to go around selling .coin "domains" (represented by NFTs), and were at risk of running into collisions. As a result, they decided to no longer sell these domains, and stop their libraries and services from resolving them.

But fear not, they said, because "Unstoppable domains are self-custodied NFTs, so you still own your .coin domain, but it won’t work with our resolution services or integrations."

That's right, folks, you'll still have your .coin NFT! It just won't resolve, or be otherwise useful in any way.
The token on the blockchain provides no control over either of the URLs, so it isn't able to fulfill reasonable expectations for the result of purchasing it.

Legal

I wrote in NFTs and Web Archiving:
There is no guarantee that the creator of the NFT had any copyright in, or other rights to, the content to which either of the links resolves at any particular time.
In December 2020 it became possible for anyone to mint and sell an NFT for real money, when OpenSea announced that "any user can mint NFTs on its platform for free". At first, OpenSea tried to approve these mints, but in March 2021 they gave up trying and plagiarism erupted. People rushed to mint and sell an NFT of any random image they found on the Web. The DeviantArt online art community was a prime victim:
"At DeviantArt, we’ve seen our users suffer at the hands of bad actors in Web3 through art infringement and theft, instead of enjoying the promise and opportunity that Web3 holds for creators," CEO Moti Levy said.

"Artists are doubly punished, first through the theft of their work, and then again by having to file endless DMCA reports to multiple NFT marketplaces," Levy added.
Note that the DMCA notice to the marketplace merely prevents that marketplace offering the plagiarized NFT for sale. Immutability means that the NFT cannot be removed from the blockchain. As usual in technology markets only a few markets dominate the NFT market. So the fact that none of them allow it to appear effectively renders it worthless.

The plagiarism problem got so bad that DeviantArt developed an automated system for detecting and DMCA-ing plagiarized NFTs, as Nica Osorio reports in DeviantArt Amps Up War On Art, NFT Theft And Infringement With Protect Protocol:
The team launched the DeviantArt Protect in August 2021 which indexes nine blockchains and so far has processed nearly 330,000 Non-Fungible Token (NFT) infringement claims and indexed more than 400 million.
Now "worth" $280
Despite this, the problem was so unmanageable that some markets just gave up:
Cent, the NFT marketplace which sold Jack Dorsey's NFT of his first tweet for $2.9 million, stopped transactions on February 6. The founder explained that people selling NFTs of content they didn't own, copies of other NFT projects, and NFTs resembling securities were "rampant" problems on the platform. "We would ban offending accounts but it was like we're playing a game of whack-a-mole... Every time we would ban one, another one would come up, or three more would come up."
In NFTs and the One Precedent-setting Law Case That Can Make or Break Them by Dario Garcia Giner argues that NFTs could improve the market for physical art in two ways:
The first is a guarantee to pay artists (and the original gallerist) royalties on a piece’s resale. The second is to limit the damaging influence of predatory flippers with resale limitations. Currently, this is practically unenforceable in the industry.
...
As a proxy for physical ownership, the NFTs smart contracts could be the repository of re-selling restrictions the buyer agrees to uphold in purchasing a physical piece. The first would be NFT royalty contracts, which guarantee a certain percentage of resale income would flow back to the original beneficiaries. The second would be re-selling restrictions, which have already been discussed in the context of tying NFTs to luxury goods.

This could make the NFT the sole channel for all sales past, present, and future.
This is all typical crypto-bro touting potential benefits. Absent a legal framework recognizing NFTs as being legal contracts conferring rights and duties, this is all so much hot air. And given the problems of NFTs I'm outlining, and their underlying pseudonymity, courts should be very reluctant to recognize them in this way.

Regulatory

Bored-Ape Creator Yuga Labs Faces SEC Probe Over Unregistered Offerings by Matt Robinson illustrates the regulatory risks of NFTs:
The SEC is examining whether certain nonfungible tokens from the Miami-based company are more akin to stocks and should follow the same disclosure rules, according to a person familiar with the matter, who asked not to be named because the probe is private. Wall Street’s main regulator is also examining the distribution of ApeCoin, which was given to holders of Bored Ape Yacht Club and related NFTs. The cryptocurrency was created in part for web3, a vision of a decentralized internet built around blockchains.
...
ApeCoin gives holders influence over another crypto-native entity known as a decentralized autonomous organization, or DAO. The idea was to give the Bored Ape community a hand in shaping the decentralized, blockchain-powered vision of the internet that venture capitalists often describe as web3. The Bored Ape DAO will use the blockchain to enable and record votes on decisions related to how the community is managed.
Voting to govern a token in the hope of making a return on investing in it is pretty clearly equivalent to an equity investment, as Amy Castor's Bored Apes Yacht Club launches Apecoin. It looks like an unregistered penny stock offering pointed out in March when ApeCoin launched.

This isn't the only regulatory problem for NFTs. In regulated markets wash trading is illegal, but as I wrote in Making Sure "Number Go Up" it is endemic in cryptocurrency markets and NFTs are no exception. The most notorious example was detailed in Nick Baker's An NFT Just Sold for $532 Million, But Didn’t Really Sell at All:
The process started Thursday at 6:13 p.m. New York time, when someone using an Ethereum address beginning with 0xef76 transferred the CryptoPunk to an address starting with 0x8e39.

About an hour and a half later, 0x8e39 sold the NFT to an address starting with 0x9b5a for 124,457 Ether -- equal to $532 million -- all of it borrowed from three sources, primarily Compound.

To pay for the trade, the buyer shipped the Ether tokens to the CryptoPunk’s smart contract, which transferred them to the seller -- normal stuff, a buyer settling up with a seller. But the seller then sent the 124,457 Ether back to the buyer, who repaid the loans.

And then the last step: the avatar was given back to the original address, 0xef76, and offered up for sale again for 250,000 Ether, or more than $1 billion.

Marketing

While the idea of a financialized Web is catnip for VCs, speculators and content industries such as game publishers, those who don't know history are doomed to repeat it. We have been down this path before, the last time with the idea of micropayments. They were all the rage in the late 90s, but in 2000 Clay Shirky wrote:
The Short Answer for Why Micropayments Fail
Users hate them.
...
Why does it matter that users hate micropayments? Because users are the ones with the money, and micropayments do not take user preferences into account.
Three years later Andrew Odlyzko provided a more detailed analysis of the failure in The Case Against Micropayments. Now, Brian Feldman recounts how this played out 22 years later in How Gamers Beat NFTs:
for the better part of the past year and a half, a whole lot of people in the games business thought NFTs sounded great. With crypto booming in the summer of 2021, NFT backers grew beyond the Bitcoin faithful to include Mark Zuckerberg’s Meta, the traditional financial industry, and a string of game publishers whose market values range from modest (Team17, about $700 million) to massive (the crypto-curious Square Enix, $5.2 billion). With games, the idea, broadly speaking, was to merge the well-established, wildly profitable act of selling digital items and the buzzy world of web3, the catchall term for a hazy mix of blockchain, cryptocurrency, and virtual-reality technologies that always seem to be over the next horizon.
So how did these dreams of avarice work out?
Gamers and industry types have waged a war of public opinion online, dismissing NFT advocates as hucksters, vowing boycotts, and generally making a scene. Protests unraveled or headed off the addition of NFTs or blockchain initiatives by Valve’s Steam marketplace, which has since banned blockchain tech; Stalker2, the Chernobyl disaster sequel; the serial-killer-vs.-survivors game Dead by Daylight; voice actor Troy Baker; Electronic Arts and Sega, which both walked back initial stated interest in NFTs; Discord, the standard chat app for gamers; and, in late July, the Microsoft-owned juggernaut Minecraft. “The initial backlash from consumers was a response to the shallow game mechanics and Ponzi-scheme-like practices that informed most of the early designs,” says Joost van Dreunen, a games industry analyst and investor who teaches about the business of video games at New York University. “It isn’t fun or sustainable.”
It wasn't just gamers who hated NFTs, as David Gerard reports:
In July 2021, CNN announced with great fanfare its plan for NFTs for News! “Vault is a Web3 project by CNN to reflect on the world events we experience together. Collect NFTs of historic artistic interpretations from digital artists.” Anyway, they’ve just shut it down. [CNN Vault, archive; Twitter]

People who spent actual money on CNN NFTs are yelling that they’ve been rugpulled. Press Gazette estimated that CNN had taken in at least $329,700 by April 2022. CNN was actively promoting spending your money at Vault up to a few weeks before the shutdown. [Press Gazette; The Verge]

CNN claim Vault was just an “experiment.” This doesn’t really match up with the detailed project roadmap they published, which extends into 2023 with a “globe” icon. [CNN, archive]

Vault was running on Dapper Labs’ Flow blockchain. CNN has offered a refund of 20% of minting price — why only 20%? — in USDC stablecoins or FLOW tokens. Flow limits stablecoin withdrawals to $10 per transaction, with a $4 transaction fee. [The Block]

Financial

Another fundamental problem with NFTs is that, because they don't actually control the resource they purport to represent, they don't provide any way to extract rent from it. Thus they cannot generate income, their value is entirely speculative, based on the Greater Fool Theory. It turned out that, given the prevalence of wash trading, most of the greater fools were the original purchaser themselves. In actual arms-length trades, the fools were definitely lesser. Pranshu Verma's They spent a fortune on pictures of apes and cats. Do they regret it? provides many examples:
An NFT of Twitter founder Jack Dorsey’s first tweet, purchased last year by an Iranian crypto investor for $2.9 million, was put up for auction in April, with bids topping out at $280. A token of a pixelated man with sunglasses and hat that sold for roughly $1 million seven months ago brought just $138,000 on May 8. A digital token of an ape with a red hat, sleeveless T-shirt and multicolored grin part of the popular Bored Ape Yacht Club — purchased for over $520,000 on April 30, was sold for roughly half that price 10 days later.
Hayes Brown explains the excess of lesser fools in NFTs are plunging in popularity? Yeah, that makes sense:
NFTs and cryptocurrency depend on two things to keep their valuations high: increasing demand and perceived scarcity.

Which leads us to two problems the market is facing. First, the number of active traders has plummeted from almost a million accounts at the start of the year to about 491,000, NBC News reported Thursday. A lack of new interest or sustained interest in an asset is rarely a good sign for its longevity.

Second, there’s been a flood of supply. “There are about five NFTs for every buyer, according to data from analytics firm Chainalysis,” the Journal reported. “As of the end of April, there have been 9.2 million NFTs sold, which were bought by 1.8 million people, the firm said.”

That excess supply makes sense when you consider that everyone and their mother have been rushing to pump out an NFT in a bid to get in on the trend.
10 months only
The result is evident in Sidhartha Shukla's NFT Trading Volumes Collapse 97% From January Peak:
Trading volumes in nonfungible tokens -- digital art and collectibles recorded on blockchains -- have tumbled 97% from a record high in January this year. They slid to just $466 million in September from $17 billion at the start of 2022, according to data from Dune Analytics. The fading NFT mania is part of a wider, $2 trillion wipeout in the crypto sector as rapidly tightening monetary policy starves speculative assets of investment flows.
Amy Castor's The NFT market hasn’t crashed — it was never not crashed stresses the lack of a real NFT market:
NFTs are illiquid assets. It’s not easy to find a new sucker everyday willing to pay millions of dollars for a JPEG — not even a JPEG, but a token on a blockchain that points to a JPEG.

This is why we see situations like the one on Feb. 8, when a seller going by @0x650d on Twitter decided to “hodl” his collection of 104 CryptoPunks at the last minute. The collection was supposed to fetch $30 million at Sotheby’s — but there were no buyers.

If you accept that the NFT market is crashing, you have to accept that the NFT market ever existed at all.

The problem with NFT data is that most of it is coming from the NFT platforms themselves. There’s no way to confirm if what they are reporting is real. And there is good reason to suspect the secondary market doesn’t exist at all — it’s just wash trading, meaning the same money is going back and forth between the same people, to pump up the prices.

Most of the activity on LooksRare, a marketplace that launched in January and went on to challenge power player OpenSea, turned out to be fake. In February, Chainalysis reported “significant” wash trading on NFT platforms. Their findings made international news. On May 4, the first day that Coinbase opened its NFT marketplace to the public, the platform had barely any users. This was after Coinbase boasted that 4 million were on the waitlist.
The crashing prices called for creativity among NFT vendors. Molly White's Former footballer Michael Owen claims his NFTs "will be the first ever that can't lose their initial value" is an example:
In what almost guarantees some fun lawsuits down the line, former footballer Michael Owen tried to hit back at "the critics" by announcing that "[his] NFTs will be the first ever that can’t lose their initial value". Owen's business partner quickly turned up to do damage control, writing "we cannot guarantee or say that you cannot lose. There is always a chance".

It appeared that Owen might have meant that there would be a lower bound on resale price of the NFTs, which is neither a new concept in NFTs (see Kaiju Kongz or Rich Bulls Club), nor does it mean the NFTs "can't lose their initial value". It just means that when the NFTs do lose their initial value, collectors can't recoup even a portion of their investment.
The buyers were equally inventive. If you ever doubted that NFTs were only about "number go up", Molly White's NFT trading fantasy league emerges to provide traders with the "sweet adrenaline" of flipping NFTs that they're missing in the bear market should set your mind at ease:
"Most of us are too poor to be spending the [ether] we have left on huge sweeps, but we still want that sweet adrenaline rush of flipping JPEGs" said Brian Krogsgard, co-founder of the Flip NFT platform, in a statement you would think might have raised a red flag or two in his own mind. Evidently NFT traders are now being pitched NFT trading fantasy leagues, where they will be able to paper trade NFTs without risking their real-life fake money. Unfortunately for the traders, the app uses actual NFT price data, so the huge NFT project bull runs that some traders experienced during the NFT mania of 2021 will likely not emerge here, either.

Criminal

One fundamental problem with the idea that "you will always own your NFT" is that it is only true while your private key is secret, and even then if you only use it to sign trustworthy transactions. Neither of these is easy to maintain. Here is Matt Levine on the Bored Ape Yacht Club:
The best and most valuable NFTs offer their owners real social benefits; owning an NFT in a popular series like CryptoPunks or Meebits gives you a sense of community, perhaps some intellectual-property rights, and a way to socialize with like-minded, uh, venture capitalists.

Owning an NFT in the Bored Ape Yacht Club series gives you a particularly valuable social experience: You get to have your Bored Ape stolen and then complain about it online, which lets you feel a sense of kinship with Seth Green and a bunch of venture capitalists who have also had their Bored Apes stolen.
He is right that the BAYC NFTs are likely to get stolen. Molly White documents the second example in two months in Bored Apes Discord compromised again, 32 NFTs stolen and flipped for $360,000:
Scammers were able to compromise the Discord account of a Bored Apes community manager, then use it to post an announcement of an "exclusive giveaway" to anyone who held a Bored Ape, Mutant Ape, or Otherside NFT. When users went to mint their free NFT, the scammers were able to steal their pricey NFTs. The scammer quickly flipped the stolen NFTs for a total of around 200 ETH (about $360,000), then began transferring funds to Tornado Cash.
Matt Levine continues:
Yeah one lesson here is that there is absolutely no link that the members of the BAYC Discord server won’t click. They just love clicking links! “Maybe this is the link that will get my ape stolen,” they hope, and this weekend they were right.

Look, this is what Bored Ape Yacht Club is! Bored Ape Yacht Club is a way to spend a lot of money to become the victim of an online theft that you can then complain about! You are not getting some other utility from pretending to own a picture of a monkey! But once it is stolen you can commiserate with Seth Green or whatever.
David Yaffe-Bellany is less sarcastic in Thefts, Fraud and Lawsuits at the World’s Biggest NFT Marketplace:
Mr. Chapman bought the nonfungible token last year, as a widely hyped series of digital collectibles called the Bored Ape Yacht Club became a phenomenon. In December, he listed his Bored Ape for sale on OpenSea, the largest NFT marketplace, setting the price at about $1 million. Two months later, as he got ready to take his daughters to the zoo, OpenSea sent him a notification: The ape had been sold for roughly $300,000.
...
In February, Eli Shapira, a former tech executive, clicked on a link that he said gave a hacker access to the digital wallet where he stores his NFTs. The thief sold two of Mr. Shapira’s most valuable NFTs on OpenSea for a total of more than $100,000.

Within hours, Mr. Shapira contacted OpenSea to report the hack. But the company never took action, he said. Since then, he has used public data to track the account that seized his NFTs and has seen the hacker sell other images on OpenSea, possibly from more thefts.
In All Their Apes Gone, Nitish Pahwa collected a long list of techniques for stealing Bored Apes NFTs, starting with the now notorious tweet from Todd Kramer. He writes:
I’ve assembled a lengthy, though surely incomplete, list of stolen apes and related scams. (Keep in mind that ape thefts alone are only a fraction all NFT and crypto thefts over time. It’s an expansive universe! Of scams!)
"It’s an expansive universe! Of scams!" really sums it up nicely.

Matt Levine's "The Crypto Story": Part 2 / David Rosenthal

The first part of my discussion of Matt Levine's The Crypto Story covered the first two of its four chapters:
  1. Ledgers, Bitcoin, Blockchain, in which he lays out the basics of Bitcoin's technology.
  2. What Does It Mean?, in which he discusses generalizations of Bitcoin, such as Ethereum.
  3. The Crypto Financial System, in which he discusses financial technologies such as exchanges, stablecoins and DeFi.
  4. Trust, Money, Community, in which he discusses the social and societal impacts.
Below the fold, I go on to look at the last two chapters. They are shorter than the first two, and my expertise is less in these areas, so there is less to write.

The Crypto Financial System

Again, Levine explains why as a financial journalist, he is interested:
The crypto system is, philosophically, one of permissionless innovation. The workings of the major blockchains are public and open-source; if you want to build a derivatives exchange or margin-lending protocol or whatever in Ethereum, you just do it. You don’t need to set up a meeting with Vitalik Buterin to get his approval. You don’t need to negotiate access and fees with the Ethereum Corp.; there is no Ethereum Corp. Anyone can try anything and see if it works.

If you’re a smart young person coming from traditional finance, this feels liberating. If you’re used to spending months negotiating credit agreements with prime brokers and setting up direct access to trade on a stock exchange, the idea that you can just do stuff in crypto, with no preliminaries, is amazing. Obviously it’s a bit alarming as well: Some of those long, slow processes in traditional finance are there to prevent money laundering or fraud or ill-considered risk-taking. But, empirically, a lot of them aren’t really preventing any of those things, or aren’t doing so in an optimal way. A lot of them are just How It’s Always Been Done. Nothing in crypto is How It’s Always Been Done; it’s all too new.
He starts by discussing what it means to "hold crypto", both the risks of "your keys, your coins" such as the loss of the private key for your wallet, and those of the "not your keys, not your coins" approach of trusting an exchange. He notes one of the issues with doing so:
There are some obvious downsides. One big one: It’s like a bank! If you got into Bitcoin because you don’t trust banks and you want to be in control of your own money, it’s somewhat weird, philosophically, to just go and trust a crypto exchange to keep your money for you.
But the alternative is to be a bank, with total responsibility for the security of your own funds. This is hard, as Nicholas Weaver discovered:
If security experts can't safely keep cryptocurrencies on an Internet-connected computer, nobody can. If Bitcoin is the "Internet of money," what does it say that it cannot be safely stored on an Internet connected computer?
If you use an exchange, your legal position is unclear, but is likely to be as an unsecured creditor. Adam Levitin takes 77 pages to examine this question in Not Your Keys, Not Your Coins: Unpriced Credit Risk in Cryptocurrency:
Cryptocurrencies are designed to address a problem of transactional credit risk—the possibility of “double spending.” The lesson here is the credit risk can arise not just from active transacting in cryptocurrency, but also from passive holding of cryptocurrency. Because this passive holding risk turns on technical details of bankruptcy and commercial law, it is unlikely to be understood, much less priced, by most market participants. The result is a moral hazard in which exchanges are incentivized to engage in even riskier behavior because they capture all of the rewards, while the costs are externalized on their customers
This uncertainty is especially true of Binance, the even more dominant exchange since buying FTX, as Levitin explains in Binance's Custodial Arrangements: Whose Keys? Whose Coins?:
Binance.us's Terms of Uses disclose absolutely nothing about its custodial arrangement for crypto holdings. From the documents on Binance.us's website, it is impossible to determine the legal relationship between Binance.us and its customers and hence the type of counterparty risk they have from dealing with the exchange. That's scary.
Levine also explains why financial institutions typically don't want to hold cryptocurrencies, but would rather use the futures markets to create derivatives that track them. Not holding them means not having to explain how you are mitigating the risks to your regulators.

I like his explanation of stablecoins:
One thing that would be cool is if crypto could keep track of who has dollars. Then you could get the benefits of crypto (decentralization, smart contracts, skirting the law) along with the benefits of dollars (your bank account isn’t incredibly volatile, you can buy a sandwich). A stablecoin is a crypto token that’s supposed to always be worth $1.51 If you have a stablecoin, then you have $1 on the blockchain. You hope.
No discussion of stablecoins is complete without Tether:
One of the longest-running and funniest controversies in crypto is about where Tether, the biggest stablecoin, keeps its money. Tether is replete with colorful characters (the Mighty Ducks guy, etc.), and they go around boasting about how transparent they are without actually saying where the money is. They also go around promising to publish an audit but never do it. They probably have the money, more or less, but they seem to be going out of their way to seem untrustworthy. Still, people trust them.
Two different types of people trust Tether enough to use it:
  • Speculators trust Tether because they have to in order to play the game; it is in effect what the bets are denominated in.
  • Traders, especially in East Asian countries, trust Tether because by avoiding their national regulations they can transact in ways that they otherwise couldn't, or would be uneconomic. Datafinnovation explains this in USDT-on-TRON, FTX & WTF Is Really Happening.
Levine makes an interesting argument about "wrapping" actual assets in tokens:
Stablecoins are “wrapped” dollars, dollars that live on the blockchain.
...
it’s a way for the crypto financial system to ingest the traditional financial system. Have a financial asset? Put it in a box and issue tokens about it. Now it’s a crypto asset. If the crypto financial system is good—if the computer programs, payment rails, and institutional structures of crypto have competitive advantages against the programs, rails, and structures of traditional finance—then some people will prefer to trade their stocks or bonds or other financial assets in the crypto system.
Levine describes algorithmic stablecoins and the mechanism behind the Terra/luna collapse using a fictious pair, Dollarcoin (Terra) and Sharecoin (Luna):
People start to want dollars rather than Dollarcoins, so some of them sell Dollarcoins for dollars on the open market. This pushes the price of Dollarcoin slightly below $1, perhaps to 99¢. Other people get nervous, so they go to the smart contract —which is supposed to keep the price of a Dollarcoin at $1—and trade Dollarcoins in for $1 worth of Sharecoins. Then they sell those Sharecoins, which pushes down the price of Sharecoin, which makes more people nervous. They trade even more Dollarcoins for Sharecoins and sell those. This pushes the price of Sharecoin lower, which creates more nervousness, which leads to more redemptions at lower Sharecoin prices and even more Sharecoin supply flooding the market. This is a well-known phenomenon in traditional finance (it happens when companies issue debt and commit to paying it back with stock), and it has the technical name “death spiral.”
I discussed this vulnerability of algorithmic stablecoins in Metastablecoins. Metastability is a concept in physics, describing a system like the one in the graphic which can have two energy states, a higher one separated from a lower one by an energy barrier. Algorithmic stablecoins depend on arbitrage to maintain their value; the "death spiral" happens when the arbitrageurs lack enough firepower to prevent the value transiting the arbitrage barrier. As we see with Terra/Luna and earlier failures, calling them "stablecoins" is inaccurate, they should be called metastablecoins.

Further, a point that Levine misses is that even fully backed stablecoins are metastable because they are subject to "bank runs". Back in May after USDT briefly traded down to 95.11 cents, Bryce Elder focused on Tether's restrictions on redemptions in Barclays to tether: the test is yet to come:
Tether’s closed-shop redemption mechanism means it cannot be viewed like a money-market fund. Processing delays can happen without explanation, there’s a 0.1 per cent conversion fee, and the facility is only available to verified customers cashing out at least $100,000.
This means that although in theory holders of USDT can redeem them for dollars from Tether's backing as Levine describes, in practice most of them can't. They can only sell their USDT for USD on an exchange, putting downward pressure on the USDT price. Barclays writes:
We think that willingness to absorb losses, even though USDT is fully collateralized and has an overnight liquidity buffer that exceeds most prime funds, suggests the token might be prone pre-emptive runs. Holders with immediate liquidity demands have an incentive (or first-mover advantage) to rush to sell in the secondary market before the supply of tokens from other liquidity-seekers picks up. The fear that USDT might not be able to maintain the peg may drive runs regardless of its actual capacity to support redemptions based on the liquidity of its collateral.
Thus even the biggest fully backed stablecoin is actually metastable.

Levine then turns to explaining DeFi, in particular with a very clear explanation of how automated market makers (AMM) work, which grew out of Vitalik Buterin's suggestion for how to solve the problem that the way market makers work in TradFi requires vastly faster and cheaper computation than Ethereum can deliver:
The mechanism would be a smart contract that holds A tokens of type T1, and B tokens of type T2, and maintains the invariant that A * B = k for some constant k (in the version where people can invest, k can change, but only during investment/withdrawal transactions, NOT trades). Anyone can buy or sell by selecting a new point on the xy=k curve, and supplying the missing A tokens and in exchange receiving the extra B tokens (or vice versa). The “marginal price” is simply the implicit derivative of the curve xy=k, or y/x.
Levine's explanation is far less cryptic; you should read it.

DeFi isn't just providing liquidity between tokens, it also involves lending. Secured loans, such as margin loans, can be implemented effectively as "smart contracts", but unsecured loans not so much:
An unsecured loan is essentially about trust. It’s about the lender trusting that she’ll be repaid not out of a pool of collateral but out of the borrower’s future income. She has to trust that the borrower will have future income and that he will pay.

Relatedly, an unsecured loan requires identity. You need to know who’s borrowing the money, what their payment history looks like, what their income is. ... If you borrow money against crypto collateral, all your lender needs to know is that the collateral is on the blockchain. If you borrow money against your future income, your lender needs to know who you are.
And so we come back to the oracle problem. Any time a "smart contract" such as an unsecured lending protocol needs to interact with the real world, it runs into the potential for "Garbage In, Garbage Out" (GiGo). Wikipedia notes that:
The underlying principle was noted by the inventor of the first programmable computing device design:
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.
— Charles Babbage, Passages from the Life of a Philosopher
There is much more in this chapter, touching on flash loans, Miners Extractable Value, and the way the Terra/Luna fiasco mirrors the 2008 financial crisis but without the contagion into the real economy. But, again, it is hard to find anything there to disagree with.

Trust, Money, Community

This chapter contains three short stories. The first is about how the way people use a system designed to eliminate the need for trust involves trusting much less trustworthy entities:
Crypto, in its origins, was about abandoning the system of social trust that’s been built up over centuries and replacing it with cryptographic proof. And then it got going and rebuilt systems of trust all over again. What a nice vote of confidence in the idea of trust.
The reason for this is that using the "trustless" system directly is so difficult and risky that only experts can do it safely. So if the system is to gain wide usage it can only be through layers that disquise the difficulties and risks. And these layers necessarily require trust. Moxie Marlinspike wrote about a great example of this in My first impressions of web3:
As it happens, companies have emerged that sell API access to an ethereum node they run as a service, along with providing analytics, enhanced APIs they’ve built on top of the default ethereum APIs, and access to historical transactions. Which sounds… familiar. At this point, there are basically two companies. Almost all dApps use either Infura or Alchemy in order to interact with the blockchain. In fact, even when you connect a wallet like MetaMask to a dApp, and the dApp interacts with the blockchain via your wallet, MetaMask is just making calls to Infura!

These client APIs are not using anything to verify blockchain state or the authenticity of responses. The results aren’t even signed. An app like Autonomous Art says “hey what’s the output of this view function on this smart contract,” Alchemy or Infura responds with a JSON blob that says “this is the output,” and the app renders it.

This was surprising to me. So much work, energy, and time has gone into creating a trustless distributed consensus mechanism, but virtually all clients that wish to access it do so by simply trusting the outputs from these two companies without any further verification.
Trust is such a great optimization that people can't believe systems wouldn't use it. This isn't just about computer systems. In 2019's The Web Is A Low-Trust Society I wrote:
Back in 1992 Robert Putnam et al published Making democracy work: civic traditions in modern Italy, contrasting the social structures of Northern and Southern Italy. For historical reasons, the North has a high-trust structure whereas the South has a low-trust structure. The low-trust environment in the South had led to the rise of the Mafia and persistent poor economic performance. Subsequent effects include the rise of Silvio Berlusconi.
But, on the issue of trust as an optimization, see Cory Doctorow's Delegating trust is really, really, really hard (infosec edition), a topic I've written about before.

The second contrasts the view of money as a social construct with the idea that cryptocurrencies are objective facts:
Your Bitcoin are yours immutably; they’re controlled only by your private key, and no government or bank can take them away from you. But the history of crypto since Satoshi has undermined this view. If you got your Bitcoin illegitimately, the government can trace them and stop you from spending them. There are still gatekeepers—crypto exchanges and fiat off-ramps and banks—that decide what you can do with your money. Crypto might be immutable and “censorship-resistant,” but its interactions with the real world are not.
"Code is law" but the people behind the code know what that saying is good for. In Deconstructing ‘Decentralization’: Exploring the Core Claim of Crypto Systems Prof. Angela Walch gets to the heart of what the claim that a system is "decentralized" actually means:
the common meaning of ‘decentralized’ as applied to blockchain systems functions as a veil that covers over and prevents many from seeing the actions of key actors within the system. Hence, Hinman’s (and others’) inability to see the small groups of people who wield concentrated power in operating the blockchain protocol. In essence, if it’s decentralized, well, no particular people are doing things of consequence.

Going further, if one believes that no particular people are doing things of consequence, and power is diffuse, then there is effectively no human agency within the system to hold accountable for anything.
In other words, it is a means for the system's insiders to evade responsibility for their actions.

The third is about the value of communities, such as the ones that surround cryptocurrencies:
A key lesson of crypto is: A bunch of people can get together online and make their community have economic value, and then capture that value for themselves. If you explain the mechanism for that, it sounds even worse. “Well, see, there’s this token of membership in the community, and it’s up 400% this week. Also the tokens are JPEGs of monkeys.”

But look, pretty soon, what are we going to sell to each other? Online communities are valuable. There’s money to be made.
In an era when Elon Musk is driving Twitter off a cliff, laying off half its staff, and Mark Zuckerberg is reacting to Meta's stock dropping 70% this year by laying off 11,000 workers, I'd treat Levine's comment skeptically.

Finally, Levine makes the case for cryptocurrencies:
A problem, and an advantage, of crypto is that it financializes everything. “What if reading your favorite book made you an investor in its stock.” Feh, it’s a story that only a venture capitalist could love. On the other hand, it’s a story that venture capitalists love. A minimalist case for crypto is:
“It’s an efficient way to get venture capitalists to put money into software projects.”
Again, I urge you to read the entire magnum opus. It is well worth your time. And, for a much shorter take on many of these issues, go read the Bank of England's John Lewis' Old problems with new assets: some of crypto’s challenges look strangely familiar, which concludes with:
New assets don’t always mean new problems or new solutions. Ironically, despite being promoted as alternatives to traditional finance, the crypto ecosystem faces many of the same problems. Some challenges relate to the underlying currencies – ideally you want a currency with stable value whose quantity can be changed to supply liquidity. But unbacked cryptocurrencies like bitcoin or ethereum which are the cornerstones of the system have the opposite properties: unstable value and a quantity that can’t be easily changed.

Other challenges relate to the system as a whole. Typically those are asymmetric: in upswings no-one wants to get out, loans get repaid, there are no margin calls, liquidity is abundant and collateral prices are rising. It’s only in downswings these issues materialise, often at the same time. And crucially, the crypto ecosystem currently lacks many of the guard rails developed over time in the regular system (capital buffers, liquidity requirements, stress tests, lender of last resort, resolution frameworks etc) to deal with them. As such, I think it is much more vulnerable when those problems emerge.

No Actual People Were Harmed In The Making Of This Market / David Rosenthal

FTX Token

The modus operandi of the crypto-bros in responding to criticism and calls for regulation is to talk about "innovation" and gaslighting about hypothetical future benefits, to deflect attention from the actual current costs of their favorite technology (see also autonomous vehicles). Below the fold I point out an egregious example of the genre.


What we see as I write this is that the anti-regulation forces (Binance) have destroyed the pro-minimal-regulation forces (FTX):

After days trying to shore up his teetering crypto empire, Bankman-Fried sought Chapter 11 bankruptcy for more than 130 entities in the FTX Group, including Alameda.

Bankman-Fried resigned as chief executive officer of the FTX Group as part of the filings, and John J. Ray III was appointed to replace him, according to a statement. Ray, a turnaround and restructuring expert, has previously served senior roles in bankruptcies including Enron.
This leaves the "investors" in FTX facing an indefinite but long wait to get any money back, if they ever do, because as Adam Levitin takes 77 pages to point out, the legal status of their "investments" in bankruptcy is obscure. What we also see is that apparently Binance suffers from exactly the same problem that it used to take down FTX:
Binance holds $74.7 billion worth of tokens of which around 40% are in its own stablecoin and native coin, according data shared by Nansen.
...
Of the $74.6 billion termed as networth, about $23 billion was in its own stablecoin BUSD and $6.4 billion in its Binance Coin, according to Nansen.

The exchange has also allocated 10.5% of its holdings in Bitcoin and 9.8% in Ether, Nansen data shows.
But the crypto-bros argue that these disasters are simply the price to be paid for the huge benefits of "financial innovation". Today's example of the genre comes from Aaron Brown in FTX Collapse Is a Feature, Not a Bug, of Financial Innovation:
The problems at FTX have already led to calls for more regulation of crypto, but there are three big problems with that idea. First is that these same disasters happen frequently in the regulated financial world. Particularly large examples lead to more regulations, but that never seems to stop people from finding new ways to make old mistakes. Second is that in all material respects relevant to these problems, FTX was already subject to regulations. FTX was not a bunch of anonymous offshore hackers nor was it run by regulation-dodging libertarians. Its three pieces were regulated, audited entities that — at least until someone proves different — complied with regulations.

The third and biggest problem is that FTX had good ideas — vetted by many smart people — about how to avoid financial disasters via technology rather than regulation. This was the main impetus for the introduction of Bitcoin after the 2008 financial crisis. We can’t dismiss those ideas because FTX failed. It’s not as if “more regulation” has any track record of success. Failure means we need more experimentation with more new ideas until we find a mix that works.

No new regulations will help FTX’s customers and creditors. They might stop someone from starting a copycat FTX, but no one is likely to do that now, nor would anyone trust it. What new regulations would do is block one of the most exciting areas of crypto innovation, which is a new type of financial exchange. Most of the promising ones are simple, pure exchanges without attached entities and that hold no customer funds. Automated market makers, frequent batch auctions, zero-knowledge orders, portfolio trading and other innovations attempt to use cryptographic security to take away the ability of people to cheat rather than just telling them not to do it and occasionally fining or jailing a few of them afterwards. These are padlocks rather than “Do Not Enter” signs. And if they prove successful, the new exchange mechanism can re-engineer trading in traditional assets as well as crypto assets.

No doubt there will be failures and scandals associated with these innovations, just as no doubt there will be failures and scandals associated with regulated financial institutions. But the innovations have the potential to fix problems and eventually eliminate them, while no one can believe that some future round of regulation will be the one to finally solve the ancient problems of finance.
So, Aaron Brown, how are these "good ideas — vetted by many smart people" working out in the actual present as opposed to the hypothetical future? Are they really managing to "avoid financial disasters via technology rather than regulation"? He even admits that, so far, they haven't. They have actually caused financial disasters. But not to worry, we'll just try something else. Eventually we'll find something that works.

What he cavalierly dismisses is that the costs of "financial disasters" aren't paid by the SBFs of this world, who escape with only a paltry few hundred million dollars, or even the Aaron Browns, but in the wrecked lives of ordinary people. He needs to read Molly White's heartbreaking collection of Excerpts from letters to the judge in the Voyager Digital bankruptcy case or the Consumer Financial Protection Bureau's 46-page Complaint Bulletin: An analysis of consumer complaints related to crypto-assets:
The majority of the more than 8,300 complaints related to crypto-assets submitted to the CFPB from October 2018 to September 2022 have been submitted in the last two years with the greatest number of complaints coming from consumers in California. In these complaints, the most common issue selected was fraud and scams (40%), followed by transaction issues (with 25% about the issue of “Other transaction problem,” 16% about “Money was not available when promised,” and 12% about “Other service problem”). In addition, analyses suggest that complaints related to crypto-assets may increase when the price of Bitcoin and other cryptoassets increase.
Actually reading the report is something that should, but likely wouldn't, upset Aaron Brown's world view. The CFPB reports that:
  • The top issue across all crypto-asset complaints was “Fraud or scam.” This issue appears to be getting worse, as fraud and scams make up more than half of “virtual currency” complaints received thus far in 2022. Some consumers stated that they have lost hundreds of thousands of dollars due to unauthorized account access. The prevalence of fraud and scam complaints raises the question of whether crypto-asset platforms are effectively identifying and stopping fraudulent transactions.
  • Consumers report many different scam types, including romance scams, “pig butchering,” and scammers posing as influencers or customer service. Crypto-assets are often targeted in romance scams, where scammers play on a victim’s emotions to extract money. According to the FTC, of all romance scam payment types, crypto-asset romance scams accounted for the highest median individual reported losses at $10,000. Some of these scammers employ a technique law enforcement refers to as “pig butchering,” where fraudsters pose as financial successes and spend time gaining the victim’s confidence and trust, coaching victims through setting up crypto-asset accounts. Some scammers try to use social media posts by crypto-asset influencers and celebrities to trick victims. Finally, lack of customer service options for many cryptoasset platforms and wallets creates opportunities for social media scams where attackers pretend to be customer service representatives to gain access to customers’ wallets and steal crypto-assets.
  • Crypto-assets are a common target for hacking. Consumers reported “SIM-swap” hacks, where an attacker intercepts SMS messages to exploit two-factor authentication, and phishing attacks, social engineering, or both. Companies oftenresponded to these complaints by stating that consumers are responsible for the security of their accounts. Crypto platforms are a frequent target of hacks by malicious actors, including certain nation-state actors. Hackers affiliated with one nation state have stolen over $2 billion in crypto-assets total, including more than $1 billion from Jan 2022 – July 2022 alone, and their hacks have included several prominent crypto platforms, including a “play to earn” crypto-asset game.
  • There are signs that older consumers are also impacted by crypto-asset frauds and scams. Older consumers report a higher rate of crypto-asset related frauds and scams compared to complaints overall: 44% versus 40%.
  • Complaints suggest that servicemembers are facing issues with crypto-asset scams. Servicemembers have submitted complaints about “SIM-swap” hacks, identity theft, and romance scams. Servicemembers have also submitted complaints about transaction problems and poor customer service at crypto-asset platforms.
  • Complaints about frauds or scams continue to rise, making up more than half of all total crypto-asset complaints received by the CFPB thus far in 2022. Crypto-asset complaints and fraud reports have also been increasing at other federal agencies: The SEC has received over 23,000 tips, complaints, and referrals regarding crypto-assets since fiscal year 2019, with a particularly sharp increase in the last two years, while crypto-asset losses reported to the FTC in 2021 were nearly sixty times more than in 2018
An increase of "nearly sixty times" in losses in three years is just the price ordinary people need to pay for the awesome benefits that accrue to the Aaron Brown of the world from "financial innovation".

The point of financial regulations is not to "avoid financial disasters". It is to prevent the inevitable financial disasters affecting ordinary people, by ensuring that their costs fall on the perpetrators.

The Stablecoin Saga / David Rosenthal

Starting in August, Datafinnovation has posted two series of posts about (meta)stablecoins, the first about Tether's reserves and the second about tracing flows of stablecoins. Below the fold, primarily for my own understanding, I try to summarize the first. I plan to return to the second in a subsequent post.

The TL;DR of the series is that, assuming Tether has turned over a new leaf and is being honest about USDT being backed by USD:
  • There should be a big pile of USD somewhere.
  • The logical place for it to be is in US banks.
  • The logical US banks for it to be in are the two US "crypto banks", Silvergate and Signature.
  • US banks have to report to regulators.
  • These reports show a set of coincidences that suggest this is where the reserves are.
The overview of USDT, US Banks & More Coincidences (August 5th) is:
First we are going to go through two US banks that hold a lot of crypto-related USD. Then we are going to look how remarkably their balance sheets resemble the quantity of USDT outstanding. And finally we are going to hint at a fascinating connection between those banks’ purported source of USD and a potential “crypto big hitter” source.
Source
The two US banks are Silvergate and Signature. First, Signature, which in 2019 started to focus on cryptocurrencies. In Q1 2022 their "digital asset" related deposits had more than doubled in a year, to about $64B. Second, Silvergate, which has almost no retail business and only one branch. Their "digital asset" related deposits, essentially their entire business, in Q1 2022 were about $13.5B. Combined, this $77.5B closely matched the issuance of Tether at that point, and these deposits had closely tracked that issuance since at least 2020.

So there has been a pile of actual USD sitting in two banks that closely matches the issuance of USDT:
This is not Tether’s cash surely. At least not directly. But what is it?

First, this proves enough cash looks to have entered the ecosystem to back Tether. Absolutely nobody is claiming these US banks are lying about their deposit liabilities to both the SEC and FDIC.
...
We are not asserting Tether has all of this cash or that they are honest. But it does look like enough cash came in one way or another. There is cash somewhere.
...
Second, it suggests a theory. Maybe Tether’s commercial paper consists of obligations of whoever owns these USD deposits. Again there is money here and if one company promises it to another there is a sense in which that is commercial paper.
Tether used to claim that most of their allegedly one-for-one backing was "commercial paper", which normally consists of a loan a company with excess cash made to a company in need of cash. This isn't like that, but:
It is a horrible abuse of the language but promising someone else your bank deposit is similarly some kind of a certificate of deposit or maybe cash-equivalent/accounts receivable thing.
Presumably, having a contract with another company saying "if we need it we can draw on your accounts at Silvergate and Signature" is enough for Tether to issue that amount of USDT. And both sides hope they won't need it. A year ago, in Anyone Seen Tether’s Billions?, Zeke Faux went looking for Tether's "commercial paper":
Elsewhere on the website, there’s a letter from an accounting firm stating that Tether has the reserves to back its coins, along with a pie chart showing that about $30 billion of its dollar holdings are invested in commercial paper—short-term loans to corporations. That would make Tether the seventh-largest holder of such debt, right up there with Charles Schwab and Vanguard Group.

To fact-check this claim, a few colleagues and I canvassed Wall Street traders to see if any had seen Tether buying anything. No one had. “It’s a small market with a lot of people who know each other,” said Deborah Cunningham, chief investment officer of global money markets at Federated Hermes, an asset management company in Pittsburgh. “If there were a new entrant, it would be usually very obvious.”
The explanation seems to be that what Tether means by "commercial paper" is not the kind of asset the commercial paper market trades.

Next, Minor Stablecoins & USDT Issuance: Even More Coincidences (August 19th) starts:
Here we analyze the flows into a collection of “minor” stablecoins and find a remarkable connection between these tokens and USDT issuance. And — importantly — we find that this pattern does not hold for USDC and BUSD. Those stablecoins are widely used and do not look like this sort of funnel into USDT.
...
What do we mean by minor stablecoins? HUSD, TUSD, USDP, GUSD. There is also USDK but it’s so tiny the word “minor” doesn’t do it justice.

GUSD & USDP likely everyone will agree are properly USD backed and perform something like the required KYC/AML checks. The other two…let’s just say questions have been raised.
First, HUSD:
HUSD is an odd stablecoin. First, it’s not particularly large in market cap terms but sure has a lot of minting and burning
...
they switched over to Stable Universal from Paxos as the stablecoin service provider

Which seems to be an odd outfit in that they still take payments into Silvergate bank:
...
Stable Universal is taking payments into the same accounts that Huobi referenced above. And they are a strange outfit in that, despite all the US banking stuff, they don’t take US customers:

So Huobi and this HUSD are a bit weird. But there is a plausible route for them to have all the USD. And, at least in part, those USD sit at US banks. Huobi Trust is a licensed (in Nevada) financial services business in the US so they should have no trouble moving money into the US banking system.
Source
Second, TUSD. The screengrab shows that 92% of their cash is at four named US banks, First Digital, Signature, Prime Trust and Silvergate:
Recognize some of those bank names? Good. From here on our we are going to assume these things are all backed by real USD and that, to a large extent, those USD migrate to Silvergate and Signature. There is ample reporting that these folks all bank at the same two places.
Source
How does all this cash relate to Tether?:
We know these stablecoins interact with the same banks as the USDT-sized pile of dollars we found last time. Now let’s compare USDT market cap against the cumulative burning of these stables:

We can see based on the shape of the combined minor stable, USDC and BUSD burns that the two “more normal looking” stables take off much later and bend more aggressively. All three are on the same secondary axis at the same scale. There is an uncanny correspondence between USDT growth and the amount of funds being burned through these minor stablecoins.
Source
There is an even more detailed coincidence:
Here is burning of the minor stables vs issuance of USDT on Ethereum and TRON:

Pretty tight connection during 2019-2020 no? These are all on the same axis.

Another coincidence: Huobi announced support for USDT on TRON in March 2019 and Poloniex in April 2019. Those are right after Tether started supporting TRON.
So, the story so far is that these minor stablecoins are used as a channel to get USD into Signature and Silvergate, where it is used to back USDT.

Stablecoin Cyclones: Mint & Burn Patterns (August 29th) starts asking where the USD flowing into the minor stablecoins is coming from:
The point here is that the mechanism we offered last time as pure speculation — that these stablecoins are used as vehicles to get USD into the “major crypto banks” — is not just plausible but consistent with a lot of on-chain data. And this data reveals direct connections among precisely the parties we would expect if that narrative is correct.
And explains what the hypothesis would imply:
If these coins were useful we would expect to see most tokens out there floating around various protocols. We would expect lots of inter-exchange transfers and flows within DeFi. But most importantly we would expect to see long routes between mints and burns as the coins are used for real things throughout their lives. Sure, some people will mint and then redeem but that makes no sense as the main use case.

Now, we already know there is a lot of burning. So the best we can hope for is a range of circuitous routes through the ecosystem as tokens experience real economic uses before finally being redeemed.

What we do not expect to find in a useful coin is a large number of short, high volume routes from mint to burn. What economic purpose does that serve? Think of cash in the real world: how often do you withdraw a lot of money from the bank, pay someone, and they then go deposit that cash back in the bank? And what is really happening there?
They examine all four coins but HUSD is the most interesting:
First we found nearly a thousand addresses and not a single one of them minted and burned. But both sides look to be proper addresses. We were able to tag the largest minter (0x5f64d9c81f5a30f4b29301401f96138792dc5f58) as Huobi and the 3rd largest minter (0x83a127952d266a6ea306c40ac62a4a70668fe3bd) as Alameda.

Also some of these wallets — like 0x5586a235b4c5eff57b8b65f77a97780d9347fa47 — look like staging wallets off of exchanges (Huobi in this case). All they do is receive tokens from Etherscan-tagged exchange wallets and then burn them. One imagines somewhere an operations person is checking the tokens arrived, redeeming the real USD, and then burning the tokens. Maybe it’s automated — it doesn’t really matter precisely how it works.

The largest burner (0x4bd0244d26df7780a56df2f20c53b507e35cb373) looks like a staging wallet out of the one we found above tagged Binance (0xE25a329d385f77df5D4eD56265babe2b99A5436). Now recall we already know everyone involved banks at the same handful of places with the suspiciously-USDT-like cash balances.

So what do we see here? We see Huobi, Alameda and Binance minting and redeeming a lot of these things out of different wallets. And it’s not like the wallets just rotate over time — many many wallets are used for years on both sides. These look very much like individual exchange-adjacent client wallets, minting and redeeming out of different sources.
...
These are billions of dollars in tokens that are not used for trading flowing from source, through a pair of exchanges in close proximity, and then back to sink.
So what is really going on?
This could not look more like an effort to obfuscate capital flows if it tried. Yes it is plausible this is all bog-standard economic activity. But recall, again, what else we know: USD at the “crypto money center” banks tracks USDT well, and USDT-on-TRON tracks these stablecoin burns well.

Now we know that the former head of TRON is involved directly in the process. We also know these burns are happening out of places that bank at the major banks — and that freshly minted tokens often make their way there before returning home to the null address.

Does this prove beyond all doubt these are conduits for USD to get redeemed at Signature/Silvergate and then back USDT? No. Does it show how closely the empirical evidence matches a world where that is happening? Yes. Yes it does.
The problem that these minor stablecoins are solving is how to turn fiat currency into balances in Tether, the most widely accepted stablecoin. The fiat buys one of the minor stablecoins at some loosely regulated exchange, and redeems it for USD at one of the "crypto-banks". The USD then "buys" USDT, not by transferring USD to Tether but by increasing the value represented by "commercial paper".

The summary of USDT-on-TRON, FTX & WTF Is Really Happening (September 7th) is:
FTX/Alameda minted nearly all the USDT-on-TRON and operate as something like a central bank or reserve manager for a shadow East Asian USD payment system. We provide convincing evidence from novel on-chain analysis that shows how a real, albeit mostly-not-kosher, crypto use case works. This data also makes plain that Binance/Cumberland runs the Ethereum part of the same ecosystem and that these two groups of parties probably coordinate their actions in some way.
...
we are going to show that this entire complex looks an awful lot like a funnel to establish backing for a USD payment network aimed at people who cannot (easily or legally, depending) hold USD or transfer them. This also exposes how USDT is split into a China-and-surroundings slice and a rest-of-world slice with a different major crypto entity handling each part.
Here is the background:
  1. USDT is used in the real world for all kinds of “grey area” stuff. This video again.
  2. Crypto is formally illegal in China. Including stablecoins.
  3. USD are generally inaccessible, or heavily restricted, in China.
  4. TRON, and Huobi Eco Chain, are big in stablecoin transactions in China and adjacent/similar places.
"This video" is a detailed description of the "shadow East Asian USD payment system" by QCP Capital, based in Malaysia. You should definitely watch it. It explains how international trade in this area is denominated in USDT because it is faster, cheaper and less visible to the authorities.

Source
Datafinnovation show that the USDT-on-TRON flows are concentrated in China-and-surroundings and dominated by Sam Bankman-Fried's companies FTX and Alameda. They are based in the Bahamas and run by Americans, so are not too concerned about facilitating activities of which the Chinese government might not approve. Whereas the USDT-on-Ethereum flows are concentrated "all over Latin America, Africa and India" and dominated by CZ's Binance and adjacent companies such as Justin Sun's and Huboi. These are based in East Asia and need to be more concerned with what the Chinese government might think. But recent events have upset this convenient duopoly. Binance's CZ threw FTX and Alameda under the bus, then backed the bus up and did it again. So the "shadow East Asian USD payment system" is at some risk, since its sponsor is bankrupt and the alternative is much more vulnerable to Chinese pressure.

Source
It is important to understand that, at least in volume terms and especially after the recent fee cut, Binance is by far the most important exchange. It hosts ten times the volume of FTX, more than the 5 next largest exchanges. FTX is #11 in volume terms, Coinbase is #15. This isn't a recent development. These graphs are from The Economist last December and show that Binance has dominated the spot market since 2019 and the derivatives market since it overtook Bitmex in 2020.

This is all a pretty convincing story, but there are some contradictions. In August Datafinnovation suggested that ~$77B at Signature and Silvergate was backing USDT. But in May Jaimie Crawley reported that Tether Cut Commercial Paper Reserve by 17% in Q1, Accountants Say:
Tether reduced its commercial paper holdings by 17% from $24.2 billion to $20.1 billion in the first quarter, according to its latest attestation report.

The majority of this $20.1 billion (around $18 billion) is comprised of A-1 and A-2 paper, which qualify as investment grade, according to the report. A list of ratings agencies grading the commercial paper wasn't specified other than, "Standard & Poor’s ratings, or equivalent ratings by Moody’s, Fitch or other nationally recognized statistical rating organizations" in the footnotes. The geographic location of the commercial paper issuers was also not found in the report.

The reduction in commercial paper has continued with a further 20% cut since April 1, which will be reflected in the second-quarter report, Tether announced Thursday. On June 30, 2021, commercial paper and certificates of deposit totaled $30.8 billion, or 49% of Tether's assets at that time.

Tether has also slightly reduced its cash deposits from $4.2 billion to $4.1 billion and increased its U.S. Treasury bond holdings from $34.5 billion to $39.2 billion since its last report.

The "other investments" category, which includes digital tokens, has remained consistent, falling slightly from $5.02 billion to $4.96 billion.
At the same time Tether's Paolo Ardoino tweeted:
If Tether had been buying government bonds there should have been an outflow of USD from their reserves, but this doesn't show up in the reports from the crypto banks. So maybe the pile of USD that Datafinnovation found in these banks isn't Tether's reserves and the coincidences that they found are really coincidences. If so, that leaves two questions. First, where are Tether's reserves? And second, whose is the big pile of USD in the crypto banks? It is related to "digital assets", but which?

This all supports the need for regulation of cryptocurrencies in general but, in particular stablecoins. Paige Smith's Fed’s Barr Says Stablecoins Are Urgent Risk Requiring Guardrails starts:
A top Federal Reserve official warned Congress that it needed to pass legislation with strong crypto guardrails to prevent future financial-stability risks.

Entities offering stablecoins -- cryptocurrencies pegged to a separate asset -- are an urgent risk that must be addressed, because they’re backed by the dollar and “really borrow the trust of the Federal Reserve,” Fed Vice Chair for Supervision Michael Barr said Wednesday before the US House Financial Services Committee.

It’s “important for Congress to step in and say you’re not permitted to offer a stablecoin unless it’s done under a strong prudential framework with Federal Reserve oversight, supervision, regulation and approval,” Barr said. “Private money can create enormous financial-stability risks, unless it’s appropriately regulated.”

Throughout the committee hearing, Barr reiterated that it’s Congress’s role to intervene. He said bills should include a strong role for Fed oversight of stablecoins, which he called a form of private money.

2022 Review / Open Library

It seems like just the other day when the Open Library welcomed its 2 millionth registered patron in 2018. This year, we zoomed past 6M registered book lovers who collectively in 2022 have borrowed 4.3M books and counting, and who have added more than 4.7M books to their reading logs. Our book catalog expanded to nearly 38M editions and we cleaned up nearly 230k low quality records.

Together, our team released a flurry of features and improvements to the Open Library service including:

Imminently coming is a game changing smart edition-search upgrade, a Yearly Reading Goals feature, support for Web Books, a significantly more usable 1-stop “My Books” page, and design improvements to the Books Page.

Want to see a full run-down of what we accomplished together?
Check out our 2022 Open Library Community Celebration video!


Important Documents

If you’ve ever wanted to dig down deeper into the Open Library’s transparent processes:


Previous Community Celebrations

For the past 3 years around October, the Open Library has recognized its contributors by hosting an Open Library Community Celebration.


Historical Yearly Highlights

In addition to the yearly community celebration, we’ve tried to make end-of-year review posts to give the community transparency into our victories, changes, and planning. In:


Gratitude. Central to these achievements were my fellow staff on the Open Library’s engineering team: Drini Cami, Jim Champ, & Chris Clauss. Equally indispensable to this year’s achievements was Lisa Seaberg from Internet Archive’s Patron Services team. Lisa is both a voice and champion for our patrons as well as the Open Library’s Lead Community Librarian who helps facilitate our community of 500 librarian contributors and our Super Librarians (Daniel, Travis, Onno, et al) who work tirelessly together to keep our library catalog organized. Charles Horn from the openlibraries.online team has been instrumental in keeping MARC records flowing into the catalog and Cari Spivack on policy support. And this year 6 Open Library Fellows — Hayoon Choi, Sam Grunebaum, Dana Fein-Schaffer, Scott Barnes, Constantina Zouni, and Teo Cheng — who selflessly committed several months of their time to improve the Open Library platform for the world, alongside a team of more than 30 volunteer developers from around the globe. Thank you, of course, to Brewster Kahle and all of our patrons and generous donors for believing in us and keeping us funded for another year. And a special thank you to a sorely missed Aaron Swartz, without whom none of this would be possible.

Happy Thanksgiving!

– The Open Library Community

Search for Books in your Reading Log / Open Library

by Scott Barnes, 2022 Open Library Fellow

As of last week’s deploy, it’s now possible to search the Open Library for the books in your reading log by navigating to the My Books page, selecting the Currently Reading, Want to Read, or Already Read bookshelf, and typing in a search query.

Keep reading to learn tips and tricks on how to effectively search for books within your reading log.

A Forward by Mek

This year the Open Library has been exceedingly lucky to collaborate with Scott Barnes, a lawyer who has reinvented himself as a very capable software engineer. We had the pleasure of meeting Scott earlier this year while he was scouring the Open Library for old rock climbing guidebooks. Ever since joining one of our community calls, he’s been surmounting challenging technical hurdles as one of our most active 2022 Open Library Fellows. As law professor Lawrence Lessig famously penned in his 1999 book Code: “Code is law”. I guess that’s why we shouldn’t be too surprised how quickly Scott became familiar with the Open Library codebase, at the precision of his work and attention to detail, and his persistence in getting code just right without getting slowed down. We hope you’ll enjoy Scott’s contributions as much as we do and learn at least one new way of using the reading log search to improve your book finding experience.

A Forward by Drini
I’m exceedingly pleased to introduce Scott Barnes to the Open Library Blog. I have had the honour of mentoring Scott throughout some of the projects on his fellowship, and have been floored by his love, passion, and skill in all things programming. Whether it’s working on user facing features (such as this one), improving code architecture, investigating performance issues, setting up infrastructure, or keeping up-to-date with new programming techniques by diving into a new programming book or topic, Scott is always excited to dive in, learn, and make an impact. And his code never fails to meet requirements while being well-architected and robust. I am so excited to see what he does next with his programming super powers! Because as far as I can tell, there’s no stopping him. Now without further ado, I’ll hand it off to Scott to talk about:

👋 Hi, my name is Scott Barnes. This year as a 2022 Open Library Fellow I collaborated with Drini Cami to develop reading log search. In this post, I’ll show different ways of effectively using the new reading log search feature, as well as technical insight into how it was engineered behind the scenes.

Profile photo for scottreidbarnes

3 Ways of Searching your Reading Log using a Web Browser

The most common way to use search your reading log is by entering a natural, free-form search query, just like you might using your favorite search engine. From the Currently Reading, Want to Read, or Already Read page, you can search for your books on that reading log shelf by submitting text describing the book’s title, author name, ISBN, or publisher. An example could be “Lord of the Rings by J.R.R Tolkien”.

If you want greater control, you can also harness the power of Apache Solr, the underlying technology which powers the Open Library search engine.

Let’s say, for example, that you’d like to find books on your reading log by a specific author named “King” but using the Natural Language mode instead returns books with “King” in the title. Using keyword search, you could search for author: king to see only books by authors named “King” (while not seeing books with “King” in the title). On the other hand, if you only want to find titles matching “King” and not the author, you could instead search for the keyword title: king. Want to find your horror books? Try subject:Horror.

For a list of common keywords, you can see the Open Library Search How-To Guide, or peek at the code behind it: worksearch/code.py.

Reading log search, like the main Open Library search, supports boolean operators, specifically AND, OR, and NOT, along with wildcards such as * and ? to match multiple characters and a single character respectively. Therefore, to search for all books matching “climb”, “climber’s”, “climbs”, etc., that were published by Sierra Club Books that you want to read, you could visit Want to Read and search for title: climb* AND publisher: sierra club books.

NOTE: the boolean operators are CaSe sensitive, so AND will work as expected, but and will not. The actual search terms themselves are not CaSe sensitive, however, so “king” and “KiNg” will return the same results.

Searching your reading log via the API

For those looking to perform programmatic searches, it’s possible to search via a web API using the following RESTful pattern: https://openlibrary.org/people/{USERNAME}/books/{SHELF}.json?q={QUERY}.

For example, to search for all titles matching “king” on my Want to Read shelf I’d query: https://openlibrary.org/people/srb36/books/want-to-read.json?q=king.

Or if I wanted to search for all titles matching climb* on that same shelf, I’d search for https://openlibrary.org/people/srb36/books/want-to-read.json?q=title:(climb*)

Behind the scenes

Now for the technical details! Reading log search was added in pull request #7052. In exploring how reading log search might be accomplished, two key things leapt to our attention:

  1. Reading log records are stored in the database, and work and edition data (i.e. “books”) are stored in Solr; and
  2. To work with the split data, we were probably doing more queries than we needed to, both in the back end itself, and within the templates that make up the pages.

The goal then was to add the ability to search the reading log, ideally while reducing the number of queries, or at least not increasing the number.

Changing the back end

Most of the heavy lifting was done in core/bookshelves.py. The challenge here was addressing the reading log records being in one database, and the edition data being in Solr.

The solution was to query the reading log database once, and then use those results to query Solr to get all the information we’d need for the rest of the process. Then we could simply pass the data around in a Python dataclasses, and then ultimately pass the results through to the templates to render for display in patrons’ browsers.

For the super curious, this is found in get_filtered_reading_log_books()

Changing the templates

As mentioned, having the data in two places had led to some excess querying, which manifested itself in the templates where we re-queried Solr to get additional data to properly display books to patrons.

However, because we had gathered all of that information at the outset, we just had to change the templates to render the query results we passed to them, as they no longer had to perform any queries.

How you can help

Volunteers not only help make Open Library special, but they help make it even more awesome. Check out Volunteering @ Open Library.

Improvements to the Main Navigation / Open Library

Dana, UX/Design Fellow on Open Library

Forward by Mek

Few aspects of a website have greater impact and receive less recognition than good navigation. If done well, a site’s main navigation is almost invisible: it’s there when patrons need it, out of the way when they don’t, and it zips patrons to the right place without making them overthink. Over the years, we’ve attempted improvements to our navigation but we haven’t had the design bandwidth to conduct user research and verify that our changes solved our patrons problem. It turns out, we still had several opportunities for improvement. That’s why this year we were incredible lucky to have collaborated with Open Library UX Design Fellow Dana Fein-Schaffer, who recently transitioned into design from a previous role as a neuropsychological researcher. Dana’s formal education and experience links a trifecta of complimentary fields — computer science, psychology, and design — and has resulted in unique perspective as we’ve endeavored to redesign several of Open Library’s core experiences: the website’s main navigation and the desktop version of our My Books page.

As an Open Library UX Design Fellow, Dana has been in charge of design direction, conducting user interviews, figma mockups, feedback sessions, and communicating decisions to stakeholders. In addition to her skill prototyping and notable problem solving capabilities, Dana’s warmth with the community, affinity for collaboration, and enthusiasm for the project has made teaming up with her a gift. While we’d rather the fellowship not end, we endorse her work with great enthusiasm and highly recommend organizations which share our values to view Dana’s portfolio and engage her for future design opportunities.

The Design Process

IMG_7214.JPG

Hello, I’m Dana Fein-Schaffer, and I’ve been working as a UX Design Fellow with Open Library over the past several months. I’m currently transitioning into UX Design because after gaining professional experience in both psychology research and software engineering, I’ve realized that UX is the perfect blend of my skills and interests. I’ve been enjoying growing my UX skillset, and working with Open Library has been a perfect opportunity for me to gain some formal experience because I love reading and was hoping to work on a book-related project. Moving forward, I’m particularly excited about working as a UX designer for an organization that focuses on social good, especially in the literary, education, or healthcare spaces! If you have a role that may be a good fit, or if you work in one of those industries and are interested in connecting, please feel free to reach out. You can also learn about me from my portfolio.

Problem

At the beginning of my project, the current navigation bar and hamburger menu looked like this: 

I met with members of the Open Library team to identify three key areas of concern: 

  1. The label “more” was not descriptive, and it was unclear to patrons what this meant
  2. The hamburger menu was not consistent with the navigation bar items
  3. The hamburger menu was confusing for patrons, especially new patrons, to navigate

Approach

Before I began my project, the Open Library team decided to implement an interim solution to the first point above. To address the concern with the “more” label, the navigation menu was changed to instead include “My Books” and “Browse.” The website analytics showed that patrons frequent their Loans page most, so for now, the “My Books” page brings patrons to the Loans page. 

To begin redesigning the main site navigation, I first created a prototype in Figma with some potential solutions that built off of this updated navigation menu:

  1. I created a dropdown menu for “My Books” that would allow patrons to select the specific page they would like to go to, rather than automatically going to the Loans page
  1. I reorganized the hamburger menu to be consistent with the navigation menu and to use the subheadings of “Contribute” and “Resources” instead of “More.” I felt that these changes would make the hamburger menu easier to navigate for both new and long-time patrons. 

After creating the prototype, my next goal was to get feedback from patrons, so I scheduled user interviews with volunteers.

User Interviews

I conducted user interviews via Zoom with four patrons to answer the following questions: 

  1. How do users feel about the My Books dropdown? 
  2. Are users using My Books and Browse from the navigation menu or hamburger menu?
  3. Are users able to effectively use the hamburger? Do they find it easier or harder to find what they’re looking for using the reorganized hamburger?

Results & Findings

  1. Three out of the four patrons preferred the dropdown, and the fourth user didn’t have a preference between the versions. The patrons enjoyed having the control to navigate to a specific section of My Books. 
  2. All users used the navigation menu at the top of the page to navigate, rather than the hamburger menu, which supported the switch to “My Books” instead of “More.” This finding also highlighted the need to make sure patrons could access the pages they wanted to access from the top navigation menu. 
  3. Finally, all four patrons found the existing hamburger menu confusing and preferred the reorganized hamburger. Some patrons specifically mentioned that the reorganized hamburger was more compact and that they felt that the headings “contribute” and “resources” were more clear than “more.”

Synthesis and a New Direction

After learning from the user interviews that patrons preferred increased granularity for accessing My Books and a more concise hamburger menu, the Open Library team began discussing the exact implementation. We wanted to keep the navigation and hamburger menus consistent; however, we also wanted to provide many options in the My Books dropdown, which made the hamburger menu less concise. 

At the same time that we were debating the pros and cons of various solutions, another Open Library UX Design fellow, Samuel G, was working on designing a My Books landing page for the mobile site. Inspired by his designs, I created mockups for a desktop version of the My Books page. Having a My Books page that summarizes patrons’ loans, holds, and reading log at a glance allows patrons to still have increased control over My Books navigation while keeping the hamburger menu concise, since all of the individual My Books items can now be condensed into one link. 

Furthermore, having a My Books landing page opens the door for more ways for patrons to interact with their reading through Open Library. For instance, I’ve created a mockup that includes a summary of a patron’s reading stats and yearly reading goal at the top of the page. 

As we work towards implementing this design, I’m looking forward to getting feedback from patrons and brainstorming even more ways to maximize the use of this page. 

Reflections

Working with the Open Library team has been an amazing experience. I’m so grateful that I got to lead a UX project from start to end, beginning with user research and ending with final designs that are ready to be implemented. Working with such a supportive team has allowed me to learn more about the iterative design process, get comfortable with sharing and critiquing my designs, and gain more experience with design tools, such as Figma. It was also a great learning experience that sometimes your projects will take an unexpected turn, but those turns help you eventually come to the best possible design solution. Thank you to everyone who provided feedback and helped me along the way, especially Mek, who was a wonderful mentor, and Sam, who was a great collaborator on our My Books mobile and desktop project!

About the Open Library Fellowship Program

The Internet Archive’s Open Library Fellowship is a flexible, self-designed independent study which pairs volunteers with mentors to lead development of a high impact feature for OpenLibrary.org. Most fellowship programs last one to two months and are flexible, according to the preferences of contributors and availability of mentors. We typically choose fellows based on their exemplary and active participation, conduct, and performance within the Open Library community. The Open Library staff typically only accepts 1 or 2 fellows at a time to ensure participants receive plenty of support and mentor time. Occasionally, funding for fellowships is made possible through Google Summer of Code or Internet Archive Summer of Code & Design. If you’re interested in contributing as an Open Library Fellow and receiving mentorship, you can apply using this form or email openlibrary@archive.org for more information.

AUT & Last Date Modified / Archives Unleashed Project

There’s been a long-standing, frequently asked question by participants of Archives Unleashed datathons and the cohort program: how do we find out the date of a resource or page?

Dates can be really hard to decipher in web archives. As a result, we tend to rely on the crawl date, which is a pretty easy thing to grab out of a WARC since it is a mandatory field. While this is something we’ve always had in aut, it’s not the date of a response or creation of a website, but instead is the date on which the crawl occurred. Not ideal for a lot of research questions I’ve seen teams ask of web collections they are exploring.

Embarrassingly my response to these questions had always been, “dates are hard in web archives,” and then shrug. I’d then point folks to one of the many great projects out of the ODU’s Web Science and Digital Libraries Research Group such as Carbon Dating The Web, as potential path forward.

Even more embarrassing, an easy solution has been right in front of my face this entire time, sitting in the HTTP headers of WARC `response` records. Sometimes, but not all the time, a resource in a WARC record HTTP header has a value for Last-Modified. This date should come from the file’s attributes, e.g. ls -lt filename. The header field is pretty easy to grab with Sparkling, and then incorporate it into the DataFrames produced by aut.

Normally, the date should be in this format: Tue, 15 Nov 1994 12:45:26 GMT (RFC1123). But of course, there are always exceptions. So we wrote a fuzzy parser utility to extract the date and format to YYYYMMDDHHMMSS so that it mirrors the same format as crawl_date.

This field is useful for web archive collections that are only crawled once. For example, in situations like Geocities permanently shutting down or the excellent work being done by the Saving Ukrainian Cultural Heritage Online project to preserve Ukrainian cultural heritage.

This functionality is now available as of the 1.2.0 release of the Archives Unleashed Toolkit, and will be available in ARCH in the coming days.

Speaking of ARCH, if you’re interested in piloting it out, please fill out this form.

Here it is in action:

import io.archivesunleashed._

val data = "/sample-data/geocities/GEOCITIES-20091027143300–00114-ia400112.us.archive.org.warc.gz"

RecordLoader.loadArchives(data, sc)
.all()
.select($"crawl_date", $"last_modified_date", $"mime_type_web_server")
.show(20, false)

+--------------+------------------+--------------------+
|crawl_date |last_modified_date|mime_type_web_server|
+--------------+------------------+--------------------+
|20091027143300| |text/html |
|20091027143259|20000923233454 |image/jpeg |
|20091027143259|20020913163029 |image/jpeg |
|20091027143300|20020211154553 |image/jpeg |
|20091027143259|19980919164703 |image/jpeg |
|20091027143259|20080125150303 |text/html |
|20091027143300|20010921224658 |image/gif |
|20091027143258|20081009015203 |image/jpeg |
|20091027143300| |text/html |
|20091027143259|20020416145103 |image/jpeg |
|20091027143300|20090223022835 |text/html |
|20091027143300|20030928090558 |image/jpeg |
|20091027143300|20091027143300 |text/html |
|20091027143300|20021203212451 |text/html |
|20091027143300| |text/html |
|20091027143300|20040530033010 |image/bmp |
|20091027143300| |text/html |
|20091027143259|20090223022352 |text/html |
|20091027143300| |text/html |
|20091027143300|20010608202736 |text/html |
+--------------+------------------+--------------------+
only showing top 20 rows

There are a few other possibilities for trying to figure out the date of a resource. Since we already use Apache Tika to identify language of a content, and mime_type_tika, we could also potentially use it to try and identify some dates from a given resource in a WARC. It could be a future project if there is a need for it.

I’ve also take the time to update the Geocities dataset again. The updated dataset includes last_modified_date columns for the file format, web pages, and text files derivatives. It also includes crawl_date format fix for domain graph derivative (YYYYMMDD instead of YYYYMMDDHHMMSS).

Here are the 20 most frequent years resources were last modified in the dataset:

Geocities: Year of Last-Modified-Date

Finally, our preliminary research showed reliable values coming from the Last-Modified headers, so we’d love to hear feedback from users on this new feature as they process their data.


AUT & Last Date Modified was originally published in Archives Unleashed on Medium, where people are continuing the conversation by highlighting and responding to this story.

AUT & Last Date Modified / Nick Ruest

There’s been a long-standing, frequently asked question by participants of Archives Unleashed datathons and the cohort program: how do we find out the date of a resource or page?

Dates can be really hard to decipher in web archives. As a result, we tend to rely on the crawl date, which is a pretty easy thing to grab out of a WARC since it is a mandatory field. While this is something we’ve always had in aut, it’s not the date of a response or creation of a website, but instead is the date on which the crawl occurred. Not ideal for a lot of research questions I’ve seen teams ask of web collections they are exploring.

Embarrassingly my response to these questions had always been, “dates are hard in web archives,” and then shrug. I’d then point folks to one of the many great projects out of the ODU’s Web Science and Digital Libraries Research Group such as Carbon Dating The Web, as potential path forward.

Even more embarrassing, an easy solution has been right in front of my face this entire time, sitting in the HTTP headers of WARC response records. Sometimes, but not all the time, a resource in a WARC record HTTP header has a value for Last-Modified. This date should come from the file’s attributes, e.g. ls -lt filename. The header field is pretty easy to grab with Sparkling, and then incorporate it into the DataFrames produced by aut.

Normally, the date should be in this format: Tue, 15 Nov 1994 12:45:26 GMT (RFC1123). But of course, there are always exceptions. So we wrote a fuzzy parser utility to extract the date and format to YYYYMMDDHHMMSS so that it mirrors the same format as crawl_date.

This field is useful for web archive collections that are only crawled once. For example, in situations like Geocities permanently shutting down or the excellent work being done by the Saving Ukrainian Cultural Heritage Online project to preserve Ukrainian cultural heritage.

This functionality is now available as of the 1.2.0 release of the Archives Unleashed Toolkit, and will be available in ARCH in the coming days.

Speaking of ARCH, if you’re interested in piloting it out, please fill out this form.

Here it is in action:

import io.archivesunleashed._

val data = "/sample-data/geocities/GEOCITIES-20091027143300-00114-ia400112.us.archive.org.warc.gz"

RecordLoader.loadArchives(data, sc)
  .all()
  .select($"crawl_date", $"last_modified_date", $"mime_type_web_server")
  .show(20, false)
+--------------+------------------+--------------------+
|crawl_date    |last_modified_date|mime_type_web_server|
+--------------+------------------+--------------------+
|20091027143300|                  |text/html           |
|20091027143259|20000923233454    |image/jpeg          |
|20091027143259|20020913163029    |image/jpeg          |
|20091027143300|20020211154553    |image/jpeg          |
|20091027143259|19980919164703    |image/jpeg          |
|20091027143259|20080125150303    |text/html           |
|20091027143300|20010921224658    |image/gif           |
|20091027143258|20081009015203    |image/jpeg          |
|20091027143300|                  |text/html           |
|20091027143259|20020416145103    |image/jpeg          |
|20091027143300|20090223022835    |text/html           |
|20091027143300|20030928090558    |image/jpeg          |
|20091027143300|20091027143300    |text/html           |
|20091027143300|20021203212451    |text/html           |
|20091027143300|                  |text/html           |
|20091027143300|20040530033010    |image/bmp           |
|20091027143300|                  |text/html           |
|20091027143259|20090223022352    |text/html           |
|20091027143300|                  |text/html           |
|20091027143300|20010608202736    |text/html           |
+--------------+------------------+--------------------+
only showing top 20 rows

There are a few other possibilities for trying to figure out the date of a resource. Since we already use Apache Tika to identify language of a content, and mime_type_tika, we could also potentially use it to try and identify some dates from a given resource in a WARC. It could be a future project if there is a need for it.

I’ve also take the time to update the Geocities dataset again. The updated dataset includes last_modified_date columns for the file format, web pages, and text files derivatives. It also includes crawl_date format fix for domain graph derivative (YYYYMMDD instead of YYYYMMDDHHMMSS).

Here are the 20 most frequent years resources were last modified in the dataset:

Geocities: Year of Last-Modified-Date

Finally, our preliminary research showed reliable values coming from the Last-Modified headers, so we’d love to hear feedback from users on this new feature as they process their data.

grief and rage makes us do funny things / Mark Matienzo

or at least makes us deal with abjection in new ways.

this was my first #TDoR since coming out. i found trans joy where i could get it this last week: new friends, new sights, writing. and saturday night i went out with a group of other gender misfits from around town and we danced, danced, danced. inadvertently i spent a chunk of the evening in what i call “trans denmother” mode: leading people to new spots, making sure people didn’t get fucked with, and ensuring people got home safely.

of course, this was the same night as q club shooting in Colorado Springs. i got home late and didn’t check the news until the morning. i was numb to it at first. then i felt hopeless, and felt my anger burn brighter than the heat of a thousand suns. it stings to know that i can’t protect my friends and family, let alone myself. i shouldn’t have to pack a stop the bleed kit in my totebag when i go out just like i shouldn’t have to leverage deescalation tactics when walking through Capitol Hill, much less anywhere else. but it’s starting to feel like i don’t have a choice.

hopelessness turned into roiling blue flame, one that allowed me to get my blood burbling. i had been avoiding dealing with my name change petition for months, scared of commiting to something in the eyes of the law, that would render my identity visible to the state. “fuck it,” i finally said, and dug up the forms from county court websites and the SSA. i filled out the paperwork, tucked it neatly into a manila folder, and headed to the bank for the fee. when the clerk helped me with the filing i couldn’t get a date for another week, because they’re not holding hearings this week because of thanksgiving.

disappointment is not the word i would use. apathy, maybe, or abjection. it hits me in the gut again. but through all this hot, rancid garbage, i finally know what i have to give the world: my love, my care, myself. to be there the way that my queer trans siblings and family have been there for me. this is for all of you. i love you. i do this for us, so you can do the same for the next crop. replenish yourself when you can, because this world will use you up and hang you out to dry.

Zotero Webinars for Librarians / Zotero

Update: All Zotero webinars are now full! We’ve capped registrations at 100 participants per session to maintain an interactive and engaging experience for our participants. The webinar will also be recorded and shared for those unable to register.

Due to the high level of interest, we’ll be offering this session again early next year, as well as sessions on other topics for both librarians and general users. Webinars will be advertised here and on Twitter.

Teaching Zotero to Undergraduate Students

Friday, December 9, 2022
10:00 AM US Eastern Time

Thursday, December 15, 2022
3:00 PM US Eastern Time

What are the research and citation management needs of undergraduate students, and how does Zotero fit in? In this 60-minute interactive session, we’ll share tips and strategies for teaching Zotero to novice researchers, based on the instructor’s experience teaching Zotero library instruction sessions at a small liberal arts college and a research university.

About the instructor

Brendan O’Connell is an Outreach Coordinator with Zotero. He is an academic librarian by training, with many years of experience in higher education and technology, most recently at Smith College and North Carolina State University in the United States. His work as a librarian has included teaching Zotero to hundreds of students, faculty, and fellow librarians in many different formats and settings.

Brendan’s work with Zotero will focus on a variety of outreach and engagement efforts, including teaching webinars that will train librarians to better teach Zotero to students and faculty at their home institutions, as well as creating tutorial materials for a general academic audience. He is passionate about open-source software and protecting user privacy, values long held in common between libraries and Zotero.

If you have ideas about how Zotero could better engage with librarians, or if there are specific webinars, training materials, or videos you’d like to see, get in touch at outreach@zotero.org.

TRUMP-E / Nick Ruest

I had a really bad idea last night.

Feed the text of tweets to Donald Trump to DALL·E 2.

What’s it look like if you just do it manually?

I grabbed a random .full_text entry from the dataset, and tried it. It didn’t work.

Mike Pence Expected Applause When He Mentioned Trump in Munich. He Got Silence.

It results in an error: “It looks like this request may not follow our content policy.”

Guess what the problem in that text???

Trump

Remove it, and you get this:

Mike Pence Expected Applause When He Mentioned in Munich. He Got Silence Mike Pence Expected Applause When He Mentioned in Munich. He Got Silence.

Creepy.

Salient.

Perfection.

Next, get an API key, install the openai package, and build on the example.

import openai
import random

openai.organization = ""
openai.api_key = ""

with open("sample-tweet.txt","r") as f:
    data = f.readlines()

text = random.choice(data)

response = openai.Image.create(
    prompt=text,
    n=1,
    size="1024x1024"
)

image_url = response['data'][0]['url']

print(image_url)
print(text)

Run it a few times.

1 at @cnnbrk what one thing will jullian Assange not do 1 at @cnnbrk what one thing will jullian Assange not do
Why is it you're tougher on Cuba but all hugs and kisses with North Korea? Why is it you're tougher on Cuba but all hugs and kisses with North Korea?
How about you monitor the release of your tax returns first. Democracy. America first. #PeeBrain How about you monitor the release of your tax returns first. Democracy. America first. #PeeBrain
Hard to believe the so called president has so much time to tweet Hard to believe the so called president has so much time to tweet

Please enjoy the nightmare fuel.

t.co / Ed Summers

People are discovering that Twitter’s archive download includes shortened t.co URLs instead of the original URLs that you tweeted. If Twitter ever goes away, the server at t.co won’t be available to respond to requests. This means your archived tweets will potentially lack some pretty significant context. Even if they don’t work (404) these original URLs are important, because you can at least try to look them up in a web archive like the Internet Archive.

It’s kind of strange that Twitter’s archive download doesn’t display the original URL because some (not all) of the t.co URLs, and their “expanded” form are present in the archived data. The client side app just doesn’t use them in the display. On the other hand maybe it isn’t so strange, since Twitter is in the surveillance capitalism business, and clicks are what corporate social media companies crave most.

Tim Hutton has written a handy tool that will rewrite your archive as a set of Markdown files, using the t.co URLs that are available. I think this is very useful, but I actually really quite like the little client-side application that Twitter provided. So I wanted something that would rewrite the t.co URLs to show the original URL instead.

Before
After

twitter-archive-unshorten is a small Python program that will examine all the JavaScript files in the archive download and rewrite the t.co short URLs to their original full URL form.

If you look closely one thing you may notice in the screenshots above is that other short URLs (e.g. bit.ly) are not unshortened. I didn’t want this program to completely unravel the short URLs since all kinds of things can go wrong when trying to resolve these. I figure if I included a bit.ly URL in my tweet it seems like that’s what the archive should show?

If you want to try it out make sure you have Python3 installed then:

  1. Make a backup of your original Twitter archive!
  2. pip3 install twitter-archive-unshorten
  3. Unzip your Twitter archive zip file
  4. twitter-archive-unshorten /path/to/your/archive/directory/

It might take a while, depending on how many tweet with URLs you have. I had 20,000 or so short URLs so it took a 2-3 hours. Once it’s finished you should be able open your Archive and interact with it without seeing the t.co URLs. The mapping of short URLs to long URLs that was discovered is saved in your archive directory as data/shorturls.json in case you need it.

t.co / Ed Summers

People are discovering that Twitter’s archive download includes shortened t.co URLs instead of the original URLs that you tweeted. If (when) Twitter goes away, the server at t.co won’t be available to respond to requests. This means your archived tweets will potentially lack some pretty significant context. Even if they don’t work (404) these original URLs are important, because you can at least try to look them up in a web archive like the Internet Archive.

It’s kind of strange that Twitter’s archive download doesn’t display the original URL because some (not all) of the t.co URLs, and their “expanded” form are present in the archived data. The client side app just doesn’t use them in the display. On the other hand maybe it isn’t so strange, since Twitter is in the surveillance capitalism business, and clicks are what corporate social media companies crave most.

Tim Hutton has written a handy tool that will rewrite your archive as a set of Markdown files, and which also uses the longer URLs that are available. I think this is super useful, but I actually really quite like the little client-side application that Twitter provided. So I wanted something that would rewrite the t.co URLs to show the original URL instead.

Before
After

twitter-archive-unshorten is a small Python program that will examine all the JavaScript files in the archive download and rewrite the t.co short URLs to their original full URL form.

If you look closely one thing you may notice in the screenshots above is that other short URLs (e.g. bit.ly) are not unshortened. I didn’t want this program to completely unravel the short URLs since all kinds of things can go wrong when trying to resolve these. I figure if I included a bit.ly URL in my tweet it seems like that’s what the archive should show?

If you want to try it out make sure you have Python3 installed then:

  1. Make a backup of your original Twitter archive!
  2. pip3 install twitter-archive-unshorten
  3. Unzip your Twitter archive zip file
  4. twitter-archive-unshorten /path/to/your/archive/directory/

It might take a while, depending on how many tweets with URLs you have. I had 20,000 or so short URLs so it took a 2-3 hours. Once it’s finished you should be able open your Archive and interact with it without seeing the t.co URLs. The mapping of short URLs to long URLs that was discovered is saved in your archive directory as data/shorturls.json in case you need it. There also should be a twitter-archive-unshorten.log file in the root of the archive, with a record of what was done.

Random aside. One interesting thing I discovered when creating this program is that there are some very short t.co URLs, for example: https://t.co/L. I also learned from Hank that somehow https://t.co/elon was (recently?) created.

remarks (DH@30, UVa) / Bethany Nowviskie

[Last week, it was my privilege to participate in an event celebrating the anniversary of centers and institutes that have — for 30 years — supported digital humanities research, scholarship, teaching, and organizing at my alma mater and first professional employer, the University of Virginia — which is to say, the people and organizations that educated me, sustained and supported my growth, and gave me so many unexpected opportunities and gifts.

A joyous reunion weekend on the Lawn was quickly followed by campus tragedy. My heart goes out to the families and friends of the victims, to students, faculty, and staff at UVa, and to all those impacted by the scourge of gun violence in a country that seems to know no other way.]

I sat down to write this at a loss: how could I put three whole decades of gratitude and indebtedness into a 5-minute opening statement? — and I decided I’d do best in pay-it-forward mode — by spending some of my time on hopes for the future.

But one thing I’m grateful for, as I reflect on the past — sitting among so many friends and mentors, fellow-travelers, students of my own — is the privilege to have witnessed the full sweep of the 30 years of DH at UVa that we’re celebrating today.

I’m not sure how many of you know that I was a 19-year-old undergrad at the founding of IATH and EText — and a work-study student in Special Collections, when we created Virginia’s first online finding aids. I also had the amazing good fortune — a scholarship kid from the sticks, a double major in English and Archaeology — to study with two of IATH’s first faculty fellows: Jerry McGann and John Dobbins. What I was privileged to observe (from these two very different sorts of excavators) was their extreme, glowing intellectual excitement at the methods and techniques they were discovering, at what the prompts and provocations of digital representation and analysis might afford.

The excitement was catching. It brought me back for grad school five years later — to help Jerry design an interface to the Rossetti Archive and build subsequent projects like NINES, to work with John Unsworth on that age-old question, “Is Humanities Computing an Academic Discipline?”, to sit around with Steve Ramsay and ponder what it would mean to be “radical bibliographers” (and could he also please teach me Perl) — and mostly just to try not to miss it — to try not to miss the one magical moment of the birth of the web, of the start of DH, of the sense that everything, everything was about to change.

For me, the focus of that excitement changed as I moved, over the years, into very different kinds of organizing and design — as I became less interested in technological innovation than in concepts of maintenance and mutual aid, and as I encountered communities outside the serpentine walls (and learned of those I had missed within them) that helped me imagine more liberatory, even Afrofuturist, digital library infrastructures. But it never went away.

Later panels will elevate the experiences of students at UVa, grad students especially. This panel honors the organizational structures and institutional investments that enabled so much interdisciplinary and — as Steve Railton reminded us last night — inter-professional exchange, foment, and spread.

But to think about organizations is to think generationally.

I hope today’s celebration encourages us to reflect not just on the power and longevity of centers and institutes, but on what or whom we choose to center when we build them, and what kinds of cultures and ideals we will deliberately institute for the future, as we craft our next iterations — either here or elsewhere.

So, what did I try to institute, when in 2007 (as a young mother of two just coming off a postdoc, and thanks to Mike Furlough, Karin Wittenborg, and so many others) I had the opportunity to direct a brand-spanking-new Scholars’ Lab? For me, it was not only a chance to help the Library imagine a next phase of work, a new life for the several centers and teams that folded into it — it was my chance to revitalize a fading local grad student culture that had given me so much. It was a chance to try to spark up again, in the moment of a little lull, the best parts of what I had felt working in DH collectives that were so different from the solitary striving encouraged of most emerging scholars.

That was our reason for founding the Praxis Program in the Scholars’ Lab — for turning what was meant to be the director’s office into an exceptionally grubby, laughter-filled, interdisciplinary grad lounge —for thinking, as the field struggled to become more diverse, about what’s tacit and embodied in DH learning and how often we continued “Speaking in Code” — and for taking a mentoring and training approach to a laboratory culture that I know endures here: of staff creating projects with folks, not for them — and building up new practitioners even more than building out digital products.

I think, honestly, that’s the story and legacy of all the centers represented on this panel. Or at least that’s where their power lies: in the people who have come up through them — in what they’ve gained or learned from the structures around them — the gaps they’ve identified, and chosen to fill — and in what they’ll do next.

Fellow Reflection: Julie Rosier / Digital Library Federation

Julie Rosier

This post was written by Julie Rosier (@redthreadtweets), who attended the 2022 DLF Forum as a Public Library Fellow.

Julie Rosier (she/her) is an aspiring digital archivist, who will receive an MS in information science with an archival focus from the University at Albany, SUNY in May 2023. She received her BA in history and Spanish from the University of Michigan. She works as a Circulation Assistant at her local public library in Northfield, MA, where she recently initiated a digital archives project to provide greater access to the library’s special collections. She also volunteers on an oral history project in conjunction with the 350th anniversary of the town. Before her recent pivot into the library field, Julie worked in higher education administration, including roles such as producer at the Juilliard School and associate director at the UMass Labor Center. Her passion for history, archives and technology has been nurtured over the years by her work as a writer, activist and community-accountable scholar. She is a strong believer in the power of stories, and in 2004 founded a company, called Red Thread Commons, to document community history from the ground up.


Re/Membering, Re-presenting, and Magnifying Black History

I am new to library and archives conferences. However, in the six weeks leading up to the DLF Forum, I attended SAA’s annual meeting and a symposium co-hosted by New England Archivists and Simmons in Boston, and Simmons in Boston and was disappointed by the predominance of whiteness in those spaces. Looking around the room at the Forum’s Opening Plenary in Baltimore, I felt grateful for the number of BIPOC colleagues in the room. It felt like a step in the right direction toward a more balanced and just representation of racial diversity.

As the conference continued, I enjoyed content that spotlighted African American history. The talk Magnifying Gwendolyn Brooks featured a team from the University of Illinois Library at Urbana-Champaign. They discussed digitizing the papers of the Pulitzer prize-winning poet. They also outlined technical challenges related to converting the pièce de résistance, a scrapbook that the team came to call the Red Album, into appropriate digital surrogates. In the end, they created a composite view to capture the entire album layout, and an item view to browse individual photos. The team mentioned the importance of cultivating a strong relationship with Nora Brooks Blakely, Brooks’ daughter, whose input in describing the photos was indispensable.

Tilting the Lens to Black Lives: Using Public Open Digital Scholarship to Develop Student Skills and Deepen Community Engagement was another presentation that I enjoyed immensely. Willa Tavernier, the Research Impact & Open Scholarship Librarian at Indiana University Bloomington, discussed her project, Land, Wealth, Liberation: The Making & Unmaking of Black Wealth in the United States. This work brings to light stories of how the racialized US socio-economic system affects black communities. Tavernier and her diverse team of students and colleagues used digital placemaking to foster engagement. They built a multi-modal exhibit on Omeka S, leveraging tools like TimelineJS and embedded audio from SoundCloud. With the goal of “re-presenting” history, Tavernier’s team created a timeline with labels such as, “Reconstruction to Tulsa” and “Urban Renewal and the Civil Rights Era” to foreground “how public policies prohibited Black Americans’ ability to own land or homes.” According to the project’s homepage, the team offers this digital resource “as a means of fostering intercultural understanding, in the hopes that it will spur meaningful conversations, and help explore paths and policies to achieve reconciliation.”

On the final day of the Forum, Jennifer Garcon presented her work in Building/Digitizing/Preserving Community Archives. She demonstrated her use of lightweight, inexpensive digital technology and preservation techniques to recover and document the lived experiences of Philadelphia’s Black and African diasporic residents. She also discussed her collaborations with Re/Member Black Philadelphia, a project out of Penn, and the Archives for Black Lives Philadelphia (A4BLiP), a loose association of over 75 information professionals responding to the issues raised by the Black Lives Matter movement. In response to an audience question about grappling with problematic actions taken by the University of Pennsylvania and Princeton University regarding the remains of child victims from the 1985 MOVE bombing, Garcon replied that academic institutions must do the work to reckon with their legacies of harm in surrounding communities. Garcon also expressed gratitude for her connection to the A4BLiP network, which allows her to deliver institutional resources directly to Black communities while promoting a post-custodial approach to community archiving.

Overall, I was excited to witness Women of Color practitioners working in opposition to archival silences and institutional erasure of African American history. Whether amplifying the legacy of an influential poet like Gwendolyn Brooks, shining a light on the oppressive impact of centuries of racialized US socio-economic conditions on Black wealth, or operating pop-up digitization sites for community members who have rightly lost faith in institutional repositories, these projects inspired me to the core. I hope to follow in their footsteps as an emerging archival professional committed to walking a path through hard truths.

 

Several 2022 DLF Forum Fellows pose in the Baltimore harbor after a fun night out on the town.Several 2022 DLF Forum Fellows pose in the Baltimore harbor after a fun night out on the town.

 

Willa Tavernier after presenting on how digital collections focusing on the stories of historically marginalized groups can be an important site for student learning and community engagement.Willa Tavernier after presenting on how digital collections focusing on the stories of historically marginalized groups can be an important site for student learning and community engagement.

 

Jennifer Garcon (l) shows workshop participant Anne Ray (r) how to use the lightweight, minimal computing digitization station, as 2022 Forum Fellow Kelley Klor observes.Jennifer Garcon (l) shows workshop participant Anne Ray (r) how to use the lightweight, minimal computing digitization station, as 2022 Forum Fellow Kelley Klor observes.

The post Fellow Reflection: Julie Rosier appeared first on DLF.

The end of Twitter? / Meredith Farkas

Bye Bye Birdie movie poster from the 1963 film

The tl:dr is in whatever online communities you frequent, do you ever ask yourself “who isn’t here? Who doesn’t feel welcome here? What barriers are there to participation?” If not, you should. What I like about Twitter is how much more diverse it was than any place I’ve been online, and also how so many of us managed to create beautiful supportive communities there. People found their people there, but it was also so open enough that anyone could join into a conversation. I learned so much from so many people on there. I plan to stay on Twitter in solidarity with my community until it burns down, you can subscribe to my blog posts as an email newsletter here, and while I have a Mastodon account (@librarianmer@m.wolfwater.com) you can follow, I’m deeply skeptical that it’s going to be a good Twitter replacement.

As I’ve embraced slow culture more and more in my own life, one thing I’ve started doing is deeply interrogating my gut reaction to things rather than feeling an urgency to get my thoughts out there or to take action. Knowing that I’ve been raised in a white supremacist, patriarchal, heteronormative, christian society, I recognize that my first impulses or instincts (and likely all of yours too) are often informed by that noxious miasma we’ve been breathing all our lives. I take my time now. I observe more. I question my assumptions. I think this comes with the humility of recognizing that I don’t have all the answers, that I sometimes get things wrong. My “gut” is fallible. It helps to not feel like I’m grasping for anything anymore, be it a better job, innocence/goodness, more followers on social media, or the approval of others. Back when I was perpetually working to fill a hole inside, created by believing I was not enough, I was constantly looking for external validation. I don’t really need that anymore; I’m good where I am.

So instead of freaking out about Elon Musk buying Twitter, I’ve been observing. It’s been interesting to watch the dynamics of library workers’ reactions to it. I guess as someone with over 8,000 followers on Twitter, I should be more panicked about this, but I’m not. I’ve long had a love-hate relationship with Twitter. As a completist, I first struggled with the idea that I would miss most of the conversation. I took issue with platforms owning our content, and the lack of deep, thoughtful, reflective conversation. I’ve railed against social media mobbing and shaming. I’ve expressed concern about the ephemeral nature of our intellectual production on social media. I hated that Twitter was killing blogging. But in recent years, I’ve actually come to love my little corner of Twitter and the people I converse with regularly. It’s come to feel like a real community, but one that’s also very open and constantly growing and changing organically. Each of our communities look somewhat different. We have followers in common, but also many unique folks. I have gotten so much support and learned so much from my community. I wouldn’t be who I am today without them. They helped me see behind the veils of white supremacy, ableism, and other systems of oppression. At the start of COVID, we helped advocate together for better safety precautions and for libraries to close — sharing resources and letters, putting pressure on institution, etc. We celebrated each other’s wins and mourned together each other’s losses. I felt like I could be myself in that space. While I used to go to conferences and had time at work to collaborate with people outside my institution on projects, neither is really a possibility for me in my current job. My community on Twitter was a lifeline for me, a way to connect with library workers who shared my values and interests. I’d found my people. I wasn’t always super-active on Twitter. I sometimes wouldn’t post for weeks, but I always felt like I had an online home to come back to. It sucks that that might soon no longer be the case.

When I look back on that golden age of library blogging that I’d so idealized at the time, what I see now is how White it was. There was very little diversity of any kind. Given the percentage of women in librarianship, it was disproportionately male-dominated. I’d believed at the time that it was bringing down barriers because it gave people like me (a brand-new librarian working at a tiny college no one has heard of in rural Vermont) the opportunity to make an international name for myself, but I had that opportunity because I ticked certain boxes, and there were still so many barriers I didn’t notice until I saw the diversity on Twitter. Not that there aren’t power dynamics on Twitter too, but it brought down, to such a huge extent, the barriers to entry and helped people (who maybe were “the only” at their place of work) find their people. This has led me to look at any space I’m in (online or in-person) and wonder who isn’t here? Who doesn’t feel comfortable here? What barriers exist to participation? It’s also made me think of how the ways I participate may encourage or discourage others from participating. I recently realized that I was talking too much in a particular committee meeting and I’m trying to make sure I take up less space there. Shut up Meredith!

I locked my Twitter account a while back when I realized that I didn’t actually want to broadcast my thoughts more widely than the community I already had. I’d also had several run-ins with trolls over the years, the TERFs by far being the most offensive and abusive in response to my American Libraries column about the Seattle Public Library’s willingness to give them a platform. But mainly, I realized that I wasn’t looking for increased fame or reach or to go viral with people I don’t know. I wasn’t looking for more speaking or writing opportunities. I wasn’t looking for a new job or to climb a ladder. I was looking for community and I’d found it with a wonderful and diverse group of library workers. If you’re part of that group, please know that I am so grateful for you.

From people’s panic over being on a platform run by someone as unethical as Musk, you’d think they’d never used a social media tool run by a toxic person who set out to do everything in their power to exploit and make money off users (hi Mark!). The fact is, Twitter was already exploiting us and in much less obvious ways. The irony of people moving over to Instagram as a less morally-compromised alternative was rich. I haven’t used Instagram in over three years because I found it to be far more toxic than any other social media tool I’ve used. It depleted me rather than filled me up and I felt sick over what it was doing to young women’s senses of self. I dumped LinkedIn years ago. I tried a few Discords and I realized they weren’t for me for a few reasons. The first is that they felt cliquey and I’m deeply uncomfortable with spaces like that. The second is the amount of time one must devote in spaces like that to become part of the community. It privileges people without caregiving roles and people who have jobs where they can be on social media all day. That isn’t me. Even if I had a job in which that was possible, I am far too easily distracted to multi-task, so I keep social media off any device but my home computer. My work Slack is distracting enough. I had the same problem with FriendFeed back in the day and watched sadly as people I was really close to before my son was born got closer to one another on there and more distant from me. I’m definitely not looking to repeat that experience.

When Musk took over, a lot of people immediately moved to Mastodon, which is a network of decentralized servers running an open source social media tool called Mastodon. It allows for posting similar to Twitter, but with a larger character limit, and people can similarly like and repost your content. When you join Mastodon, you join a particular server run by an individual, a particular community. There are communities for specific geographic areas, for specific interests (furries, gamers, GNU Linux users), for specific industries (tech, librarians/archivist/museum folks, scholars), and also more general ones. You apply to join a particular server (some are closed, some are very restrictive about who they allow to join) which is your home base, but you can follow people on any instance if you can find them. There are tools for finding former Twitter peeps, but they don’t pick up everyone and so it can be hard to find “your people” again. I heard from one friend who is on three separate Mastodon servers in order to interact with different people about different areas of her life and that sounds beyond overwhelming. What I loved about my Twitter community is that I interacted with the same people about professional stuff as about personal. This feels like a step backwards.

The first thing that makes me uncomfortable is that I’d be using a server I pretty much know nothing about. I don’t know who runs each of these instances, but it’s a human being for whom this is a hobby. What if it gets hacked? What if they just decide not to maintain it anymore? What if they decide to close up shop and your identity and content? Yes, you can switch to other instances, but what if you get no notice and everything is gone? I don’t have my email on some random server run by a single person I don’t know. I’ve also been burned by enough open source projects in the past to feel really reluctant to commit to this. It feels like building castles out of sand close to the water line. Which maybe is fine given that it’s social media, but it does give me pause.

My concerns about Mastodon aside, the thing I noticed was that nearly all of the people I knew who were huge boosters for Mastodon, who ran from Twitter first, were White. The BIPOC and LGBTQ folks and people in the disability community who I follow were bringing up valid concerns about Mastodon and I was largely seeing those concerns blown off. One White library worker on Twitter compared it to criticizing lifeboats on a sinking ship <insert eyeroll here>. All of the people who I saw panicking over Musk buying Twitter were the people with the most privilege. I saw a similar dynamic amongst non-library workers too, so it wasn’t unique to our profession. There was an interesting essay entitled “Fleeing Twitter — The “Twexodus” — is About White Liberal Fragility” that astutely analyzed this trend. But even more than that, what does it say about you if you’re abandoning Twitter, at a time when content moderation is basically gone and there’s a significant rise in hate speech, when your friends who have less privilege and whose identities make them a target of harassment are staying? If White “allies” abandon Twitter, leaving their BIPOC, queer, trans, NB, and disabled “friends,” were they ever really allies?

I wish there were a wonderful place that is an obvious replacement for Twitter for me and the other folks who are still on there. It seems like for some Mastodon or Instagram or a particular Discord community really does fill that void. But I haven’t found that thing for me and I’m not interested in broadcasting my thoughts in another space that doesn’t fulfill my need for community. I’m not looking for an “audience” anymore and that’s not to knock people who are, but that’s just not where I am in my life. If I can’t find what works for me, I might just stick with blogging. I just started putting my blog content into a substack newsletter people can subscribe to via email (I still truly don’t understand the appeal of newsletters, but I know many do like getting their content that way). So if you’d like to get my random reflections in your Inbox, here’s your chance! My husband set up a Mastodon server for me and him, which at least gets me past my concerns about some rando hosting my content, so you can also follow me at @librarianmer@m.wolfwater.com, but I can’t promise this will be a long-term option for me. I mainly have it just in case Twitter blows up, which is looking more likely by the minute.

I plan to stay on Twitter until it literally burns to the ground or until the people in my community find a new place to be. For me, the social media experience is about the specific people in my community (not about me sharing my brilliant wit and comical observations), so there is no reason for me to go anywhere else until all of my friends on Twitter are gone. If we converse on Twitter, please know that YOU are the reason I spend time there; you are what makes it worthwhile for me. My tweets are locked and only visible to my 8300 followers. I have my Twitter account pretty well secured. Nothing I write on Twitter is something I’d be embarrassed about or is important enough for me to save. I feel safe enough.

For those who have found homes in other online communities, what does it offer that keeps you there? For those who haven’t, what features would make a community feel like home for you?

Image credit: Wikipedia

Fellow Reflection: Leland Riddlesperger / Digital Library Federation

Leland Riddlesperger

This post was written by Leland Riddlesperger, who attended the 2022 DLF Forum as a Grassroots Archives and Cultural Heritage Organizations Fellow.

Leland Riddlesperger is the Associate Archivist at the Santa Barbara Mission Archive-Library (SBMAL). Since accepting this position in February of 2022, he has implemented the institution’s transition to ArchivesSpace and manages the processing and preservation of born-digital content. He holds his MSLS from Clarion University, and studied History at The College of William & Mary. Leland is an active member of DLF’s Born-Digital Access Working Group, and is currently serving a term as Co-Coordinator.


I would like to say that I am so grateful for the opportunity to attend the DLF 2022 Forum. I am thankful to CLIR for awarding me one of the fellowships to help cover the costs. My institution would have been unable to send me to the Forum without the aid I received. I feel that I greatly benefitted from what I learned at this forum, and, that in turn, my institution and our patrons benefitted as well. I could go on about every detail about this forum and how much I learned and enjoyed my time. Instead, I will focus on the two aspects I appreciated the most: the relevancy of the subject matter presented in each panel that I attended, and connecting with colleagues and fellow members of the Born-Digital Access Working Group (BDAWG). 

This assessment is not a criticism of any other conference; it’s simply that I learned much more relevant information to my job than I ever have before in this setting. The panels at this conference presented information about real-life applications and experiences. There were panels about various tools, skills, and decision-making that I can apply to my own work. 

I would like to shout out to one panel in particular that I felt was extremely informative: What’s in a DAM?: Tackling Digital Asset Management selection, implementation, and beyond. I am one part of the decision for selecting and implementing a Digital Asset Management system at my institution, so the content of this panel was right on the nose for me. I thought the three presentations were from a unique perspective as there were two different types of institutions represented as well as one from the perspective of a vendor. They spoke about their experiences throughout the planning and implementation process. I found the Q&A afterwards was also extremely helpful. I was able to ask all the questions I had, but also learn from the questions that others had who are going through the same process. It gave me a lot to think about, and I certainly feel more prepared to begin planning the DAM software selection and implementation at my institution. 

This was my first conference since before the pandemic, so it felt great to be around so many of my peers in this field. This aspect is always one of my favorite things about attending these conferences. I enjoy seeing people that I used to work with, people with which I am collaborating on projects, and meeting new people that share my interest and passion for information. This was my experience at DLF Forum 2022! I ran into colleagues from past employers. It is always nice catching up with former coworkers and hearing about the projects they are working on. I work closely with the DLF group BDAWG, and the BDAWG subgroup Visioning Access Systems (VAS). I have been virtually meeting with these people for two years in some cases. This was the first time I was able to meet in-person with the members of these groups. 

I am very grateful to DLF and CLIR for giving me the opportunity to have these wonderful experiences at DLF Forum 2022. I learned so much from the panels and all the insightful questions that were asked by the attendees as well. I thought the whole event was a huge success. It was organized well, and featured panels covering a variety of dynamic topics relevant to my field. The venue was an excellent choice for this event. I also enjoyed the location–right on the harbor in Baltimore. I was very impressed by this forum and extremely thrilled that I was able to attend. I hope to see you all at DLF Forum 2023! 

The post Fellow Reflection: Leland Riddlesperger appeared first on DLF.

Bookmarks / Ed Summers

As you already know, lots of people are leaving Twitter, and taking their archive download with them. Twitter’s archive is really quite decent, since unlike other platform downloads, it gives you a local client-side web application for viewing your content. You aren’t stuck with a bunch of impenetrable JSON and media files (yes, I’m looking at you Slack).

However, some people are noticing that the downloadable archive doesn’t include their bookmarks. Bookmarks aren’t a feature that I ever used myself, but I know many others have. If you’ve spent significant time creating bookmarks I’m sure you are reluctant to part with them. Deleting your account without having a copy seems like a bad idea.

Yes, there is an API endpoint to fetch your bookmarks. But it only returns the last 800 tweets, so depending on how many bookmarks you’ve created that might not be good enough. The endpoint returns the JSON data for your bookmarks which isn’t exactly user-friendly. Also API endpoints, and developer keys are beyond the ken of most Twitter users, so it’s really just not an option for most people.

Scraping tools like snscrape also don’t seem to support fetching user bookmarks, probably because snscrape is hard coded to use a particular access key, which is fine for collecting public information from Twitter, but won’t work when fetching your personal bookmarks. And again, installing Python, getting the snscrape utility, and then running it on the command line isn’t really a generally accessible approach.

So, what to do?

One option you might want to consider is using Webrecorder’s ArchiveWebPage Chrome extension to create an archive of your bookmarks, and then viewing the archived content locally. Once you’ve actually deleted your Twitter account you can host your bookmarks archive somewhere online, like your personal website for viewing in any (JavaScript enabled) web browser. You could even to try to archive your entire Twitter profile this way. Read on for how to do this, and why you probably should only publish your archived content after you’ve deleted your account.

Since this really is an interactive process here’s a quick video of me collecting my Twitter bookmarks with the ArchiveWebPage extension:



It’s really important to point out here that the ArchiveWebPage extension is a client-side web application. All the archived data is on your computer. There’s no copy of it up in the cloud anywhere (unless you decide to put it there later).

Speaking of the data, you can “export” the archived content, which takes it out of the browser local storage and packages it up as a WACZ file, which is similar to EPUB or Word (DOCX) file in that it’s just a ZIP file. You can view this file at any time by importing it back into ArchiveWebPage, or using the ReplayWebPage player to play it back. ReplayWebPage is also a client-side application. The data you load in is store locally–not in the cloud.

If you do want to publish your archive you can embed it on a web page of your choosing using the <replay-web-page> web component.

To be on the safe side you probably only want to do this once you’ve decided to remove your Twitter account. The web archive contains all the communications from the Twitter website, so it’s possible there might be some time-sensitive, private information in the archive that someone could use to access your account. But if you don’t have an account anymore 🤷

Here’s the HTML I’m using here on my Jekyll static site:

<script src="https://cdn.jsdelivr.net/npm/replaywebpage@1.7.2/ui.js"></script>
<replay-web-page 
  style="height: 600px"
  embed="replay-with-info"
  replayBase="/js/replaywebpage/"
  source="https://edsu-webarchives.s3.amazonaws.com/twitter-bookmarks.wacz"
  coll="2022-11-11-random"
  url="https://twitter.com/i/bookmarks" />

which displays:



More information about sharing and embedding WACZ web archives can be found in the ReplayWebPage documentation. If you run into questions please ask over in the Webrecorder Forum.


Update: as Ryan Baumann notes, it looks like Twitter’s own interface may not let you see more than 800 bookmarks :-( So you may not be able to get more than that with ArchiveWebPage. Still I think there’s an advantage to using ArchiveWebPage here since the tweets will be viewable with context and not just some JSON or CSV file?

Update: Ryan also has a Ruby program that will get around the 800 limitation by getting your bookmarks while deleting them–which is a pretty ingenious way around the API limitation. He also has a super thread where he compares some other options like the GetDewey service.

Update: See Ryan’s blogpost about the Ruby program!

Advancing IDEAs: Inclusion, Diversity, Equity, Accessibility, 2022 November 15 / HangingTogether

The following  post is one in a regular series on issues of Inclusion, Diversity, Equity, and Accessibility, compiled by Jay Weitz.

Demythologizing Thanksgiving

Photo by Aaron Burden on Unsplash

Just in time for the holiday, the Association for Library Service to Children‘s ALSC Blog offers “Thanksgiving Books Without the Myth” by Abby Johnson, Collection Development Leader at the Floyd County Library in Indiana. Johnson lists books that concentrate on current traditions, books from the viewpoint of Indigenous people, books about gatherings of family or friends in which food is shared, and books more generally about gratitude. “I know several years ago, we looked at our Thanksgiving books and determined that if we weeded the books that contained the Thanksgiving myth, we would have almost nothing left and there weren’t better books to fill the demand,” Johnson writes. “That’s getting better every year (for which I am thankful!).”

“Bringing Disability to the Forefront”

Reference librarian at Mercer County Community College (OCLC Symbol: MRR) in Trenton, New Jersey, Stephanie Sendaula offers some practical advice about “Bringing Disability to the Forefront” in a recent ACRLog posting from the Association of College and Research Libraries. She emphasizes how important it is to both “consider the needs of disabled students year-round” and “highlight books by chronically ill and disabled authors throughout the year.” So many students and fellow workers “are living with either chronic illness, disability, or both,” and so much of it can be invisible.

Muslim representation

Earlier this year, Hijabi Librarians issued a 2022 update of its free guide to Evaluating Muslims in KidLit: A Guide for Librarians, Educators, and Reviewers, by Ariana Sani Hussain, a teacher librarian in the Twin Cities metropolitan area in Minnesota, and Mahasin Abuwi Aleem, a San Francisco Bay Area children’s librarian. Acknowledging that literature is often the first exposure that children have to both Islam and Muslims, the guide intends “to be deliberate in combating tropes and stereotypes that deal in disinformation, fear-mongering, and histories rooted in ‘orientalism’ (as coined by scholar Edward Said), colonialism and white supremacy.” In two sections, the guide “identifies resources for learning more about Muslim Americans while providing context for media representation” and suggests questions with which to consider and analyze individual works and library collections.

Jewish representation

The Association of Jewish Libraries recently published Evaluating Jewish Representation in Children’s Literature, edited by Heidi Rabinowitz, a past president of AJL and Library Director for Congregation B’nai Israel in Boca Raton, Florida. “Jews have always been marginalized and Jewish books belong everywhere that diverse books are being discussed,” the free resource notes. “The Jewish community itself is also very diverse. According to GlobalJews.org, 20% of America’s 6 million Jews or 1.2 million are African American, Latino/Hispanic, Asian, Sephardic, Mizrahi, mixed race and growing. In recent years, Jewish children’s literature has started to reflect this reality.” The AJL document, which also delves a bit into materials for adults, pays homage to and takes inspiration from the aforementioned Hijabi Librarians’ guide.

LGBTQReads

Author, editor, and founder of the blog LGBTQReads.com, Dahlia Adler recommends eight recent or upcoming young adult LGBTQ+ romances and romcoms in Booklist. As an aside, the November 2022 Booklist newsletter “Quick Tips for Schools and Libraries” in turn recommends Adler’s blog as “a beautifully organized database of LGBTQ children’s and young adult books.”

Researching Asian American history

Gonzaga University (OCLC Symbol: WUG) librarians Shayna Pekala and Stephanie Plowman and political scientist Shyam K. Sriram, formerly at Gonzaga, in Spokane, Washington, but now at Canisius College (OCLC Symbol: VKC) in Buffalo, New York, together created the first “Asian American Politics” seminar to be offered at Gonzaga earlier in 2022. They write about the experience and how it ended up “Inspiring Asian American awareness through archival research” in the November 2022 issue of College and Research Libraries News (Volume 83, Number 10, pages 427-431). Requiring students “to engage physically with the library,” making research consultation with the liaison librarian mandatory, introducing the library to students in a manner that caused them to reevaluate the institution’s offerings, and offering “the freedom to develop their own research agenda,” proved a successful formula that the authors believe can be replicated elsewhere. Beyond the library instruction aspects of the project, “we believe that all colleges have a responsibility to educate students on Asian American history, which is a preemptive step in ending anti-Asian bias.”

Diversity audits

Library collection diversity audits help analyze the varieties of representation, viewpoints, and experiences that can be found in the institution’s materials. Whether conducted manually or through automated means, they can lead to more inclusive and equitable collection development. In “Using Data From Collection Diversity Audits” (Computers in Libraries, November 2022, Volume 42, Number 9, pages 15-19), four librarians recount their experiences in planning and conducting audits with the goal being that “Every new patron who walks through the door should be able to see their identity represented on your library’s shelves and display units.” Anitra Gates and Amberlee McGaughey of Erie County Public Library (OCLC Symbol: EPL) in Pennsylvania, Celia Mulder of Clinton-Macomb Public Library (OCLC Symbol: YMN) in Michigan, and Sarah Voels of Cedar Rapids Public Library (OCLC Symbol: IWR) in Iowa write about the valuable information that can be gleaned from such audits but also warn of the shortcomings and how to compensate for them.

Web accessibility indicators

Reporting on the poll of success indicators for web accessibility that they conducted in 2020, Sonya Schryer Norris of Plum Librarian LLC and Jared Oates of Niche Academy reveal the “Data Behind Key Web ACCESSIBILITY Success Indicators and Staff Training Needs” (Computers in Libraries, November 2022, Volume 42, Number 9, pages 20-24). The two most prominent shortcomings were the absence of an explicit accessibility policy that includes the name of an Americans with Disability Act contact person plus a complaint form and a procedure to follow, and the lack of alt text, which “describes images to computer users who are blind and accesses websites via screen reading software.”

Coping with book challenges

On November 17 at 5:00 p.m. Eastern, Booklist will present “Persevering in the Face of Book Challenges,” a free webinar intended to be “a frank and inclusive conversation about how we press on to do the best for our patrons and communities, for intellectual freedom, and for ourselves in these difficult circumstances.” Cindy Hohl, Director of Branch Operations at Missouri’s Kansas City Public Library (OCLC Symbol: KCP); Kansas’s Olathe East High School (OCLC Symbol: SX3) Librarian Shelly McNerney; and Gavin Downing, Kent School District librarian and winner of the Washington Library Association‘s Candace Morgan Intellectual Freedom Award, are the panelists.

The post Advancing IDEAs: Inclusion, Diversity, Equity, Accessibility, 2022 November 15 appeared first on Hanging Together.

Welcome Molly White: Library Innovation Lab Fellow / Harvard Library Innovation Lab

The Harvard Library Innovation Lab is delighted to welcome Molly White as an academic fellow.

Molly has become a leading critic on cryptocurrency and web3 issues through her blog Web3 is Going Just Great as well as her long-form research and essays, talks, guest lectures, social media, and interviews. Her rigorous, independent, and accessible work has had a substantial impact on legislators, media, and the public debate.

In her work at the Library Innovation Lab, Molly will further examine goals she shares with the web3 community she studies—such as orchestrating community-governed projects, returning power to users from advertisers and tech companies, and increasing access to financial services and reducing wealth inequality—and consider what attracts individuals to web3 and what can be learned from their experience about how to advance those causes.

We are excited for Molly to join the Lab because we share her interest in reinventing traditional institutions to better empower people, while being deeply critical of proposed replacements that may misfire or disempower people. We also admire her skill in translating technical insights into effective advocacy. As libraries continue to rethink our role—as a cultural memory, information network, and social safety net—we have a great deal to learn from her work.

Molly will be joining us for one year, as part of our Democratizing Open Knowledge program supported by the Filecoin Foundation for the Decentralized Web. Welcome, Molly!

Cautions on ISBN and a bit on DOI / CrossRef

I have been reading through the documents relating to the court case that Hachette has brought against the Internet Archives "controlled digital lending" program. I wrote briefly about this before. In this recent reading I am once again struck by the use and over-use of ISBNs as identifiers. Most of my library peeps know this, but for others, I will lay out the issues and problems with these so-called "identifiers".

"BOOK"

The "BN" of the ISBN stands for "BOOK NUMBER." The "IS" is for "INTERNATIONAL STANDARD" which was issued by the International Standards Organization, whose documents are unfortunately paywalled. But the un-paywalled page defines the target of an ISBN as:

[A] unique international identification system for each product form or edition of a separately available monographic publication published or produced by a specific publisher that is available to the public.

What isn't said here in so many words is that the ISBN does not define a specific content; it defines a  salable product instance in the same way that a UPS code is applied to different sizes and "flavors" of Dawn dish soap. What many people either do not know or may have forgotten is that every book product is given a different ISBN. This means that the hardback book, the trade paperback, the mass-market paperback, the MOBI ebook, the EPUB ebook, even if all brought to market by a single publisher, all have different ISBNs.  

The word "book" is far from precise and it is a shame that the ISBN uses that term. Yes, it is applied to the book trade, but it is not applied to a "book" except in a common sense of that word. When you say "I read a book" you do not often mean the same thing as the B in ISBN. Your listener has no idea if you are referring to a hard back or a paperback copy of the text. It would be useful to think of the ISBN as the ISBpN - the International Standard Book product Number.

Emphasizing the ISBN's use as a product code, bookstores at one point were assigning ISBNs to non-book products like stuffed animals and other gift items. This was because the retail system that the stores used required ISBNs. I believe that this practice has been quashed, but it does illustrate that the ISBN is merely a bar code at a sales point.

1970

The ISBN became a standard product number in the book trade in 1970, in the era when the Universal Product Code (UPC) concept was being developed in a variety of sales environments. This means that every book product that appeared on the market before that date does not have an ISBN. This doesn't mean that a text from before that date cannot have an ISBN - as older works are re-issued for the current market, they, too, are given ISBNs as they are prepared for the retail environment. Even some works that are out of copyright (pre-1925) may be found to have ISBNs when they have been reissued. 

The existence of an ISBN on the physical or electronic version of a book tells you nothing about its copyright status and does not mean that the book content is currently in print. It has the same meaning as the bar code on your box of cereal - it is a product identifier that can be used in automated systems to ring up a purchase. 

The Controlled Digital Lending Lawsuit

The lawsuit between a group of publishers led by Hachette and the Internet Archive is an example of two different views: that of selling and that of reading.

 

In the lawsuit the publishers quantify the damage done to them by expressing the damage to them in terms of numbers of ISBNs. This Implies that the lawsuit is not including back titles that are pre-ISBN. Because the concern is economic, items that are long out of print don't seem to be included in the lawsuit.

The difference between the book as product and the book as content shows up in how ISBNs are used. The publisher’s expert notes that many metadata records at the archive have multiple ISBNs and surmises that the archive is adding these to the records. What this person doesn’t know is that library records, which the archive is using, often contain ISBNs for multiple book products which the libraries consider interchangeable. The library user is seeking specific content and is not concerned with whether the book is a hard back, has library binding, or is one of the possible soft covers. The “book “ that the library user is seeking is an information vessel.

It is the practice in libraries, where there is more than one physical book type available, to show the user a single metadata record that doesn’t distinguish between them. The record may describe a hard bound copy even though the library has only the trade paperback. This may not be ideal but the cost-benefit seems defensible. Users probably pay little attention to the publication details that would distinguish between these products. 

 

From a single library metadata record

 

Where libraries do differentiate is between forms that require special hardware or software. Even here however the ISBNcannot be used for the library’s purpose because services that manage these materials can provide the books in the primary digital reading formats based off a single metadata record, even though each ebook format is assigned its own ISBN for the purpose of sales.

The product view is what you see on Amazon. The different products have different prices which is one way they are distinguished. A buyer can see the different prices for hard copy, paperback, or kindle book, and often a range of prices for used copies. Unlike the library user, the Amazon customer has to make a choice, even if all of the options have the same content. For sales to be possible, each of the products has its own ISBN. 

Different products have different prices


Counting ISBNs may be the correct quantifier for the publishers, but they feature only minimally in the library environment. Multiple ISBNs on a single library metadata record is not an attempt to hide publisher products by putting them together; it's good library practice for serving its readers. Users coming to the library with an ISBN will be directed to the content they seek regardless of the particular binding the library owns. Counting the ISBNs in the Internet Archive's metadata will not be a good measure of the number of "books" there using the publisher's definition of "book."


Digital Object Identifier (DOI)


I haven't done a deep study of the use of DOIs, but again there seems to be a great enthusiasm for the DOI as an identifier yet I see little discussion of the limitations of its reach. DOI began in 2000 so it has a serious time limit. Although it has caught on big with academic and scientific publications, it has less reach with social sciences, political writing, and other journalism. Periodicals that do not use DOIs may well be covering topics that can also be found in the DOI-verse. Basing an article research system on the presence of DOIs is an arbitrary truncation of the knowledge universe.

 

The End

 

Identifiers are useful. Created works are messy. Metadata is often inadequate. As anyone who has tried to match up metadata from multiple sources knows, working without identifiers makes that task much more  difficult. However, we must be very clear, when using identifiers, to recognize what they identify.


Fellow Reflection: Jerrell Jones / Digital Library Federation

Jerrell Jones

This post was written by Jerrell Jones (@j_digilib), who attended the 2022 DLF Forum as a Student Fellow.

Jerrell Jones is a digitization specialist, professional photographer, and fine art creator. He earned a Bachelor of Fine Arts degree in photography from the University of North Texas and is currently pursuing a Master of Science in Information Science from the University of North Texas. In addition to a decade of academic library service, he has served through roles in the oil and gas sector and mental health rehabilitation field focused on building sustainable partnerships across institutions, companies, and non-profit organizations. Jones is currently contributing to the University of Houston (UH) Libraries as Digitization Lab Manager where he previously served on the Libraries’ Committee on Diversity and Inclusion and currently serves on the USRA Lunar and Planetary Institute IDEA committee. As a recent ALA Spectrum Scholar, Jerrell continues his pursuit of advancing equity, diversity, and inclusion in academic libraries, digital preservation, and community archiving. When Jones is not thinking about libraries, he can be found developing a roll of film, salsa dancing, propagating plants, and keeping a close eye on commercial airliners or the next space vehicle launch.


The DLF 2022 Opening Plenary was quite engaging and I’ve returned to considering it on several occasions since the conference. The two-panelist format facilitated a vibrant conversation between the panelists that elicited audible positive reactions from the audience, leading to an active Q&A session that left us wanting more – a fantastic way to start a conference. The recurring idea was that the technologies that we’ve come to rely on, even as early as the traditional keyboard, may need a second look to remain relevant, accessible, and effective for the rapidly-changing information landscape ahead of us. Meredith Broussard highlighted that essential equipment like soap dispensers and medical equipment are often designed for users deemed as “standard” which often ignores skin color, sex, body size, and other ways that make us beautifully different. Environments that struggle to accommodate remain behind the curve and as a result, lose their power to connect. David Nemer reminded the audience that social media giants like Meta authoritatively claim that they “connect the world” when in reality, this is not possible because countless communities remain unconnected with the rest of the world based on political, economic, and algorithm-based exclusion. Communities overlap and share common ground much less often than social media claims. A lot of fascinating ground was covered in this opening plenary but what I took away from this conversation was the reminder that information professionals are not off-of-the-hook when it comes to intervening when technologies do more harm than good. Broussard mentioned that artificial intelligence has a racism problem highlighting that as computers get “smarter” they also gain the potential to harm unless humans intervene, an intervention that was deemed essential as we allow these technologies to touch more areas of our lives. 

Overall, the most memorable moments of the conference were marked by rich conversations, many of which I had in between sessions, that affirmed my work in equity, diversity, and inclusion at the intersection of library service. I valued the level of transparency many conference attendees provided when they talked at length about the challenges at their institutions, something for which I was grateful. I sat at several tables during the conference filled with brilliant individuals that traveled from HBCUs, explaining the nuances and broader impact of their work that made me proud of them, proud to be Black, and even more proud to be a Black man working in the field of libraries. Quite simply, I was affirmed in my work, empowered in my work, by strangers that gave me no indication that we were not already family. Conferences are wonderful catalysts for connection but they often fail to provide this level of peace and unconditional belonging for diverse individuals.  I was grateful that CLIR invited the 2022 Authenticity Project Fellows and provided ample time and space for these individuals to present their work and engage in a series of conversations that could result in important collaboration for years to come. To date, I’ve never been to a conference in which I felt so seen and utterly excited to collaborate with new colleagues and that experience will remain with me. 


Thank you to Texas Digital Library, sponsor of the 2022 DLF Forum Student Fellowships.

Texas Digital Library

The post Fellow Reflection: Jerrell Jones appeared first on DLF.

OCLC v. Clarivate: What was MetaDoor? What is an OCLC Record? / Peter Murray

On November 7, 2022, OCLC and Clarivate announced a settlement in their lawsuit about using WorldCat records in the embryonic MetaDoor service. This ended the latest chapter in the saga of reuse of library metadata with little new clarity. The settlement terms were not disclosed, but we can learn a little from the proceedings. First, let’s review the press releases from the parties. Then we’ll look at the transcripts of court proceedings to see if we can get closer to answers about some questions this lawsuit raises.

Clarivate’s Statement

Clarivate’s statement about the settlement is quite vague:

Clarivate continues to deny OCLCs allegations of wrong-doing and maintains that the issue lay between OCLC and its customers, who sought to co-create an efficient community platform for sharing of bibliographic records. Clarivate will not develop a record exchange system of MARC records that include records which OCLC has claimed are subject to its policy and contractual limitations. Clarivate will bear its own fees and costs.

Gordon Samson, Chief Product Officer at Clarivate insisted, “Clarivate will continue to support the goals of open research and data exchange - because we believe it is the best way to make the process of research and learning faster, more robust and more transparent. Regardless of business model, when scholarly information is easily accessible and shareable, the dots are easier to join, the connections are explicit, and collaborations are more natural and meaningful. The process of scientific discovery is faster, and it is easier to ensure research integrity and reproducibility. We know that navigating the transition to open research is important to our customers, and we remain committed to helping them make that transition as seamlessly as possible.”

- “Clarivate and OCLC Settle Lawsuit”, Clarivate press release issued November 7, 2022

It isn’t clear from this statement whether MetaDoor is done or not. (We’ll answer the “What is/was MetaDoor?” question below.) The statement, which matches the language in the OCLC statement, only says that a service that includes OCLC records will not be built. (We’ll also try to answer the “What is an OCLC record?” question below.)

OCLC’s Statement

OCLC’s statement is only a little less vague:

OCLC is pleased to announce today that it successfully defended WorldCat to protect the collaborative service developed and maintained with and for libraries worldwide.

An agreement has been reached in a lawsuit filed by OCLC in June 2022 against Clarivate and its subsidiaries in the United States District Court, Southern District of Ohio.

Though the settlement document itself is confidential, two significant elements include:

  • Clarivate, Ex Libris, and ProQuest have ceased the development and marketing of the MetaDoor MARC record exchange system developed using records that are subject to the WorldCat Rights and Responsibilities Policy.
  • Clarivate, Ex Libris, and ProQuest will promptly and permanently delete all MetaDoor work product that incorporated or was based on records subject to the Policy.

Pursuant to the confidential agreement and elements noted above, OCLC has filed a dismissal of the lawsuit.

Member libraries, publishers, data experts, and OCLC have worked collaboratively for decades to create WorldCat. Protecting this investment and infrastructure ensures innovation for all libraries and sustainability in the future.

- “OCLC and Clarivate settle lawsuit”, OCLC press release issued November 7, 2022

We’re left with the same questions (“What is/was MetaDoor?” and “What is an OCLC record?”).

To dig any further, let’s look at some of the court documents. Many of the documents are available on the CourtListener.com mirror of the federal court PACER system: OCLC Online Computer Library Center, Inc. v. Clarivate, Plc, 2:22-cv-02470. The most illuminating bits are in the transcripts of conference calls between the court and the lawyers for each side. (The transcripts are PDF documents, so the Hypothesis service is used to link to specific sections.)

What was the “MetaDoor MARC record exchange system”?

Clarivate was pretty cagey on what MetaDoor actually was. They pretty clearly say that they are not creating a database. From page 22 of the transcript of the status conference on June 24, 2022:

THE COURT: All right. In fact, what you are doing, I think, is you are developing a database of information to provide to libraries in competition with OCLC.

MS. RODMAN [counsel for Clarivate]: No, Your Honor, absolutely not.

THE COURT: And you are not doing this for educational purposes. You are doing this for business purposes. You are going to generate some kind of financial benefit by offering this information to others, are you not?

MS. RODMAN: OC – I’m sorry – MetaDoor is not, absolutely not, a database. We are not developing a database for libraries. What MetaDoor is is a software solution that lets one library share with another library. No information ever goes into MetaDoor, ever goes to the defendants as a result of MetaDoor. It simply facilitates that library-to-library transfer which is already allowed to happen, and it gives libraries a way to do it that is not a one-by-one clunky way of doing it like they currently do.

“MetaDoor is not, absolutely not, a database”—quite the definitive statement of what it isn’t, but not very clear about what it is. Marshall Breeding’s “MetaDoor: A new bibliographic service for libraries to be offered by Ex Libris | Library Technology Guides” from June 2022 has a little more information:

This new platform will differ from existing bibliographic utilities. Instead of building a massive repository of bibliographic records, the service is based on indexing and artificial intelligence technologies to identify records residing in the integrated library systems or library services platforms of participating institutions that can be shared with other libraries for copy cataloging or record enhancement.

So MetaDoor is not a big repository of metadata records; what might it be?

My MetaDoor speculation

If MetaDoor wasn’t a database of records, then what was it? At first, I thought it was some sort of metasearch engine where searches are broadcast to an array of library catalogs. But that isn’t any better than the technology we have today; these kinds of broadcast searches are known to be slower than users expect (in the age of instant answers from the web search engines) and notoriously challenging to do good relevance ranking on the fly.

Instead of a database of full records, I think MetaDoor was a database of very brief records and identifiers that facilitated fast full record retrieval from library catalogs. What follows is complete conjecture…I haven’t seen MetaDoor prototypes or talked about its architecture with anyone from Clarivate or development partner libraries. Instead, this is what I would do if I faced the kinds of design constraints presented in the court filings and was asked to build something. Let’s call it KICReD (“Known Item Cataloging REcord Discovery”), and it would have these parts:

  • From participating libraries, a metadata extract of a subset of fields—title, main entry (e.g., author), publisher, ISBN, material type, and dimensions—along with local catalog identifiers.
  • A database of facts built from the metadata extracts.
  • Z39.50 connection details from the participating libraries.
  • Browser-based application with a Z39.50 client.

Cataloging a known item with KICReD would use this workflow:

  1. The cataloger, starting with the item in hand, would search the database of facts to find records that most closely match the item.
  2. The cataloger then selects one or more records from the database of facts.
  3. The browser-based application gets Z39.50 connection details for the local catalogs of the selected records.
  4. The Z39.50 client built into the browser-based application uses the local catalog identifier to retrieve the exact record from the participating library.
  5. The browser-based application displays the full MARC record(s) for the user to select for import into the cataloger’s library.

A KICReD-like system has several advantages. First, it would be much faster than broadcast Z39.50 searching. The database of facts has enough information for the cataloger to select records that most likely match the item in hand. Searching that database would be fast…the metadata extracts would already have been gathered, clustered, and indexed. The next step, too, would be relatively fast…known item searching with the local catalog identifier with Z39.50 is a great way to pull up a MARC record.

Second, it would seem to address concerns about WorldCat record reuse. Database of Facts is a meaningful label in KICReD as facts are not subject to copyright. Compiling and holding a database of facts would seem to be a defensible position. And because the Z39.50 search is happening in the user’s browser in real-time, the full records are never compiled and so there is never an attempt to build anything that replicates WorldCat.

Third, there are some really neat things that could happen when looking at an array of records about the same item. It might be possible, for instance, to construct a composite display of all of the fields from all of the selected records…the browser-based application could build a visualization of fields that are most common alongside fields that are unique, and then select the fields that become the record downloaded into the local catalog.

Fourth, one could do some neat automation with this kind of system. For instance, instead of typing the title and author into a search box, the cataloger could just scan the ISBN barcode from the book jacket and pull up the matching record(s) from the database of facts.

If my “KICReD” guess is close to what MetaDoor was, why did Clarivate settle? The settlement agreement is sealed. so we’ll likely never know. Still, I wonder if they went too far and included the OCLC Record Number in their version of the “database of facts”. Adding that field does make clustering of identical records easier, but (as the OCLC lawyer argued) is demonstrable evidence of the record having come from WorldCat. Or perhaps there were fields in the metadata extracts touched by OCLC’s automated processing that got close to the line between “fact” and “creative effort”. A main entry, for instance, could be enhanced with birth and death dates, professional and honorific titles, meeting locations and dates, etc. One way around that would be to have the participating library send just the subfield “a”, which should be enough in the database of facts for the cataloger to select candidate records for full record retrieval from participating libraries.

Side note: if you’d like to have something like KICReD—or just have a better name for it—let me know; it is the type of system that my employer specializes in building.

What are “records which OCLC has claimed are subject to its policy and contractual limitations”?

What is an OCLC Record? Back to the court documents at page 18 of the June 24th transcript. The court is asking how (if it had to stop Clarivate from using OCLC records) could it distinguish between OCLC records and non-OCLC records:

THE COURT: And I’m having trouble with a lack of specificity as to which records your subscribers are free to provide – because they created them or someone other than OCLC created them and – and how the Court is going to be able to determine in a – in a group of data, even with an OCN number attached to it, whether it is something that the – your subscriber is – has freedom to release or does not under your subscriber agreement, and I – go ahead. Answer that.

MS. MARTINEZ [counsel for OCLC]: Yeah. I was going to say maybe this could help. So the libraries will have their initial records prior to the time that they made it to OCLC and – and before it got into its enhancement process. And so if they want to reach out to a subscriber to see about – and that’s why we – we redrafted the preliminary injunction order in the manner that we did. This is only related to our member subscribers that currently have their records interacting with the – the WorldCat database. If they want to reach out to member subscribers and talk to them about their original records before they went into the WorldCat subscriber consortium/network, we’re fine with that. If they want to talk to any non – there are thousands non-WorldCat member libraries or institutions. They are happy to do that. If they want to talk to any of their own New – SkyRiver subscribers, they are happy to do that. It is just the subset of customers whose records are currently in WorldCat, that those are the ones that we’re talking about, and I think the order very specifically relates just to those.

THE COURT: I think you are saying that all of your subscribers, that their current catalogs would – that all of the information in their current catalogs would be protected from disclosure under your subscriber agreement simply because they – because they joined up and because they have become a subscriber, that they are not permitted to share information in their catalogs with – with others.

MS. MARTINEZ: They are WorldCat records. I think that’s the difference. It’s the WorldCat records that they are not able to share outside of the WorldCat network. They can share it intra-institutional, meaning intra-institutional amongst the other WorldCat subscribers. If they have their records – their own catalog records that are non-WorldCat records, the records that they had prior to the time that they put those records into WorldCat, they are free to share those, and that’s not – that’s not prohibited by the agreement, and it’s not – it’s not even talked about in the order. They are free to share non-WorldCat records. That’s fine.

THE COURT: Yeah, but everything they have since they have become a WorldCat subscriber is going to be WorldCat records under that definition.

MS. MARTINEZ: No, they would have –

THE COURT: Because you’ve touched everything they have.

MS. MARTINEZ: No, no, no, no, no, no, no. So like even OCLC – let’s say Ohio State. Let’s use that as an example. Ohio State comes to – to OCLC and becomes a WorldCat subscriber. OCLC could even go back to that point in time before – you know, with the very limited data that those records had in it prior to the time that it got to OCLC and it enhanced those records, and you could see – it would look very different. It would be very stripped out. The library similarly could do the same. So if they have their development partners that they are working with that are OCLC WorldCat subscribers, they just can’t get their current catalogs as they exist today on the WorldCat system, but they certainly could ask them for the records prior to the time that it went into the WorldCat consortium and OCN number and enhancements occurred and get that from that snapshot in time as well.

OCLC is saying that a library could share with Clarivate the versions of records the library creates before the record is shared with OCLC. That record wouldn’t be an OCLC-enhanced record… a record untouched by OCLC processes. Once the library downloads and overlays a record from OCLC into its local catalog, it becomes an OCLC/WorldCat record.

The mention of the “OCN number” is interesting. Could the existence of an OCN in a record be a marker for an “OCLC Record”? (Transcript page 13)

THE COURT: … Do you assign OCN numbers to all the data that you have, including data created by your subscribers?

MS. MARTINEZ [counsel for OCLC]: We put an OCN number on any record that OCLC enhances. So if it comes into their database and they are going to enhance it by – kind of similarly to what I talked about on Tuesday to the Court, you know, if they are going to add headnotes or footnotes – I’m sorry – head notes, you know, pagination, they are going to change the way that the record is searched.

Perhaps, but counsel from Clarivate takes issue with OCLC “enhancements” (transcript page 15):

MS. RODMAN: … the term “enhancement” I think is a bit of a misnomer. You have to understand where this data comes from. When OCLC creates a record, it is pulling metadata from other sources, from public sources, from libraries themselves, from the Library of Congress, from publishers. Almost all of the metadata in an OCLC record comes from sources other than OCLC. OCLC pulls that in, and they add an OCN number, which is just a sequential number. [Westlaw example removed.] OCLC assigns that OCN number for purposes of deduplication, but also, Your Honor, so that it can pull in enhancements that its libraries make after it creates the record. So OCLC pulls in metadata. It assigns an OCN number. Then when its members add their own enhancements, further data, the OCN number lets OCLC grab that additional information. So it’s a constant cycle of other people, other entities, adding information about a record and OCLC pulling it in, and it uses the OCN number to do that. OCLC has no proprietary interest in this metadata.

The argument from Clarivate’s counsel is insightful but incomplete. Yes, it is true that the metadata OCLC uses to enhance records can come from sources outside of OCLC: publishers, the Library of Congress, and other libraries. But we’ve also seen the effort that OCLC puts into records to cluster similar records into works and to eliminate redundant records. I don’t think that effort is subject to copyright as it is seen as mechanical and not creative, but I could be wrong.

So that is the clarity that I and others I’ve talked to were hoping would come out of this court case…what makes a record an OCLC record, and what proprietary interest does OCLC have in an OCLC record?

Previous chapters of the OCLC record reuse saga

This isn’t the first time that questions about the ownership and reuse of WorldCat records have been raised. In late October 2008, OCLC announced that it was making changes to its Guidelines for the Use and Transfer of OCLC-Derived Records, which had been in place since 1987. This touched off a firestorm of controversy that was even visible from outside the library profession (in the form of an article in The Guardian newspaper, mentioned in a 2009 article on this blog).

In response to the profession’s outcry over the new policy, OCLC formed a Review Board of Shared Data Creation and Stewardship. That review board recommended the withdrawal of the new policy and the formation of an OCLC Record Use Policy Council. From that, we get the WorldCat Rights and Responsibilities for the OCLC Cooperative that is in place today.

If you were paying attention earlier, you might have noticed a reference to SkyRiver. That is a “full service bibliographic utility for cataloging” that was launched in 2009 by the people behind Innovative Interfaces. I think SkyRiver is a part of Clarivate now (as part of Ex Libris’ purchase of Innovate Interfaces, which was purchased by ProQuest, which was in turn purchased by Clarivate—yes, consolidation in the library automation marketplace is really that bad). SkyRiver is nominally a competitor to WorldCat for library cataloging records, but it doesn’t have nearly the depth of content and customers.

One last bit. Over the summer, the International Coalition of Library Consortia (ICOLC) published a statement on Statement on the Metadata Rights of Libraries. (I scribbled some thoughts in the margins.) The top of the statement says:

Metadata and the metadata services that describe library collections are critical in supporting content discovery, knowledge creation, and libraries’ public missions. Metadata describing library collections is not typically copyrightable, and should be considered freely shareable and reusable under most circumstances. However, some industry players restrict libraries’ rights to use such metadata through contractual terms and market influence. Such restrictive activity is out of alignment with libraries’ needs and public, not-for-profit/educational missions.

These questions have been with us for quite a while; it appears they will be with us for quite a bit longer.

Fellow Reflection: Kelley Klor / Digital Library Federation

Kelley KlorThis post was written by Kelley Klor (@MsKelleyK), who attended the 2022 DLF Forum as a Student Fellow.

Kelley M. Klor is currently pursuing a Master’s degree in library and information science (MLIS) at the University of Missouri School of Information Science & Learning Technologies. She graduated from the Missouri University of Science and Technology with a Bachelor of Arts in Biological Science in 2007. As a currently active-duty military spouse, Kelley is a food insecurity consultant with a military service nonprofit organization.

Kelley has an interest in the intersection of quantitative research, creative solutions, education, and informing policy and believes that skills developed in reference and digital library technologies can be used to create a more economically stable environment where military-connected families can thrive.
Kelley and her family live in Northern Virginia, and when she isn’t advocating, she enjoys volunteering, hiking, reading, and learning new skills with her family.


The Heart of Archives: Providing Healing Medicine in the Form of Cultural Preservation

Before library school, archival work was not top of mind. I assumed that archives meant organizing collections of papers into boxes. It took involvement in my university’s student SAA chapter to realize there’s more to archives, and decided on the archives emphasis in my program. After attending the DLF Forum, I realized that archival work is spiritual, healing, and works to capture the experiences that make our human lives meaningful. 

We live in a world of conflict. The destruction of cultural sites, monuments, and libraries is purposeful. Survivors will struggle with loss of identity. In short, when culture is destroyed, identity is destroyed. Once it’s gone, it’s impossible to get it back.

The trauma of severe violence and loss is in my genetic memory. I know the history of my ancestors and recognize the devastation of enduring the Trail of Tears, then establishing a new home. I imagine what it would be like to lose basic rituals – family dinners, movie nights with my kids, our daily walks, our nightly bedtime routine, our holiday and birthday celebrations, our trips to the library, discovering the sites around us as we move to new communities every few years. These rituals may not be super noteworthy to anyone else, but they give me identity as a person, a mother, a wife, a Cherokee woman, a military spouse, and as a library student.

Cultural rituals, practices, and basic routines are wiped away during conflict. Memory of recipes, dances, songs, religious practices are, unfortunately, fragile. The presentation, Community-led Digitization of Archives and Cultural Heritage: The Way Forward in Iraq by Khora Nanno, Helena Rose, and Peter Herdrich, captivated me because of their discussion of the preservation of oral history and intangible cultural heritage. The emphasis on building trust and making decisions jointly with the community highlights the humanity that goes into archival work.

Another presentation, Building/Digitizing/Preserving Community Archives, by Elliott Shore, Jennifer Garcon, and Fenella France, emphasized the importance of working with individuals in community archival work. Even a consideration like choosing an intentional location is a way to build trust and relationships. 

The work of archivists to preserve intangible heritage has a part in revitalizing communities, and gives people an identity and a link to their past. When archivists improve methods allowing the capture and preservation of heritage, a spiritual and psychological medicine is created. Preserving memories can enhance the human experience for people we will likely never meet.

Doing this work is a privilege, and I feel more motivation to dive in deeper. I imagine all of the opportunities that exist to preserve the heritage of Native American communities in a way that is respectful and empowering to the people involved – with their partnership and consent. I imagine the revitalization of culture and healing that could take place with scalable and reproducible workflows. 

I still have another year of library school and so much more to learn. For me, it helps to keep the human experience element in mind as my purpose for doing this work. I’m specifically interested in preserving Indigenous traditional knowledge because so much has been lost already. Thanks to CLIR for giving me the opportunity to attend the DLF Forum as a Forum Fellow, I had the great privilege of meeting so many professionals and fellow students who share this passion and motivation.


Thank you to Texas Digital Library, sponsor of the 2022 DLF Forum Student Fellowships.

Texas Digital Library

The post Fellow Reflection: Kelley Klor appeared first on DLF.

Receipts / Ed Summers

The Webrecorder team recently announced a new feature that lets you embed web archives into your own pages with a bit of JavaScript and a WACZ file. The embed includes a “receipts” pane that includes provenance information for the archive: when the resource was created, what the archive was created with, a public key for the creator, whether the WACZ is valid, the size, and a link to easily download the archive.

For example, here’s a NYTimes article that I archived using ArchiveWebPage into my “Random” collection which I’ve also got stored on Amazon S3 at https://edsu-webarchives.s3.amazonaws.com/random.wacz.


The display above was created with this bit of JavaScript and HTML that I put into the Markdown for this post (I’m still using the Jekyll static site generator).

<script src="https://cdn.jsdelivr.net/npm/replaywebpage@1.7.2/ui.js"></script>
<replay-web-page 
  style="height: 600px"
  embed="replay-with-info"
  source="https://edsu-webarchives.s3.amazonaws.com/random.wacz"
  url="https://www.nytimes.com/live/2022/11/11/world/ukraine-war-news-russia-updates"
  replayBase="/js/replaywebpage/" />

The key things to note here are the loading of the ReplayWebPage JavaScript library, and then using the newly available replay-web-page component to add the archive to the page.

This WACZ file is called “random” because it actually contains some other things that I’ve randomly collected using ArchiveWebPage. So it’s possible to embed something else by using a different url from the archive:

<replay-web-page 
  style="height: 600px"
  embed="replay-with-info"
  source="https://edsu-webarchives.s3.amazonaws.com/random.wacz"
  url="https://www.facebook.com/georgeclintonpfunk"
  replayBase="/js/replaywebpage/" />


Down at the bottom of the receipts pane is a link to download my WACZ file. Which you could replay by going over to ArchiveWebPage and importing it.

Home invasion - Mastodon's Eternal September begins / Hugh Rundle

Nobody goes there anymore. It’s too crowded.
Yogi Berra, et al

Well, it looks like it's finally happened. As news sites began reporting that Elon Musk had finalised his purchase of Twitter, the fediverse's Eternal September — hoped for and feared in equal numbers amongst its existing user base — began.

We've had waves of new people before — most recently earlier this year when Musk first announced his purchase offer — but what's been happening for the last week is different in both scale and nature. It's clear that a sizeable portion of Twitter's users are choosing to deplatform themselves en masse, and many have been directed to Mastodon, the most famous and populated software on the fediverse.

Two kinds of party

In Hobart in the late 1990s, there were basically three nightclubs. They were all sleazy to different degrees, loud to varying levels, and people went to them because that's where other people were — to have fun with friends, to attract attention, to assert their social status, and so on. This is Twitter.

I had a friend who lived in a sharehouse around the corner from one of these popular clubs. He hosted house parties on weekends. Small, just friends and a few friends of friends. This is the fediverse.

Murmuration

For those of us who have been using Mastodon for a while (I started my own Mastodon server 4 years ago), this week has been overwhelming. I've been thinking of metaphors to try to understand why I've found it so upsetting. This is supposed to be what we wanted, right? Yet it feels like something else. Like when you're sitting in a quiet carriage softly chatting with a couple of friends and then an entire platform of football fans get on at Jolimont Station after their team lost. They don't usually catch trains and don't know the protocol. They assume everyone on the train was at the game or at least follows football. They crowd the doors and complain about the seat configuration.

It's not entirely the Twitter people's fault. They've been taught to behave in certain ways. To chase likes and retweets/boosts. To promote themselves. To perform. All of that sort of thing is anathema to most of the people who were on Mastodon a week ago. It was part of the reason many moved to Mastodon in the first place. This means there's been a jarring culture clash all week as a huge murmuration of tweeters descended onto Mastodon in ever increasing waves each day. To the Twitter people it feels like a confusing new world, whilst they mourn their old life on Twitter. They call themselves "refugees", but to the Mastodon locals it feels like a busload of Kontiki tourists just arrived, blundering around yelling at each other and complaining that they don't know how to order room service. We also mourn the world we're losing.

Viral

On Saturday evening I published a post explaining a couple of things about Mastodon's history of dealing with toxic nodes on the network. Then everything went bananas. By 10pm I had locked my account to require followers to be approved, and muted the entire thread I had myself created. Before November 2022 Mastodon users used to joke that you'd "gone viral" if you got more than 5 boost or likes on a post. In an average week, perhaps one or two people might follow my account. Often nobody did. My post was now getting hundreds of interactions. Thousands. I've had over 250 follow requests since then — so many I can't bear to look at them, and I have no criteria by which to judge who to accept or reject. Early this week, I realised that some people had cross-posted my Mastodon post into Twitter. Someone else had posted a screenshot of it on Twitter.

Nobody thought to ask if I wanted that.

To users of corporate apps like Twitter or Instagram this may sound like boasting. Isn't "going viral" and getting big follower counts what it's all about? But to me it was something else. I struggled to understand what I was feeling, or the word to describe it. I finally realised on Monday that the word I was looking for was "traumatic". In October I would have interacted regularly with perhaps a dozen people a week on Mastodon, across about 4 or 5 different servers. Suddenly having hundreds of people asking (or not) to join those conversations without having acclimatised themselves to the social norms felt like a violation, an assault. I know I'm not the only one who felt like this.

It probably didn't help that every Mastodon server administrator I know, including myself, was suddenly dealing with an deluge of new registrants, requests to join (if they didn't have open registration), and then the inevitable server overloads. Aus.social buckled under the strain, going offline for several hours as the admin desperately tried to reconfigure things and upgrade hardware. Chinwag closed registrations temporarily. Even the "flagship instance" mastodon.social was publishing posts hours after they'd been written, with messages being created faster than they could be sent. I was nervously watching the file storage creep up on the ausglam.space wondering if I'd make it to the end of the weekend before the hard drive filled up, and starting to draft new Rules and Terms of Use for the server to make explicit things that previously "everybody knew" implicitly because we previously could acculturate people one by one.

Consent

I hadn't fully understood — really appreciated — how much corporate publishing systems steer people's behaviour until this week. Twitter encourages a very extractive attitude from everyone it touches. The people re-publishing my Mastodon posts on Twitter didn't think to ask whether I was ok with them doing that. The librarians wondering loudly about how this "new" social media environment could be systematically archived didn't ask anyone whether they want their fediverse posts to be captured and stored by government institutions. The academics excitedly considering how to replicate their Twitter research projects on a new corpus of "Mastodon" posts didn't seem to wonder whether we wanted to be studied by them. The people creating, publishing, and requesting public lists of Mastodon usernames for certain categories of person (journalists, academics in a particular field, climate activists...) didn't appear to have checked whether any of those people felt safe to be on a public list. They didn't appear to have considered that there are names for the sort of person who makes lists of people so others can monitor their communications. They're not nice names.

The tools, protocols and culture of the fediverse were built by trans and queer feminists. Those people had already started to feel sidelined from their own project when people like me started turning up a few year ago. This isn't the first time fediverse users have had to deal with a significant state change and feeling of loss. Nevertheless, the basic principles have mostly held up to now: the culture and technical systems were deliberately designed on principles of consent, agency, and community safety. Whilst there are definitely improvements that could be made to Mastodon in terms of moderation tools and more fine-grained control over posting, in general these are significantly superior to the Twitter experience. It's hardly surprising that the sorts of people who have been targets for harrassment by fascist trolls for most of their lives built in protections against unwanted attention when they created a new social media toolchain. It is the very tools and settings that provide so much more agency to users that pundits claim make Mastodon "too complicated".

If the people who built the fediverse generally sought to protect users, corporate platforms like Twitter seek to control their users. Twitter claims jurisdiction over all "content" on the platform. The loudest complaints about this come from people who want to publish horrible things and are sad when the Twitter bureaucracy eventually, sometimes, tells them they aren't allowed to. The real problem with this arrangement, however, is what it does to how people think about consent and control over our own voices. Academics and advertisers who want to study the utterances, social graphs, and demographics of Twitter users merely need to ask Twitter Corporation for permission. They can claim that legally Twitter has the right to do whatever it wants with this data, and ethically users gave permission for this data to be used in any way when they ticked "I agree" to the Terms of Service. This is complete bullshit of course (The ToS are inpenetrable, change on a whim, and the power imbalance is enormous), but it's convenient. So researchers convince themselves they believe it, or they simply don't care.

This attitude has moved with the new influx. Loudly proclaiming that content warnings are censorship, that functionality that has been deliberately unimplemented due to community safety concerns are "missing" or "broken", and that volunteer-run servers maintaining control over who they allow and under what conditions are "exclusionary". No consideration is given to why the norms and affordances of Mastodon and the broader fediverse exist, and whether the actor they are designed to protect against might be you. The Twitter people believe in the same fantasy of a "public square" as the person they are allegedly fleeing. Like fourteenth century Europeans, they bring the contagion with them as they flee.

Anarchism

The irony of it all is that my "viral toot thread" was largely about the fediverse's anarchist consent-based nature. Many of the newcomers saw very quickly that their server admins were struggling heroically to keep things running, and donated money or signed up to a Patreon account to ensure the servers could keep running or be upgraded to deal with the load. Admins were sending private and public messages of support to each other, sharing advice and feelings of solidarity. Old hands shared #FediTips to help guide behaviour in a positive direction. This is, of course, mutual aid.

It's very exciting to see so many people experiencing anarchic online social tools. The clever people who build ActivityPub and other fediverse protocols and tools have designed it in ways that seek to elude monopolistic capture. The software is universally Free and Open Source, but the protocols and standards are also both open and extensible. Whilst many will be happy to try replicating what they know from Twitter — a kind of combination of LinkedIn and Instagram, with the 4chan and #auspol people always lurking menacingly — others will explore new ways to communicate and collaborate. We are, after all, social creatures. I am surprised to find I have become a regular contributor (as in, code contributor 😲) to Bookwyrm, a social reading tool (think GoodReads) built on the ActivityPub protocol used by Mastodon. This is just one of many applications and ideas in the broader fediverse. More will come, that will no longer simply be "X for Fedi" but rather brand new ideas. Whilst there are already some commercial services running ActivityPub-based systems, a great many of the new applications are likely to be built and operated on the same mutual aid, volunteerist basis that currently characterises the vast majority of the fediverse.

Grief

Many people were excited about what happened this week. Newcomers saw the possibilities of federated social software. Old hands saw the possibilities of critical mass. But it's important that this isn't the only story told about early November 2022. Mastodon and the rest of the fediverse may be very new to those who arrived this week, but some people have been working on and playing in the fediverse for over a decade. There were already communities on the fediverse, and they've suddenly changed forever.

I was a reasonably early user of Twitter, just as I was a reasonably early user of Mastodon. I've met some of my firmest friends through Twitter, and it helped to shape my career opportunities. So I understand and empathise with those who have been mourning the experience they've had on Twitter — a life they know is now over. But Twitter has slowly been rotting for years — I went through that grieving process myself a couple of years ago and frankly don't really understand what's so different now compared to two weeks ago.

There's another, smaller group of people mourning a social media experience that was destroyed this week — the people who were active on Mastodon and the broader fediverse prior to November 2022. The nightclub has a new brash owner, and the dancefloor has emptied. People are pouring in to the quiet houseparty around the corner, cocktails still in hand, demanding that the music be turned up, walking mud into the carpet, and yelling over the top of the quiet conversation.

All of us lost something this week. It's ok to mourn it.

Matt Levine's "The Crypto Story": Part 1 / David Rosenthal

Even if you're not a Bloomberg subscriber you can read Matt Levine's The Crypto Story with a free registration, or here, and I urge you to do so. It is long, about 40K words, but well worth the effort. It is remarkably good - lucid, comprehensive, balanced, accurate. It even has footnotes expanding on the details where he is oversimplifying for clarity of exposition.

Levine's magnum opus is in four parts:
  1. Ledgers, Bitcoin, Blockchain, in which he lays out the basics of Bitcoin's technology.
  2. What Does It Mean?, in which he discusses generalizations of Bitcoin, such as Ethereum.
  3. The Crypto Financial System, in which he discusses financial technologies such as exchanges, stablecoins and DeFi.
  4. Trust, Money, Community, in which he discusses the social and societal impacts.
This post is not a substitute for reading the real thing. Below the fold, some commentary on fragments of the first two parts. The second two parts will have to wait for a subsequent post, because even commenting on such a massive post gets way long. Block quotes without links are from Levine's article.

Levine explains why he's writing at such length:
I don’t have strong feelings either way about the value of crypto. I like finance. I think it’s interesting. And if you like finance—if you like understanding the structures that people build to organize economic reality—crypto is amazing. It’s a laboratory for financial intuitions. In the past 14 years, crypto has built a whole financial system from scratch. Crypto constantly reinvented or rediscovered things that finance had been doing for centuries. Sometimes it found new and better ways to do things.

Often it found worse ways, heading down dead ends that traditional finance tried decades ago, with hilarious results.

Often it hit on more or less the same solutions that traditional finance figured out, but with new names and new explanations. You can look at some crypto thing and figure out which traditional finance thing it replicates. If you do that, you can learn something about the crypto financial system—you can, for instance, make an informed guess about how the crypto thing might go wrong—but you can also learn something about the traditional financial system: The crypto replication gives you a new insight into the financial original.
...
My goal here is not to convince you that crypto is building the future and that if you don’t get on board you’ll stay poor. My goal is to convince you that crypto is interesting, that it has found some new things to say about some old problems, and that even when those things are wrong, they’re wrong in illuminating ways.

Ledgers, Bitcoin, Blockchain

Levine starts from the issue of trust in conventional digital financial systems:
A system of impersonal banking in which the tellers are strangers and you probably use an ATM anyway requires trust in the system, trust that the banks are constrained by government regulation or reputation or market forces and so will behave properly.

Saying that modern life is lived in databases means, most of all, that modern life involves a lot of trust.

Sometimes this is because we know them and consider them to be trustworthy. More often it means we have an abstract sense of trust in the broader system, the system of laws and databases and trust itself. We assume that we can trust the systems we use, because doing so makes life much easier than not trusting them and because that assumption mostly works out. It’s a towering and underappreciated achievement of modernity that we mostly do trust the database-keepers, and that they mostly are trustworthy.
The conventional digital financial systems have roots back in the days of punched cards and magnetic tapes, and have evolved incrementally since, so they are an inefficient mish-mash of awkwardly cooperating systems:
What if we rewrote all the databases from scratch, in modern computer languages using modern software engineering principles, with the goal of making them interact with one another seamlessly?

If you did that, it would be almost like having one database, the database of life: I could send you money in exchange for your house, or you could send me social reputation in exchange for my participation in an online class, or whatever, all in the same computer system.

That would be convenient and powerful, but it would also be scary. It would put even more pressure on trust. Whoever runs that one database would, in a sense, run the world. Whom could you trust to do that?
So there is a strong motivation to find a system that (a) works better but (b) requires less trust. The problem is that trust, even Regan-style "trust but verify", is such an enormous optimization over a truly trust-free system, that it is effectively impossible to eliminate it in practice, despite Satoshi's claims:
What Satoshi said he’d invented was a sort of cash for internet transactions, “an electronic payment system based on cryptographic proof instead of trust, allowing any two willing parties to transact directly with each other without the need for a trusted third party.” If I want to buy something from you for digital cash—Bitcoin—I just send you the Bitcoin and you send me the thing; no “trusted third party” such as a bank is involved.

When I put it like that, it sounds as if Satoshi invented a system in which I can send you Bitcoin and nobody else is involved. What he actually invented was a system in which lots of other people are involved.
Satoshi was not just wrong about "no trusted third party", because in practice Bitcoin transactions trust a lot of entities, including the core developers, and the few large mining pools. He was also wrong about the ability of Bitcoin to facilitate transactions involving off-chain "things" such as the notorious initial pizza. Pseudonymity, immutability, and the hour-long delay before finality make such transactions extremely risky. The normal mitigation for this kind of risk is an escrow service; a trusted third party. Is it possible to replace the escrow service with a "smart contract"? Alas, no. In Irrationality, Extortion, or Trusted Third-parties: Why it is Impossible to Buy and Sell Physical Goods Securely on the Blockchain Amir Kafshdar Goharshady shows that:
assuming that the two sides are rational actors and the smart contract language is Turing-complete, there is no escrow smart contract that can facilitate this exchange without either relying on third parties or enabling at least one side to extort the other.
One of Levine's achievements is that he next provides an accurate, readable explanation of Bitcoin's technology, starting from one-way functions through hashing, public-key encryption and digital signatures to the way Bitcoin implements a ledger:
The ledger is not really just a list of addresses and their balances; it’s actually a record of every single transaction. The ledger is maintained by everyone on the network keeping track of every transaction for themselves.

That’s nice! But now, instead of trusting a bank to keep the ledger of your money, you’re trusting thousands of anonymous strangers.
...
Everyone keeps the ledger, but you can prove that every transaction in the ledger is valid, so you don’t have to trust them too much.

Incidentally, I am saying that “everyone” keeps the ledger, and that was probably roughly true early in Bitcoin’s life, but no longer. There are thousands of people running “full nodes,” which download and maintain and verify the entire Bitcoin ledger themselves, using open-source official Bitcoin software. But there are millions more not doing that, just having some Bitcoin and trusting that everybody else will maintain the system correctly. Their basis for this trust, though, is slightly different from the basis for your trust in your bank. They could, in principle, verify that everyone verifying the transactions is verifying them correctly.
In the spirit of his "The crypto replication gives you a new insight into the financial original." he makes this point:
Notice, too, that there’s a financial incentive for everyone to be honest: If everyone is honest, then this is a working payment system that might be valuable. If lots of people are dishonest and put fake transactions in their ledgers, then no one will trust Bitcoin and it will be worthless. What’s the point of stealing Bitcoin if the value of Bitcoin is zero?

This is a standard approach in crypto: Crypto systems try to use economic incentives to make people act honestly, rather than trusting them to act honestly.
Note that the incentive is inherent to the system, not layered on via reputation or legal sanctions. In Cooperation among an anonymous group protected Bitcoin during failures of decentralization Alyssa Blackburn et al analyze how this incentive operated in the early days of Bitcoin, when multiple actors had the means and opportunity, but not the motive, to subvert the system (commentary here). But it is also worth noting that later, in 2014, GHash.io for extended periods held 51% of the mining power, as Dan Goodin reports in Bitcoin security guarantee shattered by anonymous miner with 51% network power:
on June 12, GHash produced a majority of the power for 12 hours straight,
...
GHash's ascendency to a majority miner comes even as its operators pledged never to cross the 51-percent threshold. It also comes less than a year after GHash was accused of using its considerable hashing power to attack a gambling site.
As Ittay Eyal and Emin Gün Sirer wrote at the time:
Overall, there is absolutely no reason to trust GHash or any other miner. People in positions of power are known to abuse it. A group with a history of double-expenditures just blithely went past the 51% psychological barrier: this is not good for Bitcoin.
Levine continues his impressively accessible explanation by correctly explaining the need for a Sybil defense and the way the Proof-of-Work implements it, connecting it to the idea of inherent economic incentives:
Proof-of-work mining is a mechanism for creating consensus among people with an economic stake in a system, without knowing anything else about them. You’d never mine Bitcoin if you didn’t want Bitcoin to be valuable. If you’re a Bitcoin miner, you’re invested in Bitcoin in some way; you’ve bought computers and paid for electricity and made an expensive, exhausting bet on Bitcoin. You have proven that you care, so you get a say in verifying the Bitcoin ledger. And you get paid. You get paid Bitcoin, which gives you even more of a stake in the system.
And he notes that, despite all the rhetoric about Bitcoin's 21M coin limit being deflationary, the Sybil defense mechanism is necessarily inflationary:
One important point about these mining rewards is that they cost Bitcoin users money. Every block—roughly every 10 minutes—6.25 new Bitcoin are produced out of nowhere and paid to miners for providing security to the network. That works out to more than $6 billion per year. This cost is indirect: It is a form of inflation, and as the supply of Bitcoin grows, each coin in theory becomes worth a little less, all else being equal. Right now, the Bitcoin network is paying around 1.5% of its value per year to miners.

That’s lower than the inflation rate of the US dollar. Still, it’s worth noting. Every year, the miners who keep the Bitcoin system secure capture a small but meaningful chunk of the total value of Bitcoin. Bitcoin users get something for that $6 billion:
What they get is security:
When there are billions of dollars up for grabs for miners, people will invest a lot of money in mining, and it will be expensive to compete with them. And if you invested billions of dollars to accumulate a majority of the mining power in Bitcoin, you would probably care a lot about maintaining the value of Bitcoin, and so you’d be unlikely to use your powers for evil.
My only criticism of this chapter is that it leaves the impression that Satoshi Nakamoto was a genius whose invention of cryptocurrency was unprecedented. Although his overall design was very clever, it built on both:
  • The history of attempts to create a digital currency stretching back four decades detailed, for example, in Finn Brunton's Digital Cash.
  • Academic research on component technologies also dating back four decades, as detailed by Arvind Narayanan and Jeremy Clark in Bitcoin's Academic Pedigree (see here.

What Does It Mean?

One of the things that, as a finance journalist, really interests Levine is:
The wild thing about Bitcoin is not that Satoshi invented a particular way for people to send numbers to one another and call them payments. It’s that people accepted the numbers as payments.
...
That social fact, that Bitcoin was accepted by many millions of people as having a lot of value, might be the most impressive thing about Bitcoin, much more than the stuff about hashing.
And that:
Here’s another extremely simple generalization of Bitcoin:
  1. You can make up an arbitrary token that trades electronically.
  2. If you do that, people might pay a nonzero amount of money for it.
  3. Worth a shot, no?
As Bitcoin became more visible and valuable, people just … did … that? There was a rash of cryptocurrencies that were sometimes subtle variations on Bitcoin and sometimes just lazy knockoffs. “Shitcoins” is the mean name for them.
But the shitcoins were vastly less successful at being accepted as having a lot of value::
Socially, cryptocurrency is a coordination game; people want to have the coin that other people want to have, and some sort of abstract technical equivalence doesn’t make one cryptocurrency a good substitute for another. Social acceptance—legitimacy—is what makes a cryptocurrency valuable, and you can’t just copy the code for that.

That’s a revealing fact: What makes Bitcoin valuable isn’t the elegance of its code, but its social acceptance. A thing that worked exactly like Bitcoin but didn’t have Bitcoin’s lineage—didn’t descend from Satoshi’s genesis block and was just made up by some copycat—would have the same technology but none of the value.
Rank Symbol vs. BTC vs. ETH
1 BTC 1.0
2 ETH 0.49 1.0
8 DOGE 0.05
9 ADA 0.04 0.07
10 SOL 0.03 0.06
13 SHIB 0.02
Excluding metastablecoins and non-decentralized coins, and ordering by "market cap" as shown on coinmarketcap.com, we have this table. Note that the largest BTC-clone, DOGE, is "worth" one-twentieth as much, and that only because it is favored by the cult of Elon Musk. SHIB, which is a DOGE ripoff, but on Ethereum, is "worth" about 40% as much. The largest ETH competitor, ADA, is "worth" one-twelfth as much. Despite this, shitcoins have found a viable market niche as the basis for highly profitable pump-and-dump schemes.

Levine turns to the number 2 cryptocurrency:
Ethereum is a big virtual computer, and you send it instructions to do stuff on the computer. Some of those instructions are “Send 10 Ether from Address X to Address Y”: One thing in the computer’s memory is a database of Ethereum addresses and how much Ether is in each of them, and you can tell the computer to update the database.

But you can also write programs to run on the computer to do things automatically. One sort of program might be: Send 10 Ether to Address Y if something happens. Alice and Bob might want to bet on a football game, or on a presidential election, or on the price of Ether.
Source
Ethereum isn't really a "big virtual computer". It is a very small, slow, expensive virtual computer with about 500GB of storage. In the heady days of last December Nicholas Weaver pointed out that:
Put bluntly, the Ethereum “world computer” has roughly 1/5,000 of the compute power of a Raspberry Pi 4!

This might be acceptable if the compute wasn’t also terrifyingly expensive. The current transaction fees for 30M in gas consumed is over 1 Ether. At the current price of roughly $4000 per Ether this means a second of Ethereum’s compute costs $250. So a mere second of Ethereum’s virtual machine costs 25 times more than a month of my far more capable EC2 instance. Or could buy me several Raspberry Pis.
Levine acknowledges but underplays the slow part, and omits the expensive part:
The way Ethereum executes programs is that you broadcast the instructions to thousands of nodes on the network, and they each execute the instructions and reach consensus on the results of the instructions. That all takes time. Your program needs to run thousands of times on thousands of computers.

Computers and network connections are pretty fast these days, and the Ethereum computer is fast enough for many purposes (such as transferring Ether, or keeping a database of computer game characters). But you wouldn’t want to use this sort of computer architecture for extremely time-sensitive, computation-intensive applications.
Levine correctly identifies one of the big problems with "smart contracts", as Ethereum programs are called. If Alice and Bob bet on a football game, how does the program know what the result was?
The standard solution in crypto is called an “oracle.” It’s a program that will periodically query some company or website that tracks the relevant information (election results, football scores, weather, etc.) and post the answer to the Ethereum blockchain. An oracle is essentially a way to bring information from the outside world (the real world, or just the internet) onto the blockchains.
So the end result of the bet depends on what the oracle says. It is just another program, which will have bugs and vulnerabilities. In particular, it will suffer from "garbage in, garbage out", giving rise to so-called oracle manipulation attacks:
A vulnerability arises when protocols relying on oracles automatically execute actions even though the oracle-provided data feed is incorrect. An oracle with deprecated or even malicious contents can have disastrous effects on all processes connected to the data feed. In practice, manipulated data feeds can cause significant damage, from unwarranted liquidations to malicious arbitrage trades.
Molly White reports on an example from October 23rd in Oracle manipulation attack on a QuickSwap market earns exploiter $188,000:
Adding to the recent string of oracle manipulation attacks is an attack on the miMATIC ($MAI) market on the QuickSwap decentralized exchange. An exploiter was able to manipulate the spot price of assets to borrow funds, ultimately making off with 138 ETH ($188,000) that they mixed through Tornado Cash. The vulnerability was due to the use of a Curve LP oracle, which contains a vulnerability that was disclosed by a security firm earlier that month.
And another from November 2nd in Oracle attack on Solend costs the project $1.26 million :
Solend announced that an exploiter had manipulated the oracle price of an asset on their platform, allowing them to take out a loan that left the platform with $1.26 million in bad debt.
And another from November 7th in Pando exploited for $20 million:
The defi protocol Pando suffered a $20 million loss when it was exploited with an oracle manipulation attack. The protocol suspended several of its projects in response to the hack, and wrote that they hoped to negotiate with the hacker to regain some of the stolen proceeds.
These attacks are happening weekly.

Levine's discussion of Ethereum's transition to Proof-of-Stake is clear:
When we discussed proof-of-work mining, I said that crypto systems are designed to operate on consensus among people with an economic stake in the system. PoW systems demonstrate economic stake in a cleverly indirect way: You buy a bunch of computer hardware and pay for a lot of electricity and do a bunch of calculations to prove you really care about Bitcoin. PoS systems demonstrate the economic stake directly: You just invest a lot of money in Ethereum and post it as a bond, which proves you care.

This is more efficient, in two ways. First, it uses less electricity. ... Second, PoS more directly measures your stake in the system. You demonstrate your stake in Ethereum by 1) owning Ether and 2) putting it at risk28 to validate transactions. To take control of the PoS system and abuse it for your own nefarious purposes, you need to own a lot of Ether, and the more you own, the less nefarious you’ll want to be. “Proof of stake can buy something like 20 times more security for the same cost,” Vitalik has argued. ... The capital investment isn’t in computers but in the relevant cryptocurrency. The transaction is very close to: “Invest a lot of cryptocurrency and then get paid interest on that cryptocurrency.”
This is the Golden Rule, "he who has the gold makes the rules". Or Mark 4:25:
For he that hath, to him shall be given: and he that hath not, from him shall be taken even that which he hath.
And the extraordinary Gini coefficients of cryptocurrencies mean that he that hath already hath almost all of the gold. But this is mitigated slightly, as he explains, by staking services:
in fact a lot of Ethereum validation runs through crypto exchanges such as Coinbase, Kraken, and Binance, which offer staking as a product to their customers. (The biggest is a thing called Lido Finance, which isn’t an exchange but a sort of decentralized staking pool.) The customers keep their Ether with the exchange anyway, so they might as well let the exchange stake it for them and earn some interest.
Source
The problem with this is the lack of decentralization. As I write, Lido, Coinbase and Kraken between them have nearly 53% of all the stake. This isn't great, but it is actually better than Proof-of-Work Ethereum, which last November had 2 mining pools with nearly 54% of the mining power.

The reward for staking is one of the "new things to say about some old problems" that Levine finds interesting:
Crypto has found a novel way to create yield. ... You can deposit your crypto into an account, and it will pay you interest. It will pay you interest not for the reason banks generally do—because they’re lending your money to some other customer who will make use of it—but because you are, in your small way, helping to maintain the security of the transaction ledger.
Where does the interest come from? Part is from the block reward, and part is from transaction fees:
generally speaking, the more you offer to pay for gas, the faster your transaction will be executed: If Ethereum is busy, paying more for gas gets you priority for executing your transactions. It is a shared computer where you can pay more to go first.
Source
Well, yes, but there are two problems with "pay more to go first":
  • As I discussed here, there is a fixed supply of computation but a variable demand for it. The result is a variable price for computation. In practice, fees are low when few people want to transact, but spike enormously when everyone does. This chart shows that while now, in the "crypto winter" the average ETH transaction fee is around 77 cents, during the Luna/Terra crash it spiked by a factor of 32. You are in a blind auction for fees, so are motivated to overpay.
  • The fact of "pay-for-priority" is what enables front-running and other forms of Miners' Extractable Value. It is likely that the profitable trade you spotted will be front-run by a bot that spots your pending transaction and offers to pay more.
Here is an example of an MEV bot in action:
One such bot, known as 0xbadc0de, earned a windfall when a trader tried to sell 1.8 million cUSDC (USDC on the Compound protocol) – notionally worth $1.85 million – but only received $500 in assets in return due to low liquidity. The MEV bot, however, profited 800 ETH (~$1 million) from arbitrage trades surrounding the sale.
Levine returns to the financial aspects:
Here’s another important difference between Ethereum and Bitcoin. Bitcoin never raised money; Ethereum did.
...
You could imagine Vitalik saying: “OK, we are a company, we’re Ethereum Inc., we’re going to start the Ethereum blockchain and make money from it, and we’ll sell shares in Ethereum Inc. to raise the money to build the blockchain. Sell 20% of Ethereum Inc. for cash, get the cash, build the blockchain and, I don’t know, collect royalties.
...
They sold tokens. In July 2014 they sold Ether “at the price of 1000-2000 ether per BTC, a mechanism intended to fund the Ethereum organization and pay for development,” according to Ethereum’s white paper. In all, they sold about 60 million Ether for about $18.3 million, all before the technology itself went live. Ethereum’s genesis block was mined in July 2015. Today there is a total of about 122 million Ether outstanding. Some of that, like Bitcoin, comes from mining or, now, validating. But almost half of it was purchased, before Ethereum was launched, by people who wanted to bet on its success.
...
It worked out well for those investors; that 60 million Ether is worth billions of dollars today.

But as a “whole new financing model,” it’s a mixed bag. Many people, particularly securities regulators, think it’s good that startups usually can’t raise money from the general public without at least a business plan. And there’s a sense in which this sale—the Ether “pre-mine,” or “initial coin offering” (ICO)—was the original sin of crypto as a financing tool. A lot of other crypto teams copied Ethereum’s approach, writing up vague plans to build some project and then raising money by preselling tokens that would be useful if the project ever happened.
As he writes, the resulting high level of fraud was inevitable:
When ragtag groups of hackers with no business plan can raise millions of dollars from anyone with an internet connection, they all will. The odds that any particular one of those non-business-plans will succeed are low. The odds that any particular one will be a scam are high.
He then discusses the persistently wrong idea that "putting things on the blockchain" will make the world better:
How to make this connection is largely an unsolved problem, and quite possibly an unsolvable one, but also an important one. Crypto’s financial system is well-developed and has some advantages in openness and permissionless innovation over the traditional financial system. If you could ingest the physical world into that financial system, you’d have something cool.
The unsolvable part of this is the "oracle problem" that is fundamental to "smart contracts".

Levine discusses the idea of Web3 as a reaction to the centralized, closed FAANGs controlling the Internet and making lots of money:
But web3 will be when people build decentralized open community-controlled protocols for the internet again and also make lots of money. Because the decentralized protocols won’t be owned by big tech companies, but they won’t be free and owned by no one, either. They’ll be owned by their users.

Just kidding: They’ll be owned by venture capitalists who buy tokens in these projects early on and who are the biggest boosters of web3.
Levine doesn't explain why the VCs are the "biggest boosters of web3". As I explained in List And Dump Schemes it is because investing in the tokens allows them to outsource the securities fraud to the founders. He makes another good point about the financial implications of owning Bitcoin or other cryptocurrencies:
But another thing you get is a share in the Bitcoin project. Not a share of actual stock, but still a chance to profit from the success of Bitcoin. If this digital cash thing takes off, then lots of people will want Bitcoin to use to buy sandwiches, and there will be a lot of demand for Bitcoin. But only 21 million Bitcoin will ever exist. So each Bitcoin will be more valuable as more people decide to use Bitcoin as their way to transfer digital cash.

That logic never quite made sense. A convenient currency for digital cash transfer has a stable value, and the rising value of Bitcoin makes it less useful as a currency: If your Bitcoin keep going up in value, you should not spend them on sandwiches. Bitcoin as an appreciating asset will be a bad currency. Still, it worked well enough. Bitcoin is used enough for digital transfers of value that it became valuable, and early adopters got rich.

This is a key financial innovation of crypto:
Crypto built an efficient system to make the customers of a business also its shareholders.
Levine explains why this matters:
It’s hard to build a network-effects business from scratch. Early users of a network-effects business won’t get much out of it, because there’s no network yet. You might as well just wait until there are more users. The economics of crypto tokens reverse that. Early users of a crypto network get tokens cheap, and if the network takes off later, then their tokens will be worth a lot. Now there’s an advantage to being early.
But this has a major downside. The "shareholders" have two concerns. First, they are heavily invested in finding the Greater Fool to make their number go up and, second, they are worried that the Greater Fool might buy one of the multitude of other coins instead of theirs. Because even in good times the supply of Greater Fools is limited, and because the supply of alternative coins is effectively unlimited, this leads to the corrupt and toxic discourse so prevalent around cryptocurrencies. Examples include Bitcoin maximalists and the XRP army.

In practice I think Levine is wrong about how this works. It is possible that it would work for a coin used as a medium for trade, but even BTC isn't that. For the token of a "network-effects business" it doesn't work. Lets take the example of Helium Network, lavishly funded by A16Z. The idea was that the equivalent of miners would run WiFi hot-spots and be rewarded with HNT. Users of the hot-spots would buy HNT to pay for their use.

Source
First, as the "price" chart shows, Helium was another of A16Z's List And Dump Schemes (down 92% from the peak). It turns out that almost all of these token-based systems turn out to enrich VCs and insiders at the expense of what Levine calls "the customers". For the details, see Fais Khan's "You Don't Own Web3": A Coinbase Curse and How VCs Sell Crypto to Retail. Especially in the "crypto winter", it has become hard to ignore this track record.

Second, the question is "who are the customers?". The hot-spot owners are the ones who get rewarded in HNT, but they aren't the customers of the service. They are more like employees, the ones doing the work that runs the service. The customers are the ones paying actual money into the scheme, the users of the hot-spots. They only briefly hold HNT and are thus not shareholders. In fact, to the customer the idea that the token's "price" would go up is a bug, not a feature. It represents the service increasing its prices.

There is a lot more in this chapter, including discussions of DAOs, blockchain-based identity, whether "uncensorable" is achievable, NFTs and the metaverse. It is all so good its hard to find parts to comment on, so I won't.

The remaining two chapters are shorter, so the follow-up post should be shorter too (I hope)

Progress on building a National Finding Aid Network / HangingTogether

The following post is part of a series highlighting OCLC Research’s efforts for the Building a National Finding Aid Network (NAFAN) project. You can view the rest of the series here.

We’ve been a bit quiet on the blog recently regarding our work on NAFAN, mainly because we’ve been heads down on doing that work! We’ve been deep in data analysis, and are getting to the exciting part of a research project when all of the efforts of data collection come together to start producing tangible findings. Similarly, project leads at California Digital Library (CDL) and partners at Shift Collective have been hard at work with their own progress to report.

I recently joined Rachael Hu and Adrian Turner from CDL to share some of that progress in a webinar. The recording and slides from that presentation are now available.

NAFAN information session (2022-10-06) from California Digital Library on Vimeo.

In the presentation, we share updates on a number of project fronts:

  • Initial research findings from three of OCLC Research’s data collection efforts: focus group discussions with archivists, an analysis of 145K EAD encoded finding aids, and a pop-up survey of 3,300 users of regional archival aggregation websites.
  • Findings related to how a national-level finding aid network might offer services, support, and advocacy to community-based archives, surfaced in a two-day workshop hosted by Shift Collective in December 2021.
  • The high-level functional and design model envisioned for a national-level finding aid network, based on input from these and other findings in the larger project.

I encourage you to watch the video. You can also read the full report from Shift’s community convening Increasing National Discovery of Archives. And of course, keep an eye out here for further updates from our NAFAN research!

The post Progress on building a National Finding Aid Network appeared first on Hanging Together.

NDSA Leadership Refreshes Foundational Strategy, Now Includes Transparency and Openness / Digital Library Federation

The NDSA held its 10th annual Digital Preservation conference on October 12-13 in Baltimore, Maryland, you can read some highlights from NDSA Leadership on the NDSA News blog. During the opening ceremony, while sharing my remarks as Chair, I mentioned that NDSA Leadership worked over the summer and fall to refresh our foundational strategy. What better day to talk about our refreshed strategy than World Digital Preservation Day!

Our mission statement now reflects that NDSA fosters and incubates communities of practice who advocate, support, and provide expertise in digital preservation through their work, to benefit practitioners, service providers, and memory organizations, as well as the creators, owners, and users of digital content. These changes reflect a more dynamic and engaged NDSA that supports the community in furthering our collective goals.

The vision statement was revised to make the tone more future-oriented with some small edits for clarity. Although the NDSA mission statement makes it clear that NDSA is an international alliance, the vision statement now also states that NDSA provides national leadership in the United States on digital preservation. NDSA Leadership has long seen NDSA serving in this capacity, but explicitly adding it in our vision statement makes clear to everyone that NDSA wants to serve in that role.

The most significant changes were around the NDSA’s values and principles. Transparency was previously embedded within ethical behavior, but NDSA Leadership felt that it was important enough to stand on its own as a value along with openness. As our values state, “transparency is essential for trust and is needed for fruitful digital preservation partnerships.” In addition to this change, Leadership derived multiple operational principles from each value. The principles are meant to inform digital preservation practices but will also serve as guidestones to NDSA Leaders in their work to carry out the mission and achieve the vision of NDSA.

The goals and strategies & activities sections of our foundational strategy have remained unchanged, as has our 2020 strategic plan. It’s likely that these will also be updated in the next couple of years as NDSA Leadership delves into big topics like how NDSA engages with the digital preservation marketplace of service providers, long term plans for how NDSA hosts conferences, how NDSA engages with members to meet their needs, and finally, the ideal organizational alignment to help us achieve our mission and vision. 

Happy World Digital Preservation Day everyone! How are you celebrating, “data for all, for good, forever”?

~Nathan Tallman
Chair, NDSA Coordinating Committee

 

The post NDSA Leadership Refreshes Foundational Strategy, Now Includes Transparency and Openness appeared first on DLF.

Cuaderno de bitácora: Entre el espacio físico y virtual en España / HangingTogether

Gracias a Sara Noguera, OCLC, que amablemente ha proporcionado la versión en español de esta entrada de blog. La versión en inglés está disponible aquí

En septiembre de 2022, Lynn Silipigni Connaway, Sara Noguera, Francesc García Grimau y yo hicimos un recorrido por diferentes bibliotecas de España. En este post comparto algunas impresiones y reflexiones fruto de este viaje.

A la experiencia de viaje se le añadía el “toque” COVID

Llegamos a Alicante desde diferentes direcciones –Zúrich, Ámsterdam, Barcelona, Jerez– y nos encontramos de nuevo en persona después de tres años de pandemia COVID-19. Sentí como algo surrealista revivir de nuevo el trajín del aeropuerto, ponerse en “modo explorador” durante el viaje en taxi al hotel, entender cómo funcionaban los botones del ascensor, acomodarse a la habitación de hotel. Esta vez, a la experiencia de viaje se le añadía el “toque” COVID: uso obligatorio de mascarillas en los transportes, llamar taxis vía app, caminar por el paseo principal de la ciudad –Passeig Esplanada d’Espanya, con su mosaico de mármol al compás de las olas del mar– normalmente, un lugar bullicioso lleno de vendedores ambulantes hasta altas horas de la noche, ahora, desierto. “¡Ésta no es la España que recuerdo!” exclamó Lynn, “¿Qué le ha pasado a la vida nocturna española?”

Espacios vacíos y estériles. La virtualización tomando el control. Uno de los temas de nuestro estudio New Model Library: Pandemic Effects and Library Directions

Fue tan bonito verse de nuevo, cara a cara. La calidez, las risas, las conversaciones de horas y horas durante los desayunos y los paseos después de cenar. Pues claro que estuvimos en contacto estrecho durante el confinamiento –y, probablemente, trabajamos más sin tener que perder el tiempo en desplazamientos y viajes–, pero fue como ponerse al día, finalmente. Observar de nuevo aquella información que la virtualización nos había quitado, aquellas impresiones que solo se pueden sentir mediante el contacto cara a cara: postura, gestos, apariencia, vestimenta. Especialmente, pudimos ponernos al día con nuestros no tan nuevos compañeros, Sara y Francesc, con quienes aún no nos habíamos conocido en persona.

Viajar para visitar una biblioteca digital

Visita a la Biblioteca Virtual Miguel de Cervantes

Visitamos la Biblioteca Virtual Miguel de Cervantes (BVMC)…”Espera un momento”, mi jefa empezó a reírse “¿te vas de viaje para visitar una biblioteca virtual?”. La biblioteca proporciona acceso online al patrimonio cultural digitalizado del mundo hispánico. Es parte del laboratorio de Humanidades Digitales de la Universidad de Alicante. Gustavo Candela, responsable de la BVMC, nos dio la bienvenida junto a sus colegas del departamento técnico y de catalogación. Nos mostraron su Data Lab y los equipos de digitalización que tienen. Hablamos de diversos temas: sobre colecciones como datos y su uso académico. El Ministerio de Ciencia e Innovación justo había anunciado que aprobará un presupuesto que permitirá a las universidades y centros de investigación a participar en DARIAH y CLARIN, dos infraestructuras de investigación europeas, una noticia largamente esperada. Como guinda, la Biblioteca Virtual ha sido designada como colaborador en el “Plan de Recuperación, Transformación y Resiliencia (PERTE) Nueva economía de la lengua”, el cual forma parte del plan de recuperación y resiliencia de España tras la crisis derivada de la pandemia. La idea es usar el patrimonio cultural del mundo hispánico como motor lingüístico y crear un gran corpus de conocimiento para liberar el potencial de la lengua española y de las lenguas cooficiales para la investigación y la economía digitales.

Transformando nuestro taller en un evento híbrido

Entusiasmados con su energía positiva, dejamos a Gustavo y a su equipo para ir a nuestro siguiente compromiso: La presentación y el taller sobre el Nuevo Modelo de Biblioteca (NMB), generosamente acogido por José Pablo Galló León, el director de la biblioteca de la Universidad de Alicante. Pablo había escrito un texto muy reflexivo en el blog Blok de BiD titulado: “Encontraré un camino o lo haré yo mismo: el estudio New Model Library de OCLC Research sobre las consecuencias de la pandemia”. La cita atribuida a Aníbal cuando cruzó los Alpes con elefantes “Encontraré un camino o haré uno”, alude al intento que hace el estudio Nuevo Modelo de Biblioteca para retratar cómo las bibliotecas están intentando hallar un camino a seguir durante la pandemia, aun no existiendo ningún camino preestablecido. Una alegoría muy apropiada y muy bien encontrada.

Los eventos presenciales aún están lejos de volver a los niveles prepandemia

Taller sobre el Nuevo Modelo de Biblioteca en Alicante

Estábamos un poco preocupados por la asistencia presencial ya que los eventos presenciales aún están lejos de volver a los niveles prepandemia. Finalmente, pudimos ofrecer una opción en streaming y ofrecer el taller en formato híbrido y, para nuestra agradable sorpresa, tuvimos un pico de inscritos a última hora igualando la presencia online a la presencial. Los asistentes trabajaron con la Guía práctica del estudio NMB y una gráfica para reflexionar y debatir sobre las áreas de cambio a largo plazo que la pandemia catalizó. Los asistentes se llevaron con ellos los pósteres llenos de post-its para usarlo en la planificación estratégica de la biblioteca.

Durante el evento, Pablo expuso algunas de sus preocupaciones: “Es preocupante que, durante la pandemia, hubo una reducción del uso de los materiales impresos del 40% pero el uso de los recursos digitales no aumentó. En lo digital no somos tan eficientes. Tenemos que mejorar las habilidades e ideas para construir servicios digitales innovadores.” Además, comentó que en los últimos 25 años se han dedicado muy pocos recursos para renovar los edificios de las bibliotecas. De hecho, señaló: “Este es un problema importante. Tenemos que repensar los espacios de las bibliotecas para usos colaborativos. Hablando en general, necesitamos poner más atención sobre la colaboración”.

Para volverse híbrido, tanto la tecnología como la infraestructura del espacio bibliotecario necesita actualizarse y remodelarse

Estas reflexiones resonaron durante las visitas que hicimos en Madrid. Ana Albertos, subdirectora de la biblioteca de la Universidad Complutense de Madrid, comentó que a los estudiantes no les gusta consultar recursos electrónicos, prefieren soluciones más cómodas del campus virtual. Por lo que hace al servicio híbrido, tanto tecnología como la infraestructura del espacio bibliotecario necesita actualizarse y remodelarse. Los estudiantes quieren que la biblioteca abra, pero también demandan más espacios diferentes. Ricardo Santos, Director de Proceso Técnico, y Lourdes Alonso Viana, Jefa de Servicio de Coordinación y Normalización de la Biblioteca Nacional de España, se interesaron por la gestión de la equidad en una plantilla que está evolucionando al modo híbrido y en la necesidad de formar al personal.

El Instituto Cervantes y la virtualización

Durante nuestra visita relámpago en Madrid, también nos vimos con Yolanda de la Iglesia, responsable de Documentación en la red de bibliotecas del Instituto Cervantes, quien está construyendo un nuevo departamento de cultura digital. También mencionó el programa PERTE, que ayudará a financiar la digitalización y preservación de sus colecciones de patrimonio cultural para promover usos innovadores de los recursos de la lengua española. Uno de estos proyectos es el “Mapa de la traducción”, una visualización de datos de traducciones de obras en español y autores latinoamericanos de los últimos 70 años. WorldCat es la mejor fuente para este tipo de proyecto, incluso si los metadatos no están “completos”, una cualificación que se está complicando a medida que los metadatos se reutilizan cada vez más para fines distintos de los que se crearon originalmente. ¡Una cuestión interesantísima para nuestro proyecto sobre los metadatos de próxima generación!

Dejamos Madrid, cada uno hacia sus respectivas direcciones, satisfechos de ricas reflexiones, agradecidos por la hospitalidad de la gente y de los lugares que visitamos juntos.

En este tiempo tan corto, pudimos ser testimonios sobre cómo la virtualización y el uso del espacio físico poseen dinámicas propias. No acaban de convivir fácilmente, tal y como el término híbrido sugiere.

The post Cuaderno de bitácora: Entre el espacio físico y virtual en España appeared first on Hanging Together.

DLF Digest: November 2022 / Digital Library Federation

DLF Digest

A monthly round-up of news, upcoming working group meetings and events, and CLIR program updates from the Digital Library Federation.


This month’s news:

This month’s open DLF group meetings:

For the most up-to-date schedule of DLF group meetings and events (plus NDSA meetings, conferences, and more), bookmark the DLF Community Calendar. Can’t find meeting call-in information? Email us at info@diglib.org

DLF groups are open to ALL, regardless of whether or not you’re affiliated with a DLF member institution. Learn more about our working groups and how to get involved on the DLF website. Interested in starting a new working group or reviving an older one? Need to schedule an upcoming working group call? Check out the DLF Organizer’s Toolkit to learn more about how Team DLF supports our working groups, and send us a message at info@diglib.org to let us know how we can help. 

The post DLF Digest: November 2022 appeared first on DLF.

Advancing IDEAs: Inclusion, Diversity, Equity, Accessibility, 2022 November 1 / HangingTogether

The following  post is one in a regular series on issues of Inclusion, Diversity, Equity, and Accessibility, compiled by Jay Weitz.

Disenfranchisement and disengagement in libraries

Even in 2022, librarianship remains a predominantly white profession with an often shameful history of discrimination. Two librarians at Ohio State University (OCLC Symbol: OSU) in Columbus have collaborated on a pair of ALA Editions Special Reports, Narratives of (Dis)Enfranchisement: Reckoning with the History of Libraries and the Black and African American Experience and Narratives of (Dis)Engagement: Exploring Black and African American Students’ Experiences in Libraries, both published in 2022. Social Sciences Librarian Tracey Overbey and Teaching and Learning Department head Amanda L. Folk sit for an interview about their work in “Making the connection: Amanda L. Folk and Tracey Overbey discuss libraries and the Black and African American experience.” Their first volume is intended “to provide an accessible overview of the historical intersections between libraries and races to help understand the present context.” In the second volume, they report on their study of the experiences of Black students in libraries and interactions with library workers. In addition to “making the profession less white,” they “hope that Black and African American colleagues see that their histories, experiences, and identities are being heard and valued.”

Saving library funding

Photo by Holly Mindrup on Unsplash

In an August 2, 2022, election, voters in Jamestown Township, Michigan, rejected the renewal of an annual operating millage that would have funded the Patmos Library (OCLC Symbol: MIPAT). As reported in the Detroit Free Press on August 5, 2022, “A small group of conservative residents campaigned against the renewal because the library refused to remove all LGBTQ material.” Later that month, novelist Nora Roberts, who also publishes under the name J.D. Robb, donated $50,000 (apparently the limit allowed for GoFundMe donations) to the library. That was on top of thousands of considerably smaller donations from around the world that, as of mid-October, had more than replaced the defeated $245,000 millage. As grateful as they are, the Patmos Library Board reminds local voters “that what this library needs to remain open over the long term is to pass the 10-year levy renewal in November. We cannot run the Patmos Public Library for the next decade without stable taxpayer support.” A small portion of the donations will be used to campaign for the levy renewal in the November election. Less than $1000 had been available to fund the millage campaign in August. Best-selling author Roberts has been a longtime supporter of libraries, literacy, arts, and humanitarian efforts through her Nora Roberts Foundation.

Libraries and Veterans National Forum

In the United States, November 11, 2023, is Veterans Day. To better support and serve the military and veteran communities, librarians from the Texas A&M University Libraries (OCLC Symbol: TXA) created the Libraries and Veterans National Forum in collaboration with academic, public, school, state, and Veterans Administration librarians and with the support of the Institute of Museum and Library Services. Recordings of most of the sessions from the September 2021 online Forum are freely available, as are the vast collection of resources that comprise the Forum’s Toolkit. The ever-growing Toolkit may be accessed from the perspectives of the type of library, type of resource, topic, and intended audience. Collection development policies, lesson plans, best practices documents, and ideas for honoring Veterans Day are just of few of the areas covered.

Censorship in Italy

Attempts to ban books and censor libraries are hardly limited to the United States. In “Mobilising for intellectual freedom: interview with the Observatory on Censorship of the Italian Library Association,” the team at the Associazione Italiana Biblioteche (AIB) Osservatorio sulla censura answers IFLA’s questions about access to information in Italy. Section 3.2 of the 2014 version of the organization’s “Code of Ethics” states, “It is the duty of librarians to promote, individually and in association, the autonomy and efficiency of library service, as a tool of democracy and freedom. Founded in 2018, the Observatory monitors cases of censorship in Italy, studies and reports on the issue, and has established the annual “Saved Books” or “Libri Salvati” commemoration, similar to ALA’s “Banned Books Week.” As members of the Observatory say in the interview, “Our slogan could be: involve people to make them think.”

Combating inequality

During the IFLA World Library and Information Congress in Dublin, Ireland, in July 2022, the Latin America and the Caribbean Regional Division presented “Libraries and inequality. The role of regional cooperation in building fair and sustainable societies.” A report on the session and the three papers is now available on the IFLA website. Former IFLA President Glòria Pérez-Salmerón of Spain spoke on “The role of regional cooperation in building fair and sustainable societies.“ Jeimy Hernández Toscano of Colombia, who is Manager of Reading, Writing, and Libraries at the Unesco Regional Center for Book Promotion in Latin America and the Caribbean (CERLALC), spoke on “Historical debt and social justice: changing libraries’ priorities and their place in the world.” Former ALA President Loída García-Febo of Puerto Rico spoke on “Library Power: UN Sustainable Development Goals and the promise of a better world.” All together, the session highlighted library cooperation in the region and its role in “progressive building of fair and sustainable Latin American societies.”

Inclusive shelving

In an excerpt from their 2021 book Library Programming for Adults with Developmental Disabilities, authors Barbara Klipper, retired youth services librarian from Ferguson Library (OCLC Symbol: FEM) in Stamford, Connecticut, and Carrie Scott Banks, head of Inclusive Services at Brooklyn Public Library (OCLC Symbol: BKL) in New York, call attention to an often overlooked barrier to accessibility. “Special reader collections are often marginalized, shelved in back areas or even behind closed doors, reinforcing the stigma of using them,” they write, reminding us that people with developmental disabilities have the same range of reading levels and interests as everyone else. Klipper and Banks put forth several ideas for promoting access to materials through how and where they are presented in the library, shelving and labeling advice, and other relevant tips.

“The Library’s Role in a Misinformation Age”

In honor of the tenth anniversary of the South Bowie Branch of the Prince George’s County Memorial Library System (OCLC Symbol: MDK) in Maryland, Pulitzer Prize winning Miami Herald columnist and author Leonard Pitts, Jr., a resident of Bowie, delivered an address entitled “Think Again: The Library’s Role in a Misinformation Age.” On October 25, 2022, Pitts spoke about intellectual freedom, journalism, schools, and libraries.

OER and social justice

CJ Ivory and Angela Pashia, librarians at the University of West Georgia (OCLC Symbol: GWC) in Carrollton, have edited Using Open Educational Resources to Promote Social Justice, which is available — appropriately enough — in an open access edition. The volume moves beyond issues of affordability and into the realm of inequities built into both publishing and academia. As the editors state in the introduction, “The chapters in this volume cover a wide range of topics, from theoretical critiques to examples of OER development in practice to examinations of institutional support for OER development.” It includes a section on “Decolonizing Learning in the Global South” and another devoted to “Building and Decolonizing OER Platforms.” Librarians and teachers are encouraged to involve students in the creation of local open resources, thereby adding new perspectives to scholarship.

“Accessibility Assembly”

In order to further ALA’s commitment to accessibility and diversity, the Reference and User Services Association (RUSA) has collected a wide range of toolkits in its “Accessibility Assembly.” From “Accessible Communication Styles” to “Volunteers with Disabilities,” the collection offers materials on mental, physical, and sensory disabilities; library workers and users; assistive technologies and service animals; and professional recruitment.

“User” privacy

Questions of privacy and the definition of “user” have been raised by a book challenge at Colorado’s Gunnison County Libraries (OCLC Symbol: RQY). A local resident submitted a “Request for Reconsideration” regarding the graphic novel “Gender Queer” by Maia Kobabe. The library ended up retaining the book in its young adult collection, but an editor at the local Crested Butte News made a Colorado Open Records Act request that resulted in the resident’s personal data being revealed in the newspaper. The resident sued the library’s executive director for breaking the library’s privacy laws. The story goes on, and you can read about it in Jeffrey A. Roberts’ article “Newspaper editor appeals court ruling that shields the identities of people who want library books banned or reclassified” from the Colorado Freedom of Information Coalition and in Tayla Cardillo’s Intellectual Freedom Blog entry “Do People Who Fill Out ‘Request for Reconsideration’ Forms Have a Right to Privacy?

“Democracy’s Library”

The nonprofit Internet Archive (OCLC Symbol: INARC) has created “Democracy’s Library,” an ongoing compilation of over 700 collections from more than fifty local, regional, and national government organizations from around the world. At present, they claim to have brought together over half a million documents and linked to the websites of more than 200 countries. The “Government Websites of the World” portion of the larger project intends “to preserve the current government websites of every country in the world to ensure perpetual access to online government information for an informed citizenry and democratic resilience.”


The post Advancing IDEAs: Inclusion, Diversity, Equity, Accessibility, 2022 November 1 appeared first on Hanging Together.

Highlights from NDSA’s Digital Preservation Conference 2022 / Digital Library Federation

The 2022 Digital Preservation (aka DigiPres) conference in Baltimore was a huge success and a fantastic opportunity to connect with colleagues. NDSA Leadership is grateful to the Organizing Committee for putting together an engaging program, to Jes Neal and Stacey Erdman for their thoughtful leadership, and to CLIR and DLF for their support and collaboration. We wanted to take a moment to share some of Leadership’s personal highlights from the days’ events:

  • It was really energizing to be in physical space with so many people I’ve only seen on Zoom for the past few years. One of my favorite sessions was Alex Kinnaman’s presentation on recovering from a ransomware attack – it made me think a lot about cybersecurity, its impact on digital preservation, and the importance of preventative measures. I also loved Dorothy Berry’s keynote about the stories that digital preservation tells.  -Hannah Wang
  • I really enjoyed each of the “Emerging Technologies” presentations; Jasmine Mulliken’s work on digital scholarly monographs was an interesting case study of a multi-faceted approach for preserving complex digital objects. With my co-chair Lauren Work, we presented some findings from the recently published NDSA 2021 Staffing Survey Report and had an engaging Q&A with the audience about the survey and resulting report. It was great to catch up with colleagues, including those whom I hadn’t seen in a couple of years and others whom I had never met in-person before but have worked with on NDSA initiatives.  -Elizabeth England
  • Due to a family emergency, I was only able to attend the first half-day of DigiPres 2022. Still, it was an invigorating half-day allowing me to reconnect with many people I have only seen virtually in the past few years and to make new digipres friends. The interactive workshop on Mailbag was a highlight for me, it was the smoothest tech workshop I’ve seen, there were nearly no technical glitches and everyone was able to follow along on their own laptops. You could tell the audience was eager to try it out and see how Mailbag could help them with their own email preservation workflows. I regret having to miss the Fixating on Fixity session and am watching the OSF proceedings repository to see if the slide deck is uploaded.  -Nathan Tallman
  • I’ve been fortunate to befriend many colleagues over the last decade or so, and connecting with them in person for the first time in several years reaffirmed my commitment to values-based practice. Perhaps for that reason, DigiPres sessions that focused on digital preservation staff and their needs and concerns drew my attention most. In particular, Elizabeth England and Lauren Work did an excellent job parsing some of the results of the 2021 Staffing Survey—for me, emphasizing how much top-down advocacy and resource support for digital preservation is needed. That presentation was followed by a detailed discussion of George Blood’s experiments with WGBH’s Raananah Sarid-Segal and Caroline Oliveira Mango on the carbon impact of cryptographic hashes, which further reinforced how valuable the dedicated time and attention of staff is necessary to refine our practices to reduce climate impact.   -Courtney Mumma
  • Like others, I was delighted to be in Baltimore for NDSA and to meet many of my colleagues in person, many for the first time. For me, the most impactful session I attended was “A Digital Preservation Reckoning: If we don’t lead with values, where do we end?” Led by Hannah Wang, Courtney Mumma, Sibyl Schaefer, and Andrew Diamond, the session engaged the audience to consider risk and accountability for the digital preservation services we use. Lots of food for thought and valuable considerations for how we approach digital preservation as a profession. I also enjoyed Jasmine Mulliken’s presentation on digital preservation and publication, “The Story of a Digital Scholarly Publication, as Told by its Preservation Format.” Exciting and extremely useful to see a demonstration and deconstruction of how a multimodal publication might be archived, documented, and stored, and how use is affected by preservation strategies.  -Ann Hanlon

The post Highlights from NDSA’s Digital Preservation Conference 2022 appeared first on DLF.

Metadata During COVID – Part II / Digital Library Federation

 

Mandy Mastrovita, Dana Reijerkerk, and Rebecca Fried of the DLF AIG Metadata Working Group Blog subgroup, and Annamarie Klose of the DLF AIG-MWG Tools subgroup, contributed to this post.


Introduction

Metadata assessment and remediation continue to be necessary during the COVID-19 pandemic due to the prevalence of remote work opportunities and the loss of access to physical collections. Members of the DLF AIG Metadata Working Group engage in ongoing discussions of how their institutions have increased their capacity for metadata work. This post is part two of a series about COVID-19 metadata projects and activities. 

You can find part one of this series at https://www.diglib.org/metadata-during-covid/.

Return to in-library work

In some institutions, metadata has now moved to the back seat as physical collections and in-person service return to top priority. However, the demand for access to online content hasn’t dropped. On the contrary, people want more online content now that they know how to use these resources and have higher expectations for how they will be delivered. 

One metadata librarian (who asked to remain anonymous) notes that, at his institution, “there was certainly an assumption that ‘anyone can do this’ and ‘it will be an easy solution for ‘work from home.’” Kate Flynn, Digital Programs & Metadata Project Librarian at the University of Illinois at Chicago, adds, “It also caused shifts in workflows as well as new ones. You may have created metadata before but found that you were doing assessment and corrections instead.”

How do we balance these demands and shift resources to support this work?

Dana Reijerkerk, the Knowledge Management and Digital Assets Librarian at Stony Brook University Libraries, says that in March 2020, various members of Resource Management at Stony Brook University Libraries began working together to remediate and create metadata. Previously, metadata work was ad hoc and dependent on reporting lines. Some colleagues picked up this work as a way to continue operations while being remote. Interdepartmental collaboration is now the norm. For example, the digital asset manager, a cataloger, the Director of Special Collections and University Archivist, and library administrators are working to streamline metadata creation and MARC record ingests of electronic theses and dissertations. This workflow was only possible by piloting remote work and opening opportunities for collaborative asynchronous operations. 

Mandy Mastrovita, a digital projects librarian at the Digital Library of Georgia, says that since the COVID era began in 2020, our metadata unit has become much closer, thanks to the countless sidebar meetings we arranged during the pandemic. We love this! 

Building new workflows requires trial and error. Our workflows for metadata are always changing to match the skills of students, classified staff, and librarians. One size does not fit all!

We’ve spent so much time figuring out what our projects need and designing new workflows with tools like the Google suite, Teams, OpenRefine, and GitHub that let us pick up most of our workflows online. With these reinvestigations of our workflows, our metadata team has been able to cross-train in different parts of our processes more effectively. Now, we’re more flexible, and we’ve made our workflows more modular so they can work better. This work has vastly improved our adaptability.

At the beginning of the pandemic, we embarked on a transcription project to involve student workers. The materials included several collections’ worth of sermons and letters from the 19th century, handwritten in cursive. Although our students completed a good chunk of the work, we decided when we returned to the office that the student output yielded FAR less than a vendor would have for the same project. 

Becky Fried, a digital projects and metadata librarian at Union College in Schenectady, New York, discovered that COVID and the New York Pause allowed her organization to take a step back, regroup, and reorganize how digital projects and metadata are managed and captured. Metadata work mostly paused. Instead, a backlog of projects that were digitized and described but not published was the focus for remote work. Opportunities to partner with other institutional departments to support such events as the Annual Steinmetz Symposium in an online forum flourished. Those partnerships have led to rich collections of material now that in-person events have restarted. Library staff completed a transcription project that served as a trial run for an upcoming crowd-sourced transcription project using Omeka with the Scripto plug-in for an NHPRC grant received in 2021 for “Expanding and Enhancing Description and Access to the John Bigelow Papers at Union College.”

Annamarie Klose, Metadata Initiatives Librarian, The Ohio State University Libraries, notes that COVID metadata projects assisted by staff and student employees significantly impacted her work with Digital Collections (DC). Thousands of records were created and updated due to collaboration with employees who don’t typically work on metadata. Successes included ingesting more than 10,000 issues of the university’s student newspaper, The Lantern, and hundreds of audiovisual materials. However, we also learned important lessons. 

Non-metadata employees worked better on carefully structured tasks and included significant documentation to follow. In the case of The Lantern, this meant verifying that metadata spreadsheets and digital files were matched and prepped for ingest and passed quality review after ingest. In terms of assisting with description, Metadata Initiatives (MI) has to review metadata from non-metadata employees to ensure it conforms to national and international best practices. Subject analysis, in particular, requires more training and expertise to create proper metadata that adheres to established standards. 

Non-metadata employees tended to do better with tasks that involved a set list of options, e.g., limited controlled vocabulary terms, to assign related to content. There were also some successful collaborations with catalogers who don’t usually create Dublin Core metadata. For example, one cataloger remediated thousands of digitized comic strips to provide subject analysis and summaries for each. However, traditional catalogers often found working with a customized version of Dublin Core to be a new experience. MI’s work to provide remote work for staff and student employees also required the creation of new workflows and documentation to manage these efforts. 

Managing a large group of people with virtual tools, including BOX, provided challenges. A few less tech-savvy employees found the collaborative tools daunting. But some employees liked working with metadata and thought it was a welcome change of pace. Typically a small unit, MI is still getting through the backlog of work processed during the early phase of the pandemic.

A couple of great places to go if you want to hear from other metadata librarians who are trying to find solutions to their ongoing projects as they endure COVID growing pains are the AIG-MWG Slack channel and the AIG-MWG Google group.

These are both forums to ask your colleagues questions about adapting workflows, sharing resources, and working collaboratively remotely and in person. So many of us have figured out things that work within the environments we have developed at our institutions. Reaching out to other folks who work in similar situations might relieve you of having to recreate the wheel on your own.

The post Metadata During COVID – Part II appeared first on DLF.

Greater Fool Supply-Chain Crisis / David Rosenthal

Source
Louis Ashworth's The unbearable stability of bitcoin points to a mystery:
Bitcoin’s doing fine, thanks for asking.

Amid a shaky few months for markets, the world’s favourite cryptocurrency has been remarkably stable, trading in the narrowest range since late 2020
...
As might be expected, that lack of movement has coincided with a downturn in activity on most exchanges other than market leader Binance, which has cut fees to drum up more business and now hosts about a fifth of all volume
...
Stability looks good, at least relative to what’s happening elsewhere. But thinning volumes in a market that has no utility beyond store of value, where the actions of its few active daily participants are being determined by technical resistance and support levels, is not a symptom of robust health.
Below the fold I look into this persistent failure to proceed moon-wards

Vildana Hajric's headline tells the recent story — Bitcoin Breakout Stalls With Daily Trading Volume Also Tumbling:
Cryptocurrency price increases have mostly stalled following two days of gains that had spurred optimism for a more sustainable rally.

Bitcoin, the largest token by market value, declined as much as 1.5% on Thursday to once again trade around $20,000, a level it’s been stuck around for weeks. Ether was little changed, while an index tracking the 100 largest coins also fell.
Hajric continues:
Interest in crypto has waned amid a slump in prices that’s seen Bitcoin lose 70% from its all-time high of near $69,000 in November. Retail investors, in particular, have been disenchanted by the asset class. They’ve not been wading into the market in the same way they did during the first two pandemic years, with Google searches for the word “crypto” falling to the lowest levels in the past year.
Source
Why would retail investors buy? They are facing high inflation and a looming recession, their stock and bond portfolios are evaporating, and their cryptocurrency HODL-ings have evaporated even faster. It isn't just retail:
Meanwhile, institutional digital-asset products this month saw their lowest-ever volume in data going back to June 2020, with average daily trading volume dropping 34% to $61 million, according to CryptoCompare.
Most investors want to decrease their cryptocurrency leverage, fearing margin calls. Yueqi Yang reports that Crypto Broker Genesis Says Lending Plunged 80% in Third Quarter:
Crypto brokerage Genesis, reeling from a sharp decline in the digital-asset market that sent the industry into a tailspin, said it originated $8.4 billion in new loans in the third quarter, an 80% plunge from the prior three-month period. Most of the rest of its businesses also experienced substantial declines.

Total active loans slumped to $2.8 billion from $4.9 billion in the second quarter, Genesis said in its quarterly earnings report. Trading volumes also declined, with spot volume sliding 44% to $9.6 billion. Its derivatives desk traded $18.7 billion in notional value, down 30% from the prior quarter.
...
Genesis was the biggest creditor ensnared in the collapse of Three Arrows after the once highflying hedge fund failed to meet margin calls. Over the summer, the company cut staff by 20% and overhauled its leadership team. A series of senior executives have departed, including its co-head of sales and trading Matt Ballensweig and head of derivatives Joshua Lim. Most recently, its new chief risk officer Michael Patchen also left after three months.
Because there are no retail buyers, and because the institutional HODL-ers don't want to realize their losses, everyone is just HODL-ing:
Source
The proportion of Bitcoin that has been held for one year and over is at an all-time high, with more than a quarter of total supply now not having moved on-chain for at least five years, according to Genesis.

The amount of Bitcoin that hasn’t moved in over a year reaching a record of over 66% this week means that there’s “less and less BTC readily available to new entrants,” said Noelle Acheson, author of the “Crypto is Macro Now” newsletter.

But there are plenty of other signs investors remain uninvolved. The 14-day moving average of perpetuals volumes, as measured via FTX, reached its lowest point since the start of 2021, according to data compiled by Strahinja Savic at FRNT Financial. And the drop in derivatives volumes has come alongside a slump in spot volumes. Savic points out that the 14-day moving average volume of BTC/USD on the crypto platform is also at its lowest since April of last year.
Amy Castor and David Gerard explain it differently:
Crypto has crashed, and some of our readers are asking us why the price of bitcoin has been holding steady at around $19,000 to $20,000 for the past few months. Why won’t it go down further?

We think the price of bitcoin is a high wire act. If the price drops too low, some leveraged large holders could go bust. So the number needs to be kept pumped above that level. If the price goes up too far, the suckers — not just retail, but the bitcoin miners — may be tempted to cash out at last.

The idea is to pump just enough to keep the price up — but not so much that suckers dump their bitcoins directly into the pump.

If too many bagholders try to sell, what quickly becomes obvious is there are no actual buyers. At least, none with real money.

The party is over. Retail investors have all gone home, so there are no more suckers getting in line to pump the price up anymore. Coinbase’s 10-Q showed a drop in retail dollars.

In addition to a dearth of real dollars, there’s also been a dearth of fresh tethers coming in since June. That dearth lasted until October 25 — when a billion tethers were printed and prices suddenly jumped 10%, just in time to liquidate a pile of short-margin traders on FTX.
It is no wonder that investors are gun-shy:
Bitcoin derivatives — assets that derive their value from bitcoin — aren’t doing well either. The ProShares Bitcoin Strategy ETF (BITO) tracks the CME’s bitcoin futures. These are just bets in dollars on the price of bitcoin. Bloomberg Intelligence analyst James Seyffart says: “If you just want exposure to Bitcoin” — i.e., not doing anything so gauche as touching a bitcoin — “BITO is the best option in the ETF landscape, at least in the US.” But in the more than a year that it’s existed, BITO has performed even worse than bitcoin itself. BITO holders have mostly stayed holding, so its holders are just like bitcoin bagholders too. [Bloomberg]
A year ago BITO, an ETF that uses futures contracts to track the BTC "price", launched to much acclaim and $1.1B of investors' actual dollars in two days. Vildana Hajric and Katherine Greifeld's The Bitcoin Futures ETF at 1: $1.8 Billion Lured, Over Half Lost takes up the story:
But, the ETF wasn’t exactly what die-hard fans had wanted. The fund doesn’t hold Bitcoin directly. Instead, it is based on futures contracts and was filed under mutual fund rules that Securities and Exchange Commission Chairman Gary Gensler had said provided “significant investor protections.”

US regulators have been hesitant to approve a product that tracks the real coin, citing volatility and manipulation, among other things.
Source
As the graph shows, BITO did an excellent job of tracking BTC. The result was:
The fund has slumped over 70% since its launch, tracking a crypto collapse that dragged Bitcoin to around $20,000. At a year old, BITO has posted cumulative inflows of more than $1.8 billion, and yet as of Friday had just $619 million left.

“It’s been a bad year -- we’re looking at $1.2 billion burned,” said James Seyffart, a Bloomberg Intelligence analyst. “But if you just want exposure to Bitcoin, BITO is the best option in the ETF landscape, at least in the US.”
This chart shows that, once "number go down" the number of investors who "just want exposure to Bitcoin" is miniscule. The fund is just for devoted HODL-ers.

Source
Ashworth quotes a Morgan Stanley report:
Almost 1 year into the bitcoin bear market, most who bought bitcoin in 2021 are facing heavy losses and appear to be waiting for any rallies to close their position. A record number of bitcoin units haven’t been used for any transaction in the past 6 months, currently at 78% of total and this number continues to rise (Exhibit 1).

What this means, if we oversimplify a bit, is that those who bought/received bitcoin more than 6 months ago are holding onto their positions, with some likely waiting for a price recovery. For the remaining 22% of bitcoin units held by the shorter term investors who did transact bitcoin in the past 6 months, estimates suggest their average breakeven price is just over $22.3k (+7% from current but was as high as 20% a few days ago, see Exhibit 2)
The more HODL-ing and the less trading, the easier it is for the "leveraged large holders" to pump the "price" to avoid getting liquidated, but not so much that the HODL-ers sell, which is Amy Castor and David Gerard's argument.