Saturday, January 25, 2014, 3:00-4:00 p.m., A Consideration of Holdings in the World Beyond MARC	Sunday, January 26, 2014, 8:30-10:00 a.m., The Other Side of Linked Data: Managing Metadata Aggregation	Sunday, January 26, 2014, 8:30-10:00 a.m., Mapmakers

Most ALA watchers have noticed a shift from ‘invited talks’ at Interest Group and Committee meetings to requests for proposals from the chairs, from which pool the speakers are chosen. This is, of course, in parallel with changes going on with other professional conferences, and it’s an interesting shift for a number of reasons.

There’s a democratization aspect to this change–the chairs are no longer limited in their choice to people they already know about, thereby potentially increasing the possibility that new and different ideas will get an airing. Maybe this Midwinter someone will come up with an absolutely wonderful and unexpected presentation that rockets the speaker from the unknown mob to the smaller roster of interesting known speakers. This is a good thing, I believe, even though the chance of witnessing such a rocket launch are dauntingly small.

As someone who has been around long enough (and noisily, it must be said) this shift means that I don’t need to wait for invitations to do presentations based on some chair’s idea of what might interest their group (but may no longer interest me), I can go ahead and respond to the calls that are appealing to me. I’d like to think that the result is something fresh enough to be interesting for me to prepare and an audience to listen to, without being totally divorced from prior talks that represent earlier phases. An odd result of this shift in process is that speakers who submit proposals to various committees don’t generally know who else will be speaking at a particular program until after their proposal has been approved, and maybe not even then. This particular aspect has already led to some very interesting lineups at meetings across the conference.

Because I take seriously the idea of not re-using previous talks to the extent that I could become horribly boring, I tend to apply for things that allow me to explore something that isn’t unrelated to what I’ve done before, but at least requires that I rethink something or try a different approach than I’ve used before to expose what I (and the people I work with) are thinking about. I think that’s pretty much what most audiences are looking for, right?

So below are my talks for ALA Midwinter. I may be accompanied by one or another of my colleagues on a couple of these, and will surely have their help building the presentations.

A Consideration of Library Holdings in the World Beyond MARC

Of all the MARC 21 formats, Holdings was the one most clearly designed for machine manipulation. It is granular, flexible, and intended to be used at either a detailed or summary level. It has sometimes frightened potential users because it looks complex (even where it isn’t), and in its ‘native’ form is not particularly human friendly. Some of the complexity arises because there are both display and prediction aspects in the encoding, and not all library systems have developed predictive serial check-in systems supported by MARC Holdings.

Some of the bibliographic metadata efforts now going forward ignore the existing MARC Holdings, sometimes in favor of simpler solutions based on the perception of the waning need for predictive check-in for digital subscriptions. Not much effort has been expended to bring the MARC Holdings format forward into the discussions about changing requirements and re-use of existing standards.

For the ALCTS CRS Committee on Holdings Information, Saturday, January 25, 2014, 3:00-4:00 p.m.

Holdings has been an interest of mine since I was a law librarian representing the American Association of Law Libraries on MARBI. In the early computer era in libraries, where digital publication was the exception, law publishers demonstrated a great deal of creativity in their publication of updating services, from loose-leaf services and regular republication of standard tools, and law catalogers always had the best examples of holdings problems. These days, most of those materials have been subsumed by various digital tools, which have their own complexities, particularly in the context of versions, republication and compilation.

But the question remains–has what we learned from the pre-digital world of holdings functionality have relevance in the digital era?

The Other Side of Linked Data: Managing Metadata Aggregation

Most of the current activity in the library LOD world have been on publishing library data out of current silos. But part of the point of linked data for libraries is that it opens up data built by others for use within libraries, and has the potential for greater integration of library data within the larger data world. The sticking point for most librarians is that data building and distribution outside the familiar world of MARC seems like a black box, the key held by others.

Traditionally, libraries have relied on specialized system vendors to build the functionality they needed to manage their data. But the discussions I’ve heard too often result in librarians wanting vendors to tell them what they’re planning, and vendors asking librarians what they need and want. In the context of this stalemate, it behooves both library system vendors and librarians to explore the issues around management of more fine-grained metadata so that an informed dialogue around requirements can begin.

For the ALCTS Metadata Interest Group, Sunday, January 26, 2014, 8:30-10:00 a.m.

Transitioning from a rigidly record-based system to a more flexible environment where statement level information can be aggregated and managed is difficult to envision from the vantage point of our current MARC-based world. This has lead to a gap between what we know, and the wider world of linked open data we’d like to participate in. One of the critical steps is to understand how such a world might look, and what it requires of us and our systems. The goal is to be able to move some of that improved understanding to the point of innovation and development.

Mapmakers

It’s very clear that there will be no single answer to moving bibliographic metadata into the world beyond MARC, no direct ‘replacement’ for the simple walled garden we all have lived in for 40+ years. While it’s certainly true that the emerging global universe of bibliographic description has continued to expand and has seems more chaotic than ever, there are still commonalities of understanding with the world beyond our garden walls that we’re only beginning to identify. How then can we begin to expose our understanding to that universe and develop some consensus paths forward? Specifically, what are the possibilities for using semantic mapping to provide us with the flexibility and extensibility we need to build our common future.

For the ALCTS CaMMS Cataloging & Classification Research Interest Group, Sunday, Jan. 26, 10:30-11:30.

Librarians too often see ‘mapping’ and think ‘crosswalking’, but the reality is that these are quite different strategies. Crosswalking was a natural fit for the MARC environment, where the ‘one, best’ crosswalk would logically be developed centrally and implemented as part of current application needs. But the limitations of crosswalking make much less sense as we transition into a world where the Semantic Web has begun to take hold (of our heads, if not our systems!).

In the Semantic Web world, maps can contain a variety of relationships (not just the crosswalk ‘same as’), and central development and control is neither necessary nor very useful. This doesn’t mean that we’re all on our own and that collaboration isn’t still our best strategy.

[Note: At this point the final information about where these presentations will be held isn’t available, but I’ll add that when it becomes public.]

by Diane Hillmann at December 09, 2013 08:26 PM

The bootstrap3 collapse javascript plugin is actually quite flexible, but it’s very terse documentation and examples might make it a bit hard to figure out how.

The docs example shows using a data-toggle attribute on the trigger for the collapsing element, the thing that the user will click on to expand/contract. Then, in data-target on the same element,  you might supply a string id which identifies the content body, which will be shown/hidden.

If you are dynamically generating content one way or another, you might not have convenient id’s on the content bodies.

So, we might start looking at the various options the collapse plugin takes, and at using it “via Javascript; enable manually…”

There are a couple things that were confusing there to me:

1. When applying manually with `$some_element.collapse();`, you apply to the content body, not the trigger. (Even though the default, where you trigger purely with data- attributes, you apply those data attributes to the trigger).
2. Then you can make your own trigger, simply something which, on click, finds the relevant body and and triggers the collapse toggle action, maybe like `$(this).next(“.content-child”).collapse(“toggle”);`
3. The collapse plugin will add the class `collapse` to your content body, which is styled by boostrap CSS to be hidden. To avoid any JS ‘flicker’, you probably want to manually add the `collapse` class to your content body  element yourself, so it always starts out hidden.  Then, in page load, when you want to apply the collapse plugin to your body, apply it like: `some_element.collapse({toggle: false});`, to avoid toggling it’s state to visible on page load when you apply the collapse plugin to it.

This stack overflow item, with accompanying jsfiddle, pretty much provide this solution and a demonstration. But also with fairly laconic explanation, it took me a bit of head scratching and looking at source code to figure out what was actually going on here.

no-js?

Personally, I still try to follow the old-school doctrine of unobtrusive javascript, where if a user-agent has javascript turned off, everything will still display perfectly usably.

So adding the .collapse class directly to my content body produces some undesirable effects — if javascript is turned off, the body will still be hidden by the CSS, but there will be no way to show it by triggering the collapse trigger.

This problem applies to the by-the-example use of bootstrap3 collapse too, as it also has the .collapse class applied directly to the content body.

The solution is pretty simple if you use the “no-js” class technique originally demonstrated by Modernizr. 

.no-js .collapse {
   display: block;
}

 

Filed under: General

by jrochkind at December 09, 2013 06:39 PM

The Following is a guest post by the Open Knowledge Foundation Community Member Graeme Arnott originally posted his blog here

At the first meeting in Glasgow of Open Knowledge Foundation Scotland it was good to see some familiar faces from the ‘Open Badges in Scottish Education Group‘ (OBSEG). A little twitter conversation afterwards with Sheila MacNeill and Lizzie Brotherston, and we had agreed that openbadges for OKF Scotland was something that we definitely wanted to do.

I started by playing around with the iPad App version of Badge Designer on openbadges.me. It was a bit footery at first but once I’d gotten the hang of it I realised that playing with what was possible wasn’t simply a process of skills development but was also a process of working out what I wanted to be possible. So whilst it was easy enough to design individual badges I needed to think about the place of each individual badge within the proposed badge eco-system.

Play/Pause

Before getting to the point of deciding what each badge looked like however, it was necessary to consider a number of questions,

• who should get a badge?
• for what should they get that badge?
• what behaviours did we want to encourage?
• what were the unintended consequences of encouraging certain behaviours?

The answer that I came up with to the first question was that everybody involved should get a badge. This meant that there would be a badge for the speakers who did the lightning talks, the people who turned up, and the people who organised the group.

I can appreciate that someone might make the argument that says the organisers and the speakers do more ‘work’ than an attendee but that’s to see the audience as simply passive recipients of the speaker’s knowledge. It’s an outdated idea of audience, and the amount of tweets both during and after the first meeting testify not solely to the audience’s enthusiasm but to the construction of the community that will (hopefully) want to reconvene at the next meeting. It’s also the case that within hours of the first meeting Lorna Campbell had storifyed it whilst Martin Hawksey had collated tweets featuring our #OpenDataGLA. This active participation on the part of the “audience” begged the question as to whether someone who simply turned up out of curiosity should also get a badge. It was thinking about that question, and the word ‘get’, that I started to think about using the phrases of “earning a badge”, or “claiming a badge”. However, I think it needs to be acknowledged that someone who comes to every OpenDataGla meeting but doesn’t present a lightning talk or tweet is nevertheless actively contributing to the group. So the view that I’ve now comfortably settled into is that everyone involved in OKF Scotland has a role to play in making it work, and that each role, consequently, deserves acknowledgement.

To achieve this horizontal arrangement my first idea was that the design of each badge would have to be such that one wasn’t any “better” than any other. This doesn’t accord with the badge design rubric by Peter Rawsthorne where there is a clear hierarchy from ‘Introductory’ through to ‘Exemplary’. As far as I understand this though, the rubric is more specifically applicable to tasks or achievements which are increasingly more difficult, complex or demanding. Asking myself whether Rawsthorne’s hierarchy applied to OKF Scotland question helped me realise that the answer isn’t straightforwardly interrogative. During my preparations for my lightning talk at OpenDataGla/2 I asked a couple of the participants who took part in the Glasgow Women’s Library/WMUK editathon, Scottish Women on Wikipedia, if they’d come along and say a few words. I hadn’t intended to put the person under any pressure or obligation but for one person, who hates public speaking, this was a terrifying suggestion.

This situation helped me realise that whilst speaking to a room full of people is something of an everyday occurrence for me for this other person it was an absolute nightmare. What I think this means, in terms of badge design for a community, is that if that person had come along and managed to speak they would have experienced the badge as something that they had really gained rather than something they had simply been given, and that therefore the meaning of the badge will be regarded in their relative terms, regardless of the conceptual design. For me this means that there is no particular need in this case to embed a hierarchy of achievement because the actual badge earners will have their own personal take on each badge’s value.

Between starting this post, which is a number of months ago, and now, December 2013, it seems that the OKFN have started the process of refreshing their brand design. In a sense it doesn’t really matter because my general idea was that the design of the openbadges would closely follow the OKFN logo design.

The ones I’ve done so far all have the same basic design shape with a different icon and different text on the banner. I wanted to use the Open Knowledge Foundation’s colour scheme and I tried to get the hexadecimal codes from the OKFN website but in the end it didn’t matter because the openbadges.me palette is limited to the colour options that it provides. We just ended up picking what we thought was the closest.

I haven’t answered all the questions that I set out in this post, and there a number of other areas that need to be covered like issuing, and the content of the metadata but for the moment I just want to use this post to get the conversation going and see if it’s worthwhile pursuing.

Acknowledgements

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. However, the media used in this post is licensed with different CC licences. If you do share this work please respect the conditions of these different licenses.

by Guest at December 09, 2013 04:35 PM

Today the Open Knowledge Foundation and the Alliance for Lobbying Transparency are co-launching a petition endorsed by a group of transparency and civil society organisations, asking the UK government to rewrite the lobbying bill to give citizens a proper register of lobbyists in the UK

As we’ve written about before, the lobbying register proposed in the Lobbying Bill (which is currently going through parliament) only covers a small portion of the UK’s £2 billion influence industry. We think this needs to change and call on the government to rewrite the bill to address this. The full text of the petition is copied below, and you can sign up here.

If you’re interested in improving lobbying transparency in your country, you can also join our recently launched global lobbying transparency working group, co-convened with our friends at the Sunlight Foundation.

If you agree with us that citizens should be able to see who is trying to influence government decisions and to what end, please do sign and help us to share the petition to your friends and colleagues:

• http://you.38degrees.org.uk/p/secretlobbying

STOP SECRET CORPORATE LOBBYING

The government is poised to let corporate lobbyists off the hook: we have just weeks to stop them. Having been slammed for proposals that unfairly cracked down on charities, the government is now trying to pass a watered down ‘register of lobbyists’, which would do nothing to stop the malign influence of big corporations over our political system.

For a truly democratic result, all lobbyists must be obliged to say who they’re meeting, what decisions they are seeking to influence and how much they are spending. If the government is serious about rebuilding public trust, it must rewrite its proposals for a lobbying register now, and keep its promise to citizens.

Why is this important?

The UK has a £2 billion commercial lobbying industry – the third largest in the world – that is embedded in our political system. But the government’s current proposal would exclude the vast majority of commercial lobbyists and give the rest an easy ride. The new rules would apply to as little as 5% of all lobbying activity, and those they did affect would hardly have to provide any information at all.

David Cameron and Nick Clegg promised they would run the ‘the most open and transparent government in the world’ but they have yet to deliver on their promise to shine a light on secret corporate lobbying, which advances the interests of the few at the expense of the many. This is their last chance to propose a register that covers all lobbyists and includes meaningful information about their activities.

This petition is coordinated by the Open Knowledge Foundation and the Alliance for Lobbying Transparency and endorsed by Access Info, the Campaign for Freedom of Information, Campaign for Press and Broadcasting Freedom, Corporate Europe Observatory, Corporate Watch, Greenpeace, Spinwatch, the Sunlight Foundation, Unlock Democracy and War on Want. If your organisation would like to endorse the petition, please send an email to jonathan.gray@okfn.org.

by Jonathan Gray at December 09, 2013 03:56 PM

I visited my high school French teacher at the end of October. She's retired and lives in Alsace. My French stays with may, mostly unused, but something else I learned from her has stayed with me- she showed me how to eat steamed artichokes. Whenever I eat and enjoy artichokes that way, my mouth remembers. So often, we give something to other people, and it it lives on, having an impact out of proportion to the original gift. A girl in my Calculus class introduced me to Bruce Springsteen and The Wild, the Innocent and the E Street Shuffle. Wherever you are, Sarah Strong, thank you, whether you remember or not!

On October 29, I lost a friend from graduate school, Kanji Yoh (陽完治), and I've been reflecting on so many things I learned from Kanji that have remained with me over the years. But what I learned has been hard for me to articulate, because it's not like artichokes or the Boss. OK, I blame Kanji for taking me to the Opera my first and only time, so I suppose I can thank him for helping me cross that off my list. But there's a lot more.

Kanji and I both started graduate school at Stanford the same year. For me graduate school was getting back on a familiar academic track towards being a scientist. For Kanji, graduate school was a step off the conventional track. He had been an engineer at Hitachi, in Japan, at the sort of job where you were expected to work for the company your entire working life. And it was reasonable to expect the company to take care of you in return. Kanji was a rare, remarkable, courageous creature, and my advisor recognized him as such. Kanji needed something more.

Valuing that "more" is what I learned from Kanji.

Kanji and I shared a tiny, sloped-ceiling office with 2 other students in the attic of Stanford's McCullough Building for four years. We were among the first batch students of a new Professor, "Coach" Jim Harris, and we were charged with building a new research lab and launch a new research program. Kanji would ask me interesting questions, and I'd startle him with odd conjectures and amazing facts.

Kanji worked on a counterintuitive idea, to engineer p-channel transistors from layers of Gallium Arsenide instead of the usual n-channel transistors. It was a sort of semiconductor jiujitsu, flipping the weak characteristics of a material into strengths, so as to balance two complementary transistors and achieve both speed and low power. It was an elegant idea; the job of a researcher is to see whether physical reality has the same beauty as the mathematical description.

My wife and I spent a memorable Thanksgiving with Kanji, his wife and some friends. We rented a condo at Lake Tahoe, went skiing, and did a real Thanksgiving turkey. Then we did some more skiing; Kanji was so eager to get out on the slopes early, and there was a foot of fresh snow.

After a post-doc at IBM Watson Labs, Kanji got a job as an Assistant Professor in Osaka. It was a difficult time for him, because the strict Japanese academic system doesn't allow much freedom for junior Faculty. I had a chance to visit him once. We walked around his neighborhood and talked about cicadas, which were making a huge noise. I told him that New Jersey cicadas, unlike Japanese cicadas, come in broods that appear in 13 or 17 year cycles. Always a prime number to fool cyclical predators. Kanji's face lit up with wonderment at the beauty of this fact. "Really?" he said.

Kanji persevered in Osaka and got a position at Hokkaido University in Sapporo. His work there covered quantum dots and wires, graphene transistors, and spin polarized tunneling. His recent work has been on "spintronics", trying to find ways to use and manipulate electron spins to do calculations with quantum mechanics. When I left semiconductor physics 15 years ago, these things were just starting to be thought about and the idea seemed so beautiful and futuristic and impossible.

Kanji played both the violin and flute and performed with three different ensembles. He was just as serious about his music as he was about his semiconductors. He became a fan of  Soprano Anna Netrebko, and would fly around the world to go to her performances. I was a bit shocked when he first told me about this, especially the part about the protocol for delivering roses to the star after a performance. But it's inspired to take more seriously some of things I've long thought would be fun to do, but who has the time? Like visiting may high-school French teacher in Alsace and drinking Auxerrois with her friend the viticulteur.

I miss him. I'm sure Kanji's wife, a translator and prize-winning poet, and his two kids feel a great absence. They're not alone.

by Eric (noreply@blogger.com) at December 09, 2013 01:23 PM

This article by Rufus Pollock, Founder and Director of the Open Knowledge Foundation, is cross-posted from “Telefonica Digital Hub” released on 5 December 2013.

Every day we face challenges – from the personal such as the quickest way to get to the work or what we should eat to global ones like climate change and how to sustainably feed and educate seven billion people. Here at the Open Knowledge Foundation we believe that opening up data – and turning that data into insight – can be crucial to addressing these challenges, and building a society in which is everyone – not just the few – are empowered with the knowledge they need to understand and effect change.

Open data and open knowledge are fundamentally about empowerment, about giving people – citizens, journalists, NGOs, companies and policy-makers – access to the information they need to understand and shape the world around them.

Through openness, we can ensure that technology and data improve science, governance, and society. Without it, we may see the increasing centralisation of knowledge – and therefore power – in the hands of the few, and a huge loss in our potential, individually and collectively, to innovate, understand, and improve the world around us.

Open Data is data that can be freely accessed, used, built upon and shared by anyone, for any purpose. With digital technology – from mobiles to the internet – increasingly everywhere, we’re seeing a data revolution. It’s a revolution both in the amount of data available and in our ability to use, and share, that data. And it’s changing everything we do – from how we travel home from work to how scientists do research, to how government set policy.

Now much of that data is personal, data about you and what you do – what you buy (your loyalty card, your bank statements), where you go (your mobile location or the apps you’ve installed) or who you interact with online (Facebook, Twitter etc). That data should never be “open”, freely accessible to anyone – its your data, and you should control who has access to it and how it is used.

But there’s a lot of data that isn’t personal. Data like the government’s budget, or road maps, or train times, or what’s in that candy bar, or where those jeans were made, or how much carbon dioxide was produced last year … Data like this could and should be open if the governments and corporations who control it can be persuaded to unlock it.

And that’s what we’ve been doing at the Open Knowledge Foundation for the last decade: working to get governments and corporations to unlock their data and make it open.

We’re doing this because of the power of open data to unleash innovation, creativity and insight. It has potential to empower anyone – whether it is an entrepreneur, an activist or a researcher – to get access to information and use it as they see fit. For example, citizens in Ghana using data on mining to ensure they get their fair share of tax revenues to pay for local schools and hospitals, or a startup like Open Healthcare UK using drug prescription data released by the UK government to identify hundreds of millions of pounds of savings for the health services.

It’s key to remember here that real impact doesn’t come directly from open data itself – no one’s life is immediately improved by a new open data initiative or an additional open dataset. Data has to be turned into knowledge, information into insight – and someone has to act on that knowledge.

To do that takes tools and skills – tools for processing, analysing and presenting data, and skills to do that. This is why this is another key area of the Open Knowledge Foundation’s work. With projects like SchoolofData we’re working to teach data skills to those who need them most, and in Open Knowledge Foundation Labs we’re creating lightweight tools to help people use data more easily and effectively.

Finally, it’s about people, the people who use data, and the people who use the insights from that data to drive change. We need to create a culture of “open data makers“, people able and ready to make apps and insights with open data. We need to connect open data with those who have the best questions and the biggest needs – a healthcare worker in Zambia, the London commuter travelling home – and go beyond the data geeks and the tech savvy.

Image “Neon Sign Open” by Justin Cormack, CC-BY

by Rufus Pollock at December 09, 2013 12:40 PM

In my last post we considered the idea of Antifragility and how libraries might consider making our collections more anti-fragile. Today I’d like to discuss how we take the antifragile concept further by considering an antifragile approach to discovery, as well as exploring librarianship as a tool for antifragilism.

Arguably, the whole idea of libraries, particularly public libraries, is inherently one of antifragility. The public library allows those with an interest in knowledge or ideas to pursue that interest, without requiring the individual means to fund it. The more our world changes, the more valuable the public library becomes as a source of ideas, information and inspiration. Our communities become stronger and more resilient as they share and discover old skills, new ideas and inspiring stories.

Crisis librarianship

We have seen this starkly in the US since the 2008 financial meltdown. With thousands of citizens thrown out of work and personal, state and federal budgets in chaos, public libraries in the US have never been busier. Whether requiring assistance to fill out government paperwork, internet access to apply for jobs, or simply some solace and reflection, Americans have flocked to public libraries at precisely the moment the technology utopians have been declaring their irrelevance.

Public libraries played an important role in another time of crisis in the US, when Hurricane Katrina devastated New Orleans and the surrounding area. As centralised federal and state bureaucracies flailed, public libraries in a variety of states of physical repair continued to operate, with queues of citizens arriving to gain information – about family and friends, about government assistance, food, transport and housing.

Braquet also reminds us of the utility of reading for pleasure in a moment of crisis. Witness accounts from the aftermath of Katrina show that the use of libraries as a place for mental escape and a sense of normalcy was crucial to the emotional and psychological health of the survivors. This role has been remarked upon in other contexts such as the Theresienstadt Ghetto during the Second World War (also referred to by Braquet), and during the Siege of  Leningrad (when Leningraders suddenly became interested in books about the 1812 French invasion and the Russian Civil War).

Service was not just continued during these crises – the librarians stepped up and in many ways the library became better and even more relevant in the moment of crisis.

Encouraging antifragility through better discovery tools

If our job is to encourage the anti-fragile scholar then we need to design systems to enable that. Catalogues need to provide balance between serendipity and straight search/findability – using concepts like Mitchell Whitelaw’s idea of generous interfaces. There is also huge scope to expand the use of metadata. Google Scholar, for example, lists the known articles that cite each article that comes up in a search result. Imagine a library catalogue or search tool that connected the references listed in each work to create a concept map of which books/papers contributed to which others, and which claim to be more unique works standing on their own. Whilst we have tools for academic papers (and to a lesser extent, books) that rank them in terms of ‘impact factor’, what the independent scholar/autodidact really wants to know is not so much which books or papers are most influential, but which works they don’t need to bother reading. That is, which items simply discuss the same things as something she has already read? A tool like this would be less interested in how many works cite any particular work, and more interested in which works share the same influences. In some ways this is a bit like a “Wilkin Profile“, but perhaps it is more easily understood as a Literature Map for works rather than authors. We may well also see some interesting tools emerge from the work of Franco Moretti when it comes to discovering links between various works of literature.

The fragile

Despite the natural inclination of public libraries to nurture antifragility, those of us managing them can lapse into fragilising our libraries and the communities they serve. The drive to make libraries more efficient, or more ‘modern’, often results in a much more fragile and risk-exposed institution.

Consider journal aggregators – the EBSCOs and Proquests of the world. In a rush to provide access and the newfangled technology of full text keyword searching, libraries have eventually found themselves in a perpetual hostage negotiation in order to continue to provide access to academic journal articles. It was more efficient and easier to outsource digitisation to publications and aggregators, but ultimately this has resulted in a much more brittle and expensive solution.

Now we are seeing exactly the same story play out with ebook distribution platforms. Public librarians have fallen over each other to sign up to Overdrive – a platform offering digital books with all the inconvenience of physical lending including one-copy-at-a-time lending and artificial ‘wear and tear’ – but as an extra bonus, you don’t even actually own the files! Well played, librarians.

If we look at what we’re doing from an antifragility perspective, concepts like Open Access, Creative Commons and LOCKSS become even more important. From the library’s perspective, churn in the publishing industry is fantastic for a system like LOCKSS. It means that we get the benefit of high quality publications, but when the publisher goes broke the articles are released in to the wider world. Likewise, the more change, energy and experimentation there is in the use of Creative Commons licensed works, the stronger the system becomes. Open Access more generally provides benefits in this way. Released from the restriction of one publisher enforcing one way of presenting and providing access to any given journal, article or monograph, Open Access publishing opens the field to a wide open field of new possibilities. Most Open Access experiments will fail. But the more there are, the stronger and more valuable the model becomes. If DOAJ fails, other aggregators will spring up from the ashes. If Weave collapses (and I certainly hope it doesn’t), other library UX specialists will learn from whatever led to its failure and build something better. In the meantime, some clever undergraduate is probably sitting in a dorm room right now creating a new tool to take data from multiple journals and present it in a completely new and useful way. Because they can do it without having to ask permission.

Instead of being worried about instability, change and chaos, we should realise that it is what makes libraries more valuable. We can harness this by embracing creative destruction and building our systems to be strengthened by change. After all, libraries are for use.

Tagged: antifragility, creative commons, librarianship, linked data, open access, public libraries, Wilkin profile

by Hugh Rundle at December 08, 2013 08:52 PM

A new adventure of my Penguin webcomic. Today we are going to meet Mrs Penguin.Filed under: Comics Tagged: comic, Illustrator, inking, Penguin, Photoshop, soccer, webcomic

by hochstenbach at December 07, 2013 02:56 PM

Tomorrow morning at 9AM CET, the Nobel Prize Lectures in Physics will be streamed live on the internet. François Englert will speak about The BEH Mechanism and its Scalar Boson. Peter Higg's talk is entitled Evading the Goldstone Theorem. Robert Brout, the B in BEH, will not share in the Prize. Not because he didn't make a prize-worthy contribution, but because he didn't live long enough. Brout died two and a half years ago, at the age of 82. Higgs, 84, gets the name recognition because he noticed that the "mechanism" proposed by Brout and Englert implied the existence of a particle. Englert, at 81 is the youngster of the trio. 3 other physicists Tom Kibble, Gerald Guralnik and C. R. Hagen, did pretty much the same thing, but got their paper out a bit later, so they get to be "almost Nobelists".

Ironically, Brout might have won the prize if he was a woman. Women live longer than men. But it's not as if women haven't missed out on the Nobel Prize in Physics because they were women.

Two women have been awarded Nobel Prizes in Physics, Marie Curie(1903) and Maria Goeppert-Mayer (1963). 192 men are physics Nobelists, Higgs and Englert make 194.

From what I've come to understand, there is at least one case where "being a woman" is a probable explanation for a physicist's exclusion from a Physics Nobel Prize. Chien-Shiung Wu most likely deserved a share of the Nobel Prize awarded to T. D. Lee and C. N. Yang. It's also hard to explain why another physicist, Lise Meitner, didn't share the 1944 Chemistry Prize awarded to Otto Hahn.

It's not as if the landscape for prospective prizes looks very different. Of the 42 eligible "Nobel class" physicists on Thomson-Reuters "citation laureate" list  only two are women.  (not that it's the best list or anything, but it's a list, and both Lene Hau and Vera Rubin are widely respected).

Reading through the list of 83 eminent 20th century women physicists who made their contributions before 1976, it's striking to me how many of these scientists made significant contributions despite NOT BEING ALLOWED to, or being accorded second class status of some sort.

Also I was surprised to find that many of the 83 are still eligible to win a Nobel (in addition to having made major contributions, they seem to be still around!):

This is my favorite discovery from reading about the 83 eminent physicists. It's a photo of Helen Megaw, a crystallographer whose work on perovskite minerals I had encountered in my past life as a physics researcher. It was taken on the occasion of being awarded an honorary degree a year or two before her death in 2002 at age 95, but as you can see by her expression, she totally won life.

• Betsy Ancker-Johnson 86
• Margaret Burbidge 94
• Jocelyn Bell Burnell 70
• Nina Byers 83
• Yvonne Choquet-Bruhat 89
• Patricia Cladis 76
• Esther Conwell 91
• Cécile DeWitt-Morette 91
• Louise Dolan 63
• Mildred Dresselhaus 83
• Helen Thom Edwards 77
• Magda Galula Ericson 84
• Sandra Moore Faber 68
• Judy R. Franz 73?
• Mary Katharine Gaillard 74
• Gail Gulledge Hanson 66
• Evans Hayward 91?
• Caroline Stuart Littlejohn Herzenberg 81
• Shirley Ann Jackson 67
• Renata Kallosh 70
• Margaret Galland Kivelson 85
• Noemie Benczer Koller 80
• Juliet Lee-Franzini 80?
• Marcia Neugebauer 81
• Helen R. Quinn 70
• Vera Cooper Rubin 85
• Myriam P. Sarachik 80
• Johanna Levelt Sengers 80+
• Sau Lan Wu 70?

I was unable to find information about two others on the list, Christiane Bonnelle and Janine Connes.

There may be two or three future Nobelists on this list, or there may be zero. A few of the women on the list are eminent for their contributions to society, and wouldn't be considered for a Nobel. Sometimes it takes a long time for pioneering work to be recognized as such; sometimes today's hot topic seems inconsequential or worse, wrong, 10 years later. But imagine what the list would look like without our history of discouraging women who felt the calling of physics.

by Eric (noreply@blogger.com) at December 07, 2013 01:33 PM

I have had some success converting EAD and MARC into RDF/XML, and consequently I am able to literally illustrate the resulting RDF triples.

I have acquired sets of EAD files and MARC records of an archival nature. When it comes to EAD files I am able to convert them into RDF/XML with a stylesheet from the Archives Hub. I then fed the resulting RDF/XML to the W3C RDF Validation Service and literally got an illustration of the RDF, below:

hou00096.xml –> hou00096.rdf –> illustration

Transforming MARC into RDF was a bit more complicated. I first convert a raw MARC record into MARCXML with a Perl module called MARC::File::XML. I transformed the result into MODS with MARC21slim2MODS3.xsl, and finally into RDF/XML with mods2rdf.xslt. Again, I validated the results to got the following illustration:

003078076.marc –> 003078076.xml –> 003078076.mods –> 003078076.rdf –> illustration

The resulting images are huge, and the astute/diligent reader will see a preponderance of literals in the results. This is not a good thing, but it all that is available right now.

On the other hand the same astute/diligent reader will see the root of the RDF/XML pointing to a meaningful URI. This URI will be resolvable in the near future via content negotiation. This is a simple first step. The next steps will be to apply this process to an entire collection of EAD files and MARC records. After that the two other things can happen: 1) the original metadata files can begin to include URIs, and 2) the XSL used to process the metadata can employ a more standardized ontology. It is not an easy process, but it is a beginning.

Right now, something is better than nothing.

by LiAM: Linked Archival Metadata at December 06, 2013 09:03 PM

This year’s meeting’s theme is “Technology and the academic librarian: emerging, merging, and changing the game”. I didn’t take notes on everything, but all the presentations will be recorded and available with slides (if applicable) later on.

Online Instruction: Keeping it Personal

Ania Dymarz, Life Sciences Librarian (SFU)

• Context: library going through renovation, using Canvas LMS
• keeping it personal: informal tone, picture, all the avenues of contact
• the library: providing map of 3rd floor and what’s going on
• students: had to take picture and post citation to book
• take aways: consider ‘broader’ context, add sign-up drop-in session, work for distance ed?

Teaching via Skype: The Rashomon Effect

Holly Hendrigan, Liaison Librarian for Education and Arts & Social Sciences (SFU)

• wanted to offer same opportunities to class in another city e.g. guest lectures
• explained Zotero
• students used to watching people on screen
• had some issues with bandwidth
• more questions than answers
• impression not the same as feedback from instructor and number of students that came to ask questions later

To Use or Not To Use: Should Academic Libraries be Using Web 2.0 Tool to Engage Students and Faculty?

Laura Thorne, Learning Services Librarian (UBC-O)

• study from Dec 2012 to April 2013
• mid-size research based institution in Eastern Canada
• survey:
  • student use? mostly social media websites, wikis, IM
  • engage with library? 9%
  • worthwhile to? 74% yes
  • why? quickly to share updates, convenient, already using tools, instant access to help, improves engagement
• difference in values: due to awareness, need to market more
• tools seen as convenient and access
• still need to have purpose, want relevant information, engaging and dynamic, conversation and feedback
• need to be customizable and mobile
• tools aren’t replacement for in-person contact, or website
• faculty & students interested in learning more, especially to integrate into LMS
• trying too much throwing out general information
• using skype because requested by instructor and going through LMS transition

Web Literacy Standard and Open Badges

Cynthia Ng, Accessibility Librarian (CILS @ Langara)

• blog write up

Experimenting with Shifting Sands: UBC Library’s Localized Open Online Course (LOOC) Partnership

Erin Fields, Teaching & Learning Librarian (UBC)

• flexible learning: flipped classrooms, inverted, reduced class time, online, MOOCs
• UBC MOOC
• course + digital literacy units = LOOC (like MOOC, but only UBC community)
• each unit is interactive, but in different ways e.g. forums, creating something
• UBC-endorsed Open Badge Accreditation
• M101 – collectively created, continually revised based on feedback and content, variety of ways to interact with content
• user experience guides changes to units
• behind barriers because copyright
• potential area of growth
• open education and what the library needs to address to remain relevant
• badges system: difficult to implement because how to ensure fulfilled criteria, how to evaluate
• political end = biggest quagmire
• great for professional development, because meant for faculty and staff as well
• library contributes content and units as part of the library’s service

Flexible Learning Support for a First Year Business Course

Lindsay Ure, Business Liaison Librarian (UBC)

• “information sources” tutorial for students
• framework for promotion online learning objects in new ways, and to push for funding to create more
• opportunity for high impact
• how to make content relevant to students: focused on course readings, no research assignment
• catered to learning objectives: understand why high quality sources are important
• development: storyboarding & scripts, videos, quizzes
• focused on credit sources, types of information sources, finding information, evaluating information sources
• highlight specific course readings used in the course
• quizzes: pre and post-test questions, 12 graded questions
• embedded content in LMS
• successes: filled out even if forgot to make mandatory, integrated in short time frame, instructor agreed to do again
• challenges: analysis of results was time consuming, uploading videos affected quality, quiz questions not adjusted to reflect content in videos, how much students’ applying what they learned?
• next steps: revise quizzes, reorg video content, user testing with students, pilot
• long term: include in every section, integrate with blended learning model, integrate into other courses

Qualitative Data Analysis Support for Researchers

Nicole White, Head, Research Commons (SFU)

• has support for quantitative, but not qualitative
• did environmental scan, looked at what software: Nvivo (sp?), atlasTI
• what staffing to support program
• can analyze data including social media
• graduate student access to software
• hired 2 power users (graduate students) to do 1-on-1 consultations, email support, workshops
• marketing: relied on liaison librarians, email, posting on social media, graduate studies’ communication
• faculty starting to use software in own courses
• next steps: assessment: who is downloading, using; different interests and skill level
• money: ~$25k/year for software including webinars, hourly wage to graduate support

Filed under: Academic, Events

by Cynthia at December 06, 2013 07:30 PM

This was a lightning talk presented at the ALPS December 2013 meeting. I didn’t actually have any slides for this talk, simply showing the two sites along with the backpack while talking.

Morning everyone. I’m sure you’re wondering why I have a dinosaur companion. Remember elementary school when you had show and tell? Well, really, that’s what this presentation is about: telling you about something that I’ve been keeping an eye on for over a year now, and showing you the latest iteration of something I see a lot of application for in libraries and higher education.

By the way, this dinosaur was the Mozilla mascot. I’ve brought him with me today, because Mozilla instigated and organized the efforts behind this: the Web Literacy Standard.

What is it?

Shall I simply read out what it says right here?

The Web Literacy Standard is a map of competencies and skills that Mozilla and our community of stakeholders believe are important to pay attention to when getting better at reading, writing and participating on the web.

The exploring competencies are particularly relevant to libraries, such as the “search” competency:

Locating information, people and resources via the web.

The first point here says:

Using keywords, search operators, and keyboard shortcuts to make web searches more efficient.

While this might refer to using search engines, such as Google, I would say this would easily apply to searching databases, a catalogue, or a discovery layer.

How can we use it?

So we have this standard. Where does this lead us? I think the most obvious one is for assessment.

For example, we have students that come into instructional workshops. What do we want students to learn? The standard can give us guidance specifically by looking at the competencies listed. Most likely we want to pick a couple from ‘Search’ and ‘Credibility’.

While we might differ on exactly how we assess whether students have learned these skills, we can hopefully agree that these are the skills and competencies we’re trying to assess.

I won’t belabour this point, because anyone familiar with instruction and assessment will hopefully be able to see how the standard applies in helping to create assessment tools, such as questionnaires, or rubrics.

Why this one?

What is likely of more interest is why should we use this standard?

It’s

• open, public,
• community-led,  and
• collaborative.

At the Mozilla Festival this year, faculty members from various schools showed interest and began brainstorming about how they might use it in their courses. I say that it’s collaborative, because you can provide feedback at any time and contribute, meaning everyone can get involved.

While based on many existing ideas, the web literacy standard is also unique in pulling together much of what you might find under information or digital literacy into a single standard focused on the web environment, which is where students do a vast majority of their research, as well as spend a lot of time in general.

If you weren’t using a literacy standard already, then I would encourage you to start here. If you already had something in place, then I encourage you to see how your existing instructional material might align with the web literacy standard.

Into Open Badges

Mozilla didn’t stop there of course. In a separate, but related effort, there’s also the open badges system, where you can create and issue badges linked to any of the competencies.

The metadata of each badge includes:

• badge name
• description
• criteria (link to)
• issuer
• evidence (link to)
• date issued
• competency from standard (link to)
• tags (keywords)

The badge can then be used as proof of an acquired skill. A student can also collect badges from many places and then display them on a website or any other place online by managing their badges in their badges backpack.

In the near future, part of the plan is the ability to search for other people in a community who have earned a specific badge. For example, a student might search for other students who have acquired a badge related to advanced research skills to ask for help, building a community where students can seek out peers for help in specific areas.

The badges system will also allow pathways so that you can make badges that build on one other. They’re in the process of prototyping how the pathways might work, and if you’re interested, you can even join the conversation!

Take Away

I hope you’ve enjoyed this show and tell. Too often we work in silos, so I highly encourage you to get involved in the web literacy standard, open badges or both; or simply make use of them, because you’ll be joining a community that goes beyond just one library, one faculty, or one school; you’ll be joining the world.

Filed under: Academic, Events

by Cynthia at December 06, 2013 06:25 PM

And today, the Speed Scholarship Week train rolls to a stop. For dessert, I offer you a pair of posts on two topics that are near and dear to my heart: class-action settlements and Google Books.

Here at the Laboratorium, I have a post on a pending settlement in a privacy class action that uses the same dangerous jujitsu that the Google Books settlement did. Get everyone together in one class, and then use a settlement to force them to bless a program the defendant has yet to launch. This time, instead of a bookstore, its a consumer database, and instead of escaping copyright law, it escapes from the Fair Credit Reporting Act.

And meanwhile, at Publishers Weekly, I have a post on the Google Books fair use ruling. I survey the winners (Google competitors, researchers, readers, the disabled) and the losers (the Authors Guild) and reflect on how we got to this point. The world has changed remarkably since 2005, when the lawsuit was filed, and it has changed remarkably since 2008, when the settlement was announced.

Thanks for joining me this week. I’ve had fun, and hope you did too.

by James Grimmelmann (james@grimmelmann.net) at December 06, 2013 06:18 PM

Last year, in Future Conduct and the Limits of Class-Action Settlements, I expressed grave concern about settlements like the one in the Google Books case, which tried to leverage a lawsuit about searching books into a settlement about selling them. A pending privacy case. Berry v. LexisNexis Risk & Information Analytics Group, provides another good illustration of the problem.

LexisNexis, through a subsidiary, sells a report called Accurint to help debt collectors, private investigators, and others “Detect fraud. Verify identities. Conduct investigations.” A group of attorneys brought a class action, alleging that Accurint contains the kind of personal information that can trigger the Fair Credit Reporting Act, but doesn’t comply with the FCRA’s other requirements. They claimed that LexisNexis doesn’t give consumers access to their files, doesn’t let consumers fix mistakes, and doesn’t require Accurint customers to comply with the FCRA.

Under the proposed settlement, LexisNexis would split Accurint’s service for debt collectors in two. One new service, “Collections Decisioning,” would fully comply with the FRCA; the other, “Contact and Locate,” would not. Consumers will have the right to see their Contact and Locate files once a year and to leave 100-word comments, protections well short of the FCRA’s. LexisNexis’s theory is that if Contact and Locate contains only personal information that “bears a reasonable relationship to the location of a debtor or to the location of assets securing the debt for purpose of repossession” and if customers are warned not to use the reports when granting credit, the service will fall outside the FCRA’s ambit.

There are reasons to like this new program. For consumers, it’s a substantial improvement over LexisNexis’s current you get nothing! stance toward consumer rights. Half a loaf is better than none. But to use a class-action settlement to get there is troubling. The settlement doesn’t just redesign Accurint to keep it clear of the FCRA; it also rewrites the FCRA to keep it clear of Accurint. Class members “agree that from the Effective Date until the Sunset Date that Post Settlement Products shall not be ‘consumer reports’ within the meaning of the FCRA.”

This is a future-conduct release: it forgives LexisNexis for torts as yet uncommitted. It does not settle presently existing claims the class members have against LexisNexis based on Accurint. It does not settle any claims class members will have in the future based on Accurint. Instead, it allows LexisNexis to create a new service that is different from Accurint and exempts that new service from the FCRA.

LexisNexis argues that it is “entitled to the releases” in the settlement because it “should be permitted to sell Accurint® products to its customers … without fear of harassing, repeated litigation concerning its purported obligations under an inapplicable statute.” But if LexisNexis truly believes that Contact and Locate will not be covered by the FCRA, let it launch the service and defend itself in court if necessary. The rest of us have to conform our conduct to the law, as best we are able; we bear the risk that our understanding of our obligations is mistaken. Why should LexisNexis deserve anything more? Indeed, LexisNexis stands in a worse position than the rest of us: it is able to offer this settlement only because it has been sued for violating the FCRA. If breaking the law once is my ticket to a deal that lets me break the law in novel ways in the future, sign me up.

Of course, LexisNexis and the plaintiffs’ attorneys argue that the Contact and Locate deal is simply a permissible settlement of a genuine lawsuit. But this is where the differences between Accurint and Contact and Locate become significant. It is far from clear that a lawsuit about Accurint’s compliance with the FCRA could resolve the legality of Contact and Locate.

Contact and Locate doesn’t exist. No one knows what information LexisNexis thinks has a “reasonable relationship” to debt collection. No one has seen how LexisNexis will provide access to consumer files. No one knows what Contact and Locate users will actually do with the reports they request. Maybe Contact and Locate will fall outside the FCRA, and maybe it won’t. And if it doesn’t, it is inappropriate for a court to give LexisNexis prospective immunity for systematically violating the FCRA.

In an ordinary consumer lawsuit, prospective immunity would be a matter of individual consent. Parties can craft their own settlements, just as they can craft their own contracts. Here, however, the settlement’s releases will be imposed on class members through an exercise of the court’s judgment power. Consent gives contracts their legal and moral authority, but the “consent” here is purely a fiction. Rule 23 allows classes to litigate common “claims, issues, and defenses,” and to settle those “claims, issues, or defenses.” It does not confer a general power on class attorneys to bind class members to executory contracts.

To be sure, there is no shortage of cases holding that a consent decree in a class action can go beyond the relief available at trial. But that is because an individual defendant can agree to whatever onerous terms it is willing to live with. These cases simply do not speak to the scope of the releases given by the class.

As LexisNexis itself admits, “The Settlement will establish the rights and obligations between LexisNexis and class members on a going forward basis — in the absence of existing law.” (31). We have a word for that: it’s called “legislation.” LexisNexis will have obtained the equivalent of a private bill, setting up a new FCRA-lite regulatory regime just for it. But this regulation was drafted by the party it will govern, and it is being presented for approval to the branch of government worst positioned to evaluate it. Courts considering complex prospective settlements act without any of Congress’s safeguards: political accountability, legislative fact-finding, dedicated professional staff, the power to set its own agenda, the power to amend, the power to repeal.

Courts simply do not have this sort of authority. The settlement releases class members future FCRA claims based on LexisNexis’s future conduct in launching Contact and Locate. But a federal court lacks Article III jurisdiction over hypothetical claims concerning a hypothetical future service. If I tried to bring a class action against LexisNexis today, contending that Contact and Locate will violate the FCRA, my complaint would be dismissed as unripe. It wouldn’t become ripe simply because I changed the cover sheet to say “Proposed Settlement” rather than “Complaint.”

The only way to demonstrate that the court has authority to approve the LexisNexis settlement is to show that litigating the class’s Accurint claims could plausibly determine the legality of Contact and Locate. The parties have done nothing to show that it could, and some objectors have given reasons to show that it could not. The court will hold a fairness hearing on Tuesday the 10th. I hope it will probe the scope of the releases in the settlement. If it concludes that Contact and Locate’s compliance with the FCRA is not really at stake in the underlying lawsuit, it should reject this attempt to hijack its Rule 23 authority.

(There are many other provisions in the proposed settlement, and objectors have raised quite sharp concerns about many of them. I have chosen to focus on the future-conduct-release issue because it is the one on which I have something distinctive to say.)

by James Grimmelmann (james@grimmelmann.net) at December 06, 2013 06:05 PM

We've rescheduled this free Workshop for Tuesday, 7 January at 11am Eastern.

Join us for this session where Steve Meyer will talk about the principles and implementation of the digital signatures that anchor the OCLC WSKey infrastructure. We'll also show you some sample code and leave time for Q&A. You'll come away with a better understanding of how web service keys work with our APIs, some ideas for implementation, and answers to the questions you bring.

read more

by hostetls at December 06, 2013 04:24 PM

It does count as web scale if you actually index the web, right? The danish part of the web at least, which is not at all easy to define. Nevertheless Netarkivet currently has 372 TB of harvested data, which translates to roughly 10 billion (10,000,000,000) indexable entities. We have indexed about 1/100th of this in Solr as a test and that gave us an index of about 200 GB; the full corpus should thus take about 20 TB. The question is what kind of hardware it takes to search it all?

Soft requirements

Due to danish copyright and privacy laws, we cannot provide general access to the archive. This means that usage will be limited, which again means that the number of possible concurrent searches can be set to 1. Of course 2 people can search at the same time, but it is okay if that doubles the response times. We thus only need to look at latency – i.e. the time it takes from issuing a search until a result is received.

Faceting and grouping by URL would be very useful when searching the archive, but as the only current access possibility is “Give us the resource with this specific URL”, any search at all would be a great step forward.

Hardware possibilities

Our knee-jerk reaction to hardware for search is to order Solid State Drives, but with a limited amount of users, 20 TB of SSDs might be overkill. Luckily we do have an Isilon system with some terabytes to spare for testing as well as production. The Isilon is networked storage and it is very easy to expand so it would be great if that could deliver the needed storage performance. Blade servers with virtual machines deliver the CPU horse power, but as we shall see it really is all about IOPS (In/Out Operations Per Second).

SolrCloud

Building one single 20 TB index would be an intriguing endeavour but alas, only 2 billion documents/segment in Lucene/Solr. Besides, this would be quite devastating in case of index errors. Furthermore there is a high chance that it would provide lower performance compared to multiple indexes as a lot of the internal seeking operations for a single index are sequential.

Some sort of distribution seems to be the way forward and SolrCloud is first candidate (sorry, ElasticSearch, maybe next time?). We need to estimate what kind of iron it takes to handle the 20TB and we need to estimate a proper shard-count vs. shard-size ratio. Is 100 shards of 200GB better than 10 shards of 2TB? Since our content is ever-growing and never-updating we have the luxury of being able to optimize and freeze our shards as we build them one at a time. No frustrating “index time vs. search time”-prioritization here.

Numbers, please

Real Soon Now, promise. First we need to determine the bottleneck for our box. CPU or IO? To do so we measure cloud sizes vs. number of concurrent searches. By taking the response time and dividing by (#threads * #shards), we get the average response time for the individual shards. If that number plateaus independently of #threads as well as #shards, we have the maximum performance for our box.

For query testing we do an OR-query with 2-4 random danish words and light faceting. This should be representative of the real-world queries, at least for IO. As we cannot flush the cache on the Isilon box (Operations would kill us if we managed to do so), each test is preceded by a short (~100 queries) warmup run and a hope that the numbers will be consistent. Collected statistics are based on Solr-reported Query-Times and collected for all threads. We checked and that information is very close to full response times.

Confused? No worries, it gets a lot clearer when we have numbers.

Spinning numbers!

The Isilon system used spinning drives with a fair amount of combined SSD and RAM caching in front. Remember that reported times are response_time/(#threads * #shards) ms.

Testing with 200GB (75M documents) indexes, Median (50% percentile), remote Isilon

	Threads
Shards	1	2	4	8	16	32	avg
1	28	10	4	2	1	1	7
2	99	47	35	34	34	34	47
3	77	42	36	38	41	45	47
4	67	47	47	54	52	56	54
5	67	57	54	54	53	56	57
6	64	56	53	58	59	58	58
7	63	56	53	50	55	55	55
8	72	60	58	60	62	61	62
9	71	60	55	57	58	56	60
10	71	58	57	57	56	54	59
11	74	65	61	60	58	62	63
12	70	61	58	61	60	58	61
12	73	63	65	67	70	69	68
13	83	66	67	74	70	65	71
14	78	71	68	71	72	80	73
15	85	75	70	70	70	78	75
16	74	68	68	67	66	75	70
17	78	68	70	70	70	79	72
16	77	71	69	70	70	80	73

Testing with 200GB (75M documents) single segment indexes, 99% percentile, remote Isilon

	Threads
Shards	1	2	4	8	16	32	avg
1	157	55	33	25	14	39	54
2	230	104	87	85	69	68	107
3	208	98	93	82	77	84	107
4	166	111	109	104	88	95	112
5	152	129	110	93	90	94	112
6	128	130	110	100	96	98	110
7	125	124	98	85	91	92	103
8	128	113	99	104	106	95	107
9	151	107	97	94	91	99	106
10	121	106	97	100	88	90	100
11	136	109	107	102	94	109	109
12	117	109	100	100	94	95	102
12	139	108	139	107	143	174	135
13	140	111	158	121	116	112	126
14	134	119	108	122	131	119	122
15	154	152	123	120	128	116	132
16	121	109	107	100	112	104	109
17	124	115	111	112	122	117	117
16	126	119	112	104	122	118	117

Testing with 420GB (174M documents) single segment indexes, Median (50% percentile), remote Isilon

	Threads
Shards	1	2	4	8	16	32	avg
1	17	9	6	3	2	1	6
2	81	50	42	42	41	44	50
3	91	52	44	45	44	51	54
4	83	57	53	54	52	56	59
5	77	59	54	54	55	56	59
6	79	59	55	53	54	53	59
7	76	61	57	57	55	55	60
8	77	61	56	58	59	60	62
9	75	64	64	64	63	63	66

Testing with 420GB (174M documents) single segment indexes, 99% percentile, remote Isilon

	Threads
Shards	1	2	4	8	16	32	avg
1	154	85	70	38	18	15	63
2	255	126	105	97	78	72	122
3	207	165	103	86	79	79	120
4	181	118	102	98	92	82	112
5	164	114	100	91	88	83	107
6	168	123	95	90	80	78	106
7	146	117	93	80	81	85	100
8	150	108	98	91	83	92	104
9	134	114	102	103	97	107	109

Isilon observations

The average response time/shard/thread gets markedly better when going from 1 to 2 threads. This makes sense, as a single-thread is alternating between CPU-wait and IO-wait, thus not utilizing both resources fully. More threads does not improve performance by much and in a lot of cases, 32 threads are slower than 16 threads.

Our goal was to estimate the hardware requirements for the full 20TB index files. We see from the tables above that the average response times for the individual shards to grow with the number of shards and the number of threads, but not critically. The average of the medians for 8 shards of 200GB is 62 ms, while it is 73 ms for 16 shards; this is a 17% increase for double the amount of data. With a 20TB estimate, we would need 100 shards. Expecting about 20% increase for each doubling of the shard count, the average shard response time would be approximately 115 ms for a total of 11,500 ms.

This might be somewhat optimistic as the cache works significantly better on smaller datasets and because the average median is lower than the single thread median, which is more telling for the expected usage pattern.

Looking at the average median for 420GB shards, we have 59 ms for 4-5 shards and 66 ms for 9 shards, or an response time increase of only 11% for a doubling of the index size. Extrapolating from this, the full corpus of about 50 shards would have an average shard response time of approximately 85 ms for a total of 4,250 ms. As this is less than half of the estimated time for the 200GB shards, this seems suspiciously low (or the estimation for the 200GB shards is too high).

Those were medians. Repeating for 99 percentile we get 14,500 ms for 200GB shards and 5,500 ms for 420GB. Again, the 420GB measurements seems to be “too good”, compared to the 200GB ones.

Solid numbers

We do not have terabytes of unused space on Solid State Drives lying around, but a temporary reorganizing on our test machine freed just above 400GB. Enough for testing 2 * 200GB. It is important to keep in mind that this storage is both local & SDD as opposed to the Isilon box which is remote & spinning drives.

Note that the two rows in each test are for the same number of shards (2). This was done to see whether warming/caching improved response times.

Testing with 200GB (75M documents) indexes, Median (50% percentile), local SSD

	Threads
Shards	1	2	4	8	16	32	avg	total GB	avg full
2	20	8	4	3	2	3	7	400	13
2	16	8	4	3	2	2	6	400	12

Testing with 200GB (75M documents) single segment indexes, 99% percentile, local SSD

	Threads
Shards	1	2	4	8	16	32	avg	total GB	avg full
2	64	35	31	14	10	8	27	400	54
2	36	30	26	10	8	13	21	400	41

Solid observations

As opposed to spinning drives, the performance continues to improve markedly with threading up to around 16 threads. Keeping in mind that this is on a 16 core machine, this tells us that we do not have enough data to be sure whether the SSDs or the CPUs are the main bottleneck.

Note to self: It would be interesting to try and limit the number of active CPUs by using taskset.

One thing we can see is that the SSD-solution seems to be much better performing than the spinning drives one. But how much is much?

Putting it all together

With only enough free SSD for 2*200GB shards, we are forced to project the final load by threading. Our maintenance guys very temporarily freed 400GB of SSD on the Isilon box so we were able to see whether the local vs. remote parameter had a lot of influence on performance.

Keep in mind that this is a very loose projection and especially keep in mind that bigger shards are relatively faster while bigger test data are relatively slower. Still, the conclusion seems very clear: Using SSDs as storage is a whole other level of performance, when only a small part of the index can be cached in RAM.

Sweet sweet hardware

We all know that SSDs are expensive. Except that they are not. Yes they cost 10 times as much as spinning drives when looking at capacity only, but when the whole point about SSDs is speed, it is plain stupid just to look at capacity.

As our analysis above shows, scaling to satisfactory response times (median < 1 second in this case) with spinning drives would require 5-10 times the speed we currently get from Isilon. As the Isilon box is already using RAID for speed improvement (and crash recovery), this seems like quite a task. Compensating for spinning drives by increasing the amount of RAM for caching is unrealistic with 20TB of index.

It is simply cheaper and faster to base our search machine on SSDs. We can handle crashes by mounting each drive separately and using shards the same size as each drive. If a drive fails, the corresponding shard backup (which will probably be located on Isilon) takes over until the shard has been copied to a new SSD. Our hardware guys suggests

• 1x 24×2.5″ Intel JBOD2000 2U RACK, Dual PSU, dual backplane
• 24x 1TB Samsung 840 EVO
• 1x Intel RS25GB008 JBOD controller with 8 external ports
• A server to hold the controller

Estimated price 100-200,000 danish kroner or 20-40,000 USD, depending on RAM (suggested amount: 256 or 512GB as we expect to to heavy faceting and grouping). On top of that we need a dedicated indexing machine, which can also be used for other tasks.

by Toke Eskildsen at December 06, 2013 12:15 PM

One of the reasons for the slowing of Kryder's Law has been that the investment needed to get successive generations of disk technology into the market has been increasing. Assuming per-drive costs and volumes are approximately stable, this means that a technology generation has to stay in the market longer to recoup its development costs. Thus, even if the proportional density increase in each generation is the same, because the generations are spaced further apart, the result is a slower Kryder's Law.

Henry Samueli, CTO of Broadcom, makes the same point about Moore's Law. As the feature size of successive chip generations decreases, the cost of the manufacturing technology increases. And the techniques needed, such as FinFET and other 3D technologies, also slow down and increase the cost of using the manufacturing technology:

Process nodes themselves still have room to advance, but they may also be headed for a wall in about 15 years, Samueli said. After another three generations or so, chips will probably reach 5nm, and at that point there will be only 10 atoms from the beginning to the end of each transistor gate, he said. Beyond that, further advances may be impossible.
"You can't build a transistor with one atom," Samueli said. There's no obvious path forward at that point, either. "As of yet, we have not seen a viable replacement for the CMOS transistor as we've known it for the last 50 years."
... the ongoing bargain of getting more for less eventually will end, Samueli said. "We've been spoiled by these devices getting cheaper and cheaper and cheaper in every generation. We're just going to have to live with prices leveling off," he said.

Both of these are simply applications of the Law of Diminishing Returns.

by David. (noreply@blogger.com) at December 06, 2013 10:12 AM

It’s day four of Speed Scholarship Week, and the bus is still safely above 50. Today, I give you Anarchy, Status Updates, and Utopia, an essay about Bitcoin, /r/politics, Vi Hart’s YouTube channel, and other experiments in online governance. I argue that technical power—a platform operator’s ability to force changes on users—is far more deeply embedded in social software than we think, precisely because it is social. To use software together, we have to agree on what software to use, and that social process of agreement necessarily gives someone technical power. A commitment to fair treatment and democratic values, where it exists at all, comes from a community of users, not from the software they use.

The paper is my response to a strain of what I think of as technological libertarianism: the belief that freedom online can be fully protected if only we get the software right. I’m sympathetic to the goal, but the means don’t work. Software doesn’t exist in a vacuum; it serves human goals and is controlled by humans. Bitcoin’s protocols won’t protect you if enough other users agree to switch to a different blockchain. To talk about whether a design protects users from each other, we need to talk about the users, and not just the design.

I’ve been working on this paper, on and off, for six years. I’ve presented earlier versions at conferences, but never quite felt that I had it right. Now I do—or at least I’m close enough to share a draft and seek your comments. Here’s an excerpt:

But focusing on technical power raises its own question: why didn’t Marc Bragg and Mailpile head for the exit when things got bad, the way Vi Hart did? Yes, Second Life and PayPal changed the way their systems worked. But so what? Database entries only matter if they control your access to something that matters in the real world. Technical power only has bit to the extent you use a software system; walk away from the keyboard and the software can’t follow.

To understand where this argument goes wrong, consider what it suggests for our disappointed victims of technical power. Marc Bragg didn’t need Second Life: he could have drawn a picture of Taessot on a napkin and continued to enjoy his imaginary property. And Mailpile didn’t need PayPal; it could have drawn pictures of Benjamin Franklin on napkins and used those. You don’t need Facebook; just take a Sharpie to your living-room wall. You don’t need YouTube for cute cat videos; just film your own damn cat.

These suggestions are so unsatisfying because they miss the inherently social nature of social software. The fun, and the value, of these systems comes from sharing them with others. YouTube’s other users provide me with better cat videos than I could film for myself; Facebook tells me what my friends are actually up to, not just what I imagine they’re up to. Countless online journalists use social platforms to publish their work. Virtual property in Second Life, like a domain name or like a LinkedIn account, is valuable in only because it’s networked. To withdraw from the network in which the property is embedded is to give up something of real value, however virtual the property itself may be. (Mailpile’s frozen funds on PayPal are no more real, and no less, than any other form of money.)

This, then, is a point about social power: the authority enjoyed over any community by the person or entity who controls the terms on which the community comes together. The threat to boot you from YouTube if you don’t accept Google+ comments isn’t just about cat videos: it’s also about the people who make and watch those cat videos. The threat to boot you off of a mailing list isn’t just about the emails; it’s about your access to the other people on the mailing list. The threat to boot you from eBay isn’t just about the stars next to your name; it’s about the community of people who know what those stars mean, who give those stars their meaning.

I will be presenting Anarchy, Status Updates, and Utopia at the “Social Justice and Social Media” symposium at Pace Law School in March, and the essay will appear in a symposium issue of the Pace Law Review.

by James Grimmelmann (james@grimmelmann.net) at December 05, 2013 04:46 PM

This post was written by Pieter Colpaert, a member of the Open Knowledge Foundation Belgium Chapter.

The DataTank is open source software, just like CKAN, Drupal or Elastic Search, which you can use to transform a dataset into an HTTP API. Today (the 5th of December 2014), we are proud to launch the 4.0 version on which professional support will be provided. The project was started in 2008 by one of the founders of Open Knowledge Foundation Belgium. Today it remains mainly developed by OKFN Belgium, but we are welcoming new contributors from all over the world.

To get an idea of what The DataTank can do, check http://thedatatank.com or our demo server: http://demo.thedatatank.com.

With this new version of The DataTank, we hope that hackathon developers will have a tool to set up an API for their developers in no time, that start-ups will be able to combine different Open Datasets from all over the world in one Web service without trouble, that Open Data Portal developers are going to integrate The DataTank with CKAN, and that data owners are going to see a faster return on investment from publishing their data.

The platform is written in PHP using the Laravel framework. If this is a language you speak, feel free to dig in and fork us on github: http://github.com/tdt/core

Oh, did we mention it also publishes RDF where suitable? http://demo.thedatatank.com/api/dcat

If you want to use The DataTank at your organisation but you’re not a technical person, we can help you! Contact our team at info@thedatatank.com

by Guest at December 05, 2013 08:56 AM

Today, the American Library Association is joining a nationwide day of action calling for reform of the Electronic Communications Privacy Act (ECPA), the law that says the government can access your email and documents in the cloud without a warrant.

ECPA is one of the Internet’s most outdated laws – it was enacted in 1986, before most people had access to a home computer or email. While the public has been rightfully outraged over reports that the NSA accesses communications without a warrant, ECPA says that hundreds of other government agencies—like the IRS, FBI, and DEA, as well as state and local law enforcement agencies—can access many of our stored emails, private social media messages, and documents in the cloud without getting a warrant from a judge. The law flies directly in the face of our Fourth Amendment values.

Bills to reform ECPA have gained huge support in recent months from both parties in Congress. However, legislation is now being blocked by a power grab from the Securities and Exchange Commission, which is pushing for a special carve-out for regulatory agencies to get your documents from online providers without a warrant. The SEC carve-out would neuter ECPA reform.

That’s why we’re calling on the White House to break its silence and stand up for ECPA reform. We need President Obama to tell the SEC to back down in its demands for troubling new powers and make clear that the time for ECPA reform is now.

Today we ask you join us by signing this petition to the White House. It’s time for the President to join hundreds of tech companies, startups, advocates, and Members of Congress by supporting this commonsense, long overdue reform to ensure our privacy rights online.

Participating organizations:

Access
ACLU
Americans for Tax Reform
American Library Association
Center for Democracy & Technology
Demand Progress
Disconnect
Electronic Frontier Foundation
Engine Advocacy
Fight for the Future
Free Press
Google
Internet Infrastructure Coalition
Liberty Coalition
Open Technology Institute
R Street
SpiderOak
TechFreedom
Twitter

The post Day of Action to Demand ECPA Reform appeared first on District Dispatch.

by Jazzy Wright at December 05, 2013 07:17 AM

I hope somebody can give me some advice for transforming MARC to RDF.

I am in the midst of writing a book describing the benefits of linked data for archives. Archival metadata usually comes in two flavors: EAD and MARC. I found a nifty XSL stylesheet from the Archives Hub (that’s in the United Kingdom) transforming EAD to RDF/XML. With a bit of customization I think it could be used quite well for just about anybody with EAD files. I have retained a resulting RDF/XML file online.

Converting MARC to RDF has been more problematic. There are various tools enabling me to convert my original MARC into MARCXML and/or MODS. After that I can reportably use a few tools to convert to RDF:

• MARC21slim2RDFDC.xsl – functions, but even for my tastes the resulting RDF is too vanilla.
• modsrdf.xsl – optimal, but when I use my transformation engine (Saxon), I do not get XML but rather plain text
• BIBFRAME Tools – sports nice ontologies, but the online tools won’t scale for large operations

In short, I have discovered nothing that is “easy-to-use”. Can you provide me with any other links allowing me to convert MARC to serialized RDF?

by LiAM: Linked Archival Metadata at December 05, 2013 03:19 AM

Please join the LITA Membership Development Committee and members from around the country for networking, good cheer, and great fun! Expect lively conversation and excellent drinks. Cash Bar.

• Sunday, January 26, 6:00 pm – 8:00 pm
• Bar-ly, Chinatown; 101 N. 11th St, Philadelphia; (215) 922-2688

by mprentice at December 04, 2013 09:41 PM

It’s day three of Speed Scholarship Week. Today’s installment is my take on Massive Open Online Courses, or MOOCs. Over the course of 2012, I went from being a great enthusiast of the idea to frustrated and disappointed with them. The paper—The Merchants of MOOCs—is my attempt to understand why.

In a nutshell, today’s MOOCs do not actually solve the problems they claim to solve, because very little about them is new. If the MOOC mass-media model worked at scale, the Sunrise Semester or the correspondence course or Fathom.com or their other antecedents would already have disrupted higher education in the way MOOCs allegedly will. At the same time, though, the “open” that makes MOCs into MOOCs is tremendously exciting, because it can make education far more accessible and far more participatory. What has gone wrong is that MOOCs have been hijacked by the entrepreneurs and investors who see them in the financial terms of Silicon Valley startup culture. The question is not how MOOCs can make money, but rather how to sustain open education, regardless of whether it makes money.

Here is an excerpt from early in the paper:

The first claim about MOOCs is that they will allow all students to study with the very best professors. Thousands of Joe Coursepacks teach introductory calculus every year. Some of them are good; some are terrible. Replace them with a single MOOC, and it can be assigned to the clearest and most engaging lecturer. As David Brooks put it, “a few star professors can lecture to millions.”

There’s just one problem. We already have lectures from elite professors for the masses, on calculus and on many other subjects. They’re called “The Great Courses” and they come in an affordable package of 24 videos for a special limited-time price of $59.95.16 The “massive” in “MOOC” is the same as the “mass” in “mass media”: peo- ple have been using broadcast technologies to deliver education for decades. From 1957 to 1982, CBS aired Sunrise Semester, a half-hour program in the early morning featuring NYU professors delivering college-level lectures. NBC’s answer was Continental Classroom, which ran from 1958 to 1963. Nicaragua used radio for distance education in mathematics starting in 1974; dozens of countries followed its lead. The MOOC format adds little to the tools already at hand.

If anything, the MOOCs of today fall rather short of their predecessors. A recent article in The New Yorker offers a revealing look inside the making of one of Harvard’s MOOCs, “The Ancient Greek Hero.” The day before the course went live, the videos for the first lecture weren’t finished. The main video editor was a classics PhD, but don’t worry, she was trained in “digital storytelling” by Harvard’s “MOOC video guru.” And the professor, Gregory Nagy, was planning to bring a cameraman on his spring break trip to Greece to film the mists at Delphi. Why, we might ask, is the Francis Jones Professor of Classical Greek Literature scrambling to get second-rate B-roll footage? And do we really think that the resulting videos will be the pinnacle of pedagogical achievement in teaching ancient Greek literature?

And here is an excerpt from later on:

One advantage MOOCs have over these various resources is structure: the “C” stands for “course,” as in “prescribed course of study.” When you listen to Mike Duncan’s podcasts, you’re on your own: no one will notice or care if you give up after a week. But a MOOC has a meaningful sequence of checkpoints and deliverables to help students tie themselves to the mast. These is something to this point, but the contrast between MOOCs and open educational resources should not be overstated. On the one hand, MOOCs’ commitment mechanisms also often fall short. Nearly six out of seven of the students who started the Stanford AI course failed to finish, and when A.J. Jacobs signed up for eleven MOOCs for a New York Times experiment, he completed the “two courses with lighter workloads and less jargon.” On the other hand, nothing prevents layering the checkpoints and other work of a “course” on top of open resources. Many teachers who integrate the Khan Academy into their class- rooms customize how they draw on it for each student. MOOCs bundle student supervision with course content, but in an unbundled world, even that union can be questioned.

The “openness” of these other creators and communities is of an entirely different order than the openness of MOOCs. It is the freedom to take content and build on it, remixing it into other educational resources. It is the freedom to dive in and out of topics, pulling them together in ways that don’t follow the fixed rhythms of a college course. And, most of all, it is the freedom to join in, not just as a student but as a teacher, moving back and forth between learning and sharing what you have learned as you collaborate with others from around the world on their own diverse educational journeys. MOOCs are charismatic megafauna, but open education is an entire ecosystem.

The Merchants of MOOCs was written for a symposium on Legal Education Looking Forward at Seton Hall, and it is forthcoming in the Seton Hall Law Review.

by James Grimmelmann (james@grimmelmann.net) at December 04, 2013 07:24 PM

Peter van Buren understands the 1984 analogy that drove us to assume a very powerful adversary when we designed the LOCKSS system a decade and a half ago.

by David. (noreply@blogger.com) at December 04, 2013 12:43 PM

In Brief

The library catalog as it exists today is a century-old tool that presents an array of challenges for its users. The manner in which users search for information has not changed since the inception of the paper catalog and its different indices. The electronic records that comprise library catalogs are in a format that is largely the product of cataloging standards and practices designed to replicate printed catalog cards. Additionally, the syntax and terminology of the controlled vocabularies in the catalog are generally alien to its users. We live in an age of Google, and our catalogs should reflect the information-seeking behavior of today’s user, not the user of one hundred years ago.

---

Cutter and the Modern Catalog

The format of the catalog we have today was largely set forth in Charles A. Cutter’s Rules for a Dictionary Catalog, originally published in 1875. In that work, Cutter suggested that the catalog should “enable a person to find a book of which either the author, the title, or the subject is known.”¹ These suggestions mirror the typical indices found in the majority of catalogs today – author, title, or subject while also including an index of classification numbers (another suggestion of Cutter’s). Cutter fully expressed these ideas in his work with Ezra Abbott at Harvard College to create the first known card catalog.² Indeed, one interacts with these indices in a similar manner to interacting with the separate card catalogs for author, title, subject, and shelflist.

Beyond the methods of indexing and searching, the very format of library metadata in the catalog is dictated by the “analog” catalog. The determinant for structuring MARC was its direct predecessor – the catalog card itself. In her book Henriette Avram, creator of the MARC metadata schema, acknowledges that the catalog card shaped the format of MARC as the Library of Congress continued producing cards for distribution, as well as retrospectively converting existing bibliographic data in library catalogs into MARC format. If one looks at a catalog card and a record in an electronic catalog, they are remarkably similar. Beyond the metadata schema itself, the rules for description of items  perpetuate the formatting of data to make entries fit all on one card through standardized abbreviations seen in AACR2.³

The knowledge and tools needed to successfully navigate the catalog can be difficult to learn and so are rarely used by the majority of library patrons. For example, the searching help information at the University of Arkansas extends over seven different pages. Searching by subject headings can be frustrating and confusing, even for librarians. Perhaps a patron is looking for a book about early maps of the arctic regions, and so chooses to search the subject index in the catalog. The patron tries different permutations of headings “Maps — Arctic regions” or “Early maps — Arctic regions” and after a few more attempts, gives up in frustration. The “correct” heading for this is, confusingly, “Arctic regions — Maps — Works to 1800.” Not only are the terms themselves esoteric, the syntax of subject headings is a syntax unknown to many.

Despite its limitations, the library catalog has served the information seeking behaviors of its users well for the past century. If a user knows how to manipulate and structure search entries, the electronic catalog in its present form is an excellent discovery tool. Indeed, the catalog as it exists today is a reflection of a century of adjustment and manipulation by librarians and patrons. The concepts of controlled vocabulary, shelflist browsing, and convenience of searching an index are key contributions to access made by catalogers and the catalogs they create. Beyond these concepts, the curated and authoritative metadata generated by catalogers is also of significant value in the keyword search centered information seeking behaviors of internet and library users.

However, many information seekers prefer a Google search to a search in the library catalog. Based on colloquial evidence, most librarians can affirm that Google, not the library catalog, is the “first line” for information discovery. Furthermore, in a recent post on the Chronicle of Higher Education’s website, Brian Mathews further attests to user frustration with the current structure of library catalogs and their inherent difficulties in navigation and use. Google provides a single search interface for a wide variety of resources. With one search, a user can discover images, webpages, Google Scholar, Google Books and a wide variety of other online resources. Search strategies can be very informal and still return reasonably applicable results. Many libraries are attempting to address this preference through the implementation of discovery engines, such as Summon and Encore, that offer a single search interface for the information resources accessible through the library. Google is now the default resource for information seeking – so much so that it is not uncommon for librarians to encounter individuals who assume that libraries will be obsolete because of Google’s dominance.⁴

It is perhaps now a trope to refer to libraries in the age of Google. However, this does not obviate the evidence that libraries do indeed exist in an age of Google, and that by and large, individuals seeking information see Google as their primary resource, their first line of inquiry. Naturally this unnerves many librarians who see libraries and the collections they maintain as an authoritative, and perhaps definitive, repository of knowledge that can answer a broad swath of inquiries. What, then, can librarians do to address this preference for Google as a device for information discovery?

BIBFRAME

Not since the advent of the catalog as we know it today have librarians been able to fundamentally rethink the nature of the catalog and how it is used to serve the information seeking behaviors of its users. This ability comes from the ongoing development of the metadata schema referred to as BIBFRAME, which is powered by linked open data. BIBFRAME will help to free library metadata from the silos in which it has been kept for fifty years, and will also allow this data to be interoperable with a far wider web of data, in addition to allowing for a re-thinking of the design and arrangement of the catalog. Furthermore, the change in rules for metadata creation, represented by RDA, will also help library metadata to be more useful, though perhaps to a lesser extent than BIBFRAME. RDA is designed as a rule set for entering metadata into many different schema – MARC, Dublin Core, and BIBFRAME. However, without the changes that BIBFRAME represents, RDA is largely a cosmetic change to library metadata.

BIBFRAME and Linked Open Data

In May 2011, the Library of Congress announced an initiative to examine the possibilities of moving library metadata in legacy formats (chiefly MARC) into a wider web of data and knowledge available on and supported by the internet. In a report dated November 21, 2012, the Library noted that MARC has been an incredible success story in facilitating the machine automation of many library functions. However, the authors of the aforementioned report on BIBFRAME point out that MARC is perhaps outdated, and it is the responsibility of librarians to ensure that library metadata is fully integrated with existing metadata and web standards. In response to this need, the Library of Congress has put forward BIBFRAME (short for Bibliographic Framework) as a recommended replacement for MARC as a metadata schema. BIBFRAME, working in the arena of linked open data, will move library metadata into a much broader network of information. Indeed, the report highlights this change:

As libraries become part of this larger web of data, by leveraging the use of stable identifiers to reference clearly differentiated entities, focus will shift from capturing and recording descriptive details about library resources to identifying and establishing more relationships between and among resources. This includes related resources found on the web, and especially those beyond the traditional bounds of the library universe. These relationships – these links – drive the web, transforming the information space from many independent silos to a network graph that branches out in every direction. Relationships help search engines and other services to improve search relevancy and, most importantly, help users find the information they are looking for.

This quote implies, but does not specifically state, BIBFRAME’s use of linked open data for building web links and metadata into a larger “semantic web”. Proposed by Tim Berners-Lee in 2001, linked open data is the supporting framework for a larger movement called the semantic web. The semantic web, supported by linked open data, aims to describe the relationships between linked items on the internet. The vast majority of the internet is structured with links that do not describe the nature of the links between pages or objects – a typical hyperlink. In their article on the state of the semantic web, Bizer, Heath, and Berners-Lee highlight this by stating “In the conventional hypertext Web, the nature of the relationship between two linked documents is implicit, as HTML is not sufficiently expressive to enable individual entities described in a particular document to be connected by typed links to related entries.”⁵ The semantic web aims to make information use and discovery a richer and more immersive experience through the description of these links – their semantics. Supporters of the semantic web, and linked open data, see a changed information landscape with the implementation and adoption of the semantic web. Computers and networks would be aware of the nature of the links between objects – that an author created a book, as opposed to a simple link between two resources. As Bizer et al describes, “Whilst HTML provides a means to structure and link documents on the Web, RDF (resource description framework) provides a … model with which to structure and link data that describes things in the world.”⁶ Machines could then suggest related items or ideas based on what a user is seeking, and chooses to display. This concept has the potential of creating a far more contextual experience for users engaging in information seeking and discovery.

Two examples of the possible benefits of placing library metadata in the linked open data arena appear in an essay in Future Internet, as well as in an interview with Thea Lindquist, Associate Professor and History Librarian at the University of Colorado Boulder. The usefulness of this web of linked open data is further described in terms of possible impact for library catalog users:

Jane:  How do you describe to people what semantic computing might do for them?

Thea: Usually I say that it associates related concepts, increases findability, context and interoperability, enables semantically rich services (like faceted searching, content recommendations, and visualizations) and allows re-use, re-mixing and re-presenting of data. If they look puzzled, I start by comparing the current web of documents to the web of data. When you search for a term on the web of documents, the computer looks for the string of characters you entered, and it has no idea what the meaning associated with those characters are. When it finds matches, it returns the documents in which they are found, and it is up to you to slog through those and figure out if any of the matches are indeed relevant. If you look for “buck”, you could get documents about a male, antlered animal, a dollar, throwing (a rider) by bucking, giving someone a ride on your bike (this usage may be limited to Minnesota)…you get the picture. On the web of data, supported by ontological structures and intelligent applications, the computer can understand the word “buck” might have different meanings and what those might be, and it will ask you “are you interested in the monetary unit?” (among other things). If you say yes, it will direct you to the relevant data residing within documents rather than the entire document, whether the character string says “buck”, “dollar” or “single”.

As a further illustration, the BIBFRAME document itself describes the possibilities for libraries and users thusly:

As libraries become part of this larger web of data, by leveraging the use of stable identifiers to reference clearly differentiated entities, focus will shift from capturing and recording descriptive details about library resources to identifying and establishing more relationships between and among resources. This includes related resources found on the web, and especially those beyond the traditional bounds of the library universe. These relationships – these links – drive the web, transforming the information space from many independent silos to a network graph that branches out in every direction. Relationships help search engines and other services to improve search relevancy and, most importantly, help users find the information they are looking for.

As BIBFRAME structures library metadata in a way that is easily manipulated by machines, this new schema will free that metadata from the silos in which it has been stored for decades. The nature of library metadata structures has prevented the vast amounts of metadata created by catalogers and librarians over the past forty years from being easily used by tools outside of the library sphere. BIBFRAME is built on XML (eXtensible Markup Language) and RDF (Resource Description Framework), both “native” schemas for the internet. The web-friendly nature of these schemas allows for the widest possible indexing and exposure for the resources held in libraries. Metadata could be indexed by major search engines, and incorporated into conglomerations and indices of linked open data, such as Europeana. As a stopgap measure, Blacklight implementation in the local catalog is useful for exposing library metadata to search engines, but this exposure is done after the fact, and is not integrated into the schema (MARC) itself, creating additional ongoing work for catalogers. BIBFRAME has the potential to “build-in” the ability for library metadata to be indexed easily by major search engines. This ease of indexing and “freeing” of library metadata has the potential to bring users back to the catalog, and back to library websites, as authoritative starting and ending points in information seeking. The addition of the great mass of authoritative library metadata potentially will not only enhance users’ searches through search engines, but those results will also drive users back to the source of that authoritative data – library catalogs. Indicative of the “freeing” power of linked open data, John Overholt’s essay titled Five Theses on the Future of Special Collections highlights the need for library metadata to become part of the linked open data web in his referencing Europeana, to date one of the largest implementations of linked open data in a more user-friendly format. Furthermore, moving library metadata to a schema that is web-native also gives an added benefit to the user that this metadata is easily manipulated by widely available tools and programs for conducting research in the metadata itself. In a blog post on Religion in American History, Lincoln Mullen highlights the potential for research in the metadata, as well as the challenges attendant to conducting that research with metadata structured in MARC. Beyond this “freeing” of library metadata from the catalog for indices and users, the use of web-friendly schemas has other positive impacts for the user and libraries.

During a presentation and a question and answer session hosted by OCLC, Roberta Shaffer of the Library of Congress shared additional information about BIBFRAME. BIBFRAME is currently in an early testing phase. However, if librarians and other interested parties wish to participate in the early phases of this new schema, there are several options to explore at the BIBFRAME website. Perhaps the simplest way to get involved with the new schema is to sign up for the ListServ via email. Also on this website, one can see library metadata structured in BIBFRAME, as well as being able to convert MARC structured metadata into BIBFRAME through online tools. However, as Roberta Shaffer indicated, much of this work is in the early stages and is experimental. To this end, OCLC released a report in June of 2013 that explores the use of BIBFRAME in OCLC’s WorldCat database.

BIBFRAME and the Web

Because of its use of XML, BIBFRAME also has the advantage of being vastly more web-friendly than MARC, and so catalogs could be structured far more differently than they are now, and far more attentive to the needs of catalog users. Librarians have a professional and ethical obligation to their users, as the code of ethics of the American Library Association states: “We provide the highest level of service to all library users through appropriate and usefully organized resources.”

Lorcan Dempsey, Vice-President and Chief Strategist of the Online Computer Library Center (OCLC), recently posted an essay exploring the potential of the library catalog to not only be a tool for local discovery, but also for discovery of items beyond the library’s collections. The breadth of items in Dempsey’s example provides the user with a far broader array of information resources – a process that has the potential to be automated with the adaptation of library data and catalogs to linked open data. However, additional resources do not directly correlate to a better user experience for catalog users. Librarians must look not only to the resources described in the catalog, but also to the user experience.

On Tufte and Catalogs

Perhaps a useful starting point for librarians and designers to think about the design of catalogs powered by BIBFRAME is the work of Edward Tufte. Tufte is perhaps the best theoretician in modern user-centered graphic design. His four books on the display of quantitative information provide some useful guidelines for designers of these BIBFRAME powered catalogs of the future. Tufte is most closely identified with his work in visual displays of quantitative information, but his general ideas and theses can be applied to catalog design, which is the display of information. Indeed, Tufte says, “The design of statistical graphics is a universal matter – like mathematics – and is not tied to the unique features of a particular language.” However, most pertinent to the designer of the catalog is this quote from Edward Tufte:

What is to be sought in designs for the display of information is the clear portrayal of complexity. Not the complication of the simple; rather the task of the designer is to give visual access to the subtle and the difficult – that is, the revelation of the complex.⁷

A catalog displays a vast array of complex relationships in a way that users can understand. At this time, the navigation of these complex relationships is difficult for many users. Having a single search interface is an acceptable beginning to a better catalog, but this is not the only solution to the problem, nor is it the exclusive solution. A user searching these relationships must search authoritative metadata – the metadata structured by both BIBFRAME and linked open data and generated by catalogers. Users will continue using library resources only if they find them authoritative, and easy to use and understand, not the equivalent of a link farm.

Searching authoritative metadata in a single search box is a start, but remember Tufte exhorts designers to reveal the complex – to “give visual access to the subtle and the difficult.” This process, as described, is facilitated by BIBFRAME, but few users have the ability and inclination to read raw XML code. In Tufte’s essay on the cognitive style of PowerPoint, he highlights that not only words, but also graphics, provide a powerful method of describing complex ideas to users and viewers. This concept could be applied to library catalogs by not presenting metadata in a “list” or “record” format, but instead in a dynamically generated relationships graph. This graph could contextualise the metadata, so that if a user desired to know more about the creator of a given work, then the user could navigate through the graph’s rich information context. This places information resources in a rich context of information in a visual format. It is this contextualization and description of relationships that will make user experience in the catalog richer, but these also are the intended design of BIBFRAME, as stated in the BIBFRAME document:

As libraries become part of this larger web of data, by leveraging the use of stable identifiers to reference clearly differentiated entities, focus will shift from capturing and recording descriptive details about library resources to identifying and establishing more relationships between and among resources. This includes related resources found on the web, and especially those beyond the traditional bounds of the library universe. These relationships – these links – drive the web, transforming the information space from many independent silos to a network graph that branches out in every direction. Relationships help search engines and other services to improve search relevancy and, most importantly, help users find the information they are looking for.

Though the format of the catalog is obsolete, the intellectual endeavor and practice that catalogs represent is undeniably significant and important – not only to the collective memory of society, or the users of libraries, but also to the users of the internet. The formatting of metadata in these catalogs, as guided by cataloging standards, is the result of two hundred years of research, interaction, and revision by librarians. However, this incredible array of metadata is locked away in an outdated metadata schema, with this metadata duplicated and hidden in many discrete library catalogs. Though MARC was a technical innovation in its day, new metadata schema are needed to best serve the needs of our users, as well as the wider public. Librarians are uniquely poised to continue the creation of authoritative metadata that users can trust and use, while adapting that metadata to emerging technologies. Indeed, librarians are already working on the production of new schema – namely, BIBFRAME. This new standard will free library metadata from the silos in which it has been stored for far too long, as well as bringing library metadata into the wider web of linked open data. Beyond this freeing of metadata, BIBFRAME also has the potential to allow librarians and designers to fundamentally re-think the nature and experience of searching the library catalog. In the course of this new iteration of the catalog, it is librarian’s professional responsibility to take time to listen to their users, examine best practices (like those postulated in the work of Edward Tufte), and bring their own information expertise to the versions and implementations of a new generation of catalogers.

Perhaps closing with this reminder from Charles Ammi Cutter is as fitting now in thinking both about metadata standards and catalog design as it was when he first wrote it over one hundred years ago, describing the work of the cataloger in the card catalog:

The convenience of the public is always to be set before the ease of the cataloger.⁸

---

Acknowledgements

The author would like to express his deep gratitude for the invaluable help of Penny Baker, Collections Management Librarian at the Sterling and Francine Clark Art Institute (and an early user of both BIBFRAME and RDA) for her thoughtful comments and encouragement. The author also thanks his reviewers at Lead Pipe, Hugh Rundle, and Emily Ford. Also, the author’s gratitude goes to Erin Dorney for beginning the process of publication. He would also like to thank his colleagues at the University of Arkansas Libraries, specifically Cheryl Conway, Mikey King, and Elizabeth McKee for their support and guidance in writing and publication. Finally, his thanks go to Jen Dean, who always provides a patient sounding board and peerless editor.

References and Further Reading

Avram, Henriette D. MARC, Its History and Implications. Washington: Library of Congress, 1975.

Avram, Henriette D., John F. Knapp, and Lucia J. Rather. The MARC II Format; A Communications Format for Bibliographic Data. Supplement. Washington: Library of Congress, 1968.

Avram, Henriette D. The MARC Pilot Project: Final Report on a Project Sponsored by the Council on Library Resources, Inc. Washington: Library of Congress; [for sale by the Supt. of Docs., U.S. Govt. Print. Off.], 1969.

Bertin, Jacques. Semiology of Graphics. Madison, Wis: University of Wisconsin Press, 1983.

Bibliographic Framework as a Web of Data: Linked Data Model and Supporting Services, (Washington, DC: Library of Congress, 2012), accessed September 20, 2013, http://www.loc.gov/bibframe/pdf/marcld-report-11-21-2012.pdf

Bringhurst, Robert. The Elements of Typographic Style. Seattle, WA: Hartley & Marks, 2012.

Cutter, W. P. Charles Ammi Cutter. Chicago: American Library Association, 1931.

Cutter, Charles A., W. P. Cutter, Worthington Chauncey Ford, Philip Lee Phillips, and Oscar George Theodore Sonneck. Rules for a Dictionary Catalog. Washington [D.C.]: G.P.O., 1904.

Lima, Manuel. Visual Complexity: Mapping Patterns of Information. New York: Princeton Architectural Press, 2011.

Manguel, Alberto. The Library at Night. New Haven, CT: Yale University Press, 2008.

Schmidt, Aaron. Walking Paper, accessed November 18, 2013, http://www.walkingpaper.org/

Tufte, Edward R. Beautiful Evidence. Cheshire, Conn: Graphics Press, 2006.

Tufte, Edward R. Envisioning Information. Cheshire, Conn: Graphics Press, 1990.

Tufte, Edward R. The Visual Display of Quantitative Information. Cheshire, Conn: Graphics Press, 2001.

Tufte, Edward R. Visual Explanations: Images and Quantities, Evidence and Narrative. Cheshire, Conn: Graphics Press, 1997.

1. Charles A. Cutter, W.P. Cutter, Worthington Chauncey Ford, Philip Lee Phillips, and Oscar George Theodore Sonneck. 1904. Rules for a Dictionary Catalog. Washington [D.C.]: G.P.O., p. 12
2. Charles A. Cutter, “The New Catalogue of Harvard College Library,” The North American Review 108 (1869): 96-129.
3. Henriette D. Avram, 1975. MARC, its History and Implications. Washington: Library of Congress, p. 3.
4. Indeed, an examination of recent search statistics in the University of Arkansas’ Libraries catalog shows that the closest replication to a Google search – keyword searching – is far more popular. Keyword searching represents approximately 31% of patron searches, while subject searching represents approximately three percent.
5. Christian Bizer, Tom Heath and Tim Berners-Lee. “Linked Data – The Story So Far,” International Journal on Semantic Web and Information Systems (IJSWIS) 5 (2009): 3, accessed (September 20, 2013), doi:10.4018/jswis.2009081901
6. Ibid.
7. Edward Tufte, The Visual Display of Quantitative Information. (Cheshire, Conn. : Graphics Press.), 191.
8. Cutter, p. 5

by Jason W. Dean at December 04, 2013 11:30 AM

Recordings of the three Forum keynote presentations are available on LITA’s USTREAM channel.

by mprentice at December 03, 2013 09:33 PM

Filed under: Figure Drawings Tagged: art, art model, brushes, class, ipad, Life Drawing, model, Nude, nude model, Nudes

by hochstenbach at December 03, 2013 07:27 PM

The American Library Association’s (ALA) Washington Office is calling for nominations for two awards to honor individuals or groups who have championed, protected and promoted public access to government information and the public’s right to know.

The James Madison Award, named in honor of President James Madison, was established in 1986 to celebrate an individual or group who has brought awareness to these issues at the national level. Madison is widely regarded as the Father of the Constitution and as the foremost advocate for openness in government.

The Eileen Cooke Award honors an extraordinary leader who has built local grassroots awareness of the importance of access to information. Cooke, former director of the ALA Washington Office, was a tireless advocate for the public’s right to know and a mentor to many librarians and trustees.

Both awards are presented during Freedom of Information (FOI) Day, an annual event on or near March 16, Madison’s birthday.

Nominations should be submitted to the ALA Washington Office no later than January 17, 2014. Submissions should include a statement (maximum one page) about the nominee’s contribution to public access to government information, why it merits the award and one seconding letter. Please include a brief biography and contact information for the nominee.

Send e-mail nominations to Jessica McGilvray, Assistant Director for the ALA Office of Government Relations, at jmcgilvray@alawash.org. Submissions can also be mailed to:

James Madison Award / Eileen Cooke Award
American Library Association
Washington Office
1615 New Hampshire Avenue, NW
Washington, D.C. 20009-2520

The post ALA seeks nominations for 2014 James Madison awards appeared first on District Dispatch.

by jmcgilvray at December 03, 2013 05:15 PM

This is a cross-post by Julia Keserű, Lisa Rosenberg and Greg Brown from the Sunlight Foundation, originally published on the Sunlight Foundation blog.

With more and more civil society organizations in the open government universe recognizing that “thorny issues” — such as political finance transparency or surveillance — need to be tackled somewhat more vehemently, we are eager to seize the momentum and start a hopefully constructive dialogue around an embarrassingly under- or unregulated area: lobbying disclosure. A few weeks back, with the support of our friends at the Open Knowledge Foundation, we took the first steps to create a community of interested advocates, activists and academics, and launched a public working group around the world of influence.

Today, we are excited to announce our draft lobbying disclosure guidelines and invite the community to provide input on these recommendations.

The world of influence

Around the globe, there are only a handful of countries that have some sort of regulations on lobbying, and even these laws — with a few notable exceptions — are so weak and limited in their effect that it`s as though they did not exist. As always, the lack of political will is an important factor in the weak or non-existent laws that regulate lobbying disclosure in most countries, but not the only one to blame.

By its nature, lobbying is indeed very difficult to define: We know it includes efforts by those who are paid to influence decision makers in the legislative and executive branch, but is it also grassroot activism and journalism? We think it is. At least in some cases. So then should the law require journalists and activists to register in a lobbyist database? Probably not.

Adequate definitions certainly do not substitute for a healthy transparency ecosystem that is essential for reform; isolated proposals almost never get the necessary political buy-in. Even when countries do manage to create lobbying laws, many of these attempts are doomed to fail because of inappropriate definitions of who constitutes a lobbyist, what is considered lobbying activity, and who might be a target of lobbying.

Some of the definitions fail to cover a significant part of professional lobbyists who make regular contact with politicians, while others miss an entire branch, such as the executive. At another extreme, many of the proposed or existing lobbying regulations fail because of concerns that they are too broad and thus unintentionally limit the freedom of expression or “discourage all constituents from engaging” with their representatives.

The lobbying guidelines

The Sunlight Foundation has created guidelines to start a conversation and encourage the community to reach a consensus on basic principles around lobbying transparency. Experience shows that meaningful change in the world of influence is almost always driven by scandals – such as the Abramoff case in the U.S. And if there’s one thing we never run out of in politics, it’s scandals. Our somewhat pragmatic goal is thus to empower local activists in their fight for reform, so that whenever a new scandal is unfolding, they will backed up by the necessary ammunition — aka strong arguments and a powerful community.

In doing this work, we’re hoping to create a stronger starting point for reformers throughout the world, to help activists and lawmakers operate in a supportive international context, and to empower use and analysis of lobbying disclosures. On that front, we’ve already seen innovative use of civil society tools such as Influence Explorer and LobbyPlag that enable people to convert lobbying information into meaningful narratives, making it easier for individuals to understand the dynamics of the decision-making process, and thus strengthening public oversight of lobbying.

Based in part on our experiences working to make lobbying more transparent in the US, the guide is organized into four parts: first, a section on definitions of key terms and the basic principles to make a lobbyist registry more robust; second, guidance on the minimum information that should be disclosed; third, a section on how this information should be collected and in what formats it should be published; and finally, some basic recommendations on enforcement and oversight.

We need your input

Again, the guide is intended as a conversation starter. We encourage the transparency community to give their input in this commentable document and help us refine these norms in a collaborative manner. We are especially curious to hear your input on the definitions we offer, and to verify the international examples we’ve collected – we might be missing some important aspects or internal dynamics.

You can leave your comments in the document or write to us at international@sunlighftoundation.com before January 6th 2014.

We look forward to hearing from and working with many more of you in the coming weeks and months. In the meantime, we encourage you to join the lobbying working group.

by Guest at December 03, 2013 03:55 PM

The second course in Speed Scholarship Week is a polemic on the theme of Big Data. Unlike the many fine existing rants about the privacy of the flies caught in Big Data’s webs, my rant is about the fate of the spiders. To an extent not fully appreciated, Big Data applications enable (and sometimes depend on) comprehensive surveillance of their users. Every question you ask of a Big Database reveals something about yourself, and don’t think that someone isn’t writing those questions down. Indeed, they may be logging your access precisely to keep you from violating the privacy of the people the data concerns. Big Data’s two privacy problems are twins, and rivals.

Big Data’s Other Privacy Problem is forthcoming in an edited collection of essays based on a Georgetown Law symposium on Big Data and and Big Challenges for Law and Legal Information, to be published by West Academic. I use the Bloomberg Terminal scandal as an example of how Big Data surveils its users, then step back and reflect on the relationship between this privacy problem and the more familiar one of protecting data subjects. Here’s an excerpt:

Thus, since Big Data cannot be entirely defanged and its users cannot be entirely trusted, it becomes necessary to watch them at work. It seems like a natural enough response to the problem of the Panopticon. Subject privacy is at risk because Big Data users can hide in the shadows as they train their telescopes not on the stars but on their neighbors. And so we might say, turn the floodlights around: ensure that there are no dark corners from which to spy. We would demand audit trails—permanent, tamper-proof records of every query and computation.

But if we are serious about user privacy as well as about subject privacy, transparency is deeply problematic. The audit trails that are supposed to protect Big Data subjects from abuse are themselves a perfect vector for abusing Big Data users. Indeed, they are doubly sensitive, because they are likely to contain sensitive information about both subjects and users. The one-way vision metaphor of the Panopticon, then, is double-edged. Think about glasses. A common intuition is that mirrorshades are creepy, because the wearer can see what he chooses without revealing where his interest lies. Everyone is up in arms about the Google Glass-holes who wear them into restrooms. But the all-seeing Eye is a window to the soul. The Segway for your face is also a camera pointed directly at your brain that syncs all its data to the cloud. The assumption Glass users are making, presumably, is that no one else will have access to their data, and so no one else will be pondering what they’re pondering. But that’s what Bloomberg Terminal users thought, too.

This draft marks another experiment. It’s formatted to a 5.5" × 8.5" page, so if you want to save paper, you can print it two-up. If you prefer your papers with plenty of whitespace, it should still look fine printed at 100% on 8.5" × 11" paper. Let me know whether you find this style convenient or confusing or both.

by James Grimmelmann (james@grimmelmann.net) at December 03, 2013 03:26 PM

by mprentice at December 03, 2013 01:44 PM

Strategic Social Media: Creating Library Community Online will be presented by Doralyn Rossmann, Scott Young, Angela Tate, and Mary Anne Hansen of Montana State University Library. Presenters will go beyond the basics by demonstrating how to create a social media guide for developing communities on Facebook, Twitter, Tumblr, Instagram, Pinterest, and Foursquare. Find ways to integrate social media efforts into your organization’s strategic plan and to establish best practices and assessment methodologies. Emphasis includes creating a meaningful voice and a compelling personality.

Level Up Web: Modern Web Development and Management Practices for Libraries will be presented by Nina McHale and Christine Coughlan of Aten Design Group (CO). Topics will include: how to launch and execute a user-centric discovery and design process; content strategy; web governance and staffing; information architecture; managing a web environment/CMS; iterative development; and version control. Attendees will leave with tailored plans, covering both short-term wins and long-term strategies, to “level up” their own web presences when they return to their libraries. This session is intended for librarians and staff of all technical ability levels, from the administrator managing staff who perform web work to the reference librarian contributing web site content to the sysadmin who keeps the servers running. It provides a big picture of the web development process for all who are involved in the process.

You may register online for a LITA workshop by completing the ALA Midwinter registration: http://alamw14.ala.org/register-now. You may also add a workshop to your existing Midwinter registration or register for a workshop only by calling ALA Registration at 1-800-974-3084, or you may register onsite in Philadelphia.

For additional information on LITA workshops and Midwinter events, visit http://www.ala.org/lita/conferences/midwinter/2014

by mprentice at December 02, 2013 08:43 PM

True (but embellished) story:

Toni, a newly hired physicist at Bell Labs, decided to set up a new lab to do the most sensitive electrical measurements ever. So the young researcher spent hundreds of thousands of dollars building a metal cage with insulators and coatings to electrically isolate the lab from the rest of the world. When it was ready for testing, Toni's friends were summoned. They got inside the cage and Toni cranked up a high voltage power supply. "We're floating 50 thousand volts above the rest of the room!" Toni said, excitedly. "Wouldn't it be cool if we could get to a hundred kilovolts?" Toni's face had that evil grin that made them laugh, nervously. At sixty kilovolts, there was hardly a blip of noise on the oscilloscope, but the smell of ozone in the room was unmistakable. Toni didn't stop there; at 65 kilovolts there was a loud bang, the circuit breakers flipped, and the room exuded a smell of molten plastic. The cage was now arc-welded to the back wall; the expensive insulators instantly useless.

"Whoops." said Toni.

Here's a quiz for you- Which of the following do you think happened to Toni?

A. Toni was fired for her stupidity and for wasting Bell Labs' money. She never worked in Physics again.

B. Toni was mortified and considered quitting immediately. But months of taunting by colleagues who would occasionally make "bang" noises at her was too much. She is now a very successful High School teacher in Northern New Jersey.

C. Toni suddenly developed an interest in a different type of experiment, which had surprising results and today Toni is often mentioned as a candidate for the Nobel Prize in Physics

by Eric (noreply@blogger.com) at December 02, 2013 08:40 PM

Sign up here for monthly news to your inbox.

Welcome!

Hello and welcome to our latest news update on what is happening in and around the Open Knowledge Foundation. As we approach our 10th anniversary, we are reflecting on the open movement over the last decade and planning for the opportunities and challenges ahead. We are asking all of you what you think and to contribute to our community-wide survey to help us shape our future. Planning has also started for OKFestival in Berlin and now is the time to be thinking about how you would like to participate and what ideas, projects and contributions you can make to create an amazing event.

For the most up-to-the-minute info on all things Open, follow us on Twitter and on the blog.

And read our monthly Global community stories for more news – two this month: stories from Greece, Bangladesh, Argentina and Canada and Ireland and Germany.

Community Survey – share your thoughts!

What does Open Knowledge mean to you? Why are you involved? We are excited to be running our first survey to find out what you think. Help us shape the future of the Open Knowledge Foundation – we value your input. There are 24 questions and it will take approximately 10 minutes of your time. Deadline for responses is December 14, 2013 17:00 GMT. If you would like to know more, read our blogpost.
We will share the results and follow-up actions in January 2014. Thank you.

OKFestival 2014, 15-18 July, Berlin – get involved!

The Open Knowledge Festival will gather open-powered communities, projects and individuals, both experienced and newcomers, from all around the world. Now is the time to get ready and we invite you to start meeting up, in person and online to think about what amazing projects and ideas you would like to participate in, and what contributions you can make. A new website will be ready soon and we will shortly call for proposals to shape the agenda together. More information is on our blog. Follow @OKFestival, share ideas and questions using the hashtag #OKFestival or write to us, and sign up for the festival newsletter for all upcoming OKFestival news. Get meeting, sharing and making!

Making news

We have signed a MoU with the BBC, a great step for openness. The first joint initiative will be with Wikimedia in January to coordinate the first ever “speakerthon”. Using the BBC’s vast radio archive, participants will tag and select snippets of notable individuals’ voices in order to upload them to Wikipedia articles as open content. More information.

The Open Knowledge Foundation has also joined more than 30 civil society organisations to sign an open letter urging for greater transparency around the Trans-Pacific Partnership negotiations. It calls for an equal level of participation for policy makers, civil society, and members of the public with that of industry representatives.

We congratulate **Rufus Pollock for being elected an Ashoka Fellow** “the global association of the world’s leading social entrepreneurs – men and women with system changing solutions for the world’s most urgent social problems.” Rufus says that he is “honoured to accept the fellowship, but what is really exciting is seeing the work of the Open Knowledge Foundation on open data and open knowledge being recognised as a key aspect in driving positive social change in the twenty-first century”.

Spending Stories is officially launched! This new app helps citizens and journalists understand and compare financial data from news stories. If you would like to know more about the project, get in touch.

A new partnership has been announced between the French Ministry of Culture and Communication and the Open Knowledge Foundation France to map the Public Domain in France. The Guardian has called the Public Domain Review a ‘model of digital curation’ which showcases ‘the best and quirkiest texts, images and films the internet has to offer’. Check out the new PDR store with a selection of beautiful prints, t-shirts and other great gifts.

Making a difference

What does open data / open knowledge have to do with crisis mapping? Crowd-sourced digital maps created by online volunteers worldwide are powerful tools in humanitarian relief work. Volunteers worked 24/7 when Typhoon Haiyan (Yolanda) hit the Philippines. At the International Conference of Crisis Mappers (ICCM) in Nairobi, Kenya, the School of Data hosted a full day pre-conference training session as part of the mentorship programme. To learn more read the blog report.

Our “Follow the Money” website was launched at the Open Government Partnership Summit and you can read about it and also follow the livestreamed panel discussion. If you are interested in using information about public money to hold decision-makers to account, then we hope you’ll join us and sign up.

The recent event which brought together a group of open education enthusiasts has resulted in the building of an Open Education Timeline. The timeline has now been ‘put online’ using TimeMapper and we need your help to ensure that includes all the most important dates and consists of good quality data! We’d like to make it the the most comprehensive Open Education Timeline to date! If you are interested in contributing then take a look at our post.

Enter the LinkedUp “Vidi” Competition, the second second competition in the LinkedUp Challenge. We’re inviting you to design and build innovative and robust prototypes that use open data for educational purposes. There are prizes to be won and we can offer support. Find out more on the LinkedUp Challenge website.

Coming up

• Open Knowledge Foundation India is celebrating December as open knowledge month! If you’re in the area, join them.
• Learn how opening data can transform society and how to do it, at a one-day introductory course on Friday, 6 December in London – there are a few places left.
• Join the School of Data online Data Expedition to investigate the Nigerian extractives industry on December 7 with OpenOil. There is still time time to register.

Images: symbol by Natalie Swencki, CC-BY; OKFestival CC-BY; School of Data session at ihub for ICCM, CC-BY.

by Elaine Shaughnessy at December 02, 2013 05:26 PM

Welcome to Speed Scholarship Week. The first course is a palate cleanser: a paper that makes its points in a lighthearted way. I was invited to comment on a prize-winning student Note by Kyle Dolinsky of Washington and Lee, on the subject of copyright and 3D printers. The relationship between objects and the computer files that represent them is tricky on its own, but the copyright doctrines dealing with 3D objects and with computer software are notorious for their intricacy. I wondered whether there might be a way to cut through the complexity and get at the real issues involved. And then it struck me—that 3D printers are so miraculous a technology they might as well be magic. What if they were?

Indistinguishable from Magic: A Wizard’s Guide to Copyright and 3D Printing takes Clarke’s Third Law—“Any sufficiently advanced technology is indistinguishable from magic.”— as its inspiration. It imagines a world in which wizards use magic scrolls and wands to make 3D objects, and then gives a playful but rigorous discussion of how copyright law would apply. Here’s an excerpt from early on:

Ulrich has a Replicio wand. When he waves it with the right flick of his wrist, it makes a perfect duplicate of the object he waves it at.

The copyright treatment of the Replicio wand is simple. When Ulrich uses it to duplicate an object, he has created a “copy.” His only good argument that his copy is noninfringing will be that the object is not subject to copyright in the first place. The strength of this argument depends on what the object is. A first-century bust of Homer is in the public domain; duplicating it with my wand violates no one’s rights. A twenty-first-century bust of Homer Simpson is copyrighted; duplicating it with the wand makes me an infringer.

There is nothing special about three-dimensional objects in this respect. If Ulrich waves the wand at Rembrandt’s two-dimensional public-domain painting Aristotle Contemplating a Bust of Homer, his duplicate will not infringe. To be sure, some three-dimensional objects are uncopyrightable for a distinctive reason: because they are “useful articles” whose practical aspects are inseparable from their aesthetic features, such as bicycle racks and casino uniforms. But to understand the Replicio wand, we need to know only whether a three-dimensional object is copyrighted; nothing turns on why.

Indistinguishable from Magic will appear in the spring in the Washington and Lee Law Review. Download it while it’s hot!

by James Grimmelmann (james@grimmelmann.net) at December 02, 2013 04:45 PM

Work continues on the WorldShare Platform and I want to bring your attention to a couple of enhancements coming to the WorldCat Metadata API.

Note that on Saturday, 28 December 2013, there will be a scheduled service downtime for the OCLC WorldShare Platform and all associated OCLC Web services available through the Platform from 2:00 am to 8:00 am EST (-5 GMT). During the downtime (which will be minimized as much as possible), the WorldShare Platform will receive this quarter’s planned enhancements.

read more

by hostetls at December 02, 2013 04:17 PM

Struggling towards usable linked data services at SWIB13

Paraphrasing some of the challenges proposed by keynote speaker Dorothea Salo, the unofficial theme of the SWIB13 conference in Hamburg might be described as “No more ontologies, we want out of the box linked data tools!”. This sounds like we are dealing with some serious confrontations in the linked open data world. Judging by Martin Malmsten’s LIBRIS battle cry “Linked data or die!” you might even think there’s an actual war going on.

Looking at the whole range of this year’s SWIB pre-conference workshops, plenary presentations and lightning talks, you may conclude that “linked data is a technology that is maturing” as Rurik Greenall rightly states in his conference report. “But it has quite a way to go before we can say this stuff is ready to roll out in libraries” as he continues. I completely agree with this. Personally I got the impression that we are in a paradoxical situation where on the one hand people speak of “we” and “community”, and on the other hand they take fundamentalist positions, unconditionally defending their own beliefs and slandering and ridiculing other options. In my view there are multiple, sometimes overlapping, sometimes irreconcilable “we’s” and “communities”. Sticking to your own point of view without willingness to reason with the other party really does not bring “us” further.

This all sounds a bit grim, but I again agree with Rurik Greenall when he says that he “enjoyed this conference immensely because of the people involved”. And of course on the whole the individual workshops and presentations were of a high quality.

Before proceeding to the positive aspects of the conference, let me first elaborate a bit on the opposing positions I observed during the conference, which I think we should try to overcome.

Developers disagree on a multitude of issues:
Formats
Developers hate MARC. Everybody seems to hate RDF/XML, JSON-LD seems to be the thing for RDF, but some say only Turtle should be used, or just JSON.
Tools and languages
Perl users hate Java, Jave users hate PHP, there’s Python and Ruby bashing.
Ontologies
Create your own, reuse existing ones, yes or no upper ontologies, no ontologies but usable tools.
Operating systems
Windows/UNIX/Linux/Apple… it’s either/or.
Open source vs. commercial software
Need I say more?
Beer
Belgians hate German beer, or any foreign beer for that matter.
(Not to mention PDF).

OK, I hope I made myself clear. The point is that I have no problem at all with having diverse opinions, but I dislike it when people are convinced that their own opinion is the only right one and refuse to have a conversation with those who think otherwise, or even respect their choices in silence. The developer “community” definitely has quite a way to go.

Apart from these internal developer disagreements I noticed, there is the more fundamental gap between developers and users of linked open data. By “users” I do not mean “end users” in this case, but the intermediary deployers of systems. Let’s call them “libraries”.
Linked Data developers talk about tools and programming languages, metadata formats, open source, ontologies, technology stacks. Librarians want to offer useful services to their end users, right now. They may not always agree on what kind of services and what kind of end users, and they may have an opinion on metadata formats in systems, but their outlook is slightly different from the developers’ horizon. It’s all about expectations and expectation management. That is basically Dorothea Salo’s keynote’s point. Of course theoretical, scientific and technical papers and projects are needed to take linked data further, but libraries need linked data tools, focused on providing new services to their end users/customers in the world of the web, that can easily be implemented and maintained.
In this respect OCLC’s efforts to add linked data features to WorldCat is praiseworthy. OCLC’s Technology Evangelist Richard Wallis presented his view on the benefits of linked open data for libraries, using Google’s Knowledge Graph as an example. His talk was mainly focused at a librarian audience. At SWIB, where the majority of attendees are developers or technology staff, this seemed somewhat misplaced. By chance I had been present at Richard’s talk at the Dutch National Information Professional annual meeting two weeks earlier, where he delivered almost the same presentation for a large room full of librarians. There and then that was completely on target. For the SWIB audience this all may have been old news, except for the heads up about OCLC’s work on FRBR “Works” BIBFRAME type linked data which will result in published URIs for Works in WorldCat.
An important point here is that OCLC is a company with many library customers worldwide, so developments like this benefit all of these libraries. The same applies to customers of one of the other big library system vendors, Ex Libris. They have been working on developing linked data features for their so called “next generation” tools since some time now, in close cooperation with the international user groups’ Linked Open Data Special Interest Working Group, as I explained in the lightning talk I gave. Also open source library systems like Koha are working on adding linked open data features to their tools. It’s with tools like these, that reach a large number of libraries, that linked open data for libraries can spread relatively quickly.

In contrast to this linked data broadcasting, the majority of the SWIB presentations showed local proprietary development or research projects, mostly of high quality notwithstanding. In the case of systems or tools that were built all the code and ontologies are available on GitHub, making them open source. However, while it is commendable, open source on GitHub doesn’t mean that these potentially ground breaking systems and ontologies can and will be adopted as de facto standards in the wider library community. Most libraries, both public and academic, are dependent on commercial system and content providers and can’t afford large scale local system development. This also applies up to a point to libraries that deploy large open source tools like Koha, I presume.
It would be great if some of these many great open source projects could evolve into commonly used standard tools, like Koha, Fedora and Drupal, just to name a few. Vivo is another example of an open source project rapidly moving towards an accepted standard. It is a framework for connecting and publishing research information of different nature and origin, based on linked data concepts. At SWIB there was a pre-conference “VivoCamp”, organised by Lambert Heller, Valeria Pesce and myself. Research information is an area rapidly gaining importance in the academic world. The Library of the University of Amsterdam, where I work, is in the process of starting a Vivo pilot, in which I am involved. (Yes, the Library of the University of Amsterdam uses both commercial providers like OCLC and Ex Libris, and many open source tools). The VivoCamp was a good opportunity to have a practical introduction in and discussion about the framework, not in the least by the presence of John Fereira of Cornell University, one of the driving forces behind Vivo. All attendees (26) expressed their interest in a follow-up.
Vivo, although it may be imperfect, represents the type of infrastructure that may be needed for large scale adoption of linked open data in libraries. PUB, the repository based linked data research information project at Bielefeld University presented by Vitali Peil, is aimed at exactly the same domain as Vivo, but it again is a locally developed system, using another smaller scale open source framework (LibreCat/Catmandu of Bielefeld, Ghent and Lund universities) and a number of different ontologies, of which Vivo is just one. My guess is that, although PUB/LibreCat might be superior, Vivo will become the de facto standard in linked data based research information systems.

Instead of focusing on systems, maybe the library linked data world would be better served by a common user-friendly metadata+services infrastructure. Of course, the web and the semantic web are supposed to be that infrastructure, but in reality we all move around and process metadata all the time, from one system and database to another, in order to be able to offer new legacy and linked data services. At SWIB there was mention of a number of tools for ETL, which is developer jargon for Extract, Transform, Load. By the way, jargon is a very good way to widen the gap between developers and libraries.
There were pre-conference workshop for the ETL tools Catmandu and Metafacture, and in a lightning talk SLUB Dresden, in collaboration with Avantgarde Labs, presented a new project focused on using ETL for a separate multi-purpose data management platform, serving as a unified layer between external data sources and services. This looks like a very interesting concept, similar to the ideas of a data services hub I described in an earlier post “(Discover AND deliver) OR else”. The ResourceSync project, presented by Simeon Warner, is trying to address the same issue by a different method, distributed synchronisation of web resources.

One can say that the BIBFRAME project is also focused on data infrastructure, albeit at the moment limited to the internal library cataloguing workflow, aimed at replacing MARC. An overview of the current state of the project was presented by Lars Svensson of the German National Library.
The same can be said for the National Library of Sweden’s new LIBRIS linked data based cataloguing system, presented by Martin Malmsten (Decentralisation, Distribution, Disintegration – towards Linked Data as a First Class Citizen in Libraryland). The big difference is that they’re actually doing what BIBFRAME is trying to plan. The war cry “Linked data or die!” refers to the fact that it is better to start from scratch with a domain and format independent data infrastructure, like linked data, than to try and build linking around existing rigid formats like MARC. Martin Malmsten rightly stated that we should keep formats outside our systems, as is also the core statement of the MARC-MUST-DIE movement. Proprietary formats can be dynamically imported and exported at will, as was demonstrated by the “MARC” button in the LIBRIS user interface. New library linked data developments will have to coexist with the existing wider library metadata and systems environment for some time.
Like all other local projects, the LIBRIS source code and ontology descriptions are available on GitHub. In this case the mere scope of the National Library of Sweden and of the project makes it a bit more plausible that this may actually be reused on a larger scale. At least the library cataloguing ontology in JSON-LD there is worth having a look at.
To return to our starting point, the LIBRIS project acknowledges the fact that we need actual tools besides the ontologies. As Martin Malmsten quoted: “Trying to sell the idea of linked data without interfaces is like trying to sell a fax without the invention of paper”.

The central question in all this: what is the role of libraries in linked data? Developers or implementers, individually or in a community? There is obviously not one answer. Maybe we will know more at SWIB14. Paraphrasing Fabian Steeg and Pascal Christoph of hbz and Dorothea Salo, next years theme might be “Out of the box data knitting for great justice”.

by Lukas Koster at December 01, 2013 01:39 PM

Here is the complete 3-page story of the Penguin & the Polar Bear. Stay tuned for the next adventures of my unexpected comic hero.Filed under: Comics Tagged: comic, Illustrator, inking, Penguin, Photoshop, Polar Bear

by hochstenbach at November 30, 2013 08:18 PM

This issue resolved by GSW December 8 To borrow a line from Jonathan Rochkind's Bibliographic Wilderness 'EZProxy is terrible, it's just the best we've got.'  I don't think it's terrible but it's always been a pain that TOC services don't work off campus and that RSS feeds occasionally don't work at all, but this week I got a something new. GeoScienceWorld's search interface includes a world

by Alan @JCU Library (noreply@blogger.com) at November 30, 2013 07:08 PM

D. B. Cooper, the guy who hijacked a plane in 1971 and then — mid-flight — jumped into the darkness with a bundle of cash and disappeared, is celebrated on this day, the Saturday following Thanksgiving. Granted, this is mostly just a thing in Ariel Washington, where it’s said to have started in 1974, but the participants are pretty passionate about it.

As Steve Mandich says, “with his wraparound shades, skinny tie and no-nonsense expression, Cooper comes across as one cool customer.” Cool enough for comic books, anyway. Definitely cool enough for stencil art, according to annievix and Stanmason. And cool enough for nearly 40 years of festivals in Ariel Washington.

And now this is where I get sucked into the mystery. I find it odd that none of the theories described in the Wikipedia article include the man hiding in the plane and escaping on the ground in Reno Nevada. In 2007 the FBI disclosed that the reserve parachute Cooper supposedly used for his jump was “an unusable unit with an inoperative ripcord intended for classroom demonstrations, despite the fact that it had clear markings identifying it to any experienced skydiver as non-functional.”

The FBI, it seems, is convinced the man who appears to have meticulously planned every detail jumped into a rainy night with a non-functional trainer chute, rather than tossed it out the back and hid elsewhere in the plane. I suppose it would be very embarrassing for the FBI to admit that it’s possible a man that looks like The Matrix’s Agent Smith could have walked right past them once the plane was on the ground.

D. B. Cooper composites by unnamed FBI artist, Agent Smith illustrations by Neo Is The One and Dave Mottram.

by Casey Bisson at November 30, 2013 01:31 PM

This afternoon I’m sitting in the new bibliographic environment breakout session at Code4Lib BC. After taking a look at Mark Jordan’s easyLOD, I decided to play around with putting together a web service for Koha that emits RDF when fed a bib ID. Unlike Magnus Enger’s semantikoha prototype, which uses a Ruby library to convert MARC to RDF, I was trying for an approach that used only Perl (plus XS).

There were are of building blocks available. Putting them together turned out to be a tick more convoluted than I expected.

The Library of Congress has published an XSL stylesheet for converting MODS to RDF. Converting MARC(XML) to MODS is readily done using other stylesheets, also published by LC.

The path seemed clear for a quick-and-dirty prototype — make a copy of svc/bib, copy it to opac/svc/bib and take out the bits for doing updates (we’re not quite ready to make cataloging that collaborative!), and write a few lines to apply two XSLT transformations.

The code was quickly written — but it didn’t work. XML::LibXSLT, which Koha uses to handle XSLT, complained about the modsrdf.xsl stylesheet. Too new! That stylesheet is written in XSLT 2.0, but libxslt, the C library that XML::LibXSLT is based on, only supports XSLT 1.

As it turns out, Perl modules that can handle XSLT are rather thin on the ground. What I ended up doing was:

Installing XML::Saxon::XSLT2, which required…

Installing Saxon-HE, a Java XML and XSLT processor that supports XSLT 2.0, which required…

Installing Inline::Java, which required…

Installing a JDK (I happened to choose OpenJDK).

After all that (and a quick tweak to the modsrdf.xsl stylesheet, I ended up with the following code that did the trick:

#!/usr/bin/perl

BEGIN {
    $ENV{'PERL_INLINE_DIRECTORY'} = '/tmp/inline';
}

use Modern::Perl;

use CGI;
use C4::Biblio;
use C4::Templates;
use XML::Saxon::XSLT2;

my $query = new CGI;
binmode STDOUT, ':encoding(UTF-8)';

# do initial validation
my $path_info = $query->path_info();

my $biblionumber = undef;
if ($path_info =~ m!^/(\d+)$!) {
    $biblionumber = $1;
} else {
    print $query->header(-type => 'text/xml', -status => '400 Bad Request');
}

# are we retrieving or updating a bib?
if ($query->request_method eq "GET") {
    fetch_rdf($query, $biblionumber);
}

exit 0;

sub fetch_rdf {
    my $query = shift;
    my $biblionumber = shift;
    my $record = GetMarcBiblio($biblionumber);
    if  (defined $record) {
        print $query->header(-type => 'text/xml');
        my $xml = $record->as_xml_record();
        my $base = join('/',
                        C4::Context->config('opachtdocs'),
                        C4::Context->preference('opacthemes'),
                        C4::Templates::_current_language()
                       );
        $xml = transform($xml, "$base/xslt/MARC21slim2MODS3-3.xsl");
        $xml = transform($xml, "$base/xslt/modsrdf.xsl");
        print $xml;
    } else {
        print $query->header(-type => 'text/xml', -status => '404 Not Found');
    }
}

sub transform {
    my $xmlrecord = shift;
    my $xslfilename = shift;

    open my $fh, '<', $xslfilename;
    my $trans = XML::Saxon::XSLT2->new($fh);
    return $trans->transform($xmlrecord);

}

This works… but is not satisfying. Making Koha require a JDK just for XSLT 2.0 support is a bit much, for one thing, and it would likely be rather slow if used in production. It’s a pity that there’s still no broad support for XSLT 2.0.

A dead end, most likely, but instructive nonetheless.

by Galen Charlton at November 29, 2013 11:21 PM

Second and final day of Code4Lib BC’s lightning talks. Here are my notes.

Mark Jordan – DOCR/SMD

Slides Source code

• OCR clients – phones, tablets that do the work or just a plain script
• page server – controls the work

Components

• page server
  • PHP web app
  • PHP queue manager script
  • SQLite database
• OCR clients
  • clients use Tesseract OCR engine (available for Android and iOS)
  • first client is a simple Python script

how it works

• client: “im ready , give me an image”
• server: here you go
• client: OK here’s your text

What I Learned so far

• REST
• Slim microframework
• SQLite

What I want to Learn

• Android app development
• potential for generalizing OCR to other tasks

Peter Tyrrell – Parsing PDFs

• wanted to do search term highlight in PDFs
• using NYT Document Viewer (under document cloud)
• PDF2DJVU
• DVJULIBRE (DJVU to TXT, to XML, to TIF)
• Imagemagick (TIF to JPG)
• and then a whole bunch of other things to store, process, pass into viewer

John Durno – Uploading to Internet Archive via API

Links

• most ambitious digitization project: first 50 years of the British Colonist for the 150th anniversary
• ~100k images in PDF
• Acrobat would highlight for you if configured properly
• revised site as part of the next stage to digitize next 100 years
• Internet Archive: started digitizing newspapers from microfilm
• collection setup within the IA
• dumping content into IA can do it through API
• can upload metadata with it
• based on Amazon S3 API, a lot of tools already work with it
• boto library with ia-wrapper
• python script to upload
• new site is a wrapper around the IA for search purposes
• reading happens on IA’s site

Colleen Bell – ERM & LibGuides

• use identifier for subject database lists to export JSON
• use PHP script to process
• add as remote script in libguides
• can also do it for individual resources by ID
• script pulls based on IDs separated by comma
• don’t need libguides, can import into any page
• can do this as long as you can get your data

James MacGregor – Article Metrics with OJS/OMP

• PKP: scholarly publishing intitiative at SFU
• Open Journal System: WordPress for journals
• Open Monograph Press: OJS for presses
• all software is open source
• new: general overhaul statics framework and compatible with addition of PLOS metrics
• had a stats framework overhaul: gathered centrally, added features
• PKP ALM: application Ruby on Rails web app to aggregate article performance data, and plugin
• shows HTML views, PDF downloads, facebook/mendeley shares, pubmed, and more

Jonathan Schatz – The Story of BC Libraries’ IT Environments

• did field work going around to assess IT environment in BC Public Libraries within the Sitka group
• had about 1 day per library
• covered 3 federated libraries
• connectivity and network main priorities
• phones, internet, network, workstations/servers, printing, technical support, training
• had to map wifi with laptop and umbrella
• they do a lot with very little resources
• sometimes get creative e.g. creating internet routers

Paul Joseph – UBC Digital Library Framework

Status Quo

• 3 repositories: ContentDM, DSpace (IR), AtoM (for Rare books & special collections)
• in addition: separate, collections e.g. Drupal, ElasticSearch
• access and presentation in silos, defined by the applications
• Tried to improve with in-page view, facets, in-context results, tiling, rapid zoom

Introduce framework

• scalable, flexible
• plug in metadata and full text (when possible)
• use ElasticSearch
• series interactions to provide access to metadata and objects
• also service provider: OAI-PMH, Open Data API, Open Apps
• rely on external services to leverage functionality e.g. RefWorks, Disqus (social commenting)

Calvin Mah / Todd Holbrook – SFU Library Hours Database

• as part of the API
• might be something they can host, but lack of enthusiasm
• maybe could work with coop to host the tool
• or host yourself at your institution
• straight from UBC, but re-coded it
• kept data model and end user look
• can do usual hours, but can add exceptions using date range
• feeds to API system, available as JSON data
• Drupal widgets to drop almost anywhere

That’s it for today. Breakout time! When the time comes, have a safe trip home

 

Filed under: Events

by Cynthia at November 29, 2013 07:19 PM

Much of the education of a physicist or an engineer revolves around problem sets. To do a problem set, the student needs to understand some concepts to "set up the problem". There will be some calculating machinery that the student must have mastered. Problems can be difficult in a variety of ways. Often, the path from concept to solution won't be obvious. It might not be obvious which principles might be applied to a particular problem. The calculations required might be very messy. Insight or intuition may be required- you might have to guess the answer to figure out how to get to it. But often, solving a problem requires a leap of faith. You might have to work for a half hour before you know whether you've chosen the right approach to the problem. So it helps to have some self-confidence, to be ok with not knowing whether you're on the right path or totally lost. Other times, you just need the right tool, and maybe you have to invent the tool.

Real math or science doesn't come with answers in the back of the book. A researcher might work for years without knowing whether their efforts are leading them down a blind alley. The exquisite feeling you get when you've solved a really hard problem is why people become physicists, mathematicians, and engineers. It's the feeling of having eyes where once you couldn't see.

Having her self-confidence assaulted by every problem set in grad school was a challenge for "K", the applied physics Ph.D. I wrote about in Part 2. But there was that one problem set in high school that stumped everyone else in the class, but which she solved. Once you've tasted that success, you don't forget it.

Learning to ski was a revelation for K. You'd take the ski lift up to the top of a mountain, and somehow you'd end up at the bottom.

There are a lot of ways to get to the bottom of any slope. Some people like to do traverses. Some people go straight down. My method was to swallow the sheer terror, point my skis downhill, and power through some turns and some slides. I've never been a great skier so I'd get half way down and land on my butt. Gradually, I figured out how to avoid falling.

This is apparently a typical male's approach to skiing. A touch of reckless self confidence lends itself to this approach. Just watch some teenage boys on a ski slope if you doubt it.

K realized that she didn't have to ski like the guys. The part of skiing she enjoyed was carving turns. To carve a good turn, you have to put your weight downhill, which at first feels insecure, but in practice gives you more control. And having good technique gives you real confidence.

Realizing that she could approach problem sets her way really helped K get through those difficult problem sets. It was OK that she felt like she had no idea what to do while many of her male colleagues just pretended to know how to do them.  There was nothing wrong with focusing on skills and carving away the difficulty. And not break anything.

by Eric (noreply@blogger.com) at November 29, 2013 01:15 PM

Ireland: Meetups, OpenSpending projects and prominent media exposure…

In July our Irish group held Open Data Ireland Meetup #9, which was dedicated to ‘Local Government’ and was attended by around 20 people – see follow-up blogs posts among other here and here. In October this was followed up with Open Data Ireland Meetup #10 that focused on “The Way Forward for Open Data in Ireland” (see Storify stream) and just a couple of days ago #11 was held in Dublin and featured, among other, a telecast by Open Knowledge Foundation CEO Laura James.

On the side the Irish group is working on contributing to the Where Does My Money Go project, acting very actively on the Irish front in the Open Government Partnership and reported also that Fingal County Council has became the 1st local authority in Ireland in Open Spending. The next challenge is now to convince and help 33 other local authorities to follow Fingal’s example.

Lastly, the Ireland Open Knowledge Foundation Ambassador Denis Parfenov was prominently featured in The Irish Times as part of a band of Freedom of Information (FOI) advocates putting forward a critique of a proposition to allow adding fees to FOI requests.

Germany: Hackathons, Code for America collaboration, teenage hacker academy, presentations across the world and much, much more…

Our German group is so active we hardly know where to begin – and we’ve had to select only a few of their activities to fit into the Global Community Stories format. Well done Germany! Here we go:

In cooperation with the Museum of Jewish Culture, the German Digital Library, the Prussian Cultural Heritage Foundation, Wikimedia Germany, Internet and Society Co:llaboratory and iRights, the Germany group is organizing the conference Zugang gestalten! – Mehr Verantwortung für das kulturelle Erbe (Shaping Access! – More responsibility for our cultural heritage) and addressing the shift in their role of cultural heritage institutions, the public expectation towards them and the chances and challenges digitalisation means to them. The conference will take place in Berlin from November 28th – 29th.

Conferences, workshops and talks continue to be important for the mission of FragDenStaat.de project, which the group is also leading. They addressed experts and potential founders at the Friedrich-Ebert-Stiftung and attended the International Conference of Information Commissioners to exchange ideas and connect with other professionals in the area. They also organized a community workshop to educate people about FOI and teach campaigners how to use FragDenStaat.de. A talk has been submitted to the 30th Chaos Communication Congress.

Over the last month members of OKF Germany board and team furthermore participated as speakers and contributors to a range of open knowledge events, including Wikisym/Opensym and Wikimania in Hong Kong, the Service Experience Camp in Berlin, the Open Access days in Hamburg and of course at this year’s Open Knowledge Conference in Geneva.

In September, piggybacking on the elections in Germany, the group organized a hackathon titled Wahl.Daten.Helfer that invited the community to build their own “polling stations” in their hometown and to reinterpretate information about candidates, results and historic election development. Later that month they organized an open data hack weekend for teenagers: Jugendhackt. The event was co-organised with their partner Young Rewired State and took place in Berlin. The results were simply overwhelming: more than 60 young coders (between 12 and 18 years) and 25 mentors worked in 15 teams developing awesome apps based on open datasets (schools, nature, elections etc.). See photos here, videos here and projects here. If all goes as planned this will repeated in 2014!

In the near future the group has scheduled a hack day on Geodata to take place on Open Data Day next year (Feb 22 2014). Similarly, they will dive into making a Culture Hack day working with The German Museum of Technology and the German National Library – in collaboration with Wikimedia Germany. Open Knowledge Foundation Germany is also part of the international expansion of the successful Code for America Program. Together with Code for America and our main sponsor Google they will introduce the fellowship program to Germany at the beginning of 2014.

Lastly, one of the most important events in the upcoming year is of course the OKFestival 2014 and the Germans are thrilled that it will be held in Berlin! As the annual conference of the entire Open Knowledge Foundation Network, OKFestival is organised by the OKFN central, but the German group will of course contribute and help ensure making this a really great event.

…and in shorter news:

Our new Swedish Ambassador and his group recently organized their first science seminar on Open Design, and in Brazil they shared how they used the OpenSpending API to visualise Brazilian government spending. Open Knowledge Foundation Finland and Brazil have started visualizing their projects using the Hackdash software – see their activities here and here. In France, they’ve been translating School of Data into French, Ecole des Données, and in Scotland they held a very successful Meetup recently, and did a fabulous job of documenting it – catch the livestream videos here and here, read follow-up blog posts here, here and here.

In India, the group is developing a series of projects including OpenEducation, Open Data, Open Science & Research and the Vitayard Open Science Platform. Furthermore they recently participated in a workshop on Open Science and Open Data, held at the Indian Statistical Institute, Bangalore, in the beginning of October.

Many groups also got engaged around the Open Data Index release in October, as mentioned earlier herein. The release was mentioned in ao. The Guardian, The Economist and lots of local media around the world – thanks to localization efforts around our Local Groups: In Taiwan they crafted a Chinese (traditional) version and other groups made French, Egyption, Irish, German and Danish comments.

In the days to come, Open Knowledge Foundation Nepal will organize Open Election Data Week from 24th to 30th Nov 2013 – read more about that here. We’ve also heard news from our Japanese group, that they’re getting ready to launch Spending Stories in Japanese. Watch this space!

by Christian Villum at November 29, 2013 11:56 AM

There is nothing quite like the sense of sheer glee you get when you’re waiting at the border… and have been waiting at the border for a while… and then a new customs inspection lane is opened up. Zoom!

Marlene and I left Seattle this morning to go to the Code4Lib BC conference in Vancouver. Leaving in the morning meant that we missed the lightning talks, and arrived after the breakout sessions had started. Fortunately, folks were quick to welcome us, and I soon fell into the accessibility session.

Accessibility has been on my mind lately, but it’s an area that I’m starting mostly from ground zero with. I knew that designing accessible systems is a Good Idea, I knew about the existence some of the jargon and standards, and I knew that I didn’t know much else — certainly none of the specifics.

Cynthia Ng very kindly shared some pointers with me. For example, it is helpful to know that the Section 508 guidelines is essentially a subset of WCAG 1.0. This is exactly the sort of shortcut (through an apparently intimidating forest) that an expert can effortlessly give to a newbie — and having opportunities to learn from the experts is one of the reasons why I like going to conferences.

The accessibility breakout session charged itself with putting together a list of resources and best practices for accessibility and universal design. As I mentioned above, we arrived in the middle of the breakout session time, but a couple hours was more than enough time to get initial exposure to a lot of ideas and resources. It was exhilarating.

In no particular order, here is a list of various things that I’ll be following up on:

• The Accessibility Project
• Guerilla testing
• The 5 second test
• Swim lane diagrams
• The Paciello Group Blog
• Be careful about putting things in the right sidebar of a three-column layout — a lot of users have been trained by web advertising to completely ignore that region.  Similarly, a graphic with moving parts can get ignored if it looks too much like an ad.
• The Code4Lib BC accessibility group’s notes
• Having consistency of branding and look and feel can improve usability — but that can be a challenge when integrating a lot of separate systems (particularly if a library and a vendor have different ideas about whose branding should be foremost).
• Integrating one’s content strategy with one’s accessibility strategy.  To paraphrase a point that Cynthia made a few times, putting out too much text is a problem for any user.
• As with so much of software design, iterate early and often. The time to start thinking about accessibility is when you’re 20% of the way through a project, not when you’re 80% done.
• Standards can help, but only up to a point.  A website could pass an automated WCAG compliance test with flying colors but not actually be usable by anyone.

And there’s another day of conference yet!  I’m quite happy we made the drive up.

Here’s a general question to the world: what reading material do you recommend for folks like me who want to learn more about writing accessible web software?

by Galen Charlton at November 29, 2013 05:49 AM

Here are my notes from Code4LibBC Day 1 lightning talks!

Paul Joseph – Opening Talk

• The systems librarian is the bridge between different types of people and technology
• Regardless, the connection is that people have information in common
• We’re here today to provide a space and time to collaborate and jam together

Linda Woodcock – RDA: 10 Things

• RDA is to make it more operable and readable
• Adopted by LC in March 2013, LAC May 2013
• Things that will look different
  • record as exactly as seen (for the most part)
  • no more latin
  • no more abbreviating
  • media types moved (no more GMD) to another field
  • FRBR
  • precise relationships between works
  • metadata registry employed
  • improve collocation
• main differences: more granularity
• implementation: updated MARC21, decide new fields, authority files
• future consideration: global changes to make more consistent

Mark Jordan – Adding Linked Data to SFU’s Editorial Cartoons Collection

• one of the few collections using controlled subjects (not just keywords) that are linked
• 6916 subjects in total (sources: TGM 1, TGM 2, LCSH, MESH)
• added URI (identifier on web) to vocab terms
• used scripts to match LOC triples to subjects being used
• the subject stores URI, but show user the readable information (in whichever language is needed)
• MODS entry in Islandora: when adding topic/term, populate with URI in background

Marcus Emmanuel Barnes – METS-Bagger Tool

• digitized collection not managed for long term preservation
• digitized content packaging specification being developed
• normalizing existing digitized content based on specification for robust long term management
• 2 components:
  • collection normalization script
  • integrity scripts based on collection manifest
• collection normalization: processes existing files to do inventory, organize them in a consistent manner
• incorporates METS, BagIt, and UUID (universally unique identifiers)
• developed in Python
• plugins for technical and descriptive metadata
• future: addition of manifest and integrity checking tools, additional plugins

Misty De Meo – Archivematica Format Policy Registry (FPR)

• configurable repository of recognized formats
• set of rules for identifying and processing files
• open-source, separate FPR admin app
• can be used by other projects or integrated into other platforms
• customizations allowed: adding new file formats, what file extensions maps to what file formats
• customize internal software behaviour
• Next: flexible metadata extraction

Cynthia Ng – Shifting Perspectives: From Disability Accommodation to Universal Design

• blog post write up

Stefan Khan-Kernahan – DocStore

• a very small part of course reserves
• replace current course reserve (Ares)
• keep up with changing needs/workflows
• scale better for collection size
• better user experience
• better handle requests e.g. copyright approval
• IE causes problems with https default

May Chan – Hackfests for the Uninitiated

• values diversity and inclusiveness
• went into the code4lib national thinking a “lowly” newcomer, cataloguer
• but got excited from learning from the person next to her that she could also learn from researching and reading
• there is no shame in looking over someone’s shoulder
• everyone can read
• collaborative learning can be infection and mutative
• it’s what you make out of it

That’s it for the talks! Breakout time beginning with a cute animal:

Filed under: Events

by Cynthia at November 29, 2013 04:52 AM

Never heard of ALTO (Analyzed Layout and Text Object)? Fear not, it is a very simple representation of structured text from physical pages, and I mean that in a positive way. Its primary use is to hold OCRed text. A piece of ALTO XML is a list of boxes, representing pages. Each page is a list of boxes, representing blocks. Blocks can, among other things, be a list of boxes, representing lines. Each line is a list of boxes, representing words. Each word is just a word. No more Chinese boxes. For all its simplicity, there is a curious lack of simple viewers for the format. That’s where Quack comes in.

Quack is a bash hack from yours truly that grew out of control. The focus is small-scale quality assurance: A few hundred images is ok, a few thousand is manageable. It traverses a folder structure of images with corresponding ALTO-files and creates HTML-pages containing zoomable images with ALTO structure overlays. The pages can be viewed directly from the file system or (of course) put on a web server. Features are

• OpenSeadragon used for smooth zoom & pan of images of arbitrary size (arbitrary meaning at least up to 50 gigapixel in this context)
• Overlays
• Connected blocks for inspecting segmentation quality
• Grid for inspecting skewing and rotation
• Burned out high-lights and low-lights
• OCR-text on mouse over
• Histogram for each image
• Darkest and brightest greyscale value with percentage (e.g. 12% of the images is absolute black)
• Thumbs view for folders
• Burned out high-lights and low-lights overlays in thumbs view
• Iterative updates of the display files (additions does not require a full rebuild)
• Works with a fairly standard Linux setup, except for deepzoom which is optional but recommended for large images. Should work under Cygwin but this is not tested
• Dog slow for big ALTO files as bash is heavily misused for text processing and HTML templating

Quack is Apache 2.0 Open Source. Clone it from https://github.com/tokee/quack and give it a spin. It even has sample images to get you started.

by Toke Eskildsen at November 28, 2013 08:19 PM

For this presentation, I decided to speak more broadly on accessibility (rather than focus specifically on web accessibility), partly because it’s so short (5-10 minute lightning talk) and partly due to the fact that despite it being a “Code4Lib” regional, we wanted to promote cross collaboration across all skill and knowledge levels. I still used a technology example, but had physical space related examples as well.

Morning everyone [insert other standard introductory information about myself]

What is a Disability?

Let’s start with this question: what is a disability? That might be difficult to define, so how about we answer this first: what are the different types of disabilities that people can name? > And maybe this (stormtrooper) image will help.

By identifying the different types of disabilities, we have an idea of what a disability is. Now…

How Do We Know if Someone Has a Disability?

Ideas? Many of the physical and more severe levels of a disability might be visible, such as when you see someone using a

• wheelchair,
• hearing aid or signing, or
• cane or guide dog.

What Can’t We See?

Obviously, one of the biggest problems is that we can’t always see if someone has a disability. This is particularly true of what we design in our virtual spaces, since we cannot see what setup they’re using (image: TV room), or who might be trying to use our virtual services (image: lizard with phone). This leads us to the question:

How Do We Develop for People with Disabilities?

I’m referring to any tool, program, or service. When developing or revising something, we need to take into account people with disabilities. We seem to generally do this by following a development process: where frequently, it happens like this: develop the tool, add or adjust things to work with assistive technology, then launch. Then again, the launch may actually happen before taking assistive technology into consideration.

If we take websites as an example, we might develop a website, and afterwards, check that it works with screen readers, then fix things as necessary. Of course, screen readers are not the only assistive technology used with websites. That means after developing the website, we need to check it against other assistive technology. We could make a list, but in order to do that, we need to understand what assistive technology is out there.

So…

What is Assistive Technology?

an umbrella term that includes [...] devices for people with disabilities [...] by enabling people to perform tasks that they were formerly unable to accomplish, or had great difficulty accomplishing, by providing enhancements to, or changing methods of interacting with, the technology needed to accomplish such tasks. -Wikipedia

Basically, it’s a piece of technology that helps a person accomplish a task.

What Assistive Technology Do We Need to Test With?

Okay, so let’s go back to our website. We need to test it using a screen reader, oh but what about the zoom function a lot of people use? What about the people who can’t use a mouse very well, so only use a keyboard or touch screen?

Quick question: how many people here use a touch screen on a daily basis? That includes any tablets or phones.

In fact, if we go back to our basic definition that assistive technology is a piece of technology that helps a person do something, then…

All Technology is Assistive Technology

- Sara Hendren

Please stop and think about this for a moment, about how it applies to your work and your organization. This is also a request that if the development process I presented is how things are done on your team, please stop and think about how you might change things.

We all use various technology to accomplish tasks, so if we’re designing anything web-based, we need to be developing not for certain types of people, but for any potential situation and access through any potential technology (image: mobile devices).

The Solution?

“Move away from the approach of building separately for disabled users, and concern yourself with creating clean, beautiful, usable, and accessible websites.“
- Debra A. Riley-Huff (2012). Web Accessibility and Universal Design. Library Technology Reports, 48(7), 29-35.

Riley-Huff was specifically referring to websites, but her point is that we should be changing our methodology by using…

Universal Design

Universal design means designing for the greatest extent of people regardless of their age, ability or status. The term was originally coined by the architect, Ronald Mace, so the principles (image: original 7 principles) tend to be based on physical space.

One of the classic examples is the wheelchair ramp (image), but can you think of other people who might need to use this? The list might include seniors with mobility walkers, parents with strollers, and moving carts. A lot of places label it an “access ramp” or just “ramp” instead of a “wheelchair ramp”, reflecting the greater, more universal use.

Take Away

The point is that we should be thinking about our entire audience from the beginning. We’re not serving the “average user”, there is no such thing. Instead, let’s design for all.

Side Note: Design for all is actually a separate term (referring to a design philosophy focused on making services, tools, etc. usable without the need for adaptation), but covers the same idea.

I know that this seems like a lofty goal, but I hope that this presentation gave you a sense of how you, me, all of us can start shifting our perspective from accommodating for people with disabilities to designing  for everyone whether it be related to the web, technology in general, programs, services, or spaces.

Filed under: Events, Methodology, Web design

by Cynthia at November 28, 2013 06:51 PM

This is a guest blog post by Timothy Vollmer, Manager of Policy and Data at Creative Commons.

Creative Commons has finally released Version 4.0 of the license suite. It’s been two years since we began the license update process, but now it’s done. The 4.0 licenses are the most global, legally robust licenses produced by CC to date.

You can find highlights of the changes on the website. Probably the most significant improvement is the expansion in license scope to include sui generis database rights–those copyright-like rights that exist in Europe and a few other countries which are granted to those who exert some effort into compiling a database. The 4.0 licenses (of course, those aligned with the Open Definition) are now better suited for use by governments and publishers of public sector information and open data. A few other changes:

A more global license The new licenses have improved terminology that’s better understood worldwide. And there will be official translations of the CC licenses, so that users of around the world can read and understand the complete licenses in their local languages.

Common-sense attribution The licenses explicitly permit licensees to satisfy the attribution requirement with a link to a separate page for attribution information. This was already common practice on the internet and possible under earlier versions of the licenses, and Version 4.0 alleviates any uncertainty about its use.

30-day window to correct license violations All CC licenses terminate when a licensee breaks their terms, but under 4.0, a licensee’s rights are reinstated automatically if she corrects a breach within 30 days of discovering it. This is common sense and how several other open licenses handle inadvertent violations.

Clarity about adaptations The BY and BY-NC 4.0 licenses are more clear about how adaptations are to be licensed. These licenses now clarify that you can apply any license to your contributions you want so long as your license doesn’t prevent users of the remix from complying with the original license.

There is a more in-depth discussion of the various policy decisions and versioning considerations here. We extend our gracious thanks to everyone in the open licensing community who’ve helped bring 4.0 to life.

Image source: By Petey21 (Own work) [Public domain], via Wikimedia Commons

by Guest at November 28, 2013 02:35 PM

Today I found the following resources and bookmarked them on <a href=

• Free online image converter
• Online video converter

Digest powered by RSS Digest

The post Bookmarks for November 27, 2013 appeared first on What I Learned Today....

by Nicole C. Engard at November 27, 2013 08:30 PM

Space doesn’t matter, except when it does.

The other day Koha bug 11308 was filed reporting a problem with the public catalog search RSS feed. This affected just the new Bootstrap theme.

The bug report noted that when clicking on the RSS feed icon, the page rendered “not like an rss feed should”. That means different things to different web browsers, but we can use an RSS feed validation service like validator.w3.org to see what feed parsers are likely to think.

Before the bug was fixed, the W3C validator reported this:

This feed does not validate.
line 2, column 0: XML parsing error: :2:0: XML or text declaration not at start of entity [help]
<?xml version=’1.0′ encoding=’utf-8′ ?>

Of course, at first glance, the XML declaration looks just fine -- the key bit is that it is starting at the second line of the response.

Space matters -- XML requires that if an XML declaration is present, it must be the very first thing in the document.

Let's take a look at the patch, written by Chris Cormack, that fixes the bug:

--- a/koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-opensearch.tt
+++ b/koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-opensearch.tt
@@ -1,5 +1,5 @@
-[% USE Koha %]
 <?xml version='1.0' encoding='utf-8'  ?>
+[% USE Koha %]
 [% IF ( opensearchdescription ) %]
 <OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
    <ShortName>[% LibraryName |html %] Search</ShortName>

This patch moves the [% USE Koha %] Template Toolkit directive from before the XML declaration to after it. [% USE Koha %] loads a custom Template Toolkit module called "Koha"; further down in the template there is a use of Koha.Preference() to check the value of a system preference.

But why should importing a TT module add a blank line? By default, Template Toolkit will include all of the whitespace present in the template. Since there is a newline after the [% USE Koha %] directive, that newline is included in the response.

Awkward, when spaces matter.

However, Template Toolkit does have a way to chomp whitespace before or after template directives.

This means that an alternative fix could be something like this:

--- a/koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-opensearch.tt
+++ b/koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-opensearch.tt
@@ -1,4 +1,4 @@
-[% USE Koha %]
+[% USE Koha -%]

Adding a single hyphen here means that whitespace after the TT directive should be chomped -- in other words, not included in the response.

Most of the time, extra whitespace doesn't matter for the HTML emitted by Koha. But when space matters... you can use TT to control it.

by Galen Charlton at November 27, 2013 04:32 PM

What questions do you have about WSKeys?

Depending on your level of expertise with authentication, you might be wondering anything from how you pronounce “WSKey” to how we support authentication for multiple types of clients: server-side scripts, client-scripts, mobile devices. We spend a lot of time anticipating your questions and thinking about what would make it easier for you to implement WSkeys in order to use our web services. We're even working on client libraries and example code in several languages.

read more

by hostetls at November 27, 2013 02:32 PM

On Monday of this week, the Open Knowledge Foundation signed a memorandum of understanding with the BBC. The BBC also signed separate memorandums with the Europeana Foundation, the Open Data Institute and the Mozilla Foundation.

Laura James, CEO of the Open Knowledge Foundation, signs the MoU with James Purnell, BBC Director of Strategy and Digital.

The signing is an important step in cementing the relationship between the Open Knowledge Foundation and one of the world’s largest broadcasting organisations. It also marks a new commitment on the part of the BBC to embrace open data and open standards. James Purnell, Director of Strategy and Digital at the BBC, said that this memorandum signalled that the BBC was “here for audience’s interests and not just the BBC’s” and that through it the BBC plans to find “find new ways to engage audiences”.

The signing ceremony was a formal recognition of the work the five organisations had done together and paved the way for future collaborations, some of which are already in the pipeline. In January 2014, for instance, the Open Knowledge Foundation, alongside the BBC and the Wikimedia community, will be coordinating the first ever “speakerthon”. Using the BBC’s vast radio archive, participants will tag and select snippets of notable individuals’ voices in order to upload them to Wikipedia articles as open content. Developers at the BBC are keen to use the crowdsourced data to tag other parts of the archive and automatically identify where else a given individual is speaking. This initiative is a great demonstration of the kind of benefit open data and open content can have for an organisation like the BBC. It allows them to simultaneously use their rich digital archive to improve existing open resources like Wikipedia, whilst developing new and innovative ways to harness the power of their audiences to improve their own digital assets (in this case through crowdsourced voice identification).

Collaborations like the “speakerthon”, which enable audiences to be contributors as well as consumers of broadcast media, can be a cause for concern for cultural institutions, especially those like the BBC which were born in the heyday of industrial one-way broadcasting. I commend the BBC for taking these first steps to re-configuring the traditional relationship it has with its audiences in allowing them a more participatory role.

That is not to say that the idea of open is somehow alien to the BBC, quite the opposite. The BBC has a long history of supporting technological innovation and using the benefits it bring to improve access to information. Indeed, in its Charter the BBC sets two of its central purposes: “to sustain citizenship and civil society” and “to help to deliver to the public the benefit of emerging communications technologies”. In signing the memorandum of understanding on Monday, the BBC is affirming that it sees open data, open content and open standards as the key to connecting these two principles that are so deeply entrenched in its DNA.

As Bill Thompson, Head of Partnership Development at the BBC archive, said on Monday the memorandum marks only the first step in a long conversation between the five organisations. The challenge is to turn these words into actions and concrete collaborations that will unlock the potential of the BBC’s vast archive of culturally and historically-significant material.

What kind of collaborations between the BBC and the Open Knowledge Foundation would you like to see? What do you think are the possibilities for audience participation and technological innovation using open data at the BBC? Send us your ideas in the comments.

PRESS RELEASE: BBC signs Memorandum of Understanding with the Open Knowledge Foundation

by Sam Leon at November 27, 2013 12:46 PM

It is once again time to take a trip around the world and hear a bit about some of the great things that are happening in our growing network of Open Knowledge Foundation Local Groups. Due to the sheer volume of activities and updates we want to share this month, our Global Community Stories post is coming to you in two instalments. Today, we’ll set our virtual feet down in Greece, Bangladesh, Canada and Argentina and later in the week we will share updates from Germany, Ireland and others.

Bangladesh: Afterthought from OKCon, Wikipedia collaboration, impressive media coverage and lots of events…

After participating in OKCon, the Open Knowledge Conference in Geneva in September, our Bangladeshi friends wrote an article about the experience. The article was published as lead story in Bangla newspaper The Daily Prohom Alo (the highest circulated paper & leading bangla IT portal). Well done!

Also on the event side the group has been really active; they celebrated Software Freedom Day (SFD) by organizing a seminar about open source & open knowledge. Shortly after, the group organized a meetup as part of the global celebration of the 30th Anniversary of the GNU open software license. In late October they joined the Online Data Expedition on October 18-20 on Garment Factories around the world. This data expedition was arranged in collaboration with School of Data.

In other news our Ambassador in Bangladesh, Nurunnaby Chowdhury Hasive, was recently elected as Administrator of Bangla Wikipedia, which was covered in among other The Daily Star and C News Voice. Open Knowledge Foundation Bangladesh also partnered with Society for the Popularization of Science, Bangladesh (SPSB) recently and will onwards organize events and other activities together – for instance children science congresses.

Greece: Organizing app-competitions and improving the web presence…

Our Greek friends most recently held their first event as part of the Apps4Greece series in the city of Thessaloniki (see the presentation slides here – and plans are now being made to expand to other cities. Last week all co-organizers of the event participated in a press conference, where also Open Knowledge Foundation’s Sander van der Waal took part and gave a presentation. Several blog posts and publications newspapers documented the event, among other this one from the Karfitsa newspaper. Next the group is working on a complete overhaul of their website and project portfolio, stay tuned!

Argentina: Doing presentations across the country, working with School of Data and helping develop crisis situation software…

Our Argentinean friends did a School of Data presentation in August at Hacks Hackers Buenos Aires in the good company of journalists, programmers and designers from three continents. Later, in September, they held a Open Spending Meetup that was visited by Open Knowledge Foundation old-timer and Knight-Mozilla Fellow Friedrich Lindenberg – which was followed later that month by a presentation at the Buenos Aires Institute of Technology and University of La Plata. Lastly, they presented at the International Conference of Free Software in Technopolis.

The group has also started collaborations with other organizations such as Chequeado; a platform where the public discuss, study and verify president’s political speeches. The collaboration aims to put together a neat database and an application that makes it easier for citizens to engage. Further, on the project side, the group has been working on School of Data with Mexico. They set up workshops that teach how to use open data tools: Every Wednesday a specialist hosts a workshop on tools such as Open Refine, d3.js display, Timeliner, Github, Fusiontable and more. The group is also helping with the Citizen Emergency project: An application that alerts citizens in disaster situations such as during floods. Next step is to help develop the application towards a national implementation.

Canada: Starting the group, hosting meetups, contributing very actively to the Open Data Census…

Back in July, the group held Open Data Canada Meetup #1, which was dedicated to starting up the local group in Canada and was attended by around 30 people. Read some of the follow-up blog posts here and here.

The group has also very actively been contributing to the Open Data Census as part of both the sprint leading up to the G8 meeting in North Ireland in June, but also as part of the bigger sprint leading up to the Open Government Partnership Summit in London in October – shortly before which we released the Open Data Index.

They have also been supporting the GeoThink.ca research project across Canada by co-organizing a meetup for Québec with Dr. Renée Sieber (McGill University, Montréal) and Dr. Stéphane Roche (Université Laval, Québec City) – read more about that here and here.

Furthermore, they have been supporting the Popolo project to develop open government data specifications, focusing on the legislative branch of government, while remaining useful to a broad set of use cases.

Support has also been given to the use of an Open Agenda about Free-Open-Libre within an Agenda aggregator, the Agenda du Libre by FACiL.qc.ca. Lastly, the group has promoted the CKAN open source data handling platform, which has subsequently been integrated into the City of Montréal Open Data Portal that launched just a few weeks ago. One of our Canadian Ambassadors, Diane Mercier, is the City of Montréal Open Data Project Manager.

by Christian Villum at November 27, 2013 10:25 AM

This is the text of a talk I gave at the (wonderful) National Digital Forum in Wellington, New Zealand on November 27th, 2013. You can also find my slides here.

---

Hi there. Thanks for inviting me to NDF 2013, it is a real treat and honor to be here. I’d like to dedicate this talk to Aaron Swartz. Aaron cared deeply about the Web. In a heartbreaking way I think he may have cared more than he was able to. I’m not going to talk much about Aaron specifically, but his work and spirit underly pretty much everything I’m going to talk about today. If there is one message that I would like you to get from my talk today it’s that we need to work together as professionals to care for the Web in the same way Aaron cared for it.

Next year it will be 25 years since Tim Berners-Lee wrote his proposal to build the World Wide Web. I’ve spent almost half of my life working with the technology of the Web. The Web has been good to me. I imagine it has been good to you as well. I highly doubt I would be standing here talking to you today if it wasn’t for the Web. Perhaps the National Digital Forum would not exist, if it was not for the Web. Sometimes I wonder if we need the Web to continue to survive as a species. It’s certainly hard for my kids to imagine a world without the Web. In a way it’s even hard for me to remember it. This is the way of media, slipping into the very fabric of experience. Today I’d like to talk to you about what it means to think about the Web as a preservation medium.

Medium and preservation are some pretty fuzzy, heavy words, and I’m not going to try to pin them down too much. We know from Marshall McLuhan that the medium is the message. I like this definition because it disorients more than it defines. McLuhan reminds us of how we are shaped by our media, just as we shape new forms of media. In her book Always Already New, Lisa Gitelman offers up a definition of media that gives us a bit more to chew on:

I define media as socially realized structures of communication, where structures include both technological forms and their associated protocols, and where communication is a cultural practice, a ritualized collocation of different people on the same mental map, sharing or engaged with popular ontologies of representation.

I like Gitelman’s definition because it emphasizes how important the social dimension is to our understanding of media. The affordances of media, how media are used by people to do things, and how media does things to us, are just as important as the technical qualities of media. In the spirit of Latour she casts media as a fully fledged actor, not as some innocent bystander or tool to be used by the real and only actors, namely people.

When Matthew Oliver wrote to invite me to speak here he said that in recent years NDF had focused on the museum, and that there was some revival of interest in libraries. The spread of the Web has unified the cultural heritage sector, showing how much libraries, archives and museums have in common, despite their use of subtly different words to describe what they do. I think preservation is a similar unifying concept. We all share an interest in keeping the stuff (paintings, sculptures, books, manuscripts, etc) around for another day, so that someone will be able to see it, use it, cite it, re-interpret it.

Unlike the traditional media we care for, the Web confounds us all equally. We’ve traditionally thought of preservation and access as different activities, that often were at odds with each other. Matthew Kirschenbaum dispels this notion:

… the preservation of digital objects is logically inseparable from the act of their creation — the lag between creation and preservation collapses completely, since a digital object may only ever be said to be preserved if it is accessible, and each individual access creates the object anew. The .txtual Condition

Or, as my colleague David Brunton has said, in a McLuhan-esque way:

Digital preservation is access…in the future.

The underlying implication here is that if you are not providing meaningful access in the present to digital content, then you are not preserving it.

In light of these loose definitions I’m going to spend the rest of the time exploring what the Web means as a preservation medium by telling some stories. I’m hoping that they will help illuminate what preservation means in the context of the Web. By the end I hope to convince you of two things: the Web needs us to care for it, and more importantly, we need the Web to do our jobs effectively. For those of you who don’t need convincing about either of these points, I hope to give you a slightly different lens for looking at preservation and the Web. It’s a hopeful and humanistic lens, that is informed by thinking about the Web as an archive. But more on that later.

Everything is Broken

Even the casual user of the Web has run across the problem of the 404 Not Found. In a recent survey of Web citations found in Thompson Reuter’s Web of Science, Hennessey and Ge found that only 69% of the URLs were still available, and the median lifetime for a URL was 9.3 years. The Internet Archive had archived 62% of these URLs. In a similar study of URLs found in recent papers in the popular arXiv pre-print repository Sanderson, Phillips and Van de Sompel found that of the 144,087 unique URLs referenced in papers, only 70% were still available and of these, 45% were not archived in the Internet Archive, Web Citation, the Library of Congress or the UK National Archive.

Bear in mind, this isn’t the World Wild Web of dot com bubbles, failed business plans, and pivots we’re talking about. These URLs were found in a small pocket of the Web for academic research, a body of literature that is built on a foundation of citation, and written by practitioners whose very livelihood is dependent on how they are cited by others.

A few months ago the 404 made mainstream news in the US when Adam Liptak’s story In Supreme Court Opinions, Web Links to Nowhere broke in the New York Times. Liptak’s story spotlighted a recent study by Zittrain and Albert which found that 50% of links in United States Supreme Court opinions were broken. As its name suggests, the Supreme Court is the highest federal court in the United States…it is the final interpreter of our Constitution. These opinions in turn document the decisions of the Supreme Court, and have increasingly referenced content on the Web for context, which becomes important later for interpretation. 50% of the URLs found in the opinions suffered from what the authors call reference rot. Reference rot includes situations of link rot (404 Not Found and other HTTP level errors), but it also includes when the URL appears to technically work, but the content that was cited is no longer available. The point was dramatically and humorously illustrated by the New York Times since someone had bought one of the lapsed domains and put up a message for Justice Alito:

Zittrain and Albert propose a new web archiving project called perma.cc, which relies on libraries to select web pages and websites that need to be archived. As proposed perma.cc would be similar in principle to WebCite, which is built around submission of URLs by scholars. But WebCite’s future is uncertain due to a fund drive to raise money to support its operation. perma.cc also has the potential to offer a governance structure similar to how cultural heritage organizations support the Internet Archive in their crawls of the Web.

Internet Archive was started by Brewster Kahle in 1996. It now contains 366 billion web pages or captures (not unique URLs). In 2008 Google Engineers reported that their index contained 1 trillion unique URLs. That’s 5 years ago now. If we assume it hasn’t grown since then, and overlook the fact that there are often multiple captures of a given URL over time, Internet Archive contains about 37% of the Web. This is overly generous since the Web has almost certainly grown in the past 5 years, and we’re comparing apples and oranges, web captures to unique URLs.

Of course, it’s not really fair (or prudent) to put the weight of preserving the Web on one institution. So thankfully, the Internet Archive isn’t alone. The International Internet Preservation Consortium is a member organization made up of national libraries, universities, and other organizations that do Web archiving. The National Library of New Zealand is a member, and has its own Web archive. According to the list of Web archiving initiatives Wikipedia article the archive is comprised of 346 million URLs. Perhaps someone in the audience has a rough idea of how big this is relative to the size of the Kiwi Web. It’s a bit of a technical problem even to identify national boundaries on the Web. Since the National Library of New Zealand Act of 2003, the National Library has been authorized to crawl the New Zealand portion of the Web. In this regard, New Zealand is light years ahead of the United States, which still is required by law to ask for permission to collect selected, non-governmental websites.

Protocols and tools for sharing the size and makeup of these IIPC collections are still lacking, but the Memento project spurred on some possible approaches out of necessity. For the Memento prototype to work they needed to collect the URL/timestamp combinations for all archived webpages. This turned out to be difficult both for the archive to share, and to aggregate in one place efficiently–and the moment it was done it was already out of date. David Rosenthal has some interesting ideas for aggregators to collect summary data from web archives, which is used to instead provide hints about where a given URL may be archived. Hopefully we’ll see some development in this area, as it’s increasingly important that Web archives do collection development more closely, to encourage diversity of approaches, and ensure that one isn’t a single point of failure.

Even when you consider the work of the International Internet Preservation Consortium, which adds roughly 75 billion URLs (also not unique) we still are only seeing 44% of the Web being archived. And of course this is a very generous guesstimate, since the 366 billion Internet Archive captures are not unique URLs: e.g. a given URL like the BBC homepage has been fetched 13,863 times between December 21, 1996 and November 14, 2013. And there is almost certainly overlap between the various IIPC web archives and the Internet Archive.

The Archival Sliver

I am citing these statistics not to say the job of archiving the Web is impossible, or a waste of resources. Much to the contrary. I raise it here to introduce one of the archival lenses I want to encourage you to look at Web preservation through: Verne Harris’ notion of the archival sliver. Harris is a South African archivist, writer and director of the Archive at the Nelson Mandela Centre of Memory. He participated in the transformation of South Africa’s apartheid public records system, and got to see up close how the contents of archives are shaped by the power structures in which they are embedded. Harris’ ideas have a distinctly post-modern flavor, and contrast with positivist theories of the archive that assert that the archive’s goal is to reflect reality.

Even if archivists in a particular country were to preserve every record generated throughout the land, they would still have only a sliver of a window into that country’s experience. But of course in practice, this record universum is substantially reduced through deliberate and inadvertent destruction by records creators and managers, leaving a sliver of a sliver from which archivists select what they will preserve. And they do not preserve much.

I like Harris’ notion of the archival sliver, because he doesn’t see it as a cause for despair, but rather as a reason to celebrate the role that this archival sliver has in the process of social memory, and the archivist who tends to it.

The archival record … is best understood as a sliver of a sliver of a sliver of a window into process. It is a fragile thing, an enchanted thing, defined not by its connections to “reality,” but by its open-ended layerings of construction and reconstruction. Far from constituting the solid structure around which imagination can play, it is itself the stuff of imagination.

The First URL

So instead of considering the preservation of billions of URLs, lets change tack a bit and take a look at the preservation of one, namely the first URL.

http://info.cern.ch/hypertext/WWW/TheProject.html

On April 30th, 1993 CERN made (in hindsight) the momentous decision to freely-release the Web technology software that Tim Berners-Lee, Ari Luotonen and Henrik Nielsen created for making the first website. But 20 years later, that website was no longer available. To celebrate the 20th anniversary of the software release Dan Noyes from CERN led a project to bring the original website back online, at the same address using a completely different software stack. The original content was collected from a variety of places: some from the W3C, some from a 1999 backup of Tim Berners-Lee’s NeXT. While the content is how it looked then, the resurrected website isn’t running the original Web server software, it’s running a modern version of Apache.

CERN also hosted a group of 11 volunteer developers to spend 2 days coding at CERN (expenses paid) to recreate the experience of using the line mode browser (LMB). The LMB allowed users with an Internet connection to use the Web without having to install any software: they could simply telnet to info.cern.ch and start browsing the emerging Web using their terminal. These developers created a NodeJS JavaScript application that simulates the experience of using the early Web. You can even use it to navigate to other pages, for example the current World Wide Web Consortium page.

In a lot of ways I think this work illustrates James Governor’s adage:

Applications are like fish, data is like wine. Only one improves with age.

As any old school LISP programmer will tell you, sometimes code is data and data is code. But it is remarkable that this 20 year old HTML still renders just fine in a modern Web browser. This is no accident, but is the result of thoughtful, just-in-time design that encouraged the evolvability, extensibility and customizability of the Web. I think we as a community still have lots to learn from the Web’s example, and lots more to import into our practices. More about HTML in a bit.

Permalinks

Now obviously this sort of attention can’t be paid to all broken URLs on the Web. But it seems like an interesting example of how an archival sliver of the Web was cared for, respected and valued. Despite popular opinion, the care for URLs is not something foreign to the Web. For example lets take a look at the idea of the permalink that was popularized by the blogging community. As you know, a blog is typically a stream of content. In 2000 Paul Bausch at Blogger came up with a way to assign URLs to individual posts in the stream. This practice is so ubiquitous now it’s difficult to see what an innovation it was at the time. As its name implies, the idea of the permalink is that it is stable over time, so that the content can be persistently referenced. Apart from longevity, permalinks have beneficial SEO characteristics: the more that people link to the page over time, the higher its PageRank, and the more people who will find it in search results.

A couple years before the blogging community started talking about permalinks, Tim Berners-Lee wrote a short W3C design note entitled Cool URIs Don’t Change. In it he provides some (often humorously snarky) advice for people to think about their URLs, and namespaces with an eye to their future. One of Berners-Lee’s great insights was to allow any HTML document to link to any other HTML document, without permission. This decision allowed the Web to grow in a decentralized fashion. It also means that links can break when pages drift apart, and move to new locations, or disappear. But just because a link can break doesn’t mean that it must break.

The idea of Cool URIs saw new life in 2006 when Leo Sauerman and Richard Cyganiak began work on Cool URIs for the Semantic Web, which became a seminal document for the Linked Data movement. Their key insight was that identity (URLs) matters on the Web, especially when you are trying to create a distributed database like the Semantic Web.

Call them permalinks or Cool URIs, the idea is the same. Well managed websites will be rewarded by more links to their content, improved SEO, and ultimately more users. But most of all they will be rewarded by a better understanding of what they are putting on the Web. Organizations, particularly cultural heritage organization should take note — especially their “architects”. Libraries, archives and museums need to become regions of stability on the Web, where URLs don’t capriciously fail because some exhibit is over, or some content management system is swapped out for another. This doesn’t mean content can’t change, move or even be deleted. It just means we need to know when we are doing it, and say where something has moved, or say that what was once there is now gone. If we can’t do it, the websites that do will become the new libraries and archives of the Web.

Community

Clearly there is a space between large scale projects to archive the entire Web, and efforts to curate a particular website. Consider the work of ArchiveTeam, a volunteer organization formed in 2009 that keeps an eye on when websites are in danger of, actually are, closing their doors and shutting down. Using their wiki, IRC chatrooms, and software tools they have built up a community of practice around archiving websites, which have included some 60 sites, such as Geocities and Friendster. They maintain a page called the Death Watch where they list sites that are dying (pining for the fjords), or in danger of dying (pre-emptive alarm bells). These activist archivists run something called the Warrior which is a virtual appliance you can install on a workstation, which gets instructions from the Archive Team tracker about which URLs to download, and coordinates the collection. The tracker then collects statistics, that allow participants to see how much they have contributed relative to others. The collected data is then packed up as WARC files and delivered to the Internet Archive where it is reviewed by an adminstrator, and added to their Web collection.

ArchiveTeam is a labor of love for its creator Jason Scott Sadofsky (aka Jason Scott) who is himself an accomplished documenter of computing history, with films such as BBS: The Documentary (early bulletin board systems), Get Lamp (interactive fiction) and DEFCON: The Documentary. Apart from mobilizing action, his regular talks have raised awareness about the impermanence on the Web, and have connected with other like minded Web archivists in a way that traditional digital preservation projects have struggled to. I suspect that this self-described “collective of rogue archivists, programmers, writers and loudmouths dedicated to saving our digital heritage” is the shape of things to come for the profession. ArchiveTeam are not the only activists archiving parts of the Web, lets take a look at a few more examples.

Earlier this year Google announced that they were pulling Google Reader offline. This caused much grief and anger to be vented from the blogging community…but it spurred one person into action. Mihai Parparita was an engineer who helped create Google Reader at Google, but he no longer worked there, and wanted to do something to help people retain both their data and the experience of Google Reader. Because he felt that the snapshots of data provided weren’t complete, he quickly put together a project ReaderIsDead, which is also available on GitHub. ReaderIsDead is actually a collection of different tools: one for pulling down your personal Reader data from Google while the Google Reader servers were still alive, and a simple web application called ZombieReader that serves that data up, for when the Google Reader servers actually went dead. Mihai put his knowledge of how the Google Reader talked to backend Web service APIs to build ZombieReader.

Ordinarily fat client interfaces like Google Reader pose problems for traditional Web archiving tools like Internet Archive’s Heretrix. Fat Web clients are applications that largely run in your browser, using JavaScript. These applications typically talk back to Web service APIs to fetch more data (often JSON) based on interaction in the browser. Web archiving crawlers don’t typically execute JavaScript that is crawled from the Web, and have a hard (if not impossible) time simulating user behavior, which then triggers the calls back to the Web service. And of course the Web Service is what goes dead as well, so even if the Web archive has a snapshot of the requested data, the JavaScript would need to be changed to fetch it. This means the Web archive is left with a largely useless shell.

But in the case of Zombie Reader, the fat client provided a data abstraction that proved to be an excellent way to preserve both the personal data and the user experience of using Google Reader. Mihai was able to use the same HTML, CSS and JavaScript from GoogleReader, but instead of the application talking back to Google’s API he had it talk back to a local Web Service that sat on top of the archived data. Individual users could continue to use their personal archives. ZombieReader became a read-only snapshot of what they were reading on the Web, and their interactions with it. Their sliver of a sliver of a sliver.

Impermanence

Of course .com failures aren’t the only reason why content disappears from the Web. People intentionally remove content from the Web all the time for a variety of reasons. Let’s consider the strange, yet fascinating cases of Mark Pilgrim and Jonathan Gillette. Both were highly prolific software developers, bloggers, authors and well known spokespeople for open source and the Web commons.

Among other things, Mark Pilgrim was very active in the area of feed syndication technology (RSS, Atom). He wrote the feed validator and Universal Feed Parser that makes working with syndicated much easier. He also pushed the boundaries of technical writing by writing Dive Into HTML 5 and Dive Into Python 3 which were published traditionally as books, but also made available on the Web with a CC-BY license. Pilgrim also worked at Google, where he helped promote and evolve the Web with his involvement in the Web Hypertext Application Technology Working Group WHATWG.

Jonathan Gillette, also known as Why the Lucky Stiff or _why, was a prolific writer, cartoonist, artist, and computer programmer who helped popularize the Ruby programming language. His online book Why’s (Poignant) Guide to Ruby introduced people of all ages to the practice of programming with wit and humor that will literally make you laugh out loud as you learn. His projects such like Try Ruby and Hackety Hack! lowered the barriers to getting a working software development environment set up by moving it to the Web. He also wrote a great deal of software such as hpricot for parsing HTML, and the minimalist Web framework camping.

Apart from all these similarities Mark Pilgrim and Jonathan Gillette share something else in common: on October 4, 2011 and August 19, 2009 respectively they both decided to completely delete their online presence from the Web. They committed info-suicide. Their online books, blogs, social media accounts, and github projects were simply removed. No explanations were made, they just blinked out of existence. They are still alive here in the physical world, but they aren’t participating online as they were previously…or at least not using the same personas. I like to think Pilgrim and _why were doing performance art to illustrate the fundamental nature of the Web, its nowness, its fragility, it’s impermanence. As Dan Connolly said once:

The point of the Web arch[itecture] is that it builds the illusion of a shared information space.

If someone decides to turn off a server or delete a website it’s gone for the entire world, the illusion dissolves. Maybe it lives on buried in a Web archive, but it’s previous life out on the Web is over. Or is it?

It’s interesting to see what happened after the info-suicides. Why’s (Poignant) Guide to Ruby was rescued by Mislav Marohnic a software developer living in Croatia. He was able to piece the book back together based on content in the Internet Archive, and put it back online at a new URL, as if nothing had happened. In addition he has continued to curate it: updating code samples to work with the latest version of Ruby, enabling syntax highlighting, converting it to use Markdown, and more.

Similarly Mark Pilgrim’s Dive Into HTML 5 and Dive Into Python 3 were assembled from copies and re-deployed to the Web. Prior to his departure Pilgrim used Github to manage the content for his books. Github is a distributed revision control system, where everyone working with the code has a full copy of it local on their machine. So rather than needing to get content out of the Internet Archive, developers created the diveintomark organization account on Github, and pushed their clones of the original repositories there.

Much of Why and Pilgrim’s code was also developed on GitHub. So even though the master was deleted, many people had clones, and were able to work together to establish a new master. Philip Cromer created the whymirror on Github, which collected _why’s code. Jeremy Ruten created _why’s Estate which is a hypertext archive collects pointers to the various software archives, and writings that have been preserved in Internet Archive and elsewhere.

So, it turns out that the supposedly brittle medium of the Web, where a link can easily break, and a whole website can be capriciously turned off, is a bit more persistent than we think. These events remind me of Matthew Kirschenbaum’s book Mechanisms which deconstructs notions of electronic media being fleeting or impermanent to show how electronic media (especially that which is stored on hard drives) is actually quite resilient and resistent to change. Mechanisms contains a fascinating study of how William Gibson‘s poem Agrippa (which was engineered to encrypt itself and become unreadable after a single reading) saw new life on the Internet, as it was copied around on FTP, USENET, email listservs, and ultimately the Web:

Agrippa owes its transmission and continuing availability to a complex network of individuals, communities, ideologies, markets, technologies, and motives … from its example we can see that the preservation of digital media has a profound social dimension that is at least as important as purely technical considerations. Hacking Agrippa

Small Data

In the forensic spirit of Mechanisms, let’s take a closer look at Web technology, specifically HTML. Remember the first URL and how CERN was able to revive it? When you think about it, it’s kind of amazing that you can still look at that HTML in your modern browser, right? Do you think you could view your 20 year old word processing documents today? Jeff Rothenberg cynically observed

digital information lasts forever—or five years, whichever comes first

Maybe if we focus on the archival sliver instead of the impossibility of everything we’re not doing so bad.

As we saw in the cases of Pilgirm and _why the Internet Archive and other web archiving projects play an important role in snapshotting Web pages. But we are also starting to see social media companies are building tools that allow their users to easily extract or “archive” their content. These tools are using HTML in an interesting new way that is worth taking a closer look at.

How many Facebook users are there here? How many of you have requested your archive? If you navigate to the right place in your settings you can “Download a copy of your Facebook data.” When you click on the button you set in motion a process that gathers together your profile, contact information, wall, photos, synced photos, videos, friends, messages, pokes, events, settings, security and (ahem) ads. This takes Facebook a bit of time, it took a day the last time I tried it, and you get an email when it’s finished which contains a link to download a zip file. The zip file contains HTML, JPEG, MP4 files which you can open in your browser. You don’t need to be connected to the Internet, everything is available locally.

Similarly Twitter allow you to request an archive periodically, which triggers an email when it is ready for you to pick up. Much like the Facebook archive it is delivered as a zip file, which contains an easily browsable HTML package. The Twitter archive is actually more like a dynamic application, since it includes a JavaScript application called Grailbird. Grailbird lets you search your tweets, and examine tweets from different time periods. Just like the Facebook archive everything Grailbird needs is available locally, and the application will work when you are disconnected from the Internet. Although user’s avatar thumbnail images are still loaded directly from the Web. But all your tweet data is available as JavaScript and CSV. The application depends on some popular JavaScript libraries like jQuery and Underscore, but those also are bundled right with the archive. It would be nice to see Twitter release Grailbird as a project on Github as many of their other software projects are. Thinking of Grailbird as a visualization framework for tweets would allow interested parties to add new visualizations (e.g. tweets on a map, network graphs, etc). You could also imagine tools for reaching out into the network of an individual’s tweets to fetch tweets that they were replying to, and persisting them back locally to the package.

Some of you may remember that the Data Liberation Front (led by Brian Fitzpatrick at Google) and the eventual product offering Google Takeout were early innovators in this area. Google Takeout allows you to download data from 14 of their products as a zip file. The service isn’t without criticism, because it doesn’t include things like your Gmail archive or your search history. The contents of the archive are also somewhat more difficult to work with, compared to the Facebook and Twitter equivalents. For example, each Google+ update is represented as a single HTML file, and there isn’t a notion of a minimal, static application that you can use to browse them. The HTML also references content out on the Web, and isn’t as self-contained as Twitter and Facebook’s archive. But having snapshots of your Youtube videos, and contents of your Google Drive is extremely handy. As Brad Fitzpatrick wrote in 2010, Google Takeout is kind of a remarkable achievement, or realization for a big publicly traded behometh to make:

Locking your users in, of course, has the advantage of making it harder for them to leave you for a competitor. Likewise, if your competitors lock their users in, it is harder for those users to move to your product. Nonetheless, it is far preferable to spend your engineering effort on innovation than it is to build bigger walls and stronger doors that prevent users from leaving. Making it easier for users to experiment today greatly increases their trust in you, and they are more likely to return to your product line tomorrow.

I mention Facebook, Twitter and Google here because I think these archiving services are important for memory institutions like museums, libraries and archives. They allow individuals to download their data from the huge corpus that is available–a sliver of a sliver of a sliver. When a writer or politician donates their papers, what if we accessioned their Facebook or Twitter archive? Dave Winer for example has started collecting Twitter archives that have been donated to him, that meet a certain criteria, and making them public. If we have decided to add someone’s papers to a collection, why not acquire their social media archives and store them along with their other born digital and traditional content? Yes, Twitter (as a whole) is being archived by the Library of Congress, as so called big data. But why don’t we consider these personal archives as small data, where context and original order are preserved with other relevant material in a highly usable way? As Rufus Pollock of the Open Knowledge Foundation said

This next decade belongs to distributed models not centralized ones, to collaboration not control, and to small data not big data.

The other interesting thing about these services is their use of HTML as a packaging format. My coworker Chris Adams once remarked that the one format he expects to be able to read in 100 years is HTML. Of course we can’t predict the future. But I suspect he may be right. We need best practices, or really just patterns for creating HTML packages of archival content. We need to make sure our work sits on top of common tools for the Web. We need to support the Web browser, particularly open source ones. We need to track and participate in Web standardization efforts such as the W3C and the WHATWG. We must keep the usability of the archive in mind: is it easy to open up in your browser and wander around in the archive as with the Twitter and Facebook examples? And most importantly, as Johan van der Knijff of the National Library of the Netherlands discusses in his study of EPUB, it is important that all resources are local to the package. Loading images, JavaScript, etc from a remote location makes the archive vulnerable, since it becomes dependent on some part of the Web staying alive. Perhaps we also need tools like ArchiveReady for inspecting local HTML packages (in addition to websites) and reporting on their archivability?

Conclusion

So how to wrap up this strange, fragmented, incomplete tour through Web preservation? I feel like I should say something profound, but I was hoping these stories of the Web would do that for me. I can only say for myself that I want to give back to the Web the way it has given to me. With 25 years behind us the Web needs us more than ever to help care for the archival slivers it contains. I think libraries, museums and archives that realize that they are custodians of the Web, and align their mission with the grain of the Web, will be the ones that survive, and prosper. Brian Fitzpatrick, Jason Scott, Brewster Kahle, Mislav Marohnic, Philip Cromer, Jeremy Ruten and Aaron Swartz demonstrated their willingness to work with the Web as a medium in need of preservation, as well as a medium for doing the preservation. We need more of them. We need to provide spaces for them to do their work. They are the new faces of our profession.

by ed at November 27, 2013 12:59 AM

BIBFRAME Through The Looking Glass

A strong indication that something may be  amiss in BIBFRAME is that the picture drawn by its  model of the bibliographic universe  does not include books, and  as Oxford logician Charles Lutwidge Dodgson observed, “What use is a picture without books?”.

What, If Anything, Is a bf:Holding?

Rather than modelling a specific copy of a book, a BIBFRAME Holding models someone saying things that appear to be about a specific copy of a book. Surprisingly, there is nothing in BIBFRAME’s ontology that corresponds to any copy of a book that the statement might be about.

We cannot ask how many copies of books we know about (unless we are satisfied with the trivial answer “none”). We can only count how many times we have been told about copies of books.   If we have a BIBFRAME Holding from OCLC saying that UNC has a copy of a certain book, and another from UNC saying the same thing, we have two different Holdings.

From this behavior, it seems that a BIBFRAME Holding corresponds to a MARC Holdings Records, rather than a physical book, or FRBR Item.

Do BIBFRAME Annotations  Support Provenance for Holding?

One of the reasons given for  BIBFRAME Annotations is that they carry information about who asserted a statement (via the bf:annotationAssertedBy property).

Despite the  conflation  of Epistemic  and Ontological concerns that we see in bf:Holding, we cannot rely on  the annotationAssertedBy property to support provenance.  If we have one data source that gives one value for  bf:annotationAssertedBy, and a different data source that gives another, conflicting value, we  have no principled way to choose between them, unless we have other provenance information to support our decision.

The problem is readily apparent in the example used for the Holdings Annotation.

<http://xyz.org/bibframeTestCases/Annotations/Annotation/Holding>
    a bf:Holding ;
    ...
    bf:annotationAssertedBy <http://id.loc.gov/vocabulary/organizations/dlc> ;
    bf:dateOfAssertion "20130807" ;
....

The URI for the Holdings Annotation is in the namespace “http://xyz.org/”.   The host from which the document was   retrieved is in the bibframe.org domain.

There is a statement claiming that the Holdings Annotation was asserted by http://id.loc.gov/vocabulary/organizations/dlc .

There is no a priori reason to prefer this assertion  over a claim, in an equally unrelated graph, that

<http://xyz.org/bibframeTestCases/Annotations/Annotation/Holding>
     bf:annotationAssertedBy <http://id.loc.gov/authorities/subjects/sh89004503> ;

The value of a provenance mechanism that cannot be used without an additional  provenance mechanism is questionable.

Are Annotations Consistently Used For Opinions?

Section 2.1 of the Annotations document states that “Annotations are asserted in order to: [...] express opinions about a resource”.

However, this principle does not seem to be widely applied.  To take the most obvious example, “subject” is treated as a property, despite  Patrick Wilson’s generally accepted arguments that “the subject of a work” is indeterminate.

It is hard to understand the theoretical motivation for  treating subjects as objective, yet regarding cover images regarded as subjective.

Is Provenance Information Needed For All Assertions?

Since  information about a resource is only obtainable empirically, filtered through perception and mind, and  recorded empirically through misperception and typos, without recorded provenance information  we have  weak justification for any bibliographic knowledge we may claim.

The more independent sources for a statement about a sensible property, the more strongly we may be justified in  believing it.

The more trust we have in the expertize of a source with regards to a particular topic, the more we may trust their opinions that a work is or is not about a subject within that topic; however we might simultaneously accept as equally valid the opinion of a specialist in a different field that the work is about a subject in that  field.

If the goal of BIBFRAME is to support the fusion  of  descriptions from multiple different sources, then  provenance  must be supported for all assertions.  If the goal of BIBFRAME is to provide another syntax for storing monolithic records, then the overhead of decomposing these records into individual assertions would seem to require additional justification.

by ses at November 26, 2013 10:18 PM

Today I found the following resources and bookmarked them on <a href=

• LibreCat
  Open Tools for Library and Research Services

Digest powered by RSS Digest

The post Bookmarks for November 26, 2013 appeared first on What I Learned Today....

by Nicole C. Engard at November 26, 2013 08:30 PM

Thanks to Clarke who pointed out in a comment on a recent post here, that Google Scholar now has a “saved citations” citation management feature.

I haven’t done any experimenting with it; anyone have a review? What do you think, is this going to end up drawing a significant portion of our patron’s use away from other citation management alternatives (including some we pay for)?

Google Scholar Library. 

Today we’re launching Scholar Library, your personal collection of articles in Scholar. You can save articles right from the search page, organize them by topic, and use the power of Scholar’s full-text search & ranking to quickly find just the one you want – at any time and from anywhere. You decide what goes into your library and we’ll provide all the goodies that come with Scholar search results – up to date article links, citing articles, related articles, formatted citations, links to your university’s subscriptions, and more. And if you have a public Scholar profile, it’s easy to quickly set up your library with the articles you want – with a single click, you can import all the articles in your profile as well as all the articles they cite.

Filed under: General

by jrochkind at November 26, 2013 05:51 PM

It was just last week that we invited the open communities to start collaborating and warming up for the upcoming Open Knowledge Festival. Today we can already share with you the learnings and outcomes of the first OKFestival Meetup (in Berlin) – as we would love to imagine it, this was just the first in a long, diverse and busy series (no pressure, it’s all up to you!).

Folks from the Open Knowledge Foundation, Code for All, Free Software Foundation Europe, Open Bank Project, AfricaHackTrip, Hacks/Hackers, and Open Product Data facilitated a great evening of skill-sharing, peer-to-peer learning and exciting findings from each other’s projects. We spread the word about open (what is open? What does open knowledge mean? How many things around us can be open?), dug into a multitude of open projects and started discussing and writing down first ideas to be proposed for the festival (we’ll call for the community’s proposals to shape the agenda very soon – sign up for the festival’s newsletter to be the first to know when).

But which are exactly the ingredients that made the evening special? We listed them and thought about sharing with you all – and you’re welcome to have a go too.

• Diversity – the speakers came from different open communities (and countries, and languages) and we also invited people who wanted to discover more about open to join. Both newcomers and experienced folks in the open fields found a lot to learn from each other. The open ecosystem spreads itself in all directions, from open source, to open culture, government, science, education, environment and even to products we buy at the supermarket. There is so much to learn, talk about and share – and our peers can be the best source of new exciting discoveries. This doesn’t mean that multiple topics are always recommended, but inviting people with different backgrounds can often be a plus.

• Online meets offline and viceversa – some of our attendees and speakers lived in the city we organised the event in, but that would be just half of the fun. Everybody was invited to join the conversation from anywhere in the world: on Twitter (ping the OKFestival crowd anytime including #OKFestival in your tweets), the events’ etherpad (prominently and publicly displayed during the event) and we also had a speaker from another country! Our guest this time was Katelyn Rogers talking about Open Product Data from the UK (see her projected on the wall in our pics). We missed to have Brazilian friends joining due to the overlapping Encontro Nacional de Dados Abertos on the same date – but next time we’ll be together. Hint: find time zones reasonably suitable with yours and ping people passionately working in the open space to invite them to your OKFestival Meetups! (The Open Knowledge Foundation Network could be a good place to start with)

• Participatory format – we all love listening to a thrilling lecture, but sometimes attending a talk with just a short Q&A at the end just doesn’t satisfy our curiosity to know more about new topics and people. So we experimented and organized the Berlin evening as a speed geeking session. What is it? Each project (and project’s presenter) has a table with chairs around. Participants are divided into groups, and each group starts sitting at one table, and talks with the presenter for 5 minutes. After that, at the sound of a gong (a kitchen pan, in our case) it’s time for each group to move to the next table and project. And if 5 minutes were not enough, there’s time (and rehydrating drinks) immediately after. It was a lot of fun – we’ll add the format to the Events Handbook soon.

So, what are you waiting for? Wherever you are, whatever is the field of open you’re passionate about, however you want to meet (be it in an embassy, in your kitchen, or in a bar!) – run your OKFestival Meetup! And don’t forget to share all about it on the dedicated etherpad; we can’t wait to hear what open means to you, feature your greatest meetups, and make OKFestival happen with you!

by Beatrice Martini at November 26, 2013 05:22 PM

In these prolegomena I will be examining how a lack of shared understanding of these issues may have lead to certain choices that have resulted in unintended consequences in the current proposed BIBFRAME. I will show that these choices and consequences are not intrinsic to all linked data or semantic approaches to bibliographic and other metadata, and that alternatives approaches may have advantages.

I will attempt to give general explanations of some of the terms and  concepts that may not be obvious to those outside the communities in which they involved.   By attempting to bridge the gaps in understanding separating the different communities needed to make any future metadata framework, I hope to make the case that such a framework is both possible and desirable, and to allow more people to make positive contributions to the creation and deployment of such a framework.

In this introduction I give an outline of the remaining prolegomena, and look at the types of knowledge and skills that are needed to build a future Bibliographic Framework .

An outline of the prolegomena

Part II.  What is a Holding that cannot be held?

In the second prolegomenon, I use the Holdings Annotation as presented in the current incarnation of the bibframe proposal as an example some of the problematic features of the current BIBFRAME model.

Part III.  Only Connect: Why a Framework Based on Linked Data and Semantic Technologies Is Worth Pursuing.

In the third prolegomenon I discuss the how using Linked Data and Semantic Technologies can improve the quality and accuracy of existing cataloging practice, and look at some new architectures and applications that a  framework designed for networked use can enable.

Part IV.  IMEW: The Opposite of FRBR is FRBR.

I examine some properties of  FRBR-like data models.  Taking a bottom-up approach, starting with  Items, I relate the main entities in the FRBR WEMI model to concepts found in common  version  control systems used in software development.  I discuss some issues involved in inferring  properties of an Item based on it’s corresponding  Manifestation, Expression, and Work,  raise some questions as to possible identity criteria, and look at some other possible entities that may be implicit in the model .

Part V. On What The…? Ontology and Linked Data for BIBFRAME

Part IV briefly covers some concepts in linked data and ontologies, makes the point that owl and RDF are not the only targets, and that it is more important to accurately capture the semantics of what is being modeled than it is to think about what something might look like in XML. I argue for the use of a more expressive form of modeling and for the use of controlled natural language to specify the model. CNL makes it much easier for people to contribute, and makes it much easier to review.

Part VI:  Implementation: A Simple Matter Of Programming

Part V looks at some of the technical solutions available, some areas of computational complexity, and areas where work is needed. In particular, I describe a possible approach to applying Provenance information to Named Graphs that is compatible with the forthcoming RDF/1.1 recommendations.

by ses at November 26, 2013 05:14 PM

Those who have heard me talk about the Digital Public Library of America over the past six months know that I’m fond of saying that DPLA is as much a social project as a technical project. Much of what we do focuses on collaboration and coordination, which involves looking not just at technical—or legal—elements, but social ones.

It’s much easier to think of an issue solely as a technical problem (we just need to figure out how to code that properly), or as a legal problem (we just need to bind everyone under a contractual arrangement to achieve the desired outcome), than as a social issue, since the latter requires attention to more amorphous aspects such as ethics and politics. But being more nuanced about the mix of the social, technical, and legal can pay dividends.

Take DPLA’s metadata. (Please. Take our metadata. It’s all freely available on our site.) One of the questions I frequently get is why the Digital Public Library of America requires the metadata for items in our collection to be donated under a CC0 license. That license is maximally permissive; as its longer name implies, CC0 is in fact a Public Domain Dedication.

Metadata obviously has elements of the technical and legal. Without a stringent technical standard into which we normalize data from over a thousand institutions, and a serious digital infrastructure to transform that metadata into interfaces such as maps and timelines, we couldn’t work much magic. And since we are conscious of the legal realm that many cultural heritage materials exist in, we do ask for a contract that specifies CC0 for the metadata. (However, there are many who would argue that even a CC0 license is unnecessary and should not even be demanded; by its very nature, a purely descriptive set of metadata should not be copyrightable (under U.S. law), but this is a discussion for another day.)

But why not ask for the most modest of additional restrictions, such as a license where attribution is required—a license with a -BY attached to the right? If we wish to tip our hat to those who created or donated the metadata, why not legally mandate it?

Those who use, reuse, and commingle data know the complex issues that arise with even simple additional requirements such as this. Data that flows from many sources will pick up, like fallen branches in the stream, a variety of ensnaring reeds, adding significant friction and complexity to some applications. But good-meaning people still want to provide attribution, and individuals and institutions might have social expectations of receiving credit. What to do?

Move the attribution from the legal realm into the social or ethical realm by pairing a permissive license with a strong moral entreaty.

For instance, the Tate recently released metadata for 70,000 works of art, and 3500 artists. The license they put on the data was CC0. But right next to that license is this block on “Usage Guidelines”:

These usage guidelines are based on goodwill. They are not a legal contract but Tate requests that you follow these guidelines if you use Metadata from our Collection dataset.

The Metadata published by Tate is available free of restrictions under the Creative Commons Zero Public Domain Dedication.

This means that you can use it for any purpose without having to give attribution. However, Tate requests that you actively acknowledge and give attribution to Tate wherever possible. Attribution supports future efforts to release other data. It also reduces the amount of ‘orphaned data’, helping retain links to authoritative sources.

As with many other things, our friends from Europeana were out in front on this, as Tate acknowledges on their GitHub page. Here’s Europeana’s metadata use page:

These usage guidelines are based on goodwill, they are not a legal contract but Europeana requests that you follow these guidelines if you use metadata from Europeana.

All metadata published by Europeana are available free of restriction under theCreative Commons CC0 1.0 Universal Public Domain Dedication. However, Europeana requests that you actively acknowledge and give attribution to all metadata sources, such as the data providers (being a specific cultural heritage institution) and any data aggregators, including Europeana.

Give credit where credit is due.

DPLA does the same thing with our Data Best Use Practices page.

I have been calling this implied or ethical attribution. Or, if you like short and snappy symbols, think of it as CC0 (+BY) rather than CC-BY (or ODB-BY).

The cynics, of course, will say that bad actors will do bad things with all that open data. But here’s the thing about the open web: bad actors will do bad things, regardless. They will ignore whatever license you have asserted, or use technical means to circumvent your technical lock. And yes, with CC0 commercial entities as well might come and take all of that metadata—but that data includes pointers back to items and scans at libraries, archives, and museums, which are (or should be) in the business of disseminating knowledge as widely as possible. By being free with our metadata, we do not devalue those nonprofit institutions, but rather emphasize more broadly the incredible contents they hold.

The flip side of worries about bad actors is that we underestimate the number of good actors doing the right thing. It has been our experience looking at the many software developers (including commercial ones) who have used our data across the web and in DPLA-powered apps, for instance, that they have all maintained proper attribution, even though the CC0 license theoretically means that they can do with the data whatever they want.

I think CCO (+BY) is the best of both worlds: the data in a free-flowing environment that enables creativity and reuse, with attribution still maintained by the vast majority of people who consider themselves part of a social contract.

by Dan Cohen at November 26, 2013 04:19 PM

In the fall semester of sophomore and junior years, my work-study job at Princeton was to tutor freshman engineers in a study hall sponsored by the engineering school. The study hall had been created a few years before to help freshmen survive the shock of learning calculus, physics and chemistry, all at the same time.

We had a variety of students seeking help. I began to notice a distinct pattern in the sort of help that was needed by women and by men.

The typical interaction with a woman at the study hall went like this:

Woman: "I have no idea how to do this problem!'
Me: "Tell me about the problem"
Woman: [[ detailed explanation of problem ]]
Me: "How do you think you should attack the problem?"
Woman: [[ detailed plan for solving the problem ]]
Me: "Sounds good"
Woman: "Oh thanks so much, you've been so helpful!"

The typical interaction with a man at the study hall went like this:

Man: "I can't get this problem to work"
Me: "Tell me about the problem"
Man: [[ bizarre, complicated, and wrong explanation of the problem ]]
Me: [[ detailed explanation of problem that woman student just told me ]]
Man: "Oh"
Me: [[ detailed plan for solving the problem that woman student just told me ]]
Man: "Really?"
Me: "Would it kill you to try?"
Man: "but [[botched calculation]]"
Me: "Might want to check your signs"
Man: "Hey, I knew it would work!"

Of course, not every student was typical. I remember one freshman woman in particular. She would come in with a male friend. They were taking the sophomore level physics and math courses. This posed a problem for me, as I was also taking the sophomore level physics course, albeit the physics major track, rather than the engineering track. I was barely a week ahead of them. I used my "tell me about the problem" strategy, which at first seemed to satisfy them. But after a few weeks, their questions got more difficult and I was having more difficulty. So I suggested to them that the study hall wasn't really meant to help with sophomore level courses.

A few years later, I found myself a classmate of this student in grad school at Stanford; she went on to get a Ph.D. in Applied Physics. She never quite forgave me for "kicking them out of study hall".  Her story tomorrow.

by Eric (noreply@blogger.com) at November 26, 2013 09:46 AM

Jeff Rothenberg's ground-breaking 1995 article Ensuring the Longevity of Digital Documents described and compared two techniques to combat format obsolescence; format migration and emulation, concluding that emulation was the preferred approach. As time went by and successive digital preservation systems went into production it became clear that almost all of them rejected Jeff's conclusion, planning to use format migration as their preferred response to format obsolescence. Follow me below the fold for a discussion on why this happened and whether it still makes sense.

Most practitioners obsessively collected and attempted to verify format metadata, they implemented elaborate metadata schema to record the successive migrations which content would undergo, they set up registries of format convertors which would perform the migrations when they became necessary, they established format watches to warn of looming format obsolescence and, when it failed to appear, they studied how to assess the potential future vulnerability of formats to obsolescence. Faced with this rejection, Jeff's views evolved. In a talk last year, he endorsed a balanced approach of preparing for both migration and emulation.

Although I believe that the Web-driven transition from formats private to an application to formats as a publication medium invalidated Jeff's view of the likely incidence of format obsolescence, he was correct to prefer emulation as a response should it ever occur. So why didn't the practitioners agree? I believe there were a number of reasons. It was thought that:

• The creation of suitable emulators was a task for digital preservation alone.
• The skills required would be arcane.
• The effort involved in each one would be substantial.
• A large number of emulators would be required.
• Setting up the emulation environments required by obsolete formats would be difficult for readers.

Just as, in 1995, Jeff was writing as a phase change was transforming the prospect of format obsolescence, he was also writing as a phase change was transforming the environment for emulation. Looking back, he saw that:

• Except for the mainframe world, emulation was not a mainstream computing technology. (Apple was then using the technology as they moved from 68000 to PowerPC).
• Thus there were very few emulation implementors.
• Thus the creation of each emulator would consume of large chunk of digital preservation resources.
• The diversity of competing computer architectures would require a large number of emulators.
• The lack of a uniform, widely deployed graphical user interface would make deploying emulation to readers complex and fragile.

Since 1995, the picture has reversed completely:

• Emulation has become an essential part of the computing mainstream:
  • Vmware and others have made virtualization of even low-end physical hardware ubiquitous.
  • Languages such as Java have made emulation of abstract virtual machines ubiquitous.
  • The efforts of enthusiasts for preserving early computer games have made emulations of them ubiquitous.
• The techniques for implementing emulators are well-understood and widely known.
• The dominance of the x86 and ARM architectures means that the number of emulators needed is essentially fixed at the number that were needed in 1995.
• The advent of the Web has provided a uniform, widely-deployed graphical user interface with the potential to deliver emulations in a transparent, easy-to-use way.

We now have two examples of transparently delivering emulations of obsolete software environments to Web browsers. First, at IDCC13 and iPRES2013, the team from Freiburg University described their implementation of emulation as a cloud service, in which they instantiate obsolete hardware and software environments as virtual machine in a cloud computing service, using VNC or RDP to deliver the user interface to the reader's browser.

At the Internet Archive, Jason Scott and others instantiate the obsolete hardware and software environment as a virtual machine inside the reader's browser. Jason writes:

Today, the Internet Archive announces the Historical Software Archive, a collection of prominent and historically notable pieces of software, able to be run immediately in your browser.  They range from pioneering applications to obscure forgotten utilities, and from peak-of-perfection designs to industry-crashing classics.

They can do this because the advent of HTML5 has changed the language of the Web from HTML to Javascript:

JSMESS is a Javascript port of the MESS emulator, a mature and breathtakingly flexible computer and console emulator that has been in development for over a decade and a half by hundreds of volunteers. The MESS emulator runs in a large variety of platforms, but is now able to run embedded in most modern browsers, including Firefox, Chrome, Safari and Internet Explorer.

It isn't just ancient computer games that can run in your browser. At Slashdot, warmflatsprite writes:

"It seems that there have been a rash of JavaScript virtual machines running Linux lately (or maybe I just travel in really weird circles). However until now none of them had network support, so they weren't too terribly useful. Sebastian Macke's jor1k project uses asm.js to produce a very fast emulation of the OpenCores OpenRISC processor (or1k) along with a HTML5 canvas framebuffer for graphics support. Recently Ben Burns contributed an emulated OpenCores ethmac ethernet adapter to the project. This sends ethernet frames to a gateway server via websocket where they are switched and/or piped into TAP virtual ethernet adapter. With this you can build whatever kind of network appliance you'd like for the myriad of fast, sandboxed VMs running in your users' browsers. For the live demo all VMs connect to a single private LAN (subnet 10.5.0.0/16). The websocket gateway also NATs traffic from that LAN out to the open Internet."

In all these cases the reader clicks on a link and the infrastructure delivers an emulated environment that looks just like any other web page. The idea that emulation is too complex and difficult for readers is exploded.

Given the ability to run even modern operating systems with network and graphics in the reader's browser or in cloud virtual machines, it is hard to argue that format migration is essential, or even important, to delivering the reader's original experience. Instead, what is essential is a time-sequence library of binary software environments ready to be instantiated as readers request contemporary content.

Notice the document-centric assumption above, namely that the goal of digital preservation is to reproduce the original reader's experience. In 1995 this was the obvious goal, but now it is not the only goal. As I pointed out in respect of the Library of Congress' Twitter collection, current scholars increasingly want not to read individual documents but to data-mine from the dataset formed by the collection as a whole. It is true that in many cases emulation as described above does not provide the kind of access needed for this. It might be argued that format migration is thus still needed. But there is a world of difference between extracting useful information from a format (for example, bibliographic metadata and text from PDF) and delivering a pixel-perfect re-rendering of it. Extraction is far easier and far less likely to be invalidated by minor format version changes.

by David. (noreply@blogger.com) at November 26, 2013 08:00 AM