Issue contents
The online encyclopedia website Wikipedia is used by 10 percent of internet users; its English language site contains over 2.8 million articles and counting, its German site over 722,000; in total, over 9 million articles exist across more than 250 language sites (including a "Simple English" Wikipedia), all freely available to access and edit. Many contributions are made by hobbyists and amateur information enthusiasts; but many are also made by qualified academics and the dedicated "Wikipedians" who monitor, develop and constantly improve the site. While there are problems with Wikipedia, which it itself acknowledges, its relevance is clear.
In the same way that many Digital Collection developers have recognised "the dominance of the Web and of Google as means of access and resource discovery for digital libraries," isn't it Wikipedia's turn to be given serious consideration? Its articles frequently feature high-up in the results pages of Google searches. As "Web 2.0" applications are increasingly placed on the agendas of libraries looking for new ways to expose their collections to a wider audience, Wikipedia is certainly worth engaging with.
Much of what's been called "Web 2.0" can be understood simply as a diverse set of features reliant on (and certainly not replacing) Web 1.0 technologies to encourage people around the world to interact in a certain way; to participate. The position Wikipedia occupies in relation to the reference services and cataloguing schemes used by Libraries might be be seen in a similar light. It's a colourful, modern layer of abstraction reliant on familiar ideas and familiar forms. In that, it shares much with digital libraries, archives and museums.
Yet a recent OCLC study found that while 72 percent of users surveyed had used an Internet search engine when seeking information, only 30 percent had used a library website. Instead of seeing Wikipedia as a competitor of dubious merit, why not begin to see it as a possible collaborator, with positives, negatives, and a lot of potential? It's not going to go away anytime soon. Freely referencing, in a major world encyclopedia, the digital collection of which you possess the requisite knowledge, might be considered a wonderful opportunity.
Despite a much-publicised debate a few years ago on Wikipedia's accuracy as a reference source when compared with the Encyclopedia Britannica, many who mistrust Wikipedia and its affect on the information profession have not fully engaged with it; or at least, have not considered what it might share with libraries and with digital libraries in particular.
Another recent study found that "While there are serious concerns surrounding Wikipedia ... Using Wikipedia as a platform for constructing thesauri has substantial advantages over traditional domain-specific thesaurus construction."
Wikipedia's layer of abstraction relates to users' experience of information seeking and retrieval, and the ways in which they relate to the organisation offering access to the content, as well as to the techniques employed to search for that information. Wikipedia uses a hypertext/browsing model of searching to create a loose mesh of information with no intrinsic hierarchy, supporting lateral rather than downward movements. But its underlying purposes and goals are the same as those of any traditional catalogue; to guide the user to relevant information. Its interest in working towards those goals in a vibrant, open, online environment is shared by Digital Libraries around the world.
Screenshot (17 Mar 2008) of the Glasgow Digital Library homepage.
Using this rationale, I recently compiled a report into possible ways by which to assess the visibility of and references to the collections of the Glasgow Digital Library on Wikipedia. This might be seen as an adjunct to other, "harder" performance measurements, such as the collection and analysis of usage statistics and fits comfortably with the interests of the Centre for Digital Library Research, (home to the Glasgow Digital Library) as it seeks to measure its success and "assist discovery of specific GDL resources on a worldwide scale by a wide range of users" while encouraging co-operation with other organisations which provide learning resources.
While the GDL bases the organisation of its collections on formal classification schemes, it does not do so entirely rigidly; sometimes key words are determined by content editors while at other times LC authority files are used. The GDL provides an arena for many of the key research themes of the CDLR to be explored; efficient use of metadata, collection-level description, interactions with virtual learning environments, and knowledge organisation systems.
Staff at the CDLR already actively measure their collections' rankings on the Google search engine, stating that libraries must "remain flexible [and] establish procedures that will allow output and optimisation for different applications in future". In fact, it is this which first drew their attention to the growing prominence of Wikipedia in the first place; their Maxton Papers collection was pushed from 1st to 3rd place in Google's search results over a number of months, with the Wikipedia entry on Maxton supplanting theirs in 2nd place - a cause for some concern.
Screenshot (17 Mar 2008) of Google search results page (part) showing that the Maxton entry on Wikipedia has pushed the Maxton Papers down the Google results rankings.
Yet, demonstrating nicely how Wikipedia can be harnessed to direct traffic to a digital library collection, clicking into the Wikipedia article quickly reveals an external reference and hyperlink to the GDL's "Maxton Papers". Although the text on top of the link incorrectly cites the "Strathclyde University Digital Library" rather than the Glasgow Digital Library, this discovery surely takes the sting out of the tail somewhat. Rather than competing, Wikipedia actually facilitates discovery of GDL content. Which other Wikipedia pages act in a similar fashion? To what extent is the GDL currently represented on Wikipedia? Could this potential be somehow formally managed and assessed?
Which topics or people are featured in both the Glasgow Digital Library and Wikipedia, but not linked to one another? Which topics or people featured in the GDL are entirely absent from Wikipedia? How does the GDL's visibility compare with that of other digital library collections? Undertaking such an investigation is quite complex; the structure and processes of Wikipedia are multiple, diverse, and require some awareness of underlying technique if they are to be engaged with fully. The development of an appropriate research method was necessary but could not be comprehensive given my lack of technical expertise in the area of IT. Nevertheless I believe it to be illustrative and useful as it dovetails with current debates in the LIS sector over, for example, who should catalogue and how, how to measure performance in a social networking context, and how to optimise participation in library services on a global scale.[1]
The primary focus was on examining references to GDL collections at the levels of URL, collection name and keyword, with only tentative suggestions made for further work that could be done at the deeper levels of content and detail. It is based then on broad rather than detailed categorisations of content, generating illustrative and suggestive rather than exhaustive and definitive results. While references on Wikipedia to James Maxton as person were located, references to specific events significant in his political career, or to documents in the GDL referring to those events, were not.
I employed a set of distinct search techniques, each successive approach growing in its sophistication. As the approaches become more complex, more of the GDL collection was interrogated and Wikipedia was engaged with more fully, meaning more illuminating sets of results were produced. 8 GDL collections were used for my report, with Victorian Times and the Virtual Mitchell ignored as they are not wholly GDL collections.
Hypotheses were:
That there would be a lack of standardisation in; how the various links were implemented at the level of code; the terms in which references were framed; the relative importance given to certain words tagged as metadata or keywords by those adding content to the site.
That much of the Glasgow Digital Library's content would not be represented on Wikipedia, and that some would be represented more "accurately" (in simple terms) than others.
That some types of search string derived from index terms on the GDL, when used to search Wikipedia (for example, LC Headings), would match better than others the terminologies and keywords used in articles created by Wikipedia contributors.
Testing these hypotheses required devising a methodology based on link searching, counting and comparison. All results provided here were correct as of late February 2008.
The first technique was simple; a basic article search using the following search strings:
Although using a URL part is perhaps not the most common way of searching Wikipedia, it is an effective method of locating external links, due to the various ways they may be referred to in Wikipedia's MediaWiki markup language. Using the two "Library" titles more effectively mirrors the more common way of searching. However, the research is designed to be useful to digital libraries rather than replicating real conditions, which could only be done using a study group.
This created 3 quite different sets of results:
Only 6 of the 17 articles found using "Glasgow Digital Library" were common to the 18 found using "gdl.cdlr". Both those found using "Strathclyde Digital Library" were unique. (Because of the link implementation used on these articles, using the URL part did not locate them).
The proportion of the GDL's content which these 31 articles represent (taking number of .htm web pages or index.html files [about 4000 pages] as we could reasonably expect these be linked to) is 0.775 percent.
Searching only within Images using the same strings produced 1 result, found using "Glasgow Digital Library". This was an image of John Pringle Nichol. Yet a portrait of (for example) William Speirs Bruce, and a picture of the funeral procession of John Maclean, used (with attribution to the GDL) in articles were not returned in the results list.
Screenshot (17 Mar 2008) of Wikipedia image page of a picture of John Maclean's casket being removed, appropriately cited and licensed, with a link to the corresponding page in the GDL's Red Clydeside collection.
This was performed using the External link search feature available through the "special pages" of Wikipedia's "Toolbox". Only the search string "*.gdl.cdlr.strath.ac.uk" was used as the tool works on the basis of URL fragments/search patterns. This means no matter what the implementation, references to "gdl.cdlr" within articles will not be missed.
I analysed the links to find out which of the GDL's collections they referred to and found that most links were to the "Ebooks about Glasgow" collection. Results are shown in the table below.
| Collection | References |
|---|---|
| ebooks about Glasgow | 48 |
| ebooks about Scotland | 16 |
| Red Clydeside | 13 |
| Aspect | 9 |
| Springburn Virtual Museum | 9 |
| Scottish Mountaineering Club Journal | 7 |
| Maxton Papers | 3 |
| Voyage of the Scotia | 3 |
Clearly, using the External link search yields more results than a simple article search, demonstrating that the relevance assigned to a term or link varies hugely across Wikipedia's collection.
MediaWiki doesn't search the active articles, but uses a search index that is compiled from the articles every so often. Usually it will be considered more useful to users to search within the article text rather than the code which is often the only place where the GDL is visibly referred to. This may mean that while the contents of the GDL collections are being used for illustrative purposes, the Library itself is seen only as a repository; an image or article bank irrelevant to the point of the main article itself.
This will be determined by; the attitude of the user creating the page, the attitudes of those editing it; the automatic processes which examine and compile lists of descriptions and keywords to be used in the database server's full-text search algorithm, and the various ways in which references, terms and links can be implemented using MediaWiki and its conventions.
The proportion of the GDL's linkable content which these 108 articles represent is 2.7 percent.
This was performed using the Crosswiki Linksearch tool developed by Wikipedia user "Eagle101" and available on his Wikimedia Toolserver page. Crosswiki Linksearch searches only the Top 57 Wikipedia sites.
By using this tool we are able to find out which Wikipedia sites across the world refer to the GDL's collections. The only drawback of using this tool is that articles implemented in ways other than those understood by the tool's search routine may not be found (as we saw in the case of Approach 1). At present, this is the best tool available at the time of writing, and limits the need for manual searching in foreign languages.
Because it is aimed primarily at non-native English speakers, I decided to treat Wikipedia's "Simple English" as a distinct language. Results by language and collection are summarised below.
| Language | Collection | Number of Links |
|---|---|---|
| Esperanto | ebooks about Glasgow | 1 |
| French | Scotia | 1 |
| German | Red Clydeside | 1 |
| German | ebooks about Scotland | 1 |
| German | Scotia | 1 |
| Italian | ebooks about Scotland | 2 |
| Italian | Scotia | 3 |
| Romanian | ebooks about Glasgow | 1 |
| Russian | Red Clydeside | 1 |
| Simple English | Scotia | 1 |
| Slovenian | ebooks about Scotland | 1 |
| Ukrainian | GDL Homepage | 1 |
Instead of simply looking at the numbers, it is clearly worthwhile considering the specific collections and information objects that are being used by people gathering resources and references, worldwide, when they decide to create a Wikipedia article.
Screenshot (17 Mar 2008) of a Romanian Wikipedia page (part) on the engineer William J.M. Rankine provides a hyperlink to the corresponding Memoirs and Portraits of One Hundred Glasgow Men page on the Glasgow Digital Library.
In the case of the GDL:
The proportion of the GDL's linkable content which these 124 articles represent is 3.1 percent.
Chart showing linkage to Glasgow Digital Library collections by foreign language Wikipedia sites.
This was also carried out using the Crosswiki Linksearch tool to show how comparing the Glasgow Digital Library's visibility with that of other freely accessible digital library collections could possibly be used a performance indicator.
Two strategies were used - a comparison of all the digital collections of selected institutions and a comparison of individual collections.
The Collections chosen were:
And at the level of individual collections:
Because the Oxford Digital Library collections all possess separate domain names, I wasn't able to search simply for links to their entire corpus.
Undertaking a search of two Collections which I felt could reasonably be expected to garner results (Ancient Documents and Wilfred Owen) gave me a good reason to undertake a collection level analysis of one of the GDL's collections.
It is not possible to use the Crosswiki Linksearch to specify a path part; because the GDL collections merely add path names to distinguish between collections, I searched for links to "ebooks about Glasgow" across the Wikipedia sites within the results already provided using the Linksearch tool.
Although I am only examining 8 of the GDL's 10 collections, the number of collections given here and used when calculating averages is 10, for reasons of fairness and validity.
Chart showing the number of references for each collection across the top 57 Wikipedia sites. The New York Public Library is way ahead while the GDL is in last place.
Chart showing the number of references per collection held by the various digital libraries. Factoring in the number of collections held by each institution to convey a sense of proportion moves the Glasgow Digital Library from last to first place.
This was performed by compiling 4 sets of "string types" using indices and lists provided on the GDL. The string types (or heading types) from the GDL's indices created were: "Groups", "Person", "Place" and "Subject". This approach was by far the most time consuming, involving far more manual work than previous ones, but the results produced were extremely rewarding as a result. Using or creating a set of index terms representative of the entire GDL collection was impractical due to the time constraints and technical constraints involved in searching Wikipedia manually. However, I tried to ensure that sets of terms chosen from the 8 collections were as representative as possible in their distribution given the relative size and content of each collection and the extent to which indices were already provided for them on the GDL.
Some convenience was inevitable; the 100 Glasgow Men listed in the Glasgow ebooks collection were already listed on the GDL site. It seemed worthwhile using the names of these famous men as search strings rather than the list of subscribers to the book. This is also common sense. Formulating a list of names based on the Springburn Virtual Museum collection would hardly have been appropriate given its focus and in any case, to manually create one would have been impractical and contradicted attempts to use indices already provided.
These lists were not exhaustive and did not represent the entire GDL collection. They do however represent a large proportion of it and illustrate a useful method for comparing terms used by the GDL with terms present on or useful to Wikipedia and its users.
Pasting into a spreadsheet and using an "AutoFilter" function is most useful for comparing and contrasting large sets of results obtained using this method.
Two useful ways to interpret the data in general terms is to chart:
Two ways of further interpreting the data in terms of individual collections is to chart:
The percentage of search strings returning an article, by collection is the most useful measure here since the number of links to the GDL by collection has previously been examined.
The percentage of search strings returning an article, by collection, are given in the table below.
| Collection | Article present | as percentage of total search strings used |
|---|---|---|
| Aspect | 220 | 38.2 |
| Red Clydeside | 34 | 37.8 |
| Maxton | 30 | 43.5 |
| ebooks about Glasgow | 202 | 21.9 |
| ebooks about Scotland | 50 | 20.2 |
| Scotia | 28 | 25.7 |
| SMCJ | 24 | 43.6 |
| Springburn | 34 | 24.6 |
Results can be usefully analysed to determine which string types or headings already used by a digital library might most closely match, or be useful to the Wikipedia community when directing them to your resources, and when deciding which approach to take when searching for relevant content or adding relevant links to articles on the site.
All but one match to content of the GDL's Aspect collection came from "person" strings. The other came from Place: Springburn. This is because the vast majority of strings taken from the Aspect collection were type "Person", something determined by its nature. Aspect focuses on Scottish Members of Parliament and their campaign literature, rather than say, their relationship to their constituencies (although of course this information is detailed on the site). Again, with knowledge of your own collection factoring in such considerations is intuitive and merely common-sensical. The matches returned successfully from Red Clydeside term searches were split equally between "group" and "person" strings. All article matches returned from the Maxton Papers derived from "person" strings.
Screenshot (17 Mar 2008) of an index page provided by the GDL's Red Clydeside collection.
Screenshot (17 Mar 2008) of a Wikipedia search results page for articles (of varying relevance) matching a term taken from the Red Clydeside index.
Using established indices, subject terms and keywords familiar to GDL staff provides a convenient and relevant set of terms which could be used to determine:
Using search terms such as Library of Congress and Person or Organisation headings already utilised by the GDL and other libraries adds a valuable terminological consistency for researchers using Wikipedia.
Analysis of the Subject strings taken from the Scottish Mountaineering Club Journal reveal many of them to be very general - for example, "snow", "boulders" and "yachting". This gives a clearer idea of why there are few links to the SMC Journal's specialist hill-walking and mountaineering resources linked to from these article matches. The same is true, although to a much larger extent, with strings deriving from Subject Headings of the Springburn Virtual Museum. The SMC Journal is a more recent development of the Glasgow Digital Library collection which is also likely to have contributed to a lower number of matches and links.
Another point worth making of the collection: Wikipedia "Lists" of mountains/hills may contain the Gaelic "place" strings but these are not stand-alone articles so do not return a straight match. A suitably knowledgeable editor could locate such lists and add a brief article (in Wikipedia terms, a "stub") with a link.
There were a lot of ambiguities produced when using names to search for articles - especially when using historically popular common names such as James, John, Robert, Smith, Brown etc. Sifting through these can be extremely time consuming and would be difficult to automate. Articles may not exist on some persons whose names were used as search strings because those persons are unlikely ever to be deemed important enough to warrant an encyclopedia entry. (Although it is perhaps surprising quite how many types of people are represented on Wikipedia, from obscure dead ice hockey players to notable coleoptorists).
An article may exist for that person under a slightly different title - for example, "Jennie Lee" was used as a search string. An article on a silent cinema actress matches that term, while an article titled "Jennie Lee, Baroness Lee of Asheridge" refers to the former Independent Labour Party M.P.
An article on Winston Churchill may be fairly unlikely to link back to the GDL; but, his name did exist as an index term so it was objectively fair to include it when compiling lists.
These findings suggest the importance of having informed staff members engage fully with Wikipedia in order to exploit and understand its potential as an entry point to their collection. This is especially true in the case of disambiguating terms and accounting for synonyms and related terms which may be used as the title of Wikipedia articles and thus not returned using alternate search terms. For example, the "Aberdeen Marine Laboratory" string returned no results but a "Fisheries Research Services Marine Laboratory "string does.
The figures in the table above clearly represent a rich source of existing articles to which links and references might be added.
My initial hypotheses were borne out by an investigation which threw up many interesting points, both positive and problematic, about how Digital Libraries might successfully engage with Wikipedia.
Compared to others, the Glasgow Digital Library's representation on Wikipedia is good. As a measure of visibility, achieving a 3.1 percent representation in terms of GDL pages likely to be linked to across the various Wikipedia sites seems surprisingly healthy considering that no deliberate effort was made to do this. Still, there's clearly much scope to increase this figure by working with rather than against Wikipedia.
Perhaps a realistic 'representation on Wikipedia' percentage could be determined by interested Digital Library staff, and treated as a target to aim for, either within a set timeframe, or on a more casual basis, depending on staffing and budget considerations. Achieving a 5 percent representation could probably be done with little work and would certainly provide a great chance to further explore Wikipedia and Web 2.0, as well as the content of your own collection. Any results found using Approach 5 of this methodology could be used as a solid basis for choosing existing Wikipedia articles to add links to, which is far less complex and time consuming than creating new articles (which requires learning how to use MediaWiki's markup language).
Similarly the findings of Approach 4 could be used when compiling reports for management, marketing or collection development policies, in terms of "Web and Library 2.0" and as a method of clear performance measurement. You would probably need to gather more data and crunch more numbers to make the results statistically valid, but the potential is definitely there. A set of collections of similar size, or covering similar content, could be used and checked at intervals. Despite an impressively high result for articles matching terms derived from Glasgow Digital Library indices and lists, only 109 English language pages and a further 15 on 9 other language sites, link to the GDL.
Given the diverse ethnic makeup of New York's population both historically and now, and the position of New York in relation to global affairs, it might seem predictable that the New York Public Library was the best represented of all the collections, language wise. This could also relate to the size of the NYPL collection, and the more general, popular and cultural themes of much of its content. Authors gathering references to create Wikipedia articles may be more likely to be aware of the NYPL than of the GDL.
Considering the reputations of the Glasgow University Special Collections Department, and the National Library of Scotland, to have a significantly higher "average per collection" count than either of these institutions is surely a valuable statistic for the GDL in terms of its performance measurement.
But although the Glasgow Digital Library is primarily aimed at English speakers, Wikimedia's Interlanguage link facility means that some references to your collection will inevitably be spread around some of the non-English sites, even if articles do not originate in those countries. This can only be a bonus. It's certainly rewarding to see GDL content referred to in German, Romanian and French.
If a digital library were specifically interested in increasing its representation across various non- English Wikipedia sites, translation skills may be needed, or software which can translate from English to non-English characters. New articles need not be created from scratch however; links could be added to existing pages to minimise the effort involved for those without knowledge of foreign languages. If not, it would at least be a good idea to increase visibility on the "Simple English" Wikipedia to aid social inclusion ideals and to reach a wider, diverse user group.
Tidying up pages to introduce standardisation of referencing would improve Wikipedia (changing "Strathclyde Digital Library" to "Glasgow Digital Library", for example). On a philosophical level it would be satisfying for those who value the organisation of knowledge. Finding references to your collections and improving the accuracy of articles using them as sources is of interest to library professionals wishing to promote standardisation and have their content referenced in an appropriate context.
If only a relatively small number of articles currently link to you this would not be unfeasible. Content could be added as well as links to articles on related persons, meaning these topics could be greatly enriched for interested readers.
Relevant Categories could be found (such as the "List of Antarctic Expeditions" or the "List of Explorers", in the case of the GDL's Scotia collection) to increase the number of pages linking to articles referencing the GDL. This also reduces the chances of relevant articles becoming Orphaned, or languishing unseen and un-developed by others in the Wikipedia community.
The potential to engage with Wikipedia is huge and growing all the time as new tools are developed and new articles created and edited. Engaging with the site could clearly assist Digital Libraries assessing their relationship to "Web 2.0" and can provide valuable potential performance indicators relating to visibility, renown, and the uses to which collections content is being put. That a straightforward image search does not find all relevant results, may prove a complication for image libraries wishing to monitor their collections on Wikipedia whether for performance measurement or for legal reasons. They may have to rely on the Wikipedia community to enforce its own rules, which is not ideal, or else face lengthy and prohibitive search methods.
The number of pages linking to pages with links to the GDL was considered when preparing my report. This is highly relevant due to the importance of the hyperlink browsing model utilised by Wikipedia to create a multitude of entry points to their content. While a useful area for further research, in my report these figures would merely have been illustrative.
Jimmy Wales, Wikipedia's founder, has asked for help from librarians in improving the site and its content. Why not respond to his request and enter in to a mutually beneficial arrangement where digital libraries use Wikipedia to increase their visibility at the same time as assisting Wikipedia and its users by providing accurate, reliable information?
Libraries, whether digital or physical, might benefit from the increased interest in information retrieval created by the Web 2.0 and the participatory relationship created between users/producers. To quote Wales:
We do not believe that any resource tool can be reliable without scholarly input; this is why we so warmly welcome and invite the contributions of experts. It is a longstanding mistake to think of Wikipedia as being anti-elitist. Virtually every top Wikipedian I know is an elitist of the best sort: We love people who know what they are talking about. There is no contradiction here between Wikipedia and other reference sources; few people attempt to read an encyclopedia from cover to cover; but when they do search in it for information, those who have compiled it need to strive for accuracy and standardisation. Libraries can become involved by using Wikipedia as a way to promote their collections/services and engage with both local and global communities as well as encouraging awareness of the profession; this may be particularly relevant to virtual and digital libraries.
Charles Matthews, a mathematician and librarian heavily involved in Wikipedia has observed:
"From my point of view, much of the criticism aimed at Wikipedia, and perhaps some of the praise it gets, is misconceived because it takes the 'article' as the unit. Wikipedia shows its strengths in other ways, where readers:
"[2]
Wikipedia then may in fact create an increase in information seeking behaviour and an increase in people's engagement with the concept of "knowledge" - thus they create a strong potential user group for Digital Libraries. Provided that digital library managers encourage engagement with and understanding of the phenomenon, they could tap into an undeniably powerful worldwide "Web 2.0" phenomenon.
All rights reside with the author(s). WIDWISAWN recognises the author(s) as the copyright holder(s) unless specifically directed otherwise.
WIDWISAWN is delivered via the SAPIENS e-publishing service and is hosted by the Centre for Digital Library Research at the University of Strathclyde
Design and layout © CDLR 2008
Last updated: Apr 2008