Library Data Archive
Robert E. Molyneux
July 2018 update
This update recompiles the longitudinal data files for PLDF3 and PUSUM with data through FY 2016. PLDF3 now has 266,746 observations (data for one library for one year) with 10,174 libraries ever reporting. PUSUM has 1,310 observations (summary library data for one state for one year). All the PUSUM data are available from this site while the extent of the PLDF3 data have forced a change in their distribution. There are so many ways one might need these data–from all to some–that it seems most practical for anyone to tell me what they need and I will send it. The raw data files are also updated in the archive.
March 2018 update
This update is quite substantial. The last few years I have updated the longitudinal files but not the various publications. For this update, I have updated the two longitudinal files with the FY 2015 data. In addintion, I have added the various related publications to the archive. IMLS continues its good work with compiling and documenting these data. The U.S. library world benefits greatly from their work.
As I have pointed out elsewhere, there are differences between the roles of the compiler of data and that of the analyst. The archivist's role is different from that of either. When I have recompiled the annual FSCS/PLS data into PLDF3 and PUSUM, I have attempted to present the data as they were originally intended. These datasets are, in a sense, variorum editions of these data given that variant texts are considered. Note: treating the data as a text. I have made changes in those data and documented those changes extensively so that anyone following can see what I have done and correct that work if necessary. The digital archivist, though, as I see it, collects the original documents and preserves them. When I was a Webmaster at a Federal agency, we paid attention to changing laws so that obsolete information did not appear. Obsolete information could be harmful to the members of public. In the case of these data, there are files here in a variety of file types no longer used: ASCII and Access for example. One of the things that has happened since I last updated the various publications is that IMLS has gone back and converted files from older formats to newer ones. I note that there are a number of new SAS files of older data. And in FY 2015, IMLS is publishing the data in SAS and SPSS formats in addition to csv files which now appears to be the default format.
Here, all digital data and publication are presented with the exception that revised data are preserved and the dataset they replaced is not—where known. These are cases where the compiling agency discovered an error and corrected it. There are also "Users notes" listed and these seem to be small corrections or warnings about small aspects of the data. However—and we would not be dealing with data unless there were an “however”—there are exceptions. One way that the compiling agencies have indicated data revisions over the years is to include a trailing ‘a’ in the original publications' filenames and in the rare case of a revision, it is indicated by a trailing ‘b’. For example, the FY 2013 data Administrative Entity file is named pupld13a with an indicator for the file type. If it were revised, the key would be in the change of the name to pupld13b. Here, we have a problem. Note the FY 2004 data. There are four ‘a’ files: ASCII, Access, csv, and xls and a new SAS ‘b’ file. FY 2001 data have a similar problem. What is the obligation of the archivist here? It appears the SAS files have revised data but until I can run this down, I am leaving both here with this warning. I believe this is the only solution in this case.
Two publications I had from years back have reappeared with new file names. I suspect that they were rescanned. I cannot see any differences between these publications so I have kept the new ones but not put them in this archive.
The Library Research Service has graciously offered to house this archive of digital data and reports on U.S. libraries by two agencies of the U.S. government: the U.S. National Center for Education Statistics (NCES) and the U.S. Institute for Museum and Library Services (IMLS). The NCES-sponsored program behind the collection and publication of the public library data was known as the Federal State Cooperative System (FSCS). See this useful timeline of this program for more information. Having watched this effort from close up, I can say it was an impressive organization that functioned well. IMLS continues a similar program as the Public Library Survey which continues the public library data series without interruption.
There were two kinds of digital publications in this Archive originally. One type of publication was updated longitudinal files of U.S. public library data recompiled from annual data published by NCES and IMLS and the second was the raw data, documentation of those data, and reports derived from them by those two agencies. Now we are adding data, documentation, and reports on two other types of libraries compiled and published by NCES.
The context of the expansion of this Archive
This collection began when I was at the now-closed National Commission of Libraries and Information Science (NCLIS) and reflects work by a number of people both at NCLIS and also at other places, most notably NCES. There are also a number of other reports and compilations done by library associations, of course, but the data here are all in the public domain and, hence, freely available for use by anyone interested.
There were three CDs issued by NCLIS of the data in the waning days of the Commission which included the results of this collection of raw data, reports, and such on U.S. libraries by NCES. The CDs covered several types of libraries including academic libraries, what we now call school library media centers (SLMC) and State Library Agencies (that is, state libraries.) This Archive now includes the NCES publications on the academic and SLMCs collected on those CDs as well as new data and reports collected since the CDs were issued.
All materials in this archive are in the public domain.
Currently you will find here publications about:
|U.S. Public Libraries||Comments|
|Public Library Data File (PLDF3)||Longitudinal data of a universe file of U.S. public libraries, FY 1987-FY 2015. Updated through FY 2015.|
|State Summary/State Characteristics Data File (PUSUM)||Longitudinal compilation of summary data on public libraries at the state level, FY 1992-FY 2015. Updated through FY 2015.|
|Raw data, documentation, reports, and summary analysis of public libraries||A number of current and historical publications about these libraries is available. This is an opportunistic collection and is but a fraction of such publications.|
|U.S. Academic Libraries||Comments|
|Raw data, documentation, reports, and summary analysis of these academic libraries not only from the current Academic Library Survey but also from earlier series.||A substantial number of current and historical publications about these libraries is available from a variety of series and sources. This collection is also opportunistic.|
|U.S. School Library Media Centers||Comments|
|A small collection but what we found.||Even through there are 100,000+ of these libraries, there is not much available. The very size of this important universe makes analyzing it difficult.|
Introduction to the NCES/IMLS Data
The U.S. government through various agencies been involved in the compilation of library data since before the legendary 1876 publication: Public Libraries in the United States of America [common title: 1876 Report.] This is a large file. NCES and IMLS continue an old tradition.
The publications here were collected from a number of sources. Most came from the NCES Web site over time. It is a curiosity of working with library data that publications with data and reports about libraries derived from those data tend to disappear—in the old days to musty library shelves but these days, often, the memory hole. One might suppose that librarians who house, organize, and archive the world's publications might treat their own data similarly but that is not the way it is. Moreover, government agencies often do not archive data for very long. As a former Webmaster at a federal agency, there was always the worry about having obsolete information which might mislead someone. In any case, this Archive has copies of digital publications that in many cases took effort to locate and the cooperation of many people.
Public library data for 1987-1991 were no longer on the NCES site and, as indicated, came from other sources, largely from Kim Miller then at NCLIS who had collected almost everything. Similarly, a number the older data publications such as documentation and reports had gone missing and ferreting them out took some time and effort. Early NCES publications that had been scanned—mostly by the good folks at NCES—and were sent to me are also included here.
As mentioned, the Institute of Museum and Library Services (IMLS) has taken over the collection of the public library data and state library data from NCES. There is page for IMLS's Public Library Survey effort and for the State Library Administrative Agency survey (SLAA) will have current information on this series. As mentioned, publications about this latter type are not currently included in this Archive.
Interuniversity Consortium for Political and Social Research
The Inter-university Consortium for Political and Social Research ICPSR, a part of the Institute for Social Research, is located on the University of Michigan Campus. ICPSR maintains an archive of social science data. A number of files of library data in the PLDF3 file came from ICPSR, which processed them on behalf of NCES. When I was looking for the early years of the FSCS data, these were available on the ICSPR site and I collected them. I also discovered other data on other types of libraries, particularly historical academic data files and these data are available here, also. A spreadsheet lists all the files from ICPSR.
Sources of Library Data Outside the United States and a Look at Assessment of Libraries
The linked page is a work in progress and is as of September, 2016 although I expect to be delving into the North American library data in detail shortly. I have collected links to data sources I have worked with so this is an sample I hope to build on. I have also collected these data. It seems that library data tend to disappear without an archiving effort and that fact was the proximate cause of my starting this exercise about 15 years ago.
While looking at BIX (Der Bibliotheksindex)—an English-language project page gives the scope of the index although, sadly, the BIX has closed. I read in detail about the LJIndex and HAPLR and the discussion about them. See the linked page for these data sources.
July 23, 2019To the Library Research Service main page