Digital data – here today, gone tomorrow

14 August 2006

CASPAR (Cultural, Artistic and Scientific knowledge for Preservation, Access and Retrieval) More Information ROLLOVER

Project Co-ordinator Dr David Giaretta explains: "It is widely recognised that the digital information on which we all rely is actually remarkably fragile. Society needs to ensure that digitally encoded information can still be understood and used in the future when the software, systems and everyday knowledge will have changed. Things we take for granted now would otherwise be completely unfamiliar, something to be guessed at, even if we preserve the bits and bytes.

"Moreover, in many currently planned and future experiments, several orders of magnitude more data will be generated than has been collected in the whole of human history."

Of particular importance is the huge breadth of users and types of digital information against which CASPAR will be tested: science (using ESA satellite data and a variety of science data from the Central Laboratory of the Research Councils (CCLRC), cultural heritage (using data from UNESCO - the United Nations’ cultural agency) and performing arts (including data from the Institut National Audiovisuel, Groupe de Recherches Musicales (Ina-GRM) and the Institut de Recherche et de Coordination Acoustique-Musique (IRCAM) - French institutions that fostered the development of electronic music).

Satellite data Credits: ESA More Information ROLLOVER

The volume of data generated in environmental science is projected to increase radically over the next few years. ESA satellites, such as Envisat, ERS-2 and Meteosat Second Generation, are currently generating around one Petabyte of data per day. With the upcoming launch of the new MetOp satellites, the daily data volume generated by ESA will increase at an even faster rate. ESA’s mandate is to maintain archives of data gathered from satellites for 10 years after the end of the mission. Currently ESA is using funds from various ongoing programmes to maintain these historical bit streams in accessible archives.

Sustainable preservation of this information in the long term will require the logical integration of many more pieces of data and objects, such as the conditions under which the instruments were operated, the system and software environment used to gather the signal and the algorithms used for manipulating the acquisition bit stream. All this information is required systematically for all instruments and missions, in a dedicated programmatic vision.

Within the CASPAR project, selected ESA satellite data streams will be the first objects to demonstrate how the proposed preservation platform architecture can be applied to handle complex digital objects. ESA will not only provide the necessary satellite data and associated information but also the operational experience and demonstration infrastructure.

The Global Ozone Monitoring Experiment (GOME), launched onboard ERS-2 in April 1995, is set to be the first candidate. Since 1996, ESA has been delivering GOME global observations of total ozone, nitrogen dioxide and related cloud information to users via CD-ROM and the Internet.

Cultural and artistic data More Information ROLLOVER

Artistic data from Ina-GRM, which holds archives of French public radio and TV, and IRCAM will focus on electronic music, preserving components of scores, pieces of computer codes, instructions and documents indicating author’s motivations to preserve intelligibility or the minimal understanding necessary to be able to perform the work again.

Development and implementation

CASPAR’s work will include the development of key components and framework providing characterisation – including Representation Information and Preservation Description Information, virtual storage – using advanced storage technologies, and access services – including intuitive query and browsing mechanisms, and, throughout all these, exploiting the potential of semantic web.

Throughout the project, attention will be paid to related issues such as standardisation, authentication, accreditation and digital rights management, seen as critical for the operational implementation of digital preservation services. The achievements of the CASPAR project will be disseminated and promoted in worldwide communities interested in digital archiving and preservation.

Source / Credit ESA