News

posted on 09 April 2008 11:12

If you keep information for a hundred years or more there's no guarantee you'll be able to read it in 2108, because data formats, storage media and reading devices will have changed beyond recognition. Anyone got a card punch?

The SNIA has a concern about this and has dived deep into its acronym resource to respond. The SNIA DMF 100 YARS (SNIA Data Management Forum 100 Year Archive Requirements Survey) has resulted in the LT DIRP TWG, the Digital Information Retention and Preservation Technical Working Group, It has decided to focus on two aspects of the problem: (1) physical and logical data preservation and; (2) migration. 

Gary Zasman, chairman of the DMF L-T ACSI (Long-Term Archive and Compliance Storage Initiative), said: "With more than 70 percent of respondents saying they are 'highly dissatisfied' with their ability to assure that they will be able to read and interpret their retained information in 50 years, there is a critical need for standards to assist users in retaining and preserving information for the future of their organizations. The requirements to keep information long term exist in many governmental regulations and organizational practices, but the existing technologies to support those requirements are costly and complex."

There needs to be a standard way of storing long-term data that is independent of its storage medium and format. This concept is called by the SNIA LT DIRP TWG the SD-SCDF or Self-Describing Self-Contained Data Format. The SNIA states that this standard will define a 'preservation-oriented' logical container consisting of the content (the data) and associated preservation metadata, including reference information, integrity and authenticity controls, audit records and potentially event readers.

The SD-SCDF standard will be designed for implementation directly by application developers or in conjunction with the SNIA standard application to storage interface, the eXtensible Access Method (XAM). XAM's ability to store information independent of the application that created it, and SD-SCDF's ability to define the information being stored, enhances the ability for information to be properly and securely preserved and interpreted long into the future.

All SNIA member companies can join the DIRP TWG. To learn how to get involved visit here. To download a copy of the complete 100 Year Archive requirements survey report go to here.

This seems to be exactly the kind of standard setting that the SNIA is meant for. Let's hope that all enterprise content management vendors and unstructured data storage companies join in with willingness and committment to a truly open standard. We don't want to see the kind of disgraceful shenanigans attendant on the open document format.

^{[Chris Mellor.]}