Writing documents to the secondary storage, the leading system (d.3 / ecspand) passes retention periods for the documents to the d.ecs storage manager to write them to the storage system.

Please also consider that the d.3 system can store d.3 property and OCR files in addition to the documents. In current d.3 versions, you can set the retention period for these files via the configuration parameter

d3addon.ini.

The retention time in these two configuration parameters is specified in months. If the retention period is not configured via these parameters, the d.3 property and OCR-files are by default passed to the d.ecs storage manager with a retention period of 50 years (600 months).

d.ecs storage manager itself regularly writes so-called database logs (see d.ecs storage manager administration (Tab Database logging). The retention period for these database logs could be influenced in d.ecs storage manager version 2.5 with a configuration parameter. The parameter had to be specified in the configuration file of the respective d.ecs storage manager instance (INI-file). The respective entry was:

[Special]

RetentionForIRFiles=50

The value for the retention period had to be specified in years.

With d.ecs storage manager version 2.6, the retention period can be configured via the d.ecs storage manager administration (Tab Database logging). Additionally, the d.ecs storage manager version 2.6 allows to define the retention period for synchronization and system check logs in the section Miscellaneous.

The passed retention periods must be allowed by the storage system, i.e. it is essential to know the current settings of the storage system regarding the minimum, maximum and default retention period or to have them configured on the storage system so that they cover the full range of retention periods with which the documents and additional files such as d.3 property files and OCR-files could be passed from the leading system.

The different storage systems have different names for the minimum, maximum and default retention period.

Netapp/SnaplockNetapp

EMC Celerra/VNX with FLR

EMC Data Domain with RetentionlockData

Grau Data Filelock

Moreover, the different storage systems behave differently when documents with a retention period are passed with a retention period outside the range configured by the minimum and maximum retention time. Usually, the minimum and maximum retention time are set by the storage system, if a document with a lower or higher retention period is to be stored. However, it may also happen that a storage system (e.g. Data Domain with Retentionlock) rejects the document storage, if the passed retention period is outside the configured limits.

The file system-based storage modules (NetApp, Celerra, FileLock, Silent Cubes, Data Domain), a time range for the retention period can be configured in the d.ecs storage manager administration (version 2.6) (Minimum/maximum retention period in months) in which the retention period passed by the leading system (d.3/ecspand) must be specified.

The settings regarding Minimum retention period (in months) and Maximum retention period (in months) should be equivalent to the settings applied on the storage system. The, again, should match the range of retention periods configured in the document type settings (d.3/ecspand).

Allowed values for the Minimum retention period (in months) must be between one and twelve months. The Maximum retention time (in months) can be between 120 and 600 months.

If a document is passed with a retention period outside the range defined by Minimum retention period (in months) and Maximum retention period (in months), the document storage on the storage system is rejected by the respective d.ecs storage manager module. The respective storage jobs thus encounter errors. With the configuration parameter Automatically correct retention period, this behavior can be changed so that a passed retention period outside the configured range is automatically set to the values configured under Minimum/Maximum retention period (in months).

Before the parameter Automatically correct retention period is enabled, the following message must be confirmed with Yes.

With Yes you confirm and agree that the retention times passed by the leading system (d.3/ecspand) during the document storage or extension of retention times are automatically corrected by the d.ecs storage manager and that these adjusted retention periods are used for the secondary storage. Furthermore, by selecting yes you confirm that the settings you applied for the Minimum retention time (in months) and the Maximum retention time (in months) should be used for the automatic correction of the retention time.

The value for the retention period for a document passed by the leading system, is written to the DB-column TTL of the table D3SM_REFERENCES. The TTL value in the database is only changed by the d.ecs storage manager only via a so-called “Retention-job” from the leading system. Additionally, the date on which the document was actually written to the secondary storage is stored in the column RETENTION_ON_STORAGE.

Extending the retention period via retention-jobs

The d.3 system sends so called retention-jobs to the d.ecs storage manager, if the retention period for a document changes in d.3.

This can, for example, be the case, if documents are moved to another document type. You must consider in the d.3 configuration that a storage system typically only allows to extend the retention periods of a document and not to reduce them.

A retention-job contains a retention period for each document to be passed to the storage system by the d.ecs storage manager. This retention period must be between the limits (Minimum retention period (in months), Maximum retention period (in months)) defined for the storage system for the retention period to be adjusted on the s storage system.

If the retention period passed with the retention-job is outside the limits configured in the d.ecs storage manager system configuration, the retention-job will encounter an error unless the parameter Automatically correct retention period was enabled.

If the retention period passed by the d.3 system is within the configured limits, the retention period for the respective document is extended on the storage system. Having successfully performed the extension, the values for TTL and RETENTION_ON_STORAGE are also adjusted in the table D3SM_REFERENCES.

If the passed retention period is outside the configured limits and the parameter Automatically correct retention period is set, then the retention period for the document on the storage system is set to the maximum possible value (today plus Maximum retention period (in month). The value passed by d.3 for the retention period is set according to the column TTL. The adjusted (limited) value for the storage system is validated with a query to the storage system and is then written to the column RETENTION_ON_STORAGE.

Automatic extension of retention periods by the retention process

The so called retention process is a new function of the d.ecs storage manager version 2.6. This process automatically extends the retention periods on the storage system, if the remaining retention time falls below 60 days.

The retention time is extended to prevent that the document becomes deletable with expiry of the retention period. This may be necessary, if no automatic deletion takes place in d.3 after expiry of the retention period. Another use case, where the retention period for a document has to be extended automatically affects the documents where the document type was configured with the option event-based deletion.

The retention process can be configured in the d.ecs storage manager administration (see illustration below). The value selected for Automatically extend the retention by x months is of special significance here.

Allowed values must be between 6 and 24 months. The default value is 6 months. The basis for the extension of the retention period by the d.ecs storage manager is the value stored in the column RETENTION_ON_STORAGE of the table D3SM_REFERENCES. If no value is stored in the column RETENTION_ON_STORAGE, the date value from the column TTL is used as a basis to extend the retention period.

If the date value in the column RETENTION_ON_STORAGE for a document is lower than the result of the calculation of the current date plus 60 days, then the retention period is extended automatically. For this effect, the retention period of the document on the storage system is set to the date value resulting from the calculation of the current date plus the value configured for the retention process in the d.ecs storage manager administration under Extend retention time by x months.

Having applied the extension of the retention period on the storage system, the value is also adjusted in the column RETENTION_ON_STORAGE. The date value in the column TTL is not affected by this.

If no value is stored for a document in the column RETENTION_ON_STORAGE, the check, if the retention period has to be extended uses the value in the column TTL. In this case, however, the value of the retention time is determined from the storage system, before the retention period is extended.

If the retrieved retention time of the storage system is lower than the calculated date (current date plus extension configured for the retention process), the retention period is extended on the storage system. The newly set value of the retention period is updated in the column RETENTION_ON_STORAGE.

If the determined retention time from the storage system is higher than the calculated date, no extension of the retention period has to take place. However, the value in the column RETENTION_ON_STORAGE is set to the value of the document retention time on the storage system.