Open Documentation Menu

Technical process for a document migration

In this section, you can learn more about the technical process involved in a document migration with d.velop cloud migration. It will give you a better understanding of the order in which the steps are performed.

Once the migration starts, the following steps are performed:

  1. The connector queries the mapping from the source category to the target category from the d.velop cloud migration app.

  2. The connector queries the mapping from the source users to the target users from the d.velop cloud migration app.

  3. The connector queries the other configurations (e.g. Ignore hash values, etc.) from the d.velop cloud migration app.

  4. The mappings and configurations are saved in the migration database.

  5. The connector imports the document export from the hard disk.

  6. The imported documents are divided into batches that are then processed in multiple steps.

  7. For each batch, the connector transforms the metadata in the export based on the mappings of the categories, the mappings of the users and the other migration settings in the memory. Advanced transformations must be explicitly activated via the d.velop cloud migration user interface and are also logged in the migration report. The following transformations are possible:

    • Defining the file size based on the export, if the file size is empty.

    • Correcting the file size, if the file size in the metadata differs from the actual export.

    • Setting the file name property on the file name for the metadata file, if the file name is empty.

    • Bringing the numbering for the document versions into sequence (e.g. 1,2,3,4 instead of 1,3,5,6).

  8. The connector validates the metadata and document data for each batch. If there is any invalid data, validation errors occur and the affected documents are not included in the further migration process, without correction. The errors are logged in the migration database. Possible validation errors:

    • The documents or metadata are not in the correct format (e.g. corrupt JSON file, incorrect data types in the metadata, incorrect JSON format).

    • The documents and metadata do not match (e.g. the number of exported versions does not match the number in the metadata or the file size in the metadata does not match the file size in the export).

    • Mandatory fields such as the hash values, file size or file name are missing in the metadata.

      Exception: The hash values of each individual version of a document are unavailable and you explicitly set missing hash values to be ignored. In this case, validation errors do not occur and documents without hash values are still included in the further process. This is recorded in the migration report.

  9. The connector uploads the documents and metadata for each batch to the target system via the migration API of the d.velop cloud. The documents and metadata are transmitted as they are provided by the export and adjusted using the migration configuration. You can activate two options for the upload on the user interface:

    1. The full-text index and the dependent PDF files for the documents can be recreated in the cloud. In this case, any full-text indexes and dependent PDF files in the export are discarded. The use of this option is only recommended if there is not yet any plain text.

    2. The links to documents and dossiers can be recreated based on the structure definition defined in the cloud. Existing links are maintained. New dossiers are created or linked so that there are pre-existing and new links.

    These options are not activated by default. If you activate these options, this activation is also logged in the migration report. During the upload, the connector evaluates the response of the migration API (success or error) and saves these responses in the migration database.

  10. Once a migration run is complete (all the batches have been processed), the connector creates the migration report, including the attachments (e.g. list of successfully migrated, list of incorrect documents), locally as a backup PDF file. The checksums for the attachments are recorded in a PDF file so that any change to the files can be tracked later. All the described activities are logged in a migration database. For each document, a <filename>.data.log file that logs all the activities in the documents on an ongoing basis is created in the export directory.