Backup data system

This information is about backups, in the sense of a safety copy of data which you keep on your own systems and actively work with and modify.

At this stage, the system provide backup for the users in the next projects: ‘dynamics‘, ‘elliptic‘, ‘euclid‘, ‘hetdex‘ and ‘wst‘.

To do that,

  • instead to use the user folder: ‘/data/[project]/u/[user]’,
  • you can use the backup folder ‘/data/backup/[project]/u/[user].

The backup folders are storage in an independent filesystem, but you can use in the same way that your user folder. By default, the user backup folder quota is 10GB. You can check your disk quotas with the command:

$ /usr/lpp/mmfs/bin/mmlsquota --block-size auto

If you want additional backup space, please, send an email to the system administrator and we can evaluate your case: jescarti[at]mpe.mpg.de

The users can store under ‘/data/backup’ any files for which you want to have a safety copy. It is ok if the files are small, you don’t need to pack them in tar or zip files. Packing files is only necessary for the archive. It is also ok if the files change over time. Only, it doesn’t make much sense to put there purely temporary files.

Everyday at 19h, a cron job is launch and it will backup all files in these folders in TAPE storage. The backup is done incrementally, meaning that every day, only files which are new or have been modified since the previous backup will be copied to the backup server.Any files which have not been modified already got a copy in some previous backup and need not be copied again.

The retention policy for the backup TAPE system storage is the next:

  • When a file is modified, the backup will copy the new file to the
    backup server and keep any previous versions of the same file from
    the last 365 days. When one of these “previous versions” gets older
    than 365 days, it will be completely deleted from the backup
    server. Thus, you may have 365 different versions of a file which
    gets modified every day, or only a few versions of a file which gets
    modified infrequently, or only one version of a file which never
    gets modified. It is possible to recover any of the available
    versions from the backup.
  • When a file gets deleted from the ‘/data/backup’ disk, then any
    versions of it contained in the backup will also be subject to the
    above “aging” process, with one exception: the newest version will
    not be deleted after 365 days, but after 730 days instead.
  • Notice that the “aging” process described above applies only to
    “previous versions” of a file. For a file which still exists on disk
    under ‘/data/backup’, “previous versions” will expire whenever they
    gets 365 days old, but the “current version”, that is, the newest
    copy of the file as it is currently on disk, does not “age”: it only
    starts to “age” when there is a new version which makes it turn from
    “current” to “previous”.