R3D Data Manager Users Guide

Using the R3D Data Manager

Don't let this long help file scare you - using this program is as easy as 4 steps. (1) Setup your defaults (2) Select destinations (3) Select the source (4) Copy. But for more details on what each thing does, read on:

At the beginning of the show you will want to setup your preferences. This will help you move faster once the shoot starts and the footage starts flowing. Make sure all the drives you wish to copy R3D files TO are mounted and accessible. The startup the R3D Data Manager. Enter the project name, the company and the Digital Acquisition Specialist (DAS - a.k.a. you!). You can also setup the shoot day (day one of 10, or exterior shots) and the date.

Next, setup the destination directories. Simply click on "Add A Directory" and navigate to your destination directory. If you need to create a directory, then enter the name of the directory to be created there. Click on "open" and it will load that destination into an available destination spot. You can load up to 4 destinations for each copy process.

Now you can save all these items you just setup as the defaults. Then when you restart the R3D Data Manager application, it will load these defaults. Simply click on "Save as defaults" to save the project information, and "Save Destinations as Defaults" to save the destination directory targets. Now you are ready to begin your data management

Under the menu item "Preferences" there are some options that will affect how R3D Data Manager preforms.

General Tab:

• Auto-start copy when Red Media are mounted
  This option allow the copy process to start immediately when R3D Data Manager detects that a new Red Media was inserted. The copy operation will only progress if you have already setup destinations, and those destinations exist.
• Remind me to change Project Info settings for new Red Media
  With this option, the program will remind you with a popup notice to change the Project Information settings each time the program auto-starts a copy. The copy will still progress in the background while the message is displayed.
• Auto-scroll activity log window
  This enables auto-scrolling in the activity log window
• Prevent destination drives from sleeping
  This option helps to prevent consumer drives from sleeping between copy runs. When enabled, R3D Data Manager will write to a small hidden text file on each destination every 60 seconds. This file takes up very little space and is hidden from normal operations. By doing this R3D Data Manager should be able to prevent common errors with consumer drives.

R3D Files Tab:

• Report errors on R3D files only, ignore checksum errors on other files
  Using this option, only when a R3D file's checksums don't match will the status of the copy be set to error. Other files (such as quicktime files and RSX files) may change, and R3D Data Manager will report those errors as "warnings", not "errors". If there are both checksum errors on R3D files AND other files, it will report the copy process as "error". Only if there are NO errors on R3D files will it report "warning" or "ok". By disabling this option, it will report every error as an error.
• Save master checksums to alternate location
  This enables the option to save the master checksums to a different location when creating checksums from Red Media. This prevents writing to the Red Media before a copy process is made. You need to also specify a folder to save them in.
• Copy digital.*.profile and log files
  This options enables the program to copy all the digital profile file and log files found on the Red Media. This is the default behavior and recommended.
• Separate footage based on camera letter
  This option allows you to set one destination, and R3D Data Manager will handle sorting the camera rolls by the camera letter, putting all the rolls for a particular camera inside a single folder in your destination. It will put all the footage from media roll numbers that begin with A into a folder labeled A. You can select the suffix for the name of the folder, such as "Camera" for the folder name "A Cqmera". When R3D Data Manager copies media, it will create directories as needed if the folder for that camera has not yet been setup.

Reporting Tab:

• Send SMS/email on completion
  This option enables the program to send a SMS or email message to the address you specify. For this option to work, the following requirements must be met:
  • You must have an active internet connection when the copy process is complete.
  • You must have the Mail program open, with one email account configured for sending email.
  • You must specify an email account to send the email to. For SMS, please ask your cell phone provider to give you the email address to their SMS gateway.
• Send entire log via email
  This option appends the log of the copy to the email notification sent. Not recommended for SMS, as the logs are verbose.
• Include timestamp
  R3D Data Manager will append the current time when finished to the SMS message. This is recommended for SMS, since some SMS operators may not always deliver SMS's in order.
• Include destination information
  This allows you to include the destination information in the SMS message.
• Enable Growl support
  This option turns on growl reporting. For more info on Growl, see http://www.growl.info .

Project Info Tab:

• Setup Red Media Identifiers for Project Information:
  This enables you to set a variety of custom identifiers for your Red Media, and then select from them in the Project Information window when copying from a Red Media. When this option is enabled, you must set the identifiers here and use the drop down box in the project information window, as the text box is disabled. With this option disabled, the drop down box is also disabled and the text box is enabled. To add an identifier, type in the text box the name you wish, select the type of media that is, then click add. To remove an identifier, select it then press remove.

Cache-A Tab:

• Select method to communicate with Cache-A device:
  This selects the method that R3D Data Manager will communicate with the Cache-A device. You can set it to contact it via the archive name of the device, or the IP address of the device.
• Automatically begin archive process after each successful copy:
  With this option enabled, R3D Data Manager will automatically start the process of writing data to tape on the Cache-A device. R3D Data Manager will monitor the progress and alert you if there are any errors. When this option is disabled, you will need to manually start and monitor the copy process.
• Cache-A hostname:
  Enter the last two digits of the name of the archive device as shown in finder. This is normally the last two digits of the serial number.
• Cache-A remote IP address:
  Enter the IP address of the device.
• Test connection:
  This provides a way to ensure that R3D Data Manager and the Cache-A device are able to communicate. The test will alert you if it is successful.

While we are talking about preferences and defaults, we should also discuss the Destination Options. These options are located on the main screen toward the bottom right.

• If RDC folder exists
  This option allows you to tell the copy process what to do if it encounters a situation where the RDC folder already exists at the destination. This may be a case of the folder had already been copied and not properly erased off the RED media, or it may be a case where the computer accidentally create the folder again. In the former case, you will want to just bypass those folders, speeding up the copy process. However, in the latter case, you will want to have that data, but not overwrite the data that is already in the destination. So you can choose what you would like for it to do. The option to "Create a new folder" is the safest. The a new folder will be created at the destination, appended with a timestamp (i.e. A001_C001_0845.RDC). So both copies of that folder will live on the destination at the same time. You can then go back with red software and see if it was indeed a duplicate and act accordingly. The "Overwrite all" option will put the new data into the destination directory, possibly removing the old data at the destination if the file names match. The "Ignore" option will not copy anything. The "Ask" option will check if the folder is duplicated first and ask what to do ("create new directory", "overwrite", "ignore") before proceeding.
• After Copy
  This option allows you to speed up your copy process, if you are running short on time. You can skip the checking of the checksums in the destination folder, which will save you about 33% on your copy time. However, this is not recommended, as then we cannot guarantee that a correct copy was made at each destination.
• When Copying from RED Media
  This option is only enabled if the source directory is identified as RED Media. If the source is RED Media, you will have the option to remove the data from the RED media after all copy processes to all destinations are done, including verifying checksums of copied files (if not disabled). This will remove all data there, and prepare the RED Media for formatting on the camera or with R3D Data Managers "Re-format Red Media" option. Please note, if you are just copying from a hard drive, and NOT RED Media, this option will not be enabled, and no data will be removed from the source.
• Unmount Red Media
  This option will unmount all successfully copied Red Media when you click on the "Reset" button. If it is unable to unmount the media for whatever reason, it will notify you.

The actual copying process can be done using two distinct methods. The standard copy method and the auto-copy method.

The Standard Method:

So now you have a RED media full of shots, and are ready to copy files to somewhere usable. First you need to mount your RED Media. Once mounted, click on "Add to Queue". In the window that pops up, navigate to the RED Media. For camera builds 14 and earlier you should see a bunch of RDC folders in the selection window (i.e., be one directory up from the RDC folders). For camera builds 15 and later you should see the RDM folder (i.e., be one directory up from the RDM folder). Then click open.

Once you do this, a couple of things will start happening instantly. First, if it is a RED Media, R3D Data Manager will go through each RDC folder and find all your R3D files. It will then begin creating checksums for these files. You do not have to wait for this process to finish for you to continue a copy process

You can add up to 200 sources to copy at once, and R3D Data Manager will go through each sequentially. To add a another source, simply follow the "Add to Queue" steps once for each new source.

Now if you have your destination directories set, and your project, shot and RED Media info filled out, simply click on "Start Copy". The "Start Copy" button will turn to "Halt", and a wheel will begin rotating next to it, indicating that a copy process is started. Please note, the application will not start to actually copy files until the initial creating checksums phase has finished. However, it will start automatically once the checksum phase has finish - no more user intervention necessary.

If at any time during the copy process you realize that one of the Destination Options needs to be changed, you can change it at that time. It will take effect from that point forward. So, if you decide half way through to change the option to leave the files on the RED Media, it will honor that option, as long as it has not started deleting files.

Auto-Starting copies on Red Media mounts:

With the option to "Auto-start copies when Red Media are mounted", the copy process is slightly different. You first need to ensure that you have the program running before the Red Media is mounted, and that you have valid destinations setup. Only if both of those are done previous to a Red Media mount will the copy auto-start. If you dont have valid destinations setup, a notification will popup alerting you to setup destinations and manually add the source.

When R3D Data Manager detects that a new Red Media has been mounted, it will first clear the queue of old copies runs that have successfully completed. This helps to ensure that in the queue are only items that still need to be copied, not leftovers from previous copy runs. The program will not remove any sources that had any errors to any destinations - those will be left in the queue and retried.

After clearing the queue, R3D Data Manager will then go thorough the RDC folders of the new source, and start calculating checksums. At the same time, if you have asked the program to remind you, it will popup a notification to remind you to update the project info area with new information. The copy process continues behind this alert window. It is safe to update the project information while the copy process is underway.

As with the standard method above, as soon as the copy process starts, the main button in the upper left will turn to "halt" and a wheel will begin rotating right next to it, indicating the copy process has begun. Outside of the above mentioned project information, no more user interaction is necessary.

At any time, yo can stop an individual copy process from continuing. For example, if you setup 3 destination directories, but now just want to copy to one, you can stop the other 2 copy processes without affecting the third. To do this, simply hit the "halt" button next to the destination you wish to stop. This will stop the process at the next available moment - which may not be immediately, but will be before the next file is copied or checksumed.

To remove a destination directory, simply click on the "Remove" button next to the destination you wish to remove. Please note that if you wish to save this, you will need to hit "Save destinations as defaults" again, or it may appear again if you had already saved it. It is not possible to remove destinations while a copy process is in progress.

After all the RDC folders have been copied to all destinations and (optionally) all files checksumed and (optionally) all files removed off the RED Media, then it will present the user with any errors that were encountered. It will sum up the errors to all destinations in one window. It will indicate if it was a copy error or a checksum error, and which RDC folder(s) had the error. PLEASE NOTE: NO FILES WILL BE DELETED FROM RED MEDIA IF THERE ARE ANY COPY OR CHECKSUM ERRORS AT ANY TIME, REGARDLESS OF ANY OPTIONS THAT ARE SET. This is for your safety. If there are no errors, it will present the user with a window that says everything was successful.

Should you want to see exactly what happened, when, and why, there is a activity log window. From the "Window" menu, select "Activity Log". This is a detailed log of each files that was transferred, each file that was checked, and all the results. These results can be saved to a text file for future reference.

Like on set, R3D Data Manager serves a vital role in the post production process. It can copy files to multiple destinations at once, and verify that each copy of every file is valid - and is exactly as first read from the original RED Media.

Just as in the copying files for on set, you can copy files in post the same way. Simply setup your destinations, select your source, and copy. Since the source folder most likely wont be RED Media, the options for deleting R3D files will be disabled. In addition, it will not re-check the checksums on the source, rather it will use the saved checksums and verify the copies from that. If you would like more info on what each option does, please refer above.

You can select a whole days worth of files to copy, or a whole project. Simply select the folder above all the files you wish to copy. You will see that each RDM folder is listed in the queue separately, helping you to identify as copies are done and manage if there are errors.

You do have the option to verify files already in a RDC folder without having to copy them again. Simply select the folder above the RDC folders as your source. Then click on "Check R3D Checksums". If any R3D files don't verify to the saved checksum, you will be alerted to which file. It would be recommended to re-check this file, and if it fails again, assume that this file is no longer valid and restore from a backup.

Accountability and Audit Trails

This file is placed in each RDC folder and copied with copy process to each destination. This file contains a listing of files in that folder, the checksums for each file, and a unix timestamp of when that checksum was done. These would be the master checksums for each checksum type

After each file is copied, the checksums are checked and referenced to the master file (md5sums.txt). The checksum of the file at the destination is then written to this md5sums-xxxxxxxx.txt file. The file contains a checksum for each file in the folder, in the same format as md5sums.txt. This file is never referenced by R3D Data Manager, but becomes your audit trail. From these files, you can tell what the MD5 sum for each file was after every copy process. So if there is ever an error, you can see on what date it happened and from there determine the source.

This file is placed above the RDM folders (RED camera build 15 and above) or the RDC folders (RED camera build 14 and below), in the root of each destination. This file contains all the project, shot, and RED media information. In addition, it contains info about every folder transferred, each file in that folder, and all the checksums for that file. So this can then become a master list of everything that was transferred to that destination. From here, if you are looking for if a specific shot or file was transferred, you can search this text file and find the time it was transferred and the original RED Media for that shot.