Renderfarm for CS42

CS42 Spring 2009: Using Render farm

We've set up a few extra render nodes, belize-node0 through belize-node21, to help you render your animations more quickly. You may not need these extra computers, but we're making them available just in case you want them. This tutorial has instructions on how to use them.

Our setup for the renderfarm is a bit complicated. If you don't need the renderfarm, don't worry about it; you can render fine and quickly enough on the machines in the lab.


You'll follow the following procedure:
1. Correctly set your Maya render options, and save your scene file.
2. Sftp your scene file to the server. (The render farm will see the file automatically.)
3. On a render node you'll then run the renderer. The render will output many frames of your animation.
4. Sftp your frames back to your computer.
5. Compile your frames into a QuickTime movie using QuickTime Pro. From there, they can be assembled into iMovie or you can just continue to work in QuickTime Pro.

Let's look at each of the steps in detail.


1. Setting Your Maya Render Options

First select "Render Settings."

Images/SelectRenderGlobals.jpg


Now we're going to tell Maya the format for the filenames. Inside of Render Settings, go to the "Image File Output" section and set the two pop-up menus to "name.#.ext" and "PNG (png)". (PNG is an image file format.) You can save in an uncompressed format if you want, but this will be huge and take forever to transfer. When you're done, save your scene file.

Images/RenderGlobalSettings.jpg

You'll want to set a prefix for your filenames. You can do that either now in the "File Name Prefix" textfield, or you can do it later on the render node (I'll show you how in a section below). Personally, I find it easier to specify the file-name prefix on the render node, but either way works.

Here is what we mean by the "file name prefix:" Your output frames might have the names:

Bounce.0000001.png
Bounce.0000002.png
Bounce.0000003.png
etc.

The "Bounce" is the filename prefix, and you get to choose that. The number is determined by the frame padding we'll use later. The ".png" part is the file-extension, which tells us that we're using the PNG (.png) image format.


2. SFTP your scene file to the server.

Use the ftp software to SFTP your scene file to pemi.cs.dartmouth.edu.
IMPORANT: When you ftp your file, put it in the directory with your name. We want to keep everyone's files separate, because it is possible for you to delete each other's files. So please be careful!


3. Running a Rendernode.

We have access to 22 machines, belize-node0 through belize-node21. You are going pick one of them, and then login using the 'ssh' command. Open a terminal window and at the prompt:

1. Type in the following line (I arbitrarily pick node 1):

ssh your_login_id@belize-node10

Then hit the return key.

2. You might or might not see the computer ask you about a "RSA key fingerprint". If you do, just say "yes" and hit return. If you don't see this, don't worry, it just means that the computer already has a security key.

3. Next the computer will ask you for your password. Type that in and hit enter.

We're now logged into the render farm. Your screen might look like this:

Images/TerminalLogin.jpg


Navigation and File Manipulation in UNIX

Now that we're logged in, we need to know how to do a few basic operations:
1. Check to see how busy the render-node is (see how many people are logged in, running renders, etc.)
2. File manipulation, like listing files, copying file, moving files, creating/deleting files and directories.
3. Render on the render-node.

A lot of this file manipulation can be done in your sftp software. However, I'll desribe UNIX way to handle files, as well as how to check how busy the render-node is.

IMPORTANT!!! UNIX does not like spaces in it's filenames. You should use filenames that do not have spaces in them. You can replace spaces with underscore _ or a dash -.


Checking how busy the rendernode is in UNIX

Use the command "w" to see how many people are logged into the node. The JCPU and PCPU columns give you an idea of how much each user is using the node. If the usage looks high, you should logout and try again on a different node.

Images/w.jpg


Changing Directories and Listing Files in UNIX

First you'll want to list the files of the directory you are currently in. You can do that with the "ls" command (which is short for "list" directory). Type "ls" and hit return.

We can go to a directory using the "cd" command, which is short for "change directory."

When you change directories with "cd" you might occaisionally get lost. "What directory am I in now?" To find out, you can use the "pwd" command which prints out the current directory.

If you're lost in some random directory, and need to "go home" you can use the special symbol "~" (tilda). The tilda ~ represents the top directory for your account. So if you do "cd ~", then you'll be back at the top level.

One other symbol that is super-useful is ".." (two periods). The .. symbol refers to the parent directory. So "cd .." means "go up one directory." You'll use this a lot. For example, see below.

Images/UNIX.cd.jpg

Sometimes you'll want to get detailed information on your files. You can use "ls -la" which means list all the files and their details. This is like "show as list" in the OS X finder; you'll get to see file-sizes and modification dates for files and folders. File sizes are listed in blocks, not kilobytes.


Moving Files and Directories in UNIX


Suppose you have a file called "bouncingBall.mb" and you want to rename it to be "bouncingBall2.mb". Here's how you would do that. Type in,"mv bouncingBall.mb bouncingBall2.mb" and it will rename the file.
The "mv" command is pretty smart. Besides renaming files, you can use it to rename folders. The "mv" actually stands for "move file" which means it can be use to put files inside of folders. For example, let's put "bouncingBall2.mb" into the directory "Fire_Bounce". You do that with "mv bouncingBall2.mb Fire_Bounce". The computer recognizes that "Fire_Bounce" is a folder name, so instead of renaming the file, it puts it inside the folder. Here's an example.

Images/UNIX.mv.jpg


Creating Directories in UNIX

To create a new directory, you use the "mkdir" command which is short for "Make Directory". For example to make a directory called "NewScene" you would just type "mkdir NewScene". It will place the new directory inside of your current directory.

Images/UNIX.mkdir.jpg


Copying in UNIX

You can copy a file or directory in UNIX using the "cp" command. The "cp" command comes in two versions: one for plain files, and the other for copying entire directories.

Suppose we wanted to copy "bouncingBall2.mb" and give the new copy the name "bouncingBall.mb". We would type "cp bouncingBall2.mb bouncingBall.mb".

To copy a directory, you use the "cp -r" command, which is short for "copy recursive". (Recursive, meaning copying an entire hierarchy of nested folders.)

Images/Unix.cp.jpg

IMPORTANT: Be careful not to overwrite your files when using "cp" and "mv". I suggest having a separate directory for rendering scenes that is not the directory where you are archiving your work.


Deleting in UNIX

In UNIX, you use the "rm" command to delete files, and the "rm -rf" command to delete folders. "rm" stands for "remove" and "-rf" stands for "recursive, forced". Be super ultra careful when using "rm -rf". It is POSSIBLE for you to delete your ENTIRE directory! That means, you could accidently ERASE EVERYTHING. That would be a catastrophe. So tripple check before executing a "rm -rf".

Deleting a single file is pretty easy; just type in "rm bouncingBall2.mb" to delete the file "bouncingBall2.mb". The computer will ask if you want to remove the file. You can answer "yes" or just "y".

Deleting a directory is dangerous because the computer does not ask you,"Are you sure you want to delete the directory?" Instead, the computer just deletes it! I repeat, UNIX does not ask if you are sure before deleting a directory. It just deletes the directory, immediately! So you have to be extremely careful when deleting directories. You should always be a little uneasy when you use "rm -rf" and check that you are deleting the right thing.

To delete a directory called "Fire_Bounce2" you can just say,"rm -rf Fire_Bounce2" And remember that the computer will not ask,"Are you sure?" Always tripple check before "rm -rf" that you are deleting the right thing.


UNIX Commands summary

ls (list files in the current directory) "list directory"
ls -la (list files with details) "list long-format, alphabetical"
pwd (show your current directory) "present working directory"
mkdir (make a directory) "make directory"
cp (copy a file) "copy"
cp -r (copy a directory) "copy recursive"
mv (move or rename a file) "move"
rm (delete a file) "remove"
rm -rf (delete a directory (Be careful!)) "remove recursive and forced" (!)

UNIX directory Symbols (very useful!)
. (current directory)
.. (parent directory; up a directory in the file hierarchy)
~ (top directory for your account)

Remember that instead of UNIX, you can also use sftp software to manipulate files. (copy, rename, delete, make directories)


Rendering on the Render-Node

Okay, so suppose you have done the following:
1. SFTP'ed your Maya scene file to the server
2. You have logged into a render-node using the Terminal program and the "ssh" command.
3. You have used UNIX commands to make folders, and move files around.

You're now ready to render! You render using the "Render" command. Unix is case sensitive, so remember that the Render command is capitalized. Render has a billion options. To see all of the options, type "Render -help -r mr". Here is a text file of all the options for Render. You only need to know a few of these options; I'll show you which ones.

Please follow the following steps when rendering
1. Make a directory for all of your frames. You're going to generate thousands of frames; it's best to organize each clip into it's own directory. For example, you can "mkdir NewScene".
2. Move your Maya scene file into the new directory.
3. Go into your new directory.
4. Upload your entire Maya project directory somewhere on the server (later, this is what -proj will read).
5. Run the Render command. I'll provide details below.
6. Use sftp to transfer your frames back to your computer.
7. In QuickTime Pro, use the "Open Image Sequence" command under the "File" menu. Open up the first image of your sequence, and QuickTime will take all the images and put them into a single QuickTime movie. You can now save or export the movie, or put it into iMovie.

To show you how to use the Render command, I'll go through the above proceedure with an example.
Suppose I have just SFTP'ed a file "sphere.mb" and a project "Farm" into my account on the server.
Next, I'll log into a render node (say, belize-node1), and set up a sub-directory called NewScene. I'll put "sphere.mb" into the NewScene directory, and then go inside the NewScene directory. I'm now ready to render. Here's what this example would look like so far:

Images/PreRender.1.jpg

Okay, I'm now ready to render! The "Render" command uses a whole bunch of options, plus the filename of the file you want to render. Here's an example, which I'll explain. Suppose I want the following:

1. I want to render frames 10 to 40 of my Maya timeline.
2. I want the output frames to be numbered counting up, but starting with frame number 100.
3. I want to call the frames "Bounce"
4. I want the render resolution to be 320 by 240 pixels.
5. I have a Maya project directory with supportng files. In this example, let's suppose the project directory has been uploaded (by sftp) to our render farm, and that the directory is
~/Farm

Here's how I would do it:
Render -r mr -proj ~/Farm -s 1 -e 5 -rfs 100 -pad 8 -rd ~/NewScene -im Bounce -x 320 -y 240 sphere.mb

After awhile, Maya finishes, and you can see all of your frames.

Images/Render.jpg


Many of the options for Render are optional and you can leave them out. For example, if you leave out the "-x 320 -y 240" then Render will use the resolution settings of the ".mb" file.

Two options that are NOT optional are the "-rd ~/NewScene" option and the "-pad 8" option. If you leave out the "-rd ~/NewScene", then Render will put the rendered frames in ~/RazsProject or a default directory if -proj option is not used. You don't want that; you want to put the frames in the directory that you created for saving the frames, which, in this case, is NewScene.

The other option is "-pad 8". This option pads numbers with leading zeros. This is almost always a very good idea. This way, programs will read your frames in the correct order. I recommend ALWAYS using "-pad 8".

Lastly, suppose you start a Render and need to stop it in the middle of rendering. You can type control-C ("control" key, and the "C" key), and the process should stop. This will work if you run Render directly as described above. It will not work if you're running Render in the background (as explained below).


Running Render in the background (Advanced)

Suppose you want to start a Render run, logout, let it run overnight, and then get your frames later. How would you do that? You can use the unix command "nohup" to do this. Here's how you would do the previous example:

nohup Render -r mr -proj ~/Farm -s 1 -e 5 -rfs 100 -pad 8 -rd ~/NewScene -im Bounce -x 320 -y 240 sphere.mb &

Notice the line begins with "nohup" and ends with a percent symbol "&". "nohup" tells unix that the following command cannot be interrupted, and the final & symbol says "run in the background".

Before using "nohup Render ... &" be sure everything is set correctly! You can then logout and get your frames later. Be a little careful about using nohup. If a lot of Renders are running, it's a bit tricky to stop them. Control-C will not stop Render if you run it in the background. You would have to use the UNIX "kill" command to kill the process.

IMPORTANT: If you do not use "nohup", your render will stop when you log out or close the terminal.


Import Image Sequences into Quicktime

After your render finishes, you can SFTP your individual frames from the server to your lab machine. Then you can open them up in QuickTime Pro. Use the "Open Image Sequence" command under the "File" menu. You can now save your QuickTime movie.

I recommend saving your quicktime movie as "self contained". If you don't, you'll have to keep your individual frames around. Now you can use your movie in iMovie, or in QuickTime player.


Tips

Make a directory for each clip of your animation. It'll help you organize your thousands of individual frames.

Use "save as self contained" when exporting your QuickTime movies.

Always triple check before deleting directories "rm -rf" (or in ftp software).

Render a few stills on your lab machine before you render on a render-node. It's a good idea to render the first frame, the last frame, and a couple of frames in the middle. If it looks correct, then use the render farm.

You can use this page to see how much load is on each node.


Adapted from Raz Magar's tutorial from 2004