Maya Mel/API tips, hints, and other random stuff

2002-12-13

I've been doing some Maya programming recently. Seeing as how I couldn't have done it without the help of various individuals that have put their hard earned knowledge on the web I thought I'd post some of what I've derived.

Realize that the MDagPath::partialPathName() of "|foo|bar|moo" is NOT always "moo"!!!!

There are two functions, MDagPath::fullPathName() and MDagPath::partialPathName(). All dagNodes in Maya can be referenced by a DagPath and you get an ASCII representation of them using these two functions.

At first glance it appears it's like files and folders except they use vertical bar "|" to separate levels and so the first thing most people assume is that MDagPath::fullPathName() returns the full path and MDagPath::partialPathName() returns just the name. That is NOT true!!!! Read the docs closely, MDagPath::partialPathName() returns the shortest path which can still be distinguished from any other path.

In Maya, like in files and folders, as long as things are in a not in the same part of the hierarchy they can have the same names. So, if you have two dagNodes one with a full path of "|group1|group2|mymesh" and another with "|group1|group3|mymesh", if you call MDagPath::partialPathName() on the first one you'll get "group2|mymesh".

I've seen lots of code, including my own, that thought if I called MDagPath::partialPathName() I'd always get only everything after the last vertical bar.

How do you extract the bind/pose state of a bone/joint hierarchy the correct way?

First, what do I mean by correct way? When you are making a game and you have one skin characters, generally you need to store a polygon mesh and effect it by various bones. The general way to do this requires that you know the position and orientation of the bones when they are exerting no influence on the mesh. In other words, their positions and orientation at the time the mesh was connected or bound to the bones.

There are 2 common ways to do this that I know of both of which are sub optimal

1) Make your artists put the their object/character/monster in it's bind/pose position and then run some exporter that records the position and orientation of all the bones. The problem with this method is it requires work on some artist's part. Work that has to be repeated each time something is changed or each time you change the data format for your game or each time you start a new platform.

Another problem with this method is if you are making a 3D movie scene exporter it requires your artists to export their objects separate from the movie since most likely at no time during the movie are your characters in their bind/pose state

2) AliasWavefront suggests you do this by looking up the dagBindPose info and they give you a sample in their devkit called dagBindPose.cpp This fixes all of the above problems. It unfortunately has one new problem. It is possible to delete the bind/pose information from Maya. Maya will still function fine, your scene will still work but your exporter that tries to look up the dagBindPose will fail.

You could tell your artists "DON'T DELETE THE BINDPOSE" but my experience is that most artists have a hard time following rules on top of which you never know when some other process or plugin or editing method will happen to always delete that info.

So, here is what I currently do: Even if you delete the bindPose Maya still has the information stored because without it it too could not manipulate your characters. Here is how to look it up:

Assuming jntFn is a MFnIkJoint:

MObject jointNode = dagPathForJoint.node();
MFnDependencyNode fnJoint(jointNode);
MObject attrWorldMatrix = fnJoint.attribute("worldMatrix", &stat);

MPlug plugWorldMatrixArray(jointNode,attrWorldMatrix);

for (unsigned i = 0; i < plugWorldMatrixArray.numElements (); i++)
{
    unsigned connLength = 0;
    MPlugArray connPlugs;

    MPlug elementPlug = plugWorldMatrixArray [i];
    unsigned logicalIndex = elementPlug.logicalIndex();

    MItDependencyGraph dgIt(elementPlug,
                            MFn::kInvalid,
                            MItDependencyGraph::kDownstream,
                            MItDependencyGraph::kDepthFirst,
                            MItDependencyGraph::kPlugLevel,
                            &stat);
    if (MS::kSuccess == stat)
    {
        dgIt.disablePruningOnFilter();
        int count = 0;

        for ( ; ! dgIt.isDone(); dgIt.next() )
        {
            MObject thisNode = dgIt.thisNode();

            if (thisNode.apiType() == MFn::kSkinClusterFilter)
            {
                MFnSkinCluster skinFn(thisNode);

                MPlug bindPreMatrixArrayPlug =
                     sknFn.findPlug("bindPreMatrix", &stat);
                MPlug bindPreMatrixPlug =
                     bindPreMatrixArrayPlug.elementByLogicalIndex(logicalNdx,
                                                                  &stat);
                MObject dataObject;
                bindPreMatrixPlug.getValue( dataObject );

                MFnMatrixData matDataFn ( dataObject );

                MMatrix invMat = matDataFn.matrix();
                glbMat = invMat.inverse();

                // glbMat is now the world matrix for this
                // particular joint at bind/pose time
            }
        }
    }
}

Of course you only get global matrix with this method. To get a local matrix you'll have to find the same information for the parent joint and multiply it's inverse with this matrix. If you want translation, scale and rotation you'll need to do some math. One relatively easy way is to use MQuaternion, init with the matrix. You can also convert that quaerntion to an MEulerRotation if you want.

NOTE: It would probably be better to start with the mesh and see what things are influencing it rather then look at the joints directly. Unfortunately I was working with a system that was already in place and so that was not an option for me.

How do you extract the skin for a one skinned model the correct way?

Again, what do I mean buy correct way. Just like for the bind/pose of the joints/bones, to do a skinned character in a game you generally need to save off the mesh before it has been influenced by the bones.

Again, many poorer tools force the artists to manually put the character in that state before exporting, then they read the visible mesh in that state and save it out.

Even AliasWavefront again, does not give any correct examples about how to do this. They have one example that searches through the entire hierarchy for all MFn::kSkinClusterFilter nodes and then exports geometry by looking up things through MFn::kSkinClusterFilter::getInputGeometry(). First, looking up all SkinClusters is not a good example, especially if you want to write a scene exporter. Second, calling MFn::kSkinClusterFilter::getInputGeometry() will give you the mesh at the top of the input chain. That may or may not be the mesh that's is actually being deformed. What I mean by that is if your artists did not collapse their construction history before they bound their mesh to the bones then getInputGeometry() will return the mesh before all those edits. That, for example, might be a 50 polygon mesh but the actual mesh being deformed might actually have 100 polygons.

The correct way is to lookup what data is actually coming into the skinCluster. That data is the data after it's come out of the original mesh, after it's been edited, and just before it's about to be deformed by the bones.

MStatus stat;
MFnDagNode dagNode(meshDagPath);    // path to the visible mesh
MFnMesh meshFn(dagPath, &stat);     // this is the visible mesh
MObject inObj;
MObject dataObj1;

// the deformed mesh comes into the visible mesh
// through its "inmesh" plug
MPlug inMeshPlug = dagNode.findPlug("inMesh", &stat);

if (stat == MS::kSuccess && inMeshPlug.isConnected())
{
    // walk the tree of stuff upstream from this plug
    MItDependencyGraph dgIt(inMeshPlug,
                            MFn::kInvalid,
                            MItDependencyGraph::kUpstream,
                            MItDependencyGraph::kDepthFirst,
                            MItDependencyGraph::kPlugLevel,
                            &stat);

    if (MS::kSuccess == stat)
    {
        dgIt.disablePruningOnFilter();
        int count = 0;

        for ( ; ! dgIt.isDone(); dgIt.next() )
        {
            MObject thisNode = dgIt.thisNode();

            // go until we find a skinCluster

            if (thisNode.apiType() == MFn::kSkinClusterFilter)
            {
                MFnSkinCluster skinCluster(thisNode);

                // get the mesh coming into the skinCluster.  This
                // is the mesh before being deformed but after
                // being edited/tweaked/etc.

                MPlug inputPlug = skinCluster.findPlug("input", &stat);
                if (stat == MS::kSuccess)
                {
                    MPlug childPlug = inputPlug.elementByLogicalIndex(0);
                    MPlug geomPlug = childPlug.child(0);

                    geomPlug.getValue(dataObj1);

                    // let use this mesh instead of the visible one
                    meshFn.setObject(dataObj1);
                }

...

use the various MFnMesh functions to pull out the verts for this mesh.

How do you keep network plugins updatable (unlocked)?

The issue is that often at a game company you write some plugins, you put them on the net and you have each artist add the network path to their Maya plugin path. That way they all have access to the plugins. The problem is, as long as they are running Maya those plugins are locked (write−protected) so you can't update them unless you ask all your artists to quit Maya.

The simple answer is "don't let them be run off the network".

There are several ways to do that. At one company I set up a batch file with an icon on the artist's desktop. "Update Maya Plugins". This batch file would go get the plugins off the net and copy them to the artist's machine. That way the artist is never actually using the files on the net therefore they are never locked and can always be updated.

At my current job we made it more automated. Using the following script either in the user's userSetup.mel or in a script called by userSetup.mel, each time they run Maya it copies the plugins off the net to their local drive. There is no noticeable delay for us.

−−−−−Maya.env−−−−−

# Plugin Path
MAYA_PLUG_IN_PATH = $MAYA_LOCATION\our-plugins

# Mel Script Path
MAYA_SCRIPT_PATH = \\network\maya\4.5\scripts<

−−−−−−−OurStartup.mel−−−−−−−

//
// get path to user's LOCAL script folder
//
string $ourMayaVersion     = "4.5";
string $wowMayaPluginPath  = "//network/maya/" + $ourMayaVersion + "/plug-ins/";

//
// there's no way to add the user's local maya folder so let's use
// the main maya folder
//
string $myPluginDir = `getenv "MAYA_LOCATION"` + "/our-plugins";

//
// check if there is a plugin folder inside
//
if (!( `filetest  -d $myPluginDir` ))
{
    // no, so make one
    sysFile -makeDir $myPluginDir;
}

// get a list of all our plugins
string $files[] = `getFileList -folder $ourMayaPluginPath -filespec "*.mll"`;

// copy all the plugins locallly
for ($file in $files)
{
    print("copying " + $file + "\
");

    string $src = $ourMayaPluginPath + $file;
    string $dst = $myPluginDir + "/" + $file;
    sysFile -copy $dst $src;
}

MEL scripts can stay on the network since they are not locked by Maya. Also, I even went so far as the name the files on the net so they ended in something like ".mayaplugin" because I wanted to make sure no one could get lazy and just link to the files on the net.

note: It would be cool to be able to check the dates of the files about to be copied and only copy if the network files were newer but apparently there is no way to do that easily from MEL

How do you add Maya to your build process/tool path?

Call me crazy but it seems to me the smartest way to make a game is to make your tools start with the source data and convert it to the game data. That means for example if you have a photoshop file you need to have made into a texture you put that photoshop file some place in your conversion environment, makefile, whatever, and it gets converted to a texture. This way, your artists only have to deal with one file, the photoshop file. The same for Maya. You start with the Maya .MB file, you put it in your conversion environment, add it to your make file or batch files or whatever and it gets built into game data.

For whatever reason, it seems like 80 or 90% of the companies out there don't do this. Instead they expect their artists to hand export the data. Load up Photoshop, load in your texture, save as ".tim" or ".myimageformat" Load up your Maya file, select a specific node, export it with our custom exporters, give me the exported file.

That is an atrocious way to do things. There are all kinds of problems with that method

  • Since the original file is not needed to build the game people often forget which file the source is from. (was it zombie_ab_0012_hand_left.mb or zombie_bcd_0233_update.mb??)
  • Often, not always, exporters are context and state sensitive. The correct node must be selected, the time line set, the time range set etc before exporting. Forget something and the data will be bad and you spend sometimes hours trying to figure out why.
  • If you are making a multi-platform game, even if the first version is only one platform, when it comes time to make data for the second platform your artists will have to hand export all that data again
  • If you change the format of your exported data your artists have to hand re-export the data again.

The point is, exporting the data by hand is BAD BAD BAD.

So, how do you export the data automatically as part of your build process? Easy: Use Mayabatch. Mayabatch is a separate program from Maya. It's installed by default as of version 7.0. It requires no license unless you are rendering with mental ray so you can put in on any machine. It also loads none of the UI so it runs much faster. Here's some perl I use:

#!perl
#
use strict;
use warnings;

use IO::Handle;                     # 5.004 or higher

#
# unixifyPath ($path)
#
# change \\ to /
#
sub unixifyPath
{
   my $path = $_[0];

   $path =~ s/\\\\/\\//g;

   return $path;
}

my $origFilename   = "mymayafile.mb";
my $fileToExportTo = "exportedfile.dat";
my $melFilename    = "tempfile.mel";
my $melLogFilename = "tempfile.log";

my $outfh = IO::Handle-&gt;new();
open ($outfh, "&gt;" . $melFilename) or
    die ("couldn't open $! ", $melFilename, "\
");

# this line loads the scene
print $outfh 'file -o "', unixifyPath($origFilename), "\\";\
";
# this line runs our exporter
print $outfh 'ourExporter "', unixifyPath($fileToExportTo), "\\";\
";

close ($outfh);

my $cmd = "mayabatch -nosplash -log \\"" . $melLogFilename .
          "\\" -script \\"" . $melFilename . "\\"";
system ($cmd);

It writes out a script to load the scene then execute our plugin. Of course you could write out more melscript to select the correct node or set the timeline etc.

It also tells maya to write a log file which puts all of your cout << or printfs into the log.

Note that Maya does not return a correct return value so you'll need to make your plugin either write some kind of status file or output something to the log file

Also note that Maya only likes forward slashes in its filenames.

How do you get a list of textures in the current scene FOR REAL?

most docs will tell you

string files[] = `ls -type "file"`;
for (file in files)
{
    //get the fileTextureName
    string filename = `getAttr (file + ".fileTextureName")`;

    print (filename +"\n");
}

is all you need. Unfortunately that's not the case. If we have animated textures those will not be include. The function GetListOfAllPossibleTextures below attepts to find those. Note that there is no perfect solution. The solution below is one that works for me.

/*************************************************************************
                             GetFrameFile
 *************************************************************************/
/**
    @brief  Find a file by filename and frameNumber

    given a filename with a number already in it, attemps to change
    that number to the given frameNumber and check for that file's
    existance.  If found returns the new filename

    Only supports these formats

       name#.ext
       name.#.ext
       name####.ext
       name.####.ext

    @param  filename
    @param  frameNumber

    @return found filename or "" nothing found

    @see

*/
/* ----------------------------------------------------------------------- */

proc string GetFrameFile (string filename, int frameNumber)
{
    string result = "";

    // get extension
    string ext = fileExtension(filename);

    // exit if no extension
    if (size(ext) > 0)
    {
        // remove the extension
        ext  = "." + ext;
        string work = substring(filename,  1, size(filename) - size(ext));

        // get trailing numbers
        string numbers = match("[^0-9][0-9]+", work);

        if (size(numbers) > 0)
        {
            // remove the trailing numbers
            numbers = substring (numbers, 2, size(numbers));
            work    = substring (work, 1, size(work) - size(numbers));

            // try name#.ext and name.#.ext first
            string test = work + frameNumber + ext;
            if (`file -q -ex test`)
            {
                result = test;
            }
            else
            {
                // try name####.ext and name.####.ext
                string zeros = "000000000000000000000";
                string fnum  = frameNumber;

                int totalDigits = size(numbers);
                int haveDigits  = size(fnum);
                int needDigits  = totalDigits - haveDigits;

                test = work + startString(zeros, needDigits) + fnum + ext;
                if (`file -q -ex test`)
                {
                    result = test;
                }
            }
        }
    }

    return result;
}

/*************************************************************************
                        CheckForTextureFrames
 *************************************************************************/
/**
    @brief  Check for texture files by frame number

            Checks in the given direction for files that match
            the given filename starting at the given frameNumber.

            In other words, if you pass in

            x:/folder/file.15.tga, 20, 1

            it will check for

            x:/folder/file.15.tga
            x:/folder/file.16.tga
            x:/folder/file.17.tga
            x:/folder/file.18.tga
            x:/folder/file.19.tga
            x:/folder/file.20.tga
            ...

            it will continue until it gets an error but ignore the first 5
            errors. It will not check negative file numbers.

            the reason we skip errors is because it's the default for maya to
            set the frame extension to the timeline frame number.  It's
            also common for artists to start numbering from frame 1 so
            if the timeline is set to frame 0 the file maya is looking for
            will not exist.

    @param  filename
    @param  frameNumber
    @param  direction   1 or -1

    @return string array of files found

    @see

*/
/* ----------------------------------------------------------------------- */

proc string[] CheckForTextureFrames (
    string filename,
    int frameNumber,
    int direction)
{
    string textures[];
    int ignoreErrCount = 5;

    while(1)
    {
        ignoreErrCount--;

        string frameFile = GetFrameFile(filename, frameNumber);
        if (size(frameFile) > 0)
        {
            textures[size(textures)] = frameFile;
        }
        else
        {
            if (ignoreErrCount &lt;= 0)
            {
                break;
            }
        }

        frameNumber += direction;
        if (frameNumber &lt; 0)
        {
            break;
        }
    }

    return textures;
}

/*************************************************************************
                     GetListOfAllPossibleTextures
 *************************************************************************/
/**
    @brief  Get a list of all possible textures in a scene

        most docs will tell you

            string files[] = `ls -type "file"`;

        is all you need.  Unfortunately that's not the case.  If we have
        animated textures those will not be include.  This function
        attepts to find those.

        Since anything could be driving the frameExtension on an
        animated texture (curve, expression, etc..)
        the only true way to find which textures are used is to
        run the animation and check the result every frame
        but there is no way to tell how many frames to run for.  Just
        because timeline is set to 1-100 doesn't mean the animation
        doesn't actually run 1-500 (or even 423-437)

        so....

        we just have to make a guess.  For every texture that has
        it's useFrameExtension set I start with the current
        frameNumber and check for existing textures -5 to +5
        frames.  If I find an existing texture file I keep
        checking in that direction.  Hopefully that will find
        all the textures.  It could mean frames not actually used
        well be included in the list.

        worse, as far as I can tell, Maya doesn't provide a function
        that turns a filename + frame number into one of their
        supported formats.  At least not from mel.  There's the
        MRenderUtil::exactFileTextureName but I have not been
        able to get it to work

        worse yet, the maya docs say name#.ext is not supported
        yet my artists are using that format and it's working.

        for now I'm only going to support these formats

          name#.ext
          name.#.ext
          name####.ext
          name.####.ext

        maya also supports

          name.ext.#
          name.ext.####

        and

          name.#
          name.####

        but both of those are left over from unix days when programmers
        thought all users could remember what kind of file some file
        was with no ext and no metadata.  Probably the same programmers
        that thought users would like case sensitive file systems.
        I don't personally know any such users &#128539;

    @return

*/
/* ----------------------------------------------------------------------- */

global proc string[] GetListOfAllPossibleTextures()
{
    string textures[];

    string files[] = `ls -type "file"`;

    for (file in files)
    {
        string filename = getAttr(file + ".fileTextureName");

        // now, is this marked as animated.

        if (getAttr(file + ".useFrameExtension"))
        {
            int currentFrame = getAttr(file + ".frameExtension") +
                                        getAttr(file + ".frameOffset");

            // check down
            textures = stringArrayCatenate(
                            textures,
                            CheckForTextureFrames(filename, currentFrame, -1));
            // check up
            textures = stringArrayCatenate(
                            textures,
                            CheckForTextureFrames(filename, currentFrame,  1));
        }
        else
        {
            textures[size(textures)] = filename;
        }
    }

    textures = stringArrayRemoveDuplicates(textures);

    return textures;
}

It turns out that doesn't cover it either. "file −q −l" will also get you referenced files as well as textures used as guides in your various views. I'll look into fixing the example above to use that instead.

What other resources for Mel and Maya API programming are there?

I'm sure there are more but these are the ones I know about

Comments
Metroid Prime
Super Mario Second Reality