Hedberg Games

Art Pipeline Part 2

Posted On: 2019-01-21

By Mark

This is part 2 in a series about the art pipeline I am building. If you haven't already read it, I recommend checking out part 1, as that describes the "what" and "why", while this post is entirely focused on the "how". Furthermore, since this post will be focused on the "how", I will be including a lot of code samples and detailed explanations. (Consider that fair warning: today's post will be really long.)

Writing a Gimp Plugin

The Gimp art program uses a plugin system that allows anyone to write programs to run inside of Gimp. Plugins can be authored in a lot of different languages, but for this particular task I chose to use python, since it is a good language for rapid prototyping/iteration (and I have a bit more experience with python.)

The first step to writing the plugin was setting up the Integrated Development Environment (IDE). Normally python can be authored without an IDE (any old text editor will work) but I like coding with type-ahead support and automated documentation (it's much faster than looking up the documentation online or in the source code.) Regardless of which IDE you use (I happened to use Visual Studio) it should be roughly the same process: create a new project of the type Python, and set the external references to include Gimp's Python libraries. Gimp's python libraries are included in the Gimp install folder - in my case (Gimp version 2.8) it was under the path Gimp 2/32/lib/gimp/2.0/python. (If you're having trouble finding it, it is the folder containing the gimpfu.py file, so you can search for that. Once added, you can confirm it works by adding from gimpfu import *to an empty python script, and the IDE should indicate it can resolve the reference (for example, by displaying the documentation if you mouse over gimpfu.)

Plugin Registration

Gimp plugins are generally made up of two parts: the registration with the system and the code that will execute. I will write little about the registration (it's basically the same as you'd find in any tutorial for Gimp Python plugin.) Suffice to say that all registration is basically a call to register with a bunch of information like what it's called and who owns the copyright. The only interesting thing to mention is that in the list of parameters, I added a parameter (PF_DIRNAME, "root_directory", "Output directory", "/tmp") which tells Gimp to prompt the user with a directory picker called "Output directory". For completeness, here's the whole registration:

#Name of the method
register("export_character", 
#Description
"Exports to a collection of png images, organized by layer folders.", 
#Detailed description
"Exports to a collection of png images, organized by layer folders.",  
#author
"hedberggames",     
#copyright owner
"hedberggames",  
#year
"2019",           
#Name displayed in the menu
"Export Character", 
#Supported image types
"RGB*, GRAY*",    
[
   #First parameter of the method, the image itself
   (PF_IMAGE, "image", "Input image", None),    
   #Second parameter of the method, the "drawable" (I don't actually use this)
   (PF_DRAWABLE, "drawable", "Input drawable", None),                 
   #Third parameter: the folder we are going to save the files into
   (PF_DIRNAME, "root_directory", "Output directory", "/tmp")],       
#Results returned (empty because we don't return anything)
[],      
#The method that should execute when the user clicks "Ok"
plugin_main,                                                          
#Which menu should this appear under (when an image is open, it should appear at the bottom of the "Image" menu)
menu="<Image>/Image")

Parsing Names

In the actual plugin logic, the first thing it does is determine the character's name, based on the file name. Since I parse names at several points (file names and layer names), I have separated the logic out into it's own method parse_name. The method makes a few assumptions (that the first instance of the - character is the place to split the name, that the name is only two parts) but I generally prefer convention over configuration, so I figured it will be fine to hard-code those assumptions as long as it works for all use cases (which it currently does.) The code for splitting is a bit long-winded, but the end result is to store the values in the category and descriptor variables:

index = name.find("-")
if(index == -1):
    category = name
    descriptor = ''
else:
    category = name[0:index]
    descriptor = name[index + 1:len(name)]
Additionally, I also use a regular expression to make sure these values don't contain any invalid characters (they will be used for file names, so \ or similar symbols could cause unwanted behaviour):
import re
regex = "[^a-z0-9_ ]"
category = re.sub(regex, "", category.lower())
descriptor = re.sub(regex, "", descriptor.lower())
Finally, I return the results, using a dictionary (I have found dictionaries tend to be more maintainable than arrays or tuples when dealing with unordered collections):
return {'category':category, 'descriptor':descriptor}

In the plugin code, the first time I use parse_name is right at the start, to get the character name from the file name:

#Get character name from file name
character_name = parse_name(image.name)["category"]
While the character name won't be used until much later, the remainder of the code is using nested loops, so I prefer to get it once before any of the loops run.

Core loops

The images I am using are expected to have a specific structure: each pose (aka frame of animation) for the character is in a different layer group. Within each group, all the layers are listed individually (there are no sub-groups). Although this pattern is arbitrary, and may change in the future (specifically I may find a situation where sub-groups are useful,) I am currently writing the code assuming that pattern is used. I am choosing to code those assumptions into the plugin primarily because coding flexibility to support different patterns at this time would not be worthwhile (if I need flexibility, it's best to code it when I understand what kind I will need, which will be after using the existing code extensively.) Although I wrote the code to match the existing assumptions, I did try to keep the loops and the actual logic visibly distinct, that way it will be easy to break into its individual parts whenever I revisit it to make it more flexible. The actual code for this is mostly self-documenting, using variables names that match their purpose, and comments to clarify code to handle edge cases:

#assume top level folders are the poses
all_poses = image.layers
for pose in all_poses:
    if type(pose) != gimp.GroupLayer:
        #This is an invidual layer, not a group.
        #Skip over it, as it is likely a background 
        #or reference photo
        pass
    else:
        #Begin processing each pose

Within each pose is multiple layers, and they are generally organized to make modifying them as convenient as possible. Some of those layers correspond to individual images that will be rendered by the engine (such as the character's hair) while other layers are intended to be part of a set of layers that will be merged together before appearing in-engine (such as merging the character's limbs onto the body.) To accommodate this, I have something I refer to as a "layer category", which is a series of layers that should be merged together. I designate the "layer category" using the layer's name (using the - as a separator, so that I can use the parse_name method from earlier.) The plugin uses this information to organize the layers by their categories:

#A pose is a group of individual layers
#Loop through the layers an store them in the correct category,
#so that all layers in a category can be merged together to make 
#one image
all_categories = OrderedDict()

for layer in pose.layers:
   name_parts = parse_name(layer.name)
   category = name_parts["category"]
   if category not in all_categories:
       all_categories[category] = []
   all_categories[category].append(layer) 

#We now have all the categories for this pose, each one containing
#a list of layers.

Adjusting the offset

Each pose is positioned on a grid, similar to how they would appear in a sprite sheet. This is convenient for creating and comparing the poses, but when exporting to a separate image, the layers retain their original position, which places them outside the bounds of the image. To work around this issue, I recorded the relative offset of the entire layer group (in the "description" part of the name) that way I can use it to correctly re-positioned each individual layer during the export process. Recording the relative offset in the layer name is currently a manual (and therefore error prone) process, so I keeping an eye out for other ways to effectively capture this information. 

name_parts = parse_name(pose.name)
pose_name = name_parts["category"]

#Calculate layer offset from the layer's name
#This will be used to correctly position the layer
#when merging the image together
pose_offsets = name_parts["descriptor"].split("x")
pose_offsets[0] = int(pose_offsets[0])
pose_offsets[1] = int(pose_offsets[1])

Merging Image Categories

In order to export the layers, I copy their information into a new (invisible) image, perform any required merges, and then save that image as a new file. There are a a lot of individual tasks associated with doing this, but most of them are quite straight-forward. As such, I will approach this section slightly differently: I'll provide the code first, and follow it with a brief description of what it's doing (and any particular details that may be worth noting.) I will also include the comments, as they succinctly describe wherever I am including assumptions in the code.

for index, category_name in enumerate(all_categories):
Loops over the ordered dictionary of layer categories. index is the position in the list and category_name is the key for that entry (which is the name of that particular category). 

#Merging the layers together to form a single image per category
RGB = 0
#Assume the top layer in a category is the width and height of
#the entire image (having varied dimensions across layers 
#causes bugs regardless, so this is safe to assume)
width = all_categories[category_name][0].width
height = all_categories[category_name][0].height
#A temporary, in-memory image.
#Eventually this image will be saved to the file system
temp_image = pdb.gimp_image_new(width, height, RGB)
Using pdb.gimp_image_new, the plugin creates an image. (The user won't see it - images need to be assigned to a display in order to appear to the user.) Everything the plugin does (merging, saving, etc) will be using this temporary image, so that it doesn't impact the original one.

layers = all_categories[category_name]
#Copy layers in reverse order
#since we will insert each layer on top of the previous one
for layer in reversed(layers):
    #reassign the layer variable to be the layer in the temporary image
    layer = pdb.gimp_layer_new_from_drawable(layer, temp_image)
    parent = None
    layer_index = 0
    pdb.gimp_image_insert_layer(temp_image, layer, parent, layer_index)
 
    #Fix up the offsets (so they aren't floating somewhere off
    #the page)
    #Need to Unlink the layers first, so we don't risk moving
    #any other layers
    pdb.gimp_item_set_linked(layer, False)
    modified_offsets = (layer.offsets[0] - pose_offsets[0], layer.offsets[1] - pose_offsets[1])
    pdb.gimp_layer_set_offsets(layer, modified_offsets[0],modified_offsets[1])
As mentioned before, a layer category is a group of layers that need to be merged together. To do that, the plugin copies the layer to the new image, but this also copies the layer's offsets (where the layer is relative to the entire image). The offset required for use in this temporary image is then calculated, based on the offset for that layer and the offset for the entire pose.

#flatten the image
#Note: a runtime error here usually means an issue with the
#offsets (clip to image causes a runtime error if all the 
#layers are out of bounds)
clip_to_image = 1
flattened_layer = pdb.gimp_image_merge_visible_layers(temp_image, clip_to_image)
Once all the layers for that layer category is copied, they are flattened into a single layer. The resultant layer will then be used when we get to saving the file (the next section.)

Saving the image

Finally, it's time to save the image! To recap, the image is all the layers for one layer category (such as "hair" or "clothing"), merged together and repositioned so that they are in a single image. The image will be saved inside the root_directory (the one the user chose using the UI), and in nested sub-folders based on the character and pose names. Of course, the folders may not exist, so we may need to create them. Also, when we perform this save, it overwrites any existing files with the same name (which is generally desirable, as this same plugin creates them the first time, and then can update them if they are changed). Lastly, it is worth mentioning that I created the save as a separate method (it is very verbose, so the main plugin is much more readable with those details moved elsewhere.)

def save_png(img, layer, root_directory, character_name, pose_name, category_name, index):
    #Note: Right now, we don't use the index, but we may need it in the future
    #(ie.  for automation around combining the layer categories back together)
 
    #right now each possible folder is hardcoded (there is surely a better way to do this)
    if not os.path.exists(os.path.join(root_directory, character_name)):
        os.mkdir(os.path.join(root_directory, character_name))
    if not os.path.exists(os.path.join(root_directory, character_name, pose_name)):
        os.mkdir(os.path.join(root_directory, character_name, pose_name))
    
    #Options for the save menu
    filename = os.path.join(root_directory, character_name, pose_name, category_name + '.png')
    raw_filename = filename
    interlace = 0
    compression = 9
    bkgd = 0
    gama = 0
    offs = 0
    phys = 0
    time = 0
    comment = 0
    svtrans = 1
 
    pdb.file_png_save2(img , layer, filename, raw_filename, interlace, compression, bkgd, gama, offs, phys, time, comment, svtrans)

Cleanup

After saving the image, the plugin deletes the image, as this is required to avoid leaking memory. (In more polished code it should do this via a finally block, but when debugging it is actually helpful to have the image remain in memory if an error occurs, since you can then use the console to attach a display to it and investigate the issue.) This is the last action performed on each layer category. 

 pdb.gimp_image_delete(temp_image)

To Be Continued

That's it, all the logic of the new plugin laid bare. Hopefully it's been useful (or at least interesting) to see all those details. Also, while I had originally planned to have Part 2 encompass both the Gimp and Unity plugins, looking at the length, it seems saving Unity for part 3 would best. Tune in next week, for the details of the Unity side of the automation.

Complete code

Also, for those that are interested, I've included the complete code below (just expand it to see):

from gimpfu import *
from collections import OrderedDict
import os
 
def parse_name(name):
    """Splits the name along the seperator character (currently '-')
    Returns a dictionary containing "category" and "descriptor", so that implementation can change without impacting calling code
    """
    
    #I am sure this can be done better, but I am still a stranger to python
    index = name.find("-")
    if(index == -1):
        category = name
        descriptor = ''
    else:
        category = name[0:index]
        descriptor = name[index + 1:len(name)]
        
    import re
    #Sanitize input (why doesn't python have any built-in methods
    #for sanitizing a user generated filename?)
    regex = "[^a-z0-9_ ]"
    category = re.sub(regex, "", category.lower())
    descriptor = re.sub(regex, "", descriptor.lower())
    
    return {'category':category, 'descriptor':descriptor}
 
def save_png(img, layer, root_directory, character_name, pose_name, category_name, index):
    """Saves a single png image for the specified character, pose, and category.
    Automatically generates folders if they don't exist
    """
    #Note: Right now, we don't use the index, but we may need it in the future
    #(ie.  for automation around combining the layer categories back together)
 
    #TODO: Make this scale better
    #(right now each possible folder is hardcoded)
    if not os.path.exists(os.path.join(root_directory, character_name)):
        os.mkdir(os.path.join(root_directory, character_name))
    if not os.path.exists(os.path.join(root_directory, character_name, pose_name)):
        os.mkdir(os.path.join(root_directory, character_name, pose_name))
    
    #Options for the save menu
    filename = os.path.join(root_directory, character_name, pose_name, category_name + '.png')
    raw_filename = filename
    interlace = 0
    compression = 9
    bkgd = 0
    gama = 0
    offs = 0
    phys = 0
    time = 0
    comment = 0
    svtrans = 1
 
    pdb.file_png_save2(img , layer, filename, raw_filename, interlace, compression, bkgd, gama, offs, phys, time, comment, svtrans)
   
def plugin_main(image, drawable, root_directory):
    """Main entry point of the plugin.
    Loops through the layer folders in the current file, generating png files for their contents.
    """
 
    #Get character name from file name
    character_name = parse_name(image.name)["category"]
 
    #assume top level folders are the poses
    all_poses = image.layers
    for pose in all_poses:
        if type(pose) != gimp.GroupLayer:
             #This is an invidual layer, not a group.
             #Skip over it, as it is likely a background
             #or reference photo
            pass
        else:
            #Begin processing each pose
            
            #A pose is a group of individual layers
            #Loop through the layers an store them in the correct category,
            #so that all layers in a category can be merged together to make
            #one image
            all_categories = OrderedDict()
            
            for layer in pose.layers:
                name_parts = parse_name(layer.name)
                category = name_parts["category"]
                if category not in all_categories:
                    all_categories[category] = []
                all_categories[category].append(layer) 
                
            #We now have all the categories for this pose, each one containing
            #a list of layers.
            
            name_parts = parse_name(pose.name)
            pose_name = name_parts["category"]
 
            #Calculate layer offset from the layer's name
            #This will be used to correctly position the layer
            #when merging the image together
            pose_offsets = name_parts["descriptor"].split("x")
            pose_offsets[0] = int(pose_offsets[0])
            pose_offsets[1] = int(pose_offsets[1])
 
            for index, category_name in enumerate(all_categories):
                #Merging the layers together to form a single image per
                #category
                RGB = 0
                #Assume the top layer in a category is the width and height of
                #the entire image (having varied dimensions across layers
                #causes bugs regardless, so this is safe to assume)
                width = all_categories[category_name][0].width
                height = all_categories[category_name][0].height
            
                #A temporary, in-memory image
                #Eventually this image will be saved to the file system
                temp_image = pdb.gimp_image_new(width, height, RGB)
 
                layers = all_categories[category_name]
                #Copy layers in reverse order
                #since we will insert each layer on top of the previous one
                for layer in reversed(layers):
                    #reassign the layer variable to be the layer in the
                    #temporary image
                    layer = pdb.gimp_layer_new_from_drawable(layer, temp_image)
                    parent = None
                    layer_index = 0
                    pdb.gimp_image_insert_layer(temp_image, layer, parent, layer_index)
 
                    #Fix up the offsets (so they aren't floating somewhere off
                    #the page)
                    #Need to Unlink the layers first, so we don't risk moving
                    #any other layers
                    pdb.gimp_item_set_linked(layer, False)
                    modified_offsets = (layer.offsets[0] - pose_offsets[0], layer.offsets[1] - pose_offsets[1])
                    pdb.gimp_layer_set_offsets(layer, modified_offsets[0],modified_offsets[1])
 
                #flatten the image
                #Note: a runtime error here usually means an issue with the
                #offsets (clip to image causes a runtime error if all the
                # layers are out of bounds)
                clip_to_image = 1
                flattened_layer = pdb.gimp_image_merge_visible_layers(temp_image, clip_to_image)
 
                save_png(temp_image, flattened_layer, root_directory, character_name, pose_name, category_name, index)
 
                pdb.gimp_image_delete(temp_image)
            
 
    return
 
register("export_character",
        "Exports to a collection of png images, organized by layer folders.",
        "Exports to a collection of png images, organized by layer folders.",
        "hedberggames",
        "hedberggames",
        "2019",
        "Export Character",
        "RGB*, GRAY*",
        [(PF_IMAGE, "image", "Input image", None),
            (PF_DRAWABLE, "drawable", "Input drawable", None),
            (PF_DIRNAME, "root_directory", "Output directory", "/tmp")],
        [],
        plugin_main,
        menu="<Image>/Image")
 
main()

< Previous Next >