I'm using python-fu to script a sequence of animation frames in Partha's Gimp 2.7.5 for win 32 (Vista) (Thanks!) It's expected to be a relatively long sequence when done, probably several thousand frames, but the animation is pretty simple, very algorithmic, just a series of layer pans, zooms, fades, a little text, and drawing and stroking a growing path, over and over -- a glorified slideshow. I don't mind if it takes overnight to render. Anyway, I got the first version to load, and start writing frames out as .pngs (targeting a later assembly process using ffmeg). But it crashes gimp entirely around frame 175 or so (it starts out quick, and slows noticeably near the end). Turns out it leaks memory pretty badly, and it dies right about when task manager shows the memory use in the gimp process just passing 2GB. Signed 32-bit integer pointers, I guess.

Image is 1280x720 RGB, largest layers are 4800x3400 or so. Maybe 10 layers at most at any one time.

The original intent was to manipulate, copy, and create layers as needed from within a single image, make a new layer from visible, write that out, then remove old layers and start over for the next frame. I can get individual commands and short sequences to appear to have the desired effect when typed into the console (after a lot of reviewing source on github -- this is a pretty cranky API; don't get me started on the *_insert_* methods "parent" parameters!). Thinking that I was missing something about properly managing gimp or python, I tried isolating a much simpler set of commands, basically copying a reference layer from one image into a new displayed image, then destroying that new image and starting over. Just entering them into the console repeatedly I can see the same gimp process memory usage jump up each iteration, at first accumulating just under 1MB on each step, then apparently growing steadily larger. I can see from the layer toolbox that my layer count doesn't seem to grow larger than intended, though I suppose there could be layers not properly inserted or deleted that might not show in the toolbox.

I'm stumped on how to use python gimp to create this animation (and I came to gimp because synfig couldn't handle all my over-one hundred source images). I haven't found any example scripts that deal with this scale yet. Do the experts on this board have any corrections to what I'm doing wrong, or any tips or tricks that I might apply to manage this issue? For example, if it's not my script, are certain methodologies more leaky than others (such as opacity changes vs visibility changes, copy-pasting visible to a re-used layer vs creating a new layer from visible, layer copies within an image vs across images, layer creation/destruction vs image creation/destruction)?

Here's the simple command sequence that demonstrates the "leak". I start with an image containing an RGB layer 4800x3400 or so at layer[1] (there's a transparent top layer). Open task manager and watch memory use in the gimp process. Open the python-fu console, initialize gp =gimp.pdb and src =gimp.image_list()[0], then iterate the following:

img =gp.gimp_image_new (1280, 720, 0) dsp =gp.gimp_display_new (img)
lyr =gp.gimp_layer_new_from_drawable (src.layers[1], img) img.insert_layer (lyr, position =0)
gp.gimp_display_delete (dsp)

I see memory use jump after each iteration. Any help is appreciated.

twv@

And now, the rest of the story. I've completed this project, and everything worked out. Turns out that my perceived "memory leak" was nothing more than normal growth of the undo stack; disable undos, and the problem goes completely away. As for gimp_image_delete(), it occurs as part of deleting the last display of an image, just as documented, so it wasn't necessary (or possible) to call it explicitly.

In fact, using the Gimp for this project worked out quite well (based in part on the ovation the resulting video received). Just over 7100 frames representing 24 frames-per-second HD-720, rendered in a touch over 8 hours on an older quad-core Intel. Beyond what I mentioned above, I called layer copies and saturation and brightness and blurs for drop shadows, rotation transforms, bump mapping, feathered dissolves, and loaded around 50 .png images with transparency into layers over the course of the animation. The only problems that remained were with the API. I guess there's a lot of history there, but inconsistencies in it make it a chore to learn. And when you call a function potentially thousands of times, you can't tolerate the pointlessly repeating "deprecated" dialog attached to many functions -- I got it the first time: it's deprecated, but it's still there _now_.

Some of my API issues are just gripes. A very few of my layers were the size of my image; almost all were either larger or smaller. It took me an embarrassing amount of time to understand the differences between coordinate systems and why they had to be so -- x, y usually reference the image's upper left (in my mind, the image isn't a thing the way layers are things; the image merely acts like a window view on your layers), except when transforming a layer, in which case x, y reference the layer's upper left. At least x and y always grew right and down. But of the functions I used that involved angles, each one seemed to zero or rotate in a different direction, and both degrees and radians appear for different functions in the API. I spent some effort trying to be careful about using int (round (x)) for integer vs float parameters, but had hoped the API would secretly do this for me, in either direction, and it might, but the docs don't hint at this. I tripped a number of times over opacity going from 0. to 100., though alpha goes only from 0. to 1. (the latter makes more sense to me for both).

But a few things might be broken. I haven't found a way to specify a dummy "parent" group, so had to code image.insert_layer (layer, position =0); but gimp_image_insert_vectors() is uncallable as far as I can tell, so I had to pre-create a named vector object and keep re-using it. Most functions pass around objects instead of IDs, except gimp_vectors_remove_stroke(), which actually takes an ID (I noticed some other function return an ID; I can't recall which one it was, but I know it's there).

Besides a little grumbling, that I could (eventually) get the Gimp to do as many things as it did was amazing, and allowed me to create a very unique presentation which could not be confused with something out of Powerpoint, and made a hack like me look like a pro -- if only to people who don't use the Gimp.

-------- Original Message -------- Subject: Python FU Scripting -- Memory Leak? or How to release python gimp objects properly to manage memory usage? Date: Wed, 15 Feb 2012 23:42:31 -0500 From: Tom Vrankar
To: gimp-developer-list@gnome.org

I'm using python-fu to script a sequence of animation frames in Partha's Gimp 2.7.5 for win 32 (Vista) (Thanks!) It's expected to be a relatively long sequence when done, probably several thousand frames, but the animation is pretty simple, very algorithmic, just a series of layer pans, zooms, fades, a little text, and drawing and stroking a growing path, over and over -- a glorified slideshow. I don't mind if it takes overnight to render. Anyway, I got the first version to load, and start writing frames out as .pngs (targeting a later assembly process using ffmeg). But it crashes gimp entirely around frame 175 or so (it starts out quick, and slows noticeably near the end). Turns out it leaks memory pretty badly, and it dies right about when task manager shows the memory use in the gimp process just passing 2GB. Signed 32-bit integer pointers, I guess.

Image is 1280x720 RGB, largest layers are 4800x3400 or so. Maybe 10 layers at most at any one time.

The original intent was to manipulate, copy, and create layers as needed from within a single image, make a new layer from visible, write that out, then remove old layers and start over for the next frame. I can get individual commands and short sequences to appear to have the desired effect when typed into the console (after a lot of reviewing source on github -- this is a pretty cranky API; don't get me started on the *_insert_* methods "parent" parameters!). Thinking that I was missing something about properly managing gimp or python, I tried isolating a much simpler set of commands, basically copying a reference layer from one image into a new displayed image, then destroying that new image and starting over. Just entering them into the console repeatedly I can see the same gimp process memory usage jump up each iteration, at first accumulating just under 1MB on each step, then apparently growing steadily larger. I can see from the layer toolbox that my layer count doesn't seem to grow larger than intended, though I suppose there could be layers not properly inserted or deleted that might not show in the toolbox.

I'm stumped on how to use python gimp to create this animation (and I came to gimp because synfig couldn't handle all my over-one hundred source images). I haven't found any example scripts that deal with this scale yet. Do the experts on this board have any corrections to what I'm doing wrong, or any tips or tricks that I might apply to manage this issue? For example, if it's not my script, are certain methodologies more leaky than others (such as opacity changes vs visibility changes, copy-pasting visible to a re-used layer vs creating a new layer from visible, layer copies within an image vs across images, layer creation/destruction vs image creation/destruction)?

Here's the simple command sequence that demonstrates the "leak". I start with an image containing an RGB layer 4800x3400 or so at layer[1] (there's a transparent top layer). Open task manager and watch memory use in the gimp process. Open the python-fu console, initialize gp =gimp.pdb and src =gimp.image_list()[0], then iterate the following:

img =gp.gimp_image_new (1280, 720, 0) dsp =gp.gimp_display_new (img)
lyr =gp.gimp_layer_new_from_drawable (src.layers[1], img) img.insert_layer (lyr, position =0)
gp.gimp_display_delete (dsp)

I see memory use jump after each iteration. Any help is appreciated.

twv@