Hi -- I'm trying to do some simple work on a 0.5GP image in GIMP 2.9.5 (on Linux). (Image is ~23k by 22k pixels, 16bit RGBA).

I can open it fine, and it currently has two partial-canvas layers that are cross faded into each other (I'm manually stitching two slices of a big hugin panorama.)

The RAM is just about maxed -- I have 16GB, and it's enough to do some simple things to the image but I can't do much else.

I need to flatten the image into one layer so I can have a hope of doing further processing on it, but just having the image open puts it too close to the RAM limit to make that possible: I can't flatten the image, I can't export as PNG, etc. Whatever I try, the RAM soon maxes out and the system grinds to a halt, necessitating a hard power cycle.

I tried using 'convert' to convert the .xcf to PNG; no luck, as it doesn't work with 2.9.X XCF files. Maybe there is another route along these lines? (For some reason GIMP won't let me save as an older-version XCF, saying that the file is using incompatible features; not sure what: it's just two layers, one with a layer mask. The layers aren't full canvas size, maybe that's it?)

Any tricks I might be able to pull? Splitting out individual channels? Managing or virtualizing memory somehow? (I notice that my swap is never used -- not sure if it's misconfigured or if it's just not possible to use it in this case for whatever reason...) I don't care if the export procedure is slow, I just need to do it once to get the image down to a manageable size in RAM... maybe I could whip up a VM and give it tons of disk-backed virtual memory?

I tried all this on Windows, too (using a Partha GIMP) -- it actually exported something, but it was mostly black with bits of corrupt tiles here and there. Then I tried again with a larger tile cache and it crashed the OS and corrupted my bios somehow... yikes.

Unfortunately I can't afford more RAM.

Thanks for any ideas!

-c

I don't think this will solve the problem you described, but I have found that clearing the undo history after every operation done on open files frees a lot of memory in cases where there is /almost/ too much data for the system to cope with.

Thanks, great tip -- As you say, I think it might not help here because I'm opening the file and immediately trying to export (i.e. no undo history) but that's great to keep in mind for the future.

Another possibility: Crop the images you are trying to merge down to just the bits that overlap, process those, then add the cropped parts back in.

Ah! Yes! I don't think I can combine them in gimp (due to RAM), and I don't think I can even crop the image now (same RAM overflow issue) but I bet I can crop to the overlap areas, do the overlap, export that, and then combine the three sections (top, overlap, and bottom) with the "montage" command. Thanks for the insight!

@Ken & @Liam Re: swap -- I did try this with a 16GB swap enabled and it never seemed to swap; but I'll try it with overcommit enabled as Liam suggested (you implied that that will trigger GIMP to fork the PNG export?). Currently, when it maxes out on RAM, everything slows to a crawl; as in, moving the mouse causes the mouse to update its position once every few seconds; takes minutes just to move the mouse to the corner of the screen, etc. GUI interaction (e.g. trying to close a window or switch to a v-term) doesn't seem possible, no matter how long I wait. In other words, the system is brought to its knees. Is that what heavy swapping can look like, and what I should expect even if overcommit is enabled? I just want to know when I should let it go overnight and expect that it's slowly working through the file, and when it's actually just spinning wheels and not getting anything done...

I will also play with tile cache size and so forth (it has been at 8GB; undo history was also at 8GB, but again, I was just opening and then exporting...)

@Rick:

Just out of curiosity, why such a huge file? What do you intend to do with the finished product if you could get your computer to work (if you don't mind me asking)?

Very fair question. :-) I just wanted to make a zoomable 'gigapixel' image for the heck of it. No true demand that it be so huge. Maybe I'll print it large-scale some day if it turns out well. For now I'm mainly just honing my skills on the techniques and learning what my computer can handle. (And the only reason it's sliced in two parts is that hugin can't write out the image in one piece without running out of RAM.)

Thanks, everyone, for the help, -c

Hey, just wanted to follow up on the gigapixel editing...

I tried the overcommit_memory thing, clearing undo history, making a big swap, changing tile size, etc. Nothing seemed to help with my particular image on my particular system, but I appreciate the tips. Guess I just need a lot more memory. :-)

I wasn't able to do any editing or adjustment on the image, since most tools would crash out the program, but I was able to stitch it all together in a mostly satisfying way by piecing it out and using montage. I then put it online using OpenSeadragon.

I think for now I will not work with such large images. It's just too painful. :-) But it's nice to have some tools to work with medium size stuff.

Thanks for the help!

-Casey

P.S. - Here's the pic, if anyone's interested: http://caseyconnor.org/pub/image/osd/nfalls/

Slightly edited (small) version is here .