RSS/Atom feed Twitter
Site is read-only, email is disabled

Suggested in memory image data structure.

This discussion is connected to the gimp-developer-list.gnome.org mailing list which is provided by the GIMP developers and not related to gimpusers.com.

This is a read-only list on gimpusers.com so this discussion thread is read-only, too.

14 of 14 messages available
Toggle history

Please log in to manage your subscriptions.

Suggested in memory image data structure. Rogier Wolff 06 Jul 23:34
  Suggested in memory image data structure. Liam R E Quin 06 Jul 23:51
   Suggested in memory image data structure. Rogier Wolff 07 Jul 06:41
    Suggested in memory image data structure. Stephen Norris 07 Jul 06:44
    Suggested in memory image data structure. Bill Skaggs 07 Jul 06:54
     Suggested in memory image data structure. Rogier Wolff 07 Jul 08:00
      Suggested in memory image data structure. Sven Neumann 07 Jul 08:45
       Suggested in memory image data structure. Rogier Wolff 07 Jul 08:59
        Suggested in memory image data structure. Sven Neumann 07 Jul 09:14
     Suggested in memory image data structure. Rogier Wolff 07 Jul 08:45
      Suggested in memory image data structure. Sven Neumann 07 Jul 09:05
  Suggested in memory image data structure. The Peach 06 Jul 23:54
  Suggested in memory image data structure. Martin Nordholts 07 Jul 05:53
   Suggested in memory image data structure. Martin Nordholts 07 Jul 05:58
Rogier Wolff
2008-07-06 23:34:40 UTC (over 16 years ago)

Suggested in memory image data structure.

Hi,

I'm trying to touch up the result of a panorama shot I took. This takes quite long, mostly because the image is too big to fit into memory.

I would like to suggest a way of storing the image data in memory in such a way that it doesn't take so much memory. Or at least, it doesn't require so much memory to be presently paged in.

Currently images are commonly stored as an array of rows of pixels.

With an image width of more than 1400 pixels, if I need to access a vertical line of pixels, this will touch a different machine memory page for every single line. Thus to access say 100 pixels, 400 kbytes of core memory needs to be touched (from swap if memory is tight).

First, a page on a machine is usually about 4k. So the basic units might be 4k. On the other hand, having the units be say 12k doesn't cost much in terms of IO time, but makes things in binary fit a lot easier.

So, instead of storing images in an array of rows of pixels, we can store an image as an array of blocks. If a block is 64x64 pixels, with 24bit data, the block takes 12k of memory. Now, touching 100 vertical pixels requires only two blocks of 12k to be in memory. This is a big improvement.

Next, if we want to display an image that is say 5000x12000, my screen has only one or two megapixels. So scanning all 60 megapixels to display this is a waste of CPU time and memory.

To improve this, several scaled down versions (say by a factor of two each time) can be kept. Thus of my 5000x12000 image, a 2500x6000, 1250x3000, and a 625x1500 version should be kept. To display up to 625 vertical pixels of my full image, should use the last version. This means that less than 3 megabytes of memory needs to be touched to display my image. On a reasonable computer this is unnoticably fast.

As each scaled down version uses 4x less memory, they cost little to build and to maintain.

Oh. If I zoom into a 100x100 area of my big (5000x12000) image, the old way of storing images would require 100 or 200 pages of image memory to be present in core. My suggestion only 2 or 4.

If all plugins and internal image manipulation functions are "aware" of the old way of storing images, then I'm afraid that this is a major programming undertaking. If however, everybody accesses the image through accessfunctions (getpixel, putpixel?) then things are a whole lot easier.

Once you start manipulating images, of course, my suggestion makes things a bit slower. However, I seem to be waiting a lot for my drive instead of "doing what humans do best", so I think this would mean a speedup on the whole. But of course, if things easily fit into memory it might slow things down a bit. So possibly a runtime switch between the storage methods is desirable.

Roger.

Liam R E Quin
2008-07-06 23:51:24 UTC (over 16 years ago)

Suggested in memory image data structure.

On Sun, 2008-07-06 at 23:34 +0200, Rogier Wolff wrote:

I'm trying to touch up the result of a panorama shot I took. This takes quite long, mostly because the image is too big to fit into memory.

Working with large images does require a little extra care.

It helps to set the gimp tile cache (in preferences) to a fairly large value. E.g. I use a gigabyte or more. But when I bought extra RAM for my desktop, it made a huge difference.

Currently images are commonly stored as an array of rows of pixels.

GIMP does in fact use image tiles.

Best,

Liam

The Peach
2008-07-06 23:54:00 UTC (over 16 years ago)

Suggested in memory image data structure.

On Sun, 6 Jul 2008 23:34:40 +0200 Rogier wrote:

I'm trying to touch up the result of a panorama shot I took. This takes quite long, mostly because the image is too big to fit into memory.

nice one Roger, I did have great problems on the same issue with sat maps with an older version of gimp (2.2 if I remember correctly), since then I gave up. I never inspected that deep as you did, so I don't even know if the svn version has any improvements (but I noticed a speedup in applying an image manipulation filter).

Martin Nordholts
2008-07-07 05:53:19 UTC (over 16 years ago)

Suggested in memory image data structure.

Rogier Wolff wrote:

Hi,

Currently images are commonly stored as an array of rows of pixels.

Hi

No, they are stored in a tile based structure, check out app/base/tile-manager.h

[Description of mipmapping and a wish to implement it in GIMP]

The projection is mipmapped since GIMP 2.4, check out app/base/tile-pyramid.h

Best regards, Martin Nordholts

Martin Nordholts
2008-07-07 05:58:57 UTC (over 16 years ago)

Suggested in memory image data structure.

Rogier Wolff wrote:

Hi,

Currently images are commonly stored as an array of rows of pixels.

I just realized I missed the word 'commonly', so I would like to adjust my answer to:

Yes but not in any serious image editing application like GIMP. In GIMP images and layers etc are stored in tile based structures.

- Martin

Rogier Wolff
2008-07-07 06:41:05 UTC (over 16 years ago)

Suggested in memory image data structure.

On Sun, Jul 06, 2008 at 05:51:24PM -0400, Liam R E Quin wrote:

Currently images are commonly stored as an array of rows of pixels.

GIMP does in fact use image tiles.

But then why, on a machine with 700Mb RAM, does EVERYTHING take so very long? My image is "only 50Mpixel".

Roger.

Stephen Norris
2008-07-07 06:44:53 UTC (over 16 years ago)

Suggested in memory image data structure.

On Mon, 2008-07-07 at 06:41 +0200, Rogier Wolff wrote:

On Sun, Jul 06, 2008 at 05:51:24PM -0400, Liam R E Quin wrote:

Currently images are commonly stored as an array of rows of pixels.

GIMP does in fact use image tiles.

But then why, on a machine with 700Mb RAM, does EVERYTHING take so very long? My image is "only 50Mpixel".

Roger.

What size is your tile cache?

Stephen

Bill Skaggs
2008-07-07 06:54:50 UTC (over 16 years ago)

Suggested in memory image data structure.

On Sun, Jul 6, 2008 at 9:41 PM, Rogier Wolff wrote:

But then why, on a machine with 700Mb RAM, does EVERYTHING take so very long? My image is "only 50Mpixel".

The minimum memory usage for a 50 Mpixel RGBA image is 400 MB, and that's if you only have a single layer. So given the amount of memory needed for other purposes in a modern operating system, it's likely that gimp is swapping very frequently. 700 MB is low to be running Gimp in the first place, and much more so if you're going to work with large images. Gimp is going in directions that
should eventually make it work better in this kind of scenario, but for now and the
near future, the best advice is to get more RAM.

-- Bill

Rogier Wolff
2008-07-07 08:00:18 UTC (over 16 years ago)

Suggested in memory image data structure.

On Sun, Jul 06, 2008 at 09:54:50PM -0700, Bill Skaggs wrote:

On Sun, Jul 6, 2008 at 9:41 PM, Rogier Wolff wrote:

But then why, on a machine with 700Mb RAM, does EVERYTHING take so very long? My image is "only 50Mpixel".

The minimum memory usage for a 50 Mpixel RGBA image is 400 MB, and

Fine. So you're saying that gimp wastes a factor of two of memory. Fine! probably neccesary for something. No problem with that.

that's if you only have a single layer. So given the amount of memory needed for other purposes in a modern operating system, it's likely that gimp is swapping very frequently. 700 MB is low to be running Gimp in the first place, and much more so if you're going to work with large images. Gimp is going in directions that should eventually make it work better in this kind of scenario, but for now and the near future, the best advice is to get more RAM.

So.... What I started out saying is that my brain is not prepared to look at those 50 megapixels at one time. My screen is only 1.3 megapixels.

So, I understand that loading the big image takes some time and swapping. Then, moving the viewable window to the area I want to change may cause some swapping. But from then on, things should be working snappy. However everything I do takes lots of time.

Indeed for the short run, I could buy more memory. But this computer does not earn me money doing GIMP. It's fine for all the other stuff that I bought it for.

If I want I can probably use the machine with 8G at work for this.

What apparently happens in the GIMP developper community is that everyone buys loads of extra memory, and then the developpers don't feel the pain of GIMP's inefficient use of memory.

So, is the tiling very inefficient? Do the parameters differ enormously from what I suggested? Does gimp reference the whole image with every thing I do? If so, that would explain why I seem to need the whole image in memory, while I'm only working on a small part.

Roger.

Sven Neumann
2008-07-07 08:45:04 UTC (over 16 years ago)

Suggested in memory image data structure.

Hi,

On Mon, 2008-07-07 at 08:00 +0200, Rogier Wolff wrote:

The minimum memory usage for a 50 Mpixel RGBA image is 400 MB, and

Fine. So you're saying that gimp wastes a factor of two of memory. Fine! probably neccesary for something. No problem with that.

GIMP needs to allocate the memory to hold your image data and it needs to keep the projection (that's what you see with all layers merged) in memory. On top of that there's some extra memory for the selection mask. We could save the memory for the projection in the special case of a single-layered image with the layer mode set to Normal and 100% opacity. There's even code to share the tiles in this special case but it had to be disabled because it is broken. Perhaps someone can fix it one day, that would be a nice improvement.

So, I understand that loading the big image takes some time and swapping. Then, moving the viewable window to the area I want to change may cause some swapping. But from then on, things should be working snappy. However everything I do takes lots of time.

All operations are applied on the full image data. We use a mipmap for the projection which helps a lot for panning through zoomed-out views of large images. But as soon as you perform an operation on the image, the full-scale image data needs to be touched. It's non-trivial to avoid this. But it is on the long-term plan to implement working on the scaled-down version. We hope to be able to address this when GIMP makes full use of GEGL. If you want to help, I suggest that you contribute to the GEGL development.

Sven

Rogier Wolff
2008-07-07 08:45:43 UTC (over 16 years ago)

Suggested in memory image data structure.

On Sun, Jul 06, 2008 at 09:54:50PM -0700, Bill Skaggs wrote:

On Sun, Jul 6, 2008 at 9:41 PM, Rogier Wolff wrote:

But then why, on a machine with 700Mb RAM, does EVERYTHING take so very long? My image is "only 50Mpixel".

The minimum memory usage for a 50 Mpixel RGBA image is 400 MB, and that's if you only have a single layer. So given the amount of memory needed for other purposes in a modern operating system, it's likely that gimp is swapping very frequently. 700 MB is low to be running Gimp in the first place, and much more so if you're going to work with large images. Gimp is going in directions that should eventually make it work better in this kind of scenario, but for now and the near future, the best advice is to get more RAM. -- Bill

OK. I've scaled my image down by a factor of three on both axes, so I now have a workable amount of pixels/RAM.

I forgot to photograph part of the sky. So I need to stitch some blue to the image I have.

I've used the color picker, and the fill gradient tool to create some sky pixels that look almost ok.

If I'm zoomed in a bit, and somehow the whole image is selected, if I try to create a selection, I always "move the selection" instead of creating a new selection. I have to zoom out to be able to "invite" part of "outside the image" into view, so that I can click outside the image to start a new selection. There is probably something I'm missing here.

Next, the line between the photographed sky and the gradient is sometimes visible. So I want to smudge this a bit.

First I thought there was a tool that would even out the pixels under the cursor, to do this.

However the smudge tool seems to work differently. I have to drag to create a source and then let go where I want to drag those pixels. Fine.

So click on one side of the "edge", drag to the edge, let go. (with the biggest fuzzy cicle tool). After doing that two or three times, my view suddenly shifts, the middle of the image is in view, and apparently I've smudged a whole horizontal line. Part of my picture is completely smudged over the whole width of the image. It happens every three or four times I try to smudge that edge away.

Is this a bug? Or am I missing something again?

Roger.

Rogier Wolff
2008-07-07 08:59:32 UTC (over 16 years ago)

Suggested in memory image data structure.

On Mon, Jul 07, 2008 at 08:45:04AM +0200, Sven Neumann wrote:

But as soon as you perform an operation on the image, the full-scale image data needs to be touched.

Agreed. This is much easier to program. However, when I'm zoomed in to say a 4 megapixel part of the image, and the selection is in view, then if I do something a max of 4 Mpixels, of 4 bytes * 2 need to be touched. I have plenty of RAM for that.

It's non-trivial to avoid this. But it is on the long-term plan to implement working on the scaled-down version. We hope to be able to address this when GIMP makes full use of GEGL. If you want to help, I suggest that you contribute to the GEGL development.

So, when on my original image, I select part, and use the "fill with gradient" tool, I first get a progress bar that moves in about 30 seconds from 0 to 100%. Then I have to wait about 30 seconds for the swapping to stop. The fill could be done on the small scale cache, allowing the GUI to be updated and then a background process can update the actual image, without me having to wait for it to complete. Is that similar to what GEGL intends to do?

Roger.

Sven Neumann
2008-07-07 09:05:22 UTC (over 16 years ago)

Suggested in memory image data structure.

Hi,

it would have helped a lot if you had changed the subject of your mail, since apparently you have changed the topic completely. Also your mail should have better been send to the gimp-user list. Please do that next time you have questions on using GIMP.

On Mon, 2008-07-07 at 08:45 +0200, Rogier Wolff wrote:

If I'm zoomed in a bit, and somehow the whole image is selected, if I try to create a selection, I always "move the selection" instead of creating a new selection. I have to zoom out to be able to "invite" part of "outside the image" into view, so that I can click outside the image to start a new selection. There is probably something I'm missing here.

Have a look at the Layers dialog. I guess you have a Floating Selection, probably as a result from using the Scale tool.

Next, the line between the photographed sky and the gradient is sometimes visible. So I want to smudge this a bit.

First I thought there was a tool that would even out the pixels under the cursor, to do this.

However the smudge tool seems to work differently. I have to drag to create a source and then let go where I want to drag those pixels. Fine.

Sorry, but this is not how the Smudge tool works.

So click on one side of the "edge", drag to the edge, let go. (with the biggest fuzzy cicle tool). After doing that two or three times, my view suddenly shifts, the middle of the image is in view, and apparently I've smudged a whole horizontal line. Part of my picture is completely smudged over the whole width of the image. It happens every three or four times I try to smudge that edge away.

Is this a bug? Or am I missing something again?

Are you sure you are using GIMP 2.4? We changed the paint tool behavior to not auto-scroll when the brush tip hits the edge of the view. What you are describing sounds like the behavior of GIMP 2.2.

Sven

Sven Neumann
2008-07-07 09:14:55 UTC (over 16 years ago)

Suggested in memory image data structure.

Hi,

On Mon, 2008-07-07 at 08:59 +0200, Rogier Wolff wrote:

So, when on my original image, I select part, and use the "fill with gradient" tool, I first get a progress bar that moves in about 30 seconds from 0 to 100%. Then I have to wait about 30 seconds for the swapping to stop. The fill could be done on the small scale cache, allowing the GUI to be updated and then a background process can update the actual image, without me having to wait for it to complete. Is that similar to what GEGL intends to do?

Yes.

Sven