My working assumption is that GEGL should be as consistent as possible with GIMP.

gegl/gegl/buffer/gegl-sampler-cubic.c, for example, contains the following code:

----------------------------------------------------------------------

... gegl_sampler_cubic_get (GeglSampler *self, gdouble x, gdouble y, void *output) {
...
gint dx,dy;
...
dx = (gint) x;
dy = (gint) y;
sampler_bptr = gegl_sampler_get_ptr (self, dx, dy); ...

----------------------------------------------------------------------

Ignoring the issue of the asymetric behavior of int casting when one crosses 0 vs the "right boundary" (see gegl-sampler-linear.c or gegl-sampler-sharp.c for a fix) I want to double check that pixels are understood to be centered at points with integer coordinates (for example (0,0) is a possible pixel location, but not (.5,.5) throughout GEGL and GIMP (excluding plug-ins, of course). If there are exceptions, are they relevant to resamplers and abyss policies?

If there is a discrepancy between GEGL and GIMP, I will have Adam and Eric align GEGL with GIMP as much as possible.

Nicolas Robidoux Universite Laurentienne.

Martin Nordholts writes:
> Nicolas Robidoux wrote:
> > My working assumption is that GEGL should be as consistent as possible > > with GIMP.
>
> That seems weird to me, why not go for the "best" resampling approach > and ignore how GIMP does it?

Well, it certainly makes my life easier if I don't have to try to be aligned with how GIMP generally does things.

However, I am a bit concerned that users may expect a certain GENERAL behavior, and be upset if they don't get it. So, I'd like feedback regarding: "Coke Classic or New Coke"?

Alignment is more or less independent of "best resampling methods" (although currently all samplers implemented are "point" samplers and consequently are more friendly to the "center-based physical coordinate" approach).

Every resampling method can pretty much be used with every "image alignment convention." Some resampling methods may be a bit trickier to implement with one convention than another or may lose some desirable properties (most notably near the boundary, although this also depends on the abyss policy). But with a bit of tweaking, reasonable results can be obtained irregardless of the "image alignment convention" provided one makes sure that things are consistent.

To give an example, (classic) ImageMagick uses the usual "exact area transformation" convention (corner-based physical coordinates, which are friendly to box filtering) even though most of its many resamplers are point resamplers (if image is N pixel wide and its geometrical width is understood to be from 0 to 1, then the first pixel center is understood to be located at position 1/(2*N)), but I routinely compile it with minor source code changes so that when scaling an image it uses the usual "point transformation" convention (center-based physical coordinates, which are friendly to nearest neighbour) (under the above conditions, the horizontal position of the center of the first pixel is 0).

In order to avoid undesirable/unexpected shifts, it is important to make sure that the convention used inside the sampler is the same as the one used by, say, a scaling operator. These issues are addressed, for example, in

http://www.vtk.org/Bug/view.php?id=6558

From the sampler code already in GEGL, I assumed that the GEGL

convention is

"center-based physical coordinates:" image starts at 0, and the first pixel is centered at 0; in addition, assuming unit inter-pixel distance, the last pixel is located at N-1 if there are N pixels in total in the horizontal direction.

The most common alternative being:

"corner-based physical coordinates," which is commonly expressed in two different ways:

version 1: image starts at -.5, first pixel is centered at 0, the last pixel is located at N-1 and the image ends at N-.5.

version 2: image starts at 0, first pixel is centered at .5, the last pixel is located at N-.5 and the image ends at N.

So, my questions are:

A) Is "center-based physical coordinates" indeed the intended convention for GEGL?

B) For GIMP?

C) If the intended conventions are different, should I align GEGL's convention with GIMP's? Or leave it be?

----------------------------------------------------------------------

Apologies for the longish email, but I'd like to set this issue to rest.

Nicolas Robidoux
Laurentian University

Note 1: The first version I gave for "corner-based physical coordinates" makes things the same for the resampler, but different for an operation which scales the image on the basis of physical size instead of pixel size. The second version affects both the resampler and the scale operation.

Note 2: One can emulate one convention with the other.

Nicolas Robidoux Universite Laurentienne

This Summer, I mentor to GSoC projects, one which concerns general purpose resampling (more precicely, resampling tasks which are not dominated by downsampling), and one which is tailored for resampling tasks in which good downsampling behavior is more important.

Let me describe one aspect of how I intend to set each of them up which some people may find unusual. The point is to generate discussion now as opposed as "after the coding is done."

----------- Motivation:
-----------

When downsampling, the best image alignment convention is "corner-based physical coordinates" (that is: image extent goes from 0 to N, first pixel center is at .5; another version of the same convention: image extent from -.5 to N-.5, first pixel center is at 0). For example, exact area box filtering, the simplest decent downsampler, works more naturally with this convention.

When upsampling, the best image alignment convention is "center-based physical coordinates" (that is: image extent goes from 0 to N-1, first pixel center is at 0). For example, bilinear, the simplest decent upsampler, works more naturally with this convention.

The convention is used inside the sampler to determine the "physical" location of the sampling point relative to the pixel centers of the input image.

----------------------------------------------------- What I plan to have Adam Turcotte and Eric Daoust do: -----------------------------------------------------

The "tuned for upsampling" method will use the "corner-based coordinates" convention (Adam), and the "tuned for downsampling" method will use the "center-based coordinates" convention (Eric).

A side effect of this choice is that if one downsamples with one and upsamples with the other, the resulting image will not be aligned with the original. (Of course, if one uses the same resampler for both, the results will be aligned, unless the calling routine (e.g., scale) messes things up, which I think it won't.)

In my opinion, this is a small price to pay.

Nicolas Robidoux Universite Laurentienne

A more radical opinion regarding image size conventions:

Although this is not part of the GSoCs, I would also suggest that the image scaling operation use the convention most suited for the task (irregardless of the sampler used), possibly differently depending on the direction.

Unless I hear protests, I may just have Adam or Eric implement things this way.

Nicolas Robidoux
Laurentian University

Hello Martin:

Martin Nordholts writes: > ...
> Is it a lot of work to change convention in a resampler afterwards? > Would it be possible to parameterize this so you could even toggle > pixel-center convention during runtime?

I actually came to more or less the same conclusion.

To summarize: Ideally, when downsampling, one should use the pixel coordinate convention usually associated with area sampling (e.g., corner-based coordinates). When upsampling, the usual point sampling convention (center-based coordinates). In addition, the conventions should ideally be consistent between the operation which calls the sampler (e.g., scaling operation, or rotation operation, or warping operation) and the sampler itself (it is possible to mix and match without large ill effects but it is preferable to be consistent).

What this suggests to me is one of the following two possibilities:

Option 1: Samplers, and the operations which call samplers, have a toggle which specifies which convention to use. Given that the amount of different code in the two cases is very small, this would seem to me to be the best solution.

Option 2: Alternately, there could be two versions of the relevant routines (which, obviously, would have mostly identical code), one for each convention. Then, the convention would be determined by which version of scale (say) is called.

Option 3: I am wondering whether there is machinery already in gegl. I haven't figured out what

/gegl/gegl/operation/gegl-operation-point-filter.c

and

/gegl/gegl/operation/gegl-operation-area-filter.c

do, but maybe they have to do with a way of toggling the desired coordinate/bounding box behavior in operations?

----------------------------------------------------------------------

Eric and/or Adam:

In the grand scheme of things, can you figure out what

/gegl/gegl/operation/gegl-operation-point-filter.c

and

/gegl/gegl/operation/gegl-operation-area-filter.c

do?

Can you email me pointers to the code that actually does scaling, rotating, and similar warping (I am pretty sure perspective is not implemented in gegl)? That is, can you email me a list of the code which calls the samplers, and start figuring out how they work? Especially, what kind of image convention they use (answer this last questionmay be easier for Adam than for Eric)?

Be aware that my c++ (as opposed to my c) is quite weak. I need hand-holding here.

Thanks,

Nicolas Robidoux Universite Laurentienne

Hello Øyvind:

I wrote:

Be aware that my c++ (as opposed to my c) is quite weak. I need hand-holding here.

I realized after sending that this was not "right." What I really meant is that my object oriented coding is weak in C (and C++).

Nicolas Robidoux Laurentian University

Øyvind Kolås writes:
> ...
> I agree that the convention should be that pixel sampling sites are at > integer coordinates. I also feel it would be natural to let the extent > of a pixel be half way towards the neighbour pixel sites, this however > has the issue of not matching the display GIMP provides when zoomed > into the image.
> ...
> GEGL might in the future also offer other resamplers for blitting / > display purposes, the best would of course be if all resamplers could > be plugged in and interchangeably used for different purposes, at the > moment the resampling that happens in gegl_node_blit does not use the > resamplers to do its job for the general cases it probably shouldn't > either since it needs to be as fast as possible.

Thank you! I finally figured it out:

Image size conventions are actually decoupled from the pixel-coordinates convention. (Although they often are linked, they don't need to be.)

The resamplers can all use one convention (center-based physical coordinates, as you suggest, and as I prefer). But the operations which call resamplers can have an optional toggle, with defaults dependent on the called resampler.
image convention).

Nicolas Robidoux Universite Laurentienne