Hi,

On Tue, 2007-02-20 at 12:54 +0100, Helmut Jarausch wrote:

I'd like to help to keep the healing tool alive.

That would be very much appreciated since otherwise we would likely disable the heal tool for the 2.4 release.

Finally I don't understand what the healing tool has to do with a 'brush' (as it is currently implemented)

Painting with a brush is just a way to select an area. And a very nice and intuitive one also. The problem with the current implementation is however that it tries to heal while you are painting. This doesn't quite work. I think it would be better if, while you are painting, the tool would work like the clone tool. Then, when the mouse is released, the healing algorithm should be applied on the area selected by the paint stroke.

A bug has been reported when source and destination areas are of different depth. Since, IHMO, the healing tool doesn't make sense in that case, I'd suggest to simply abort with an informative error message.

Why does it not make sense to heal a region in an image using a texture from another image, or from another layer in the same image? The clone tool supports this nicely, so it seems to make some sense that the heal tool supports it as well.

The current implementation needs an alpha channel but I don't understand why?

The implementation needs to be able to deal with an alpha channel. I don't know the current code very well, but I think that for simplicity everything is promoted to RGBA.

Sven

I had our evaluation notes dug out to see what we have to say about this.

I'd like to help to keep the healing tool alive.

That would be very much appreciated since otherwise we would likely disable the heal tool for the 2.4 release.

yep, we can see it does not work right now.

Finally I don't understand what the healing tool has to do with a 'brush' (as it is currently implemented)

Painting with a brush is just a way to select an area. And a very nice and intuitive one also.

we agree.

The problem with the current implementation is however that it tries to heal while you are painting. This doesn't quite
work. I think it would be better if, while you are painting, the tool would work like the clone tool. Then, when the mouse is released, the healing algorithm should be applied on the area selected by the paint stroke.

hmmm, how would you know you had done enough healing?

A bug has been reported when source and destination areas are of different depth. Since, IHMO, the healing tool doesn't make sense in that case, I'd suggest to simply abort with an informative error message.

Why does it not make sense to heal a region in an image using a texture
from another image, or from another layer in the same image?

we saw in the workspace observation users who have collections of skin texture to do cloning with. Since healing is an refined extension of that, I can see the use of this.

--ps

principal user interaction architect man + machine interface works

http://mmiworks.net/blog : on interaction architecture

Hi,

On Tue, 2007-02-20 at 20:55 +0100, peter sikking wrote:

I think it would be better if, while you are painting, the tool would work like the clone tool. Then, when the mouse is released, the healing algorithm should be applied on the area selected by the paint stroke.

hmmm, how would you know you had done enough healing?

Perhaps we could do the healing from an idle handler. Then the user wouldn't have to release the mouse button to see the effect but could still paint fast without having to wait for the algorithm to finish.

Sven

Helmut wrote:

Painting with a brush is just a way to select an area. And a very nice
and intuitive one also. The problem with the current implementation is
however that it tries to heal while you are painting. This doesn't quite
work. I think it would be better if, while you are painting, the tool would work like the clone tool. Then, when the mouse is released, the healing algorithm should be applied on the area selected by the paint stroke.

This has two disadvantages. First, you won't see the real effect until you're finished anyway.

this I can agree with.

Second, this might give many disjoint sets of points, which is difficult to handle in the algorithm.

whenever I have to choose between the user sweating a bit or the developer, it will be the one that starts with a d...

let's have our priorities screwed in the right way around.

There are a bunch of selection tools including the quick mask. Using these it's much easier and quicker to select a region than painting it with a brush.

boy, am I glad we went out and did workplace observation as part of the project I am running. that means I can say with confidence that the brush is the way to go. actually healing is applied extremely local, with the tiniest brush, to take out only the irregularity, and not to destroy the all important texture around this spot.

In most cases the lasso tool would do. Since the healing tool is of more 'global' nature, being forced to use a local tool like a brush seems misleading.

What could make sense would be to use a selection (one in the destination area and a translated one in the source area) first, compute the 'healed' area without pasting it into the destination area, yet. Now the brush could be optionally used to copy parts of the healed area into the destination area.

now that we got the brush part nailed, we need to find something for the user to set the source area. To me as an interaction architect the healing brush looks like a smart clone tool on steroids.

that means we can use exactly the same interaction as the clone tool.

both tools need a bit of cleaning up in the tool options I think.

set a source location, heal a tiny bit of the photo. set a new source location, heal a tiny bit of the photo. set a new source location, heal a tiny bit of the photo. set a new source location, heal a tiny bit of the photo. set a new source location, heal a tiny bit of the photo. set a new source location, heal a tiny bit of the photo. set a new source location, heal a tiny bit of the photo. set a new source location, heal a tiny bit of the photo.

looks as straightforward as it gets to me.

A bug has been reported when source and destination areas are of different depth. Since, IHMO, the healing tool doesn't make sense in that case, I'd suggest to simply abort with an informative error message.

Why does it not make sense to heal a region in an image using a texture
from another image, or from another layer in the same image? The clone
tool supports this nicely, so it seems to make some sense that the heal
tool supports it as well.

The main difference between the cloning tool and the healing tool relies
on first computing the quotient of the pixel values in the destination area by those in the source area. Then, these factors are 'smoothed out'
by solving a Poisson equation. Finally the source pixel values are multiplied by these factors before they are put into the destination area. Currently the R,G,B values of the destination and source area are divided by each other. How can this be done if there are, say 3 values in the destination area but only 1 value in the source area.

"the healing brush looks like a smart clone brush on steroids." the solution is already there in the clone tool...

--ps

principal user interaction architect man + machine interface works

http://mmiworks.net/blog : on interaction architecture

On Tuesday 20 February 2007 16:28, peter sikking wrote: snip

boy, am I glad we went out and did workplace observation as part of the project I am running. that means I can say with confidence that the brush is the way to go. actually healing is applied extremely local, with the tiniest brush, to take out only the irregularity, and not to destroy the all important texture around this spot.

In most cases the lasso tool would do. Since the healing tool is of more 'global' nature, being forced to use a local tool like a brush seems misleading.

What could make sense would be to use a selection (one in the destination area and a translated one in the source area) first, compute the 'healed' area without pasting it into the destination area, yet. Now the brush could be optionally used to copy parts of the healed area into the destination area.

now that we got the brush part nailed, we need to find something for the user to set the source area. To me as an interaction architect the healing brush looks like a smart clone tool on steroids.

that means we can use exactly the same interaction as the clone tool.

This is exactly the way this works in photoshop and I can't think of any reason to do it differently. In fact this is exactly what I would expect as a user.

snip

"the healing brush looks like a smart clone brush on steroids." the solution is already there in the clone tool...

--ps

principal user interaction architect man + machine interface works

http://mmiworks.net/blog : on interaction architecture

Helmut wrote:

Hal wrote:

I wrote:

that means we can use exactly the same interaction as the clone tool.

This is exactly the way this works in photoshop and I can't think of any
reason to do it differently. In fact this is exactly what I would expect as a user.

Let me explain the difference between the clone tool and the healing tool in more detail.

[snip] I get the picture, smart blending clone tool on steroids...

Therefore one does need an area D which includes the defective parts but whose boundary must not contain any defective pixels. Please have a look at the example given on page 3 of http://www.tgeorgiev.net/Invariant.pdf

can we first of all agree that no photographer would put the source cursor in a highlight area when repairing a shadow area? I am sure that T. (doesn't he have a first name?) did that to show off how cool the algorithm is.

If the healing tool were to applied on the fly while the brush is over a
defective part, its boundary will most probably contain defective pixels.

Let the computer do the work. Add just one pixel around the user brushed area, and bingo! condition met.

So, if the 'brush style' of the healing tool is a must, I'd suggest to do away with the healing tool altogether - as a separate tool. Instead one could add a post-processing option to the clone tool which tries to
call for the chameleon described above.

now there is an idea: healing as a clone option, one tool less. cool.

Of course the post-processing part is should be unnoticeable by the user.
I can live with a bit of cheating where the user sees an immediate feedback that is just cloning++, and within 0,5 seconds the real healing algorithm catches up.

For that to implement, one would need to gather the (set-)union of all the regions having been touched by the brush in the source as well as in
the destination area.

no. a photographer will be concerned with image fidelity and be moving the source cursor all the time. you will get many tiny disjunct and fundamentally different (different source offset) healed areas. which can be calculated individually, which will be temporised by the speed the user works.

In case of your long scratch: even in the unlikely case of the user not using several source cursors along its path, the actual brushing of the scratch will be a painstaking affair meaning relatively slow S and D area growth.

Use these assumptions for your optimisation.

Forget rectangles, the brushed user selections are it. looks ideal that the user is specifying which pixels belong to S and D.

Remember that for the high-end photo manipulation tasks we are designing for here, the users are so discriminating that in principle no computer algorithm is good enough to do things automatic. However that must be painful to a mathematician, we saw this during the user observations.

So the healed area will be the absolute minimum, to not destroy photo-realism. Transparency will be used at the edges of the brush to blend the healing, guided by the eye of the user.

which is all that counts at the end...

--ps

principal user interaction architect man + machine interface works

http://mmiworks.net/blog : on interaction architecture

This discussion seems to be going astray a bit. Let me try to supply some suggestions and information.

Item: Peter is, no doubt unintentionally, getting into questions of implementation, as opposed to user interface, when he writes about rectangles. The brush only specifies the area to which healing is *applied* -- the region over which the differential equation is solved is invisible to the user.

Item: The whole point of this tool is to use a brush. Healing using a selection might be quite valuable, but it would be a different tool. However -- to repeat -- the brush only specifies the region to which healing is applied, not the region used to perform the healing computations.

Item: The user needs to receive feedback during brush motion. As Sven points out, the feedback does not have to be perfect, but a user needs to be able to tell at least approximately what is happening.

Item: There is no reason why healing has to be cumulative. At least during an individual painting operation (i.e., while the mouse button is held down), it is possible to retain the initial state, and accumulate the set of brush locations, using the totality to define the region for healing. Some of the existing paint tools already work this way.

Item: The main healing code is in app/paint/gimpheal.c. You don't actually need a broad knowledge of GIMP internals to understand the code there. The most important thing you need to know about is pixel regions, and the easiest way to learn that is to start by reading "How to write a GIMP plug-in", which you can find at:

http://developer.gimp.org/writing-a-plug-in/1/index.html

The interface to pixel regions for plug-ins is not identical to the one in the core, but it is very similar, and the concepts are the same.

-- Bill

______________ ______________ ______________ ______________ Sent via the CNPRC Email system at primate.ucdavis.edu

As I've been using the healing brush very intensively while retouching since it appeared , so could I add my 5c?

The healing brush tool works rather pretty now from user's POV. How much does it follow the other tools logic - it doesn't mean much for a user when he has got acquainted with the tool action and have used to it. The healing brush is quite usable now (it is used very often along with the clone tool) and deserves staying in the release even in its current state. It works and its action is hard to be replaced with any other tool. If anybody asked me - or any other user - what would I prefer: a perfectly polished tool in the future or a _working_ tool now which would be polished in the same future? I guess the answer is trivial.

What do I miss from the healing brush now? I'll try to express.

1. As the healing brush uses some data from outer regions of the brush and this results in the picture appearance (f.i. using it close to lips will make a skin near mouth reddish) I'd like to see the outline of the whole area which is used for approximations as a dim outer circle (optional).

2. I'd like to have an ability to adjust the "texturation" of the healing area manually. I always use the clone and healing brush tools with 50% opacity as it is much more controllable and the result looks better (conforms the whole appearance better). Rather often the healing brush result is too smooth and looks vanished. In this case I use clone tool to preliminary filling the area and then use healing brush to smooth it a little. So if I could adjust (increase) the texture of the source area it could help in many cases. This is the one and only solution when some parts of the skin are fully redrawn due overexposure. They should have natural look, which is due to texture also.

3. Personally I do not suffer very much from the fact the tool works once without dragging. As the process is interactive and incremental the possible benefits of productivity could be rather small.

4. If it is possible, I'd like to see a common source for the clone and healing brush tools as they are often used consequently at the same area. I mean switching between them should not require clicking Ctrl+mouse again.

Hi,
weskaggs@primate.ucdavis.edu (2007-02-21 at 0837.29 -0800):

This discussion seems to be going astray a bit. Let me try to supply some suggestions and information.

Another one: the paper describes a method to fix pixels based in other image area, but the related family of apps (PS v7 and newer, PS Elements v3 and newer) provides at least three tools that seem to do what the paper talks about:

- healing brush, behaves like clone tool plus extra processing performed when the stroke is finished. Rather interactive and quick, if you mastered clone, this one is easy to use.

http://video.about.com/graphicssoft/How-to-fix-blemishes-.htm Notice the clock cursor, that is the extra processing, with old computer and big area you clearly see it does clone and secs later it blends.

- patch tool, behaves like a reversed order copy & paste (first select area to fix, then source). Allows knowing both areas perfectly thus avoiding one typical problem of clone: a big stroke covers all you want... but starts to get pixels from source you do not want. The extra processing is done at the end of the motion, too.

http://video.about.com/graphicssoft/The-Patch-Tool.htm Notice how it looks like copy and then blends.

- spot healing brush, behaves like paint tool. A drop down decides what the program uses as "inspiration" for the fix. Simpler than clone but prone to not doing what you want if used in really big areas or complex images (thus the name "spot", best used for dust in sky of a photo, ie).

http://www.totaltraining.com/videos/mov/TPSE3_Healing_Brush.mov Notice some undo and redos until the guessing is correct.

Looking at the paper without looking at the tools is a bit strange, you already accepted looking at the inner part, better look at the full picture then. Or at least set the context of what is going to be copied, what not, what is going to be completly new or what is going to be improved.

GSR

GSR - FR wrote:

Hi,
weskaggs@primate.ucdavis.edu (2007-02-21 at 0837.29 -0800):

http://video.about.com/graphicssoft/How-to-fix-blemishes-.htm Notice the clock cursor, that is the extra processing, with old computer and big area you clearly see it does clone and secs later it blends.

- patch tool, behaves like a reversed order copy & paste (first select area to fix, then source). Allows knowing both areas perfectly thus avoiding one typical problem of clone: a big stroke covers all you want... but starts to get pixels from source you do not want. The extra processing is done at the end of the motion, too.

http://video.about.com/graphicssoft/The-Patch-Tool.htm Notice how it looks like copy and then blends.

- spot healing brush, behaves like paint tool. A drop down decides what the program uses as "inspiration" for the fix. Simpler than clone but prone to not doing what you want if used in really big areas or complex images (thus the name "spot", best used for dust in sky of a photo, ie).

http://www.totaltraining.com/videos/mov/TPSE3_Healing_Brush.mov Notice some undo and redos until the guessing is correct.

Looking at the paper without looking at the tools is a bit strange, you already accepted looking at the inner part, better look at the full picture then. Or at least set the context of what is going to be copied, what not, what is going to be completly new or what is going to be improved.

GSR

The resynthesizer plugin

http://registry.gimp.org/plugin?id=457

can do a lot of the healing functionality (and more). I've used it for everything from removing small areas and filling with a good likeness to major reconstruction of parts of photos. It's biggest downfall is that the documentation for it isn't very good; especially for replacing large sections.

Jim

Hi,

On Wed, 2007-02-21 at 12:34 +0100, Helmut Jarausch wrote:

If the healing tool were to applied on the fly while the brush is over a defective part, its boundary will most probably contain defective pixels. Think of long thin scratch, say several hundreds of pixels long, and you're using a brush of say 20 pixels in diameter.

I wasn't suggesting to apply it on the fly but to apply it on the region touched in one stroke of the tool. As far as I know that is also what PS is doing.

So, if the 'brush style' of the healing tool is a must, I'd suggest to do away with the healing tool altogether - as a separate tool. Instead one could add a post-processing option to the clone tool which tries to call for the chameleon described above.

It can still be a separate tool to the user. A lot of our tools are internally very similar. Think of tools more than a way for a user to select a certain set of parameters and algorithms. Even if the heal tool is just the clone tool with an extra option, it can still be an extra tool. Just like the Pencil tool is the same as the Paintbrush tool internally, it only has one parameter set differently.

Sven