Hello again,

if you're still interested in the deformation tool I mentioned earlier, I would be glad to implement it into GIMP during GSoC 2013.

Since last time I did some background research on another deformation algorithms and played with their implementation in various software. I also experimented with performance of my proposed algorithm and a precision of it.

There exist quite a lot of nice deformation algorithms at present. I experimented with some of them to find pros and cons of my proposed deformation algorithm.

I also talked to my former computer graphics teacher about them to gain some more insight into them.

Deformation tools I experimented with: - Interactive (Rigid) Moving Least Squares implemented in Fiji (an image processing package based on ImageJ). - Free Transform Tool implemented in Krita 2.7 pre-alpha (MLS as well) - Puppet Warp in Photoshop CS6 trial - Igarashi's As Rigid As Possible shape manipulation (Java demo) - Bounded Biharmonic Weights (MATLAB demo)

Deformations of realistic objects should be as rigid as possible (deformation includes only translation and rotation). All deformation tools above use that.

All mentioned deformation tools solve some system of equations, hence we get the result pose immediately. Because of that we must specify some initial conditions - positions of control points. That could be a drawback for a user in some situations, because to get desired pose, many control points have to be added and manipulated.

My proposed deformation algorithm (based on [1]) doesn't need these conditions, because it's based on different (iterative) approach. With this algorithm it's really like you're manipulating a real world object. It feels like there is some physics (springs) involved in it but there isn't – it's just a geometric method. Thanks to that the deformation is predictable. I think that from user perspective, intuitiveness, fast and easy manipulation and predictability of final result is very important.

About precision – previously mentioned deformation algorithms are more precise than my proposed algorithm, but it depends on lattice element (square or triangle) size. With decreasing lattice element size precision of result is increasing, but inefficiency of the iterative process is increasing as well. There has to be some balance set.

About performance – since I have been eager about the performance in C I made a pure C version of the computing core of the deformation algorithm. It seems like it's going to be about 2 times faster than Java implementation but without any additional optimizations – so there is still a space for speed-up.

There are a lot of details that will be covered in my proposal.

I know that one video clip is sometimes more than a thousand of words so I'm attaching a link to it – [2]. An example of deformation using Fiji is not included in the video clip because it's the same as what Krita does. Igarashi's ARAP and Bounded Biharmonic Weights demos aren't included as well because it isn't easy (or possible) to load specified image into them – they are just demos.

Any suggestions or advices will be appreciated.

Marek

References: [1] Daniel Sýkora, John Dingliana, Steven Collins, As-rigid-as-possible image registration for hand-drawn cartoon animations, Proceedings of the 7th International Symposium on Non-Photorealistic Animation and Rendering, August 01-02, 2009, New Orleans, Louisiana

[2] http://youtu.be/k3gN6aZWc-8

Hello Sven,

thanks for the reply.

How does the proposed method work - with more complex objects like a linear rope or cartoon drawings? (See for instance some puppet warp examples on [1])

I have looked at examples.

The proposed method is very good for bending objects or manipulating articulated objects - doing deformation of that tree in examples shouldn't be a problem for this method.

It's not so good in shrinking of objects - but there can be an easy extension added to it which can handle shrinking. It has some limitations, though. I think that Bounded Biharmonic Weights method is better at that but it has some other disadvantages.

Sometimes it would be useful if the deformation behaved similar to Moving Least Squares method (Fiji, Krita). Again with an easy extension (using weights from MLS) that is not a problem - it behaves very similarily.

I'm attaching a video clip [1] presenting these extensions on image of Mona Lisa.

- work with non-solid and non-transparent backgrounds like in photographs? Will it fill the uncovered parts of the image in a content-aware manner or can it be integrated with - for instance - the Resynthesizer plugin [2] to achieve this? Will the result still be plausible regarding lighting or can the user do this easily afterwards?

If we want to move subject's parts (e.g. arms as in Sintel example [2]) freely we always have to cut it or select it. It's the same as in Photoshop - to remove a background, something like Content Aware Fill in Photoshop or Resynthesizer in GIMP has to be used.

- in comparison to GIMPs cage transform?

Cage transform should be more precise (when every pixel is manipulated) but it's suitable for different kinds of deformations.

It would also be good if the tool worked with high bit color depth and thus were in GEGL
and already connected to the GIMP UI.

It should use GEGL but I don't know how fast it will be - I don't have so much experience with it. But I assume that every additional layer has it's overhead. Do you have any experience with that?

I wanted to make a plug-in first but I feel like UI in plug-in is not so user-friendly. I don't know...

Regards, Marek

[1] http://youtu.be/0_nl9fSTfMY [2] http://youtu.be/ieaHvHAcNRE