Since Roman has offered to improve the metadata handling in GIMP, Sven asked me to document what my plans were for the metadata editor and the JPEG plug-in. Well, these plans are best summarized in the mail that I sent last year:

http://lists.xcf.berkeley.edu/lists/gimp-developer/2007-November/018992.html http://article.gmane.org/gmane.comp.video.gimp.devel/12503/

Basically, every single line of that message is still relevant today. So I will not quote the whole message again, but I encourage those who are interested in metadata to read it entirely.

This follow-up message may also be interesting, regarding which Exif tags should be updated automatically by GIMP and why GIMP should never ask the user if an image should be rotated automatically or not:

http://lists.xcf.berkeley.edu/lists/gimp-developer/2007-November/019100.html http://article.gmane.org/gmane.comp.video.gimp.devel/12610/

That's about it for the plans. You should stop reading now if you are not interested in the details. :-)

---------- scary details below this line ----------

Maybe a few points from the first message (2.6 roadmap) should be clarified a bit. In that message, I wrote:

+ migrate some parts of the metadata core to a library that can be used by the main app. as well as some plug-ins

Currently, the metadata is stored in an XMP packet inside a GIMP parasite attached to the image. While XMP is a good format for storage in files, it is not ideal for manipulation (updates and queries). It would be better it the metadata tree could remain in memory while the image is open, instead of being converted to/from XMP all the time. Also, by having this tree attached to the image and accessible directly by the core for updates and queries, it would be possible to have the metadata updated in real-time whenever the image is modified by some tools or plug-ins. This would allow some of the metadata to be displayed in the image info window, or maybe in the status bar of the image (more and more programs are doing that).

Currently, the metadata tree is only available inside the metadata editor, which is implemented as a plug-in. This results in several round-trips via the PDB whenever something must be updated. For example, assuming that Exif would now be handled correctly, the current implementation would lead to the following scenario when a JPEG image is saved:

- the JPEG plug-in calls "plug-in-metadata-set" to update some metadata - this is passed to GIMP
- GIMP starts the metadata editor plug-in - the metadata editor asks GIMP to get the parasite attached to the image - the parasite (XMP) is parsed and stored in a tree - the tree is updated according to what was requested - the tree is encoded into a parasite (XMP) - the metadata editor asks GIMP to attach the parasite to the image, then it exits
- (if there is some window currently displaying the metadata associated with that image, it is *not* updated) - the JPEG plug-in is informed that the operation was successful - the JPEG plug-in calls "plug-in-metadata-encode-exif" in order to get an Exif block from the updated metadata - this is passed to GIMP
- GIMP starts the metadata editor plug-in again - the metadata editor asks GIMP to get the parasite attached to the image - the parasite (XMP) is parsed and stored in a tree - the metadata editor updates some metadata that must be updated when an image is saved
- the updated tree is encoded into a parasite (XMP) - the metadata editor asks GIMP to attach the updated parasite - (if there is some window currently displaying the metadata associated with that image, it is *not* updated) - the contents of the tree are encoded in Exif - the metadata editor returns an array with the Exif block and exits - GIMP returns the array to the JPEG plug-in - the JPEG plug-in can now save the file with the Exif block, then exit - (if there is some window currently displaying the metadata associated with that image, the user must remember to refresh the view)

And here is what would happen if the metadata tree could be managed by the GIMP core, but we still assume that the conversion to/from Exif is done by an external plug-in:

- the JPEG plug-in calls "gimp-metadata-set" to update some metadata - this is passed to GIMP, which updates the metadata attached to the image
- (if there is some window currently displaying the metadata associated with that image, it *will be* updated) - the JPEG plug-in is informed that the operation was successful - the JPEG plug-in calls "gimp-metadata-encode-exif" in order to get an Exif block from the updated metadata - GIMP updates some metadata that must be updated when an image is saved - (if there is some window currently displaying the metadata associated with that image, it *will be* updated) - GIMP starts the metadata editor plug-in - the metadata editor makes several calls to "gimp-metadata-get" in order to retrieve the parts of the metadata that are relevant for Exif - GIMP returns the requested data (from its internal tree) - the metadata editor encodes that information in an Exif block - the metadata editor returns an array with the Exif block and exits - GIMP returns the array to the JPEG plug-in - the JPEG plug-in can now save the file with the Exif block, then exit

Furthermore, here is what would happen if the metadata tree could be managed by the GIMP core and some common metadata operations (such as the conversion to/from Exif) would be in a "libgimp-metadata" library linked directly with the file plug-ins that need it:

- the JPEG plug-in calls "gimp-metadata-set" to update some metadata - this is passed to GIMP, which updates the metadata attached to the image
- (if there is some window currently displaying the metadata associated with that image, it *will be* updated) - the JPEG plug-in is informed that the operation was successful - the JPEG plug-in calls "gimp-update-metadata-for-save" in order tell GIMP that the image is about to be saved - GIMP updates some metadata that must be updated when an image is saved - (if there is some window currently displaying the metadata associated with that image, it *will be* updated) - the JPEG plug-in makes several calls to "gimp-metadata-get" in order to retrieve the parts of the updated metadata that are relevant for Exif - the JPEG plug-in calls the library functions for encoding that metadata into an Exif block (the previous step could even be performed by the library functions so that the JPEG plug-in would not have to care). - the JPEG plug-in can now save the file with the Exif block, then exit

Sorry for the long details, but using an example like this is probably the best way to explain why it would make sense to have some parts of the metadata handling performed by a library, and why it would be better if the metadata tree could be managed by the GIMP core instead of being constantly converted to/from XMP and stored in a parasite.

In my message from last year, I also wrote:

+ convert EXIF to XMP
+ convert XMP to EXIF

The initial goal of the code that I started writing (and never finished) for the conversion to/from Exif was to get rid of the dependency on libexif. That library had caused several severe stability problems in the past and most developers (including myself) wanted to replace it by some code that was more stable.

However, the known bugs in libexif have now been fixed (some updates have not been released yet, but that's another issue) and after studying how some people (and the programs they use) are using the Exif data, I am now convinced that preserving the Exif MakerNotes is very important. This is the trickiest part of Exif and this is where most of the time was spent in libexif or other libraries (exiv2, etc.). Since parsing and updating MakerNotes is a difficult task, I think that it is better to leave that job to libexif instead of re-inventing the wheel. So contrary to what I originally planned, I think that the code for converting to/from Exif should be based on that library. This will also make our code smaller and easier to maintain.

-Raphaël

On Wed, 23 Jul 2008 11:20:41 +0200, Roman Joost wrote:

On Tue, Jul 22, 2008 at 04:50:41PM +0200, Raphaël Quinet wrote:

[...]
Currently, the metadata tree is only available inside the metadata editor, which is implemented as a plug-in. This results in several round-trips via the PDB whenever something must be updated.

[...]

In my unexperienced point of view, it looks like this should be clarified first, before I try to improve the current metadata editor. IMHO your last point seems to be the way to go, all other attempts like the first one seems to be totally inefficient and error-prune.

Yes, this is also my point of view. The best way to go is to have the metadata tree available in the core with some PDB calls allowing plug-ins to get/set data, and also have a "libgimp-metadata" library that can be used by the file plug-ins for performing tasks such as converting between Exif and XMP. The editor itself could still be a plug-in, but the lower-level parts of metadata handling would be included in the library and another part of it would be in the GIMP core.

To recap, the three scenarios are: 1) metadata editor/viewer + metadata format converter + storage are all done by a plug-in (current situation); 2) metadata editor/viewer + metadata format converter done by a plug-in, but storage handled in GIMP core; 3) metadata editor/viewer done by a plug-in, metadata format conversions in a GIMP metadata library, and storage handled in GIMP core. In the longer term, we can also think about a scenario 4 in which parts of the metadata viewer/editor could be handled by the core, but that is not for the short term anyway.

Although I think that it makes a lot of sense to have the metadata tree inside the core as in scenario 3, Sven seems to disagree so I would like to have his opinion before deciding how to proceed.

So contrary to what I originally planned, I think that the code for converting to/from Exif should be based on that library. This will also make our code smaller and easier to maintain.

My current plan was to provide a read-only view of the currently stored metadata inkl. a button to update the view. This included the conversion from Exif to XMP. Currently from my point of view - correct me if I'm wrong - the conversation will be called from the imagefile plugin (e.g. jpeg-load), after the Exif data is attached as a parasite. The metadata exif-decode procedure will convert the Exif data to XMP than, to be able to display it as XMP data in the metadata browser.

I think that you got it right. :-) In scenario 1 above, when a file is loaded the Exif block should be converted to XMP immediately (or more precisely, it should be converted to a metadata tree, which happens to be serialized as XMP) and then the metadata parasite is attached to the image. Later, when the user starts the metadata editor or viewer, the parasite is retrieved from the image and decoded into a metadata tree that can be displayed to the user.

Note that in scenarios 2 and 3, it would not be necessary to convert the metadata tree into XMP and store it in a parasite, because that tree would be managed by the core. The metadata viewer would only have to retrieve the relevant elements for display or editing. In fact, in scenarios 2 and 3 the metadata viewer/editor would not have to care about the XMP syntax: it would only need to know the name and data type of the things that it wants to display.

Now after the pros and cons about handling metadata is clarified by your mail, how different do we set our priorities to move forward for this editor and the metadata management? I think we should setup a roadmap first, setup tasks afterwards and start implementing them.

Well, the thing that could change the plans significantly is a decision about whether or not we integrate the metadata tree inside the core. And I would like to get Sven's comments about that.

If we start with scenario 2 (for example), then it should not be too hard to switch to scenario 3 later. However, if we start with 1, then it is not easy to migrate to 2 or 3 later. Most of the new code would have to be rewritten differently.

-Raphaël