Current status on migration to gettext/po
This discussion is connected to the gimp-docs-list.gnome.org mailing list which is provided by the GIMP developers and not related to gimpusers.com.
This is a read-only list on gimpusers.com so this discussion thread is read-only, too.
Current status on migration to gettext/po
Hi list,
I just want to move a discussion I had with Ulf D. Ehlert from private to public to invite more people.
As for the change of the build system: I created a branch named 'xml2po-support' on which we're currently working. There is a custom Makefile developed by Ulf which uses a few code snippets from publican to create po/pot directories. Ulf can perhaps write a bit more about that and his plans with it. In my opinion it should be generated by autoconf, like our current Makefile.am creates the Makefile.
If you want to try out the current branch create a checkout by:
svn co svn+ssh://romanofski at svn.gnome.org/svn/gimp-help-2/branches/xml2po-support
The old 'src' directory is moved into 'oldsrc' and a profiled, English only version is kept under 'src'. From this sources po/pot files are created if you run for example:
make pot-de make po-de
the basis of the German translation is created.
I'm on vacation of around 14 days from today and will probably hardly check my e-mails. Though I'm eager to hear other peoples comments and will probably continue to work on the migration after I'm back.
Cheers,
Current status on migration to gettext/po
On Wed, Aug 06, 2008 at 05:00:12PM +0200, Roman Joost wrote:
Hi list,
I just want to move a discussion I had with Ulf D. Ehlert from private to public to invite more people.
As for the change of the build system: I created a branch named 'xml2po-support' on which we're currently working. There is a custom Makefile developed by Ulf which uses a few code snippets from publican to create po/pot directories. Ulf can perhaps write a bit more about that and his plans with it. In my opinion it should be generated by autoconf, like our current Makefile.am creates the Makefile.
If you want to try out the current branch create a checkout by:
svn co svn+ssh://romanofski at svn.gnome.org/svn/gimp-help-2/branches/xml2po-support
The old 'src' directory is moved into 'oldsrc' and a profiled, English only version is kept under 'src'. From this sources po/pot files are created if you run for example:
make pot-de make po-de
the basis of the German translation is created.
I'm on vacation of around 14 days from today and will probably hardly check my e-mails. Though I'm eager to hear other peoples comments and will probably continue to work on the migration after I'm back.
Cheers,
I'm missing something.
I've done the checkout (version same to the trunk - 2525 - right?).
Done
make pot-it
in the root (where there is the Makefile).
I do not see any special Makefile.
Nothing happens. What am I doing wrong?
(I'm feeling a little dumb...)
Current status on migration to gettext/po
Hi Marco,
Marco Ciampa (Samstag, 9. August 2008, 23:44):
On Wed, Aug 06, 2008 at 05:00:12PM +0200, Roman Joost wrote:
The old 'src' directory is moved into 'oldsrc' and a profiled, English only version is kept under 'src'. From this sources po/pot files are created if you run for example:
make pot-de make po-de
No, "pot-*" is not a valid target. Just try
make po-de
which implies "make pot" and will create/update the pot files as well as the po files for "de".
make xml-de
(implies "make po-de") will create/update localized XML files,
make html-de
(implies "make xml-de") creates/updates localized HTML files.
I'm missing something.
I've done the checkout (version same to the trunk - 2525 - right?). Donemake pot-it
Try "make po-it" (see above).
You may also try to use the "-n" make option (e.g. "make -n html-it") which will just print the commands without executing them.
Creating a single po file should also work: For a given source file, e.g. "src/filters/foo/bar.xml", replace "src/" with "po/it/" and ".src" with ".po", and try
make po/it/filters/foo/bar.po
Or for making a single localized xml file:
make xml/it/filters/foo/bar.xml
in the root (where there is the Makefile). I do not see any special Makefile.
Nothing happens.
?? You should get an error message:
make: *** No rule to make target `pot-it'. Stop.
What am I doing wrong?
Hmm, not enough coffee? ;-)
Bye,, Ulf
BTW, does anybody know how to checkout a single file, or how to skip directories when checking out? I have a very slow dial-up connection and don't want to download e.g. 'src/', 'oldsrc/', or 'images/'. Is this possible with subversion?
Current status on migration to gettext/po
? ???, 10/08/2008 ? 12:41 +0200, Ulf-D. Ehlert ?????:
Hi Marco,
Marco Ciampa (Samstag, 9. August 2008, 23:44):
On Wed, Aug 06, 2008 at 05:00:12PM +0200, Roman Joost wrote:
The old 'src' directory is moved into 'oldsrc' and a profiled, English only version is kept under 'src'. From this sources po/pot files are created if you run for example:
The problem is that changes were made in Makefile, while it's rewritten after configure and changes go away. I don't see any reason for this really. Right now, don't run autogen.sh or configure after branch checkout. If you did, checkout again.
BTW, does anybody know how to checkout a single file, or how to skip directories when checking out? I have a very slow dial-up connection and don't want to download e.g. 'src/', 'oldsrc/', or 'images/'. Is this possible with subversion?
svn checkout \
svn+ssh://svn.gnome.org/svn/gimp-help-2/branches/xml2po-support/Makefile
for example.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: =?koi8-r?Q?=FC=D4=C1?= =?koi8-r?Q?_=DE=C1=D3=D4=D8?=
=?koi8-r?Q?_=D3=CF=CF=C2=DD=C5=CE=C9=D1?=
=?koi8-r?Q?_=D0=CF=C4=D0=C9=D3=C1=CE=C1?=
=?koi8-r?Q?_=C3=C9=C6=D2=CF=D7=CF=CA?=
=?koi8-r?Q?_=D0=CF=C4=D0=C9=D3=D8=C0?=
Url : /lists/gimp-docs/attachments/20080810/17a047bc/attachment.bin
Current status on migration to gettext/po
? ???, 10/08/2008 ? 12:41 +0200, Ulf-D. Ehlert ?????:
Hi Marco,
Marco Ciampa (Samstag, 9. August 2008, 23:44):
On Wed, Aug 06, 2008 at 05:00:12PM +0200, Roman Joost wrote:
The old 'src' directory is moved into 'oldsrc' and a profiled, English only version is kept under 'src'. From this sources po/pot files are created if you run for example:
make pot-de make po-de
No, "pot-*" is not a valid target. Just try
make po-de
which implies "make pot" and will create/update the pot files as well as the po files for "de".
make xml-de
(implies "make po-de") will create/update localized XML files,
make html-de
(implies "make xml-de") creates/updates localized HTML files.
I'm missing something.
I've done the checkout (version same to the trunk - 2525 - right?). Donemake pot-it
Try "make po-it" (see above).
You may also try to use the "-n" make option (e.g. "make -n html-it") which will just print the commands without executing them.
Creating a single po file should also work: For a given source file, e.g. "src/filters/foo/bar.xml", replace "src/" with "po/it/" and ".src" with ".po", and try
make po/it/filters/foo/bar.po
Or for making a single localized xml file:
make xml/it/filters/foo/bar.xml
in the root (where there is the Makefile). I do not see any special Makefile.
Nothing happens.?? You should get an error message:
make: *** No rule to make target `pot-it'. Stop.
What am I doing wrong?
Hmm, not enough coffee? ;-)
Another thing I can't agree with is that po files are created for each xml file in sources. For me it seems like a waste of time, resources and this will drop one of main advantages of gettext - you can't share translation for a single string across document.
I'd better recommend you to create per-chapter po file with
xml2po -o pot src/chapter/*.xml
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: =?koi8-r?Q?=FC=D4=C1?= =?koi8-r?Q?_=DE=C1=D3=D4=D8?=
=?koi8-r?Q?_=D3=CF=CF=C2=DD=C5=CE=C9=D1?=
=?koi8-r?Q?_=D0=CF=C4=D0=C9=D3=C1=CE=C1?=
=?koi8-r?Q?_=C3=C9=C6=D2=CF=D7=CF=CA?=
=?koi8-r?Q?_=D0=CF=C4=D0=C9=D3=D8=C0?=
Url : /lists/gimp-docs/attachments/20080810/1c1c65ad/attachment.bin
Current status on migration to gettext/po
On Sun, Aug 10, 2008 at 12:41:32PM +0200, Ulf-D. Ehlert wrote:
Hi Marco,
Marco Ciampa (Samstag, 9. August 2008, 23:44):
On Wed, Aug 06, 2008 at 05:00:12PM +0200, Roman Joost wrote:
The old 'src' directory is moved into 'oldsrc' and a profiled, English only version is kept under 'src'. From this sources po/pot files are created if you run for example:
make pot-de make po-de
No, "pot-*" is not a valid target. Just try
make po-de
which implies "make pot" and will create/update the pot files as well as the po files for "de".
make xml-de
(implies "make po-de") will create/update localized XML files,
make html-de
(implies "make xml-de") creates/updates localized HTML files.
I'm missing something.
I've done the checkout (version same to the trunk - 2525 - right?). Donemake pot-it
Try "make po-it" (see above).
You may also try to use the "-n" make option (e.g. "make -n html-it") which will just print the commands without executing them.
Creating a single po file should also work: For a given source file, e.g. "src/filters/foo/bar.xml", replace "src/" with "po/it/" and ".src" with ".po", and try
make po/it/filters/foo/bar.po
Or for making a single localized xml file:
make xml/it/filters/foo/bar.xml
in the root (where there is the Makefile). I do not see any special Makefile.
Nothing happens.?? You should get an error message:
make: *** No rule to make target `pot-it'. Stop.
What am I doing wrong?
Hmm, not enough coffee? ;-)
Thank you.
I have another problem:
make po-it
.
.
.
Creato po/it/glossary/c.po.
pot/glossary/d.pot
po/it/glossary/d.po
Creato po/it/glossary/d.po.
pot/glossary/e.pot
Traceback (most recent call last):
File "/usr/bin/xml2po", line 855, in
CurrentXmlMode.preProcessXml(doc,msg)
File "/usr/share/xml2po/docbook.py", line 146, in preProcessXml
root = doc.getRootElement()
File "/var/lib/python-support/python2.5/libxml2.py", line 4158, in
getRootElement
if ret is None:raise treeError('xmlDocGetRootElement() failed')
libxml2.treeError: xmlDocGetRootElement() failed
po/it/glossary/e.po
msginit: errore durante l'apertura di "pot/glossary/e.pot" in lettura:
Nessun file o directory
make: *** [po/it/glossary/e.po] Error 1
BTW, does anybody know how to checkout a single file, or how to skip directories when checking out? I have a very slow dial-up connection and don't want to download e.g. 'src/', 'oldsrc/', or 'images/'. Is this possible with subversion?
?
why not use:
svn co /path/.../file
?
Am I missing something?
Current status on migration to gettext/po
Nickolay V. Shmyrev (Sonntag, 10. August 2008, 13:45):
BTW, does anybody know how to checkout a single file, or how to skip directories when checking out? I have a very slow dial-up connection and don't want to download e.g. 'src/', 'oldsrc/', or 'images/'. Is this possible with subversion?
svn checkout \
svn+ssh://svn.gnome.org/svn/gimp-help-2/branches/xml2po-support/Makef ilefor example.
That's what I tried, but it didn't work: "... refers to a file, not a directory"
(I can use 'wget' to download files, but then I can't commit.)
I think I got it -- with:
svn checkout --non-recursive svn+ssh://.../branches/xml2po-support
.../branches/xml2po-support/
all files have been downloaded, but no directory. :-)
Nickolay V. Shmyrev (Sonntag, 10. August 2008, 14:47):
Another thing I can't agree with is that po files are created for each xml file in sources. For me it seems like a waste of time, resources and this will drop one of main advantages of gettext - you can't share translation for a single string across document.
I'd better recommend you to create per-chapter po file with
xml2po -o pot src/chapter/*.xml
Yes, you are definitely right. So far we have just extracted and modified the essential parts of the publican Makefile(s). Changing the build process to create e.g. only one pot/po file for every directory (how to create per-chapter files?) should be easy.
The main (and most interesting) problem is how to save the translations, since adding all translations to the po files with cut & paste is obviously too much work.
It looks like 'xml2po' can do it if the English text and the translation have "the same structure" [xml2po --help], but I'm afraid that for many of our XML source files this is not true. (I didn't check it, though.)
I thought (hoped) Roman was working on it...
Ulf
Current status on migration to gettext/po
On Sun, Aug 10, 2008 at 10:33:22PM +0200, Marco Ciampa wrote:
I have another problem:
make po-it .
.
.
Creato po/it/glossary/c.po.
pot/glossary/d.pot
po/it/glossary/d.po
Creato po/it/glossary/d.po.
pot/glossary/e.pot
Traceback (most recent call last):
File "/usr/bin/xml2po", line 855, in CurrentXmlMode.preProcessXml(doc,msg) File "/usr/share/xml2po/docbook.py", line 146, in preProcessXml root = doc.getRootElement()
File "/var/lib/python-support/python2.5/libxml2.py", line 4158, in getRootElement
if ret is None:raise treeError('xmlDocGetRootElement() failed') libxml2.treeError: xmlDocGetRootElement() failed po/it/glossary/e.po
msginit: errore durante l'apertura di "pot/glossary/e.pot" in lettura: Nessun file o directory
make: *** [po/it/glossary/e.po] Error 1
resolved with:
touch po/it/glossary/e.pot
perhaps it is missing some check in Makefile...
Current status on migration to gettext/po
On Mon, Aug 11, 2008 at 06:46:06PM +0200, Marco Ciampa wrote:
On Sun, Aug 10, 2008 at 10:33:22PM +0200, Marco Ciampa wrote:
I have another problem:
make po-it .
.
.
Creato po/it/glossary/c.po.
pot/glossary/d.pot
po/it/glossary/d.po
Creato po/it/glossary/d.po.
pot/glossary/e.pot
Traceback (most recent call last):
File "/usr/bin/xml2po", line 855, in CurrentXmlMode.preProcessXml(doc,msg) File "/usr/share/xml2po/docbook.py", line 146, in preProcessXml root = doc.getRootElement()
File "/var/lib/python-support/python2.5/libxml2.py", line 4158, in getRootElement
if ret is None:raise treeError('xmlDocGetRootElement() failed') libxml2.treeError: xmlDocGetRootElement() failed po/it/glossary/e.po
msginit: errore durante l'apertura di "pot/glossary/e.pot" in lettura: Nessun file o directory
make: *** [po/it/glossary/e.po] Error 1resolved with:
touch po/it/glossary/e.pot
wrong...sorry. The right command was:
touch pot/glossary/e.pot
Current status on migration to gettext/po
On Mon, Aug 11, 2008 at 07:05:33PM +0200, Marco Ciampa wrote:
On Mon, Aug 11, 2008 at 06:46:06PM +0200, Marco Ciampa wrote:
On Sun, Aug 10, 2008 at 10:33:22PM +0200, Marco Ciampa wrote:
I have another problem:
make po-it .
.
.
Creato po/it/glossary/c.po.
pot/glossary/d.pot
po/it/glossary/d.po
Creato po/it/glossary/d.po.
pot/glossary/e.pot
Traceback (most recent call last):
File "/usr/bin/xml2po", line 855, in CurrentXmlMode.preProcessXml(doc,msg) File "/usr/share/xml2po/docbook.py", line 146, in preProcessXml root = doc.getRootElement()
File "/var/lib/python-support/python2.5/libxml2.py", line 4158, in getRootElement
if ret is None:raise treeError('xmlDocGetRootElement() failed') libxml2.treeError: xmlDocGetRootElement() failed po/it/glossary/e.po
msginit: errore durante l'apertura di "pot/glossary/e.pot" in lettura: Nessun file o directory
make: *** [po/it/glossary/e.po] Error 1resolved with:
touch po/it/glossary/e.pot
wrong...sorry. The right command was:
touch pot/glossary/e.pot
Another "strange" thing I have just noted is that it wants
pot/key-reference-fr.pot
to be present to correctly finish the
make po-it
command!
This too is resolved with a simple "touch".
bye
Current status on migration to gettext/po
After finishing last command (make po-it), I've done:
make xml-it
and
touch key-reference-fr.xml touch key-reference-it.xml
just to obtain:
$make xml-it
xml/it/glossary/e.xml
Traceback (most recent call last):
File "/usr/bin/xml2po", line 855, in
CurrentXmlMode.preProcessXml(doc,msg)
File "/usr/share/xml2po/docbook.py", line 146, in preProcessXml
root = doc.getRootElement()
File "/var/lib/python-support/python2.5/libxml2.py", line 4158, in
getRootElement
if ret is None:raise treeError('xmlDocGetRootElement() failed')
libxml2.treeError: xmlDocGetRootElement() failed
xml/it/glossary/k.xml
Traceback (most recent call last):
.
.
xml/it/glossary/n.xml
.
.
.
xml/it/glossary/w.xml
.
.
.
xml/it/glossary/z.xml
so my little trick just moved the problem from the make po-it to the make xml-it command.
:-(
Current status on migration to gettext/po
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 11.08.2008 um 19:37 schrieb Marco Ciampa:
After finishing last command (make po-it), I've done:
...
....
.
xml/it/glossary/n.xml
.
.
.
xml/it/glossary/w.xml
.
.
.
xml/it/glossary/z.xmlso my little trick just moved the problem from the make po-it to the make
xml-it command.:-(
this problem is caused by the extraction of the en language. Some of the files -> the one you mentioned, are simply not docbook xml files. Probably there was a glitch when doing the en extraction in the first place...
Greetings, lexA
Current status on migration to gettext/po
Axel Wernicke (Montag, 11. August 2008, 19:52):
this problem is caused by the extraction of the en language. Some of the files -> the one you mentioned, are simply not docbook xml files. Probably there was a glitch when doing the en extraction in the first place...
They are, but they do not contain English content, so the resulting XML files after profiling were empty.
I have added some checks to the Makefile.
One more (minor) item for the TODO list: we will have to find a way to sort the translated glossary entries.
Ulf
Current status on migration to gettext/po
On Mon, Aug 11, 2008 at 01:50:25PM +0200, Ulf-D. Ehlert wrote:
Nickolay V. Shmyrev (Sonntag, 10. August 2008, 14:47):
Another thing I can't agree with is that po files are created for each xml file in sources. For me it seems like a waste of time, resources and this will drop one of main advantages of gettext - you can't share translation for a single string across document.
I'd better recommend you to create per-chapter po file with
xml2po -o pot src/chapter/*.xml
Yes, you are definitely right. So far we have just extracted and modified the essential parts of the publican Makefile(s). Changing the build process to create e.g. only one pot/po file for every directory (how to create per-chapter files?) should be easy.
This could be a good compromise...IMHO.
The main (and most interesting) problem is how to save the translations, since adding all translations to the po files with cut & paste is obviously too much work.
I agree.
It looks like 'xml2po' can do it if the English text and the translation have "the same structure" [xml2po --help], but I'm afraid that for many of our XML source files this is not true. (I didn't check it, though.)
Then this could be done by the translators...the cost of the migration should be divided to all...
I thought (hoped) Roman was working on it...
IMHO a great work ... please go on!
unsuscribe
unsuscribe
Current status on migration to gettext/po
Hi,
On Mon, Aug 11, 2008 at 01:50:25PM +0200, Ulf-D. Ehlert wrote:
Nickolay V. Shmyrev (Sonntag, 10. August 2008, 13:45): Nickolay V. Shmyrev (Sonntag, 10. August 2008, 14:47):
Another thing I can't agree with is that po files are created for each xml file in sources. For me it seems like a waste of time, resources and this will drop one of main advantages of gettext - you can't share translation for a single string across document.
I'd better recommend you to create per-chapter po file with
xml2po -o pot src/chapter/*.xml
Would that actually speed things a little bit up in comparison to merging the translations for every single po file?
[...]
The main (and most interesting) problem is how to save the translations, since adding all translations to the po files with cut & paste is obviously too much work.
I was on vacation - sorry - but back on the construction scene now ;)
I spend a few hours on figuring out a possible solution to save the translations by studying various sources (Nickolays proposed patch (bug 369510), the Mail conversation with Ulf who already had a look at xml2po and xml2po itself).
I ended up by do the following scenario:
1. Profile the translation language as well into a subdirectory (e.g. de):
cd oldsrc xsltproc --stringparam profile.lang "de" ../stylesheets/profile.xsl introduction.xml > introduction.xml.new mv introduction.xml.new ../de/introduction.xml
2. Reuse the translation to merge it into one po file:
cd .. xml2po -r de/introduction.xml --output="po/de/foo.po" src/introduction.xml
The foo.po now contains the english original msgids and the German translation. Did someone tried this actually? Ulf maybe?
The only problem here is (of course), that the generated po file doesn't map 1:1 to the English original strings. This needs to be corrected by the author itself. Could still be a lot of work, but we don't have to copy & paste the whole manual.
This means, we can do this in one shot: create for each translation a directory with the profiled xml files in it. Merge the translations to po files with English as a reference and check. The steps could be made manualy. Handy would be a small script for the authors of course.
It looks like 'xml2po' can do it if the English text and the translation have "the same structure" [xml2po --help], but I'm afraid that for many of our XML source files this is not true. (I didn't check it, though.)
I thought (hoped) Roman was working on it...
Yep doing it now :)
Cheers!
Current status on migration to gettext/po
Roman Joost (Sonntag, 24. August 2008, 15:22):
Would that actually speed things a little bit up in comparison to merging the translations for every single po file?
It would speed up translating.
I spend a few hours on figuring out a possible solution to save the translations by studying various sources (Nickolays proposed patch (bug 369510), the Mail conversation with Ulf who already had a look at xml2po and xml2po itself).
I ended up by do the following scenario:
1. Profile the translation language as well into a subdirectory (e.g. de):
cd oldsrc xsltproc --stringparam profile.lang "de" ../stylesheets/profile.xsl introduction.xml > introduction.xml.new mv introduction.xml.new ../de/introduction.xml
2. Reuse the translation to merge it into one po file:
cd .. xml2po -r de/introduction.xml --output="po/de/foo.po" src/introduction.xml
The foo.po now contains the english original msgids and the German translation. Did someone tried this actually? Ulf maybe?
Yes, I tried it for a sample file. It worked like expected, but ...
The only problem here is (of course), that the generated po file doesn't map 1:1 to the English original strings. This needs to be corrected by the author itself.
That's the problem!
My sample file contained e.g. the following index entries:
...
...
...
...
Filter...
...
"en" and "de" don't match, and "xml2po -r" produces more or less useless output.
Misssing sectinfo (revhistory) entries will certainly cause similar problems.
So IMHO we had to add one more step:
0. Check and edit every single source file...
Ulf
Current status on migration to gettext/po
On Mon, Aug 25, 2008 at 08:42:00PM +0200, Ulf-D. Ehlert wrote:
So IMHO we had to add one more step:
0. Check and edit every single source file...
Yes, I've already done some cleanup in this direction (but I've just started, others are doing this by years!) trying to figure how to "fill the gaps" sometimes with some TODO, renaming screenshots files to the same of english ecc.
I think this is the only way... I think we all should try to help to do this before anythingelse.
Current status on migration to gettext/po
Hi Ulf,
On Mon, Aug 25, 2008 at 08:42:00PM +0200, Ulf-D. Ehlert wrote:
Roman Joost (Sonntag, 24. August 2008, 15:22):
Would that actually speed things a little bit up in comparison to merging the translations for every single po file?
It would speed up translating.
Okey - we should keep this in mind. I wonder how we will do this. This would certainly mean that we need to put the chapters into their own single files. Currently it's mostly split into sections... But I think this could be done after the migration into a cleanup process.
There is one point still open though, the integration into autoconf of the Makefile. Ulf - you put a lot of effort in the Makefile. Do you think you could spent a bit more time on this? If you want to discuss this topic further, I think it'll make sense to do this in another threat.
I spend a few hours on figuring out a possible solution to save the translations by studying various sources (Nickolays proposed patch (bug 369510), the Mail conversation with Ulf who already had a look at xml2po and xml2po itself).
[...]The foo.po now contains the english original msgids and the German translation. Did someone tried this actually? Ulf maybe?
Yes, I tried it for a sample file. It worked like expected, but ...
The only problem here is (of course), that the generated po file doesn't map 1:1 to the English original strings. This needs to be corrected by the author itself.
That's the problem!
My sample file contained e.g. the following index entries:
[...]
..."en" and "de" don't match, and "xml2po -r" produces more or less useless output.
Yeh - I know.
Misssing sectinfo (revhistory) entries will certainly cause similar problems.
So IMHO we had to add one more step:
0. Check and edit every single source file...
We have to do this manually unfortunately. There is no better way around this IMHO. A program would also only be able to just guess. Let's face the advantages: we do have some time for clean up ;)
How do we want to proceed? Do the authors (please yell know ;) want to do a migration for themself, using a small script or should someone of us (Ulf or me probably) should do the migration?
If no one of the authors votes for step 1, than I propose a migration using step 2 the next weekend.
We do need a fixed date anyways, where we start the migration. That means, that after the date only the work and the cleanup on the po files continues.
It's probably a good time to do another release before the migration (which would be 2.4.2), so we can start documenting GIMP 2.6 after our release and our cleanup (which would be 2.6.0 probably). But that's future sound...
Cheers,
Current status on migration to gettext/po
Hi Marco,
On Mon, Aug 25, 2008 at 09:16:52PM +0200, Marco Ciampa wrote:
On Mon, Aug 25, 2008 at 08:42:00PM +0200, Ulf-D. Ehlert wrote:
So IMHO we had to add one more step:
0. Check and edit every single source file...
Yes, I've already done some cleanup in this direction (but I've just started, others are doing this by years!) trying to figure how to "fill the gaps" sometimes with some TODO, renaming screenshots files to the same of english ecc.
I think this is the only way... I think we all should try to help to do this before anythingelse.
Do you want to 'clean up' the XML code to a 1:1 mapping before the migration? Would make sense though...
Greetings,
Current status on migration to gettext/po
On Mon, Aug 25, 2008 at 10:20:27PM +0200, Roman Joost wrote:
Hi Marco,
On Mon, Aug 25, 2008 at 09:16:52PM +0200, Marco Ciampa wrote:
On Mon, Aug 25, 2008 at 08:42:00PM +0200, Ulf-D. Ehlert wrote:
So IMHO we had to add one more step:
0. Check and edit every single source file...
Yes, I've already done some cleanup in this direction (but I've just started, others are doing this by years!) trying to figure how to "fill the gaps" sometimes with some TODO, renaming screenshots files to the same of english ecc.
I think this is the only way... I think we all should try to help to do this before anythingelse.
Do you want to 'clean up' the XML code to a 1:1 mapping before the migration? Would make sense though...
I can help. I whish that there should be an automatic tool/script (or simply just a sequence of commands) to speedup che "xml simmetry test" i.e. the testing that every language enabled part is similar to the english "template". Doing this just looking at the sources is IMHO impractical.
Something like:
symtest chapter.xml
with output:
chapter.xml (the cleaned source)
chapter.xml.orig
chapter.xml.rej
similar as a patch output.
Current status on migration to gettext/po
Roman Joost (Montag, 25. August 2008, 22:19):
There is one point still open though, the integration into autoconf of the Makefile. Ulf - you put a lot of effort in the Makefile. Do you think you could spent a bit more time on this?
You mean merging the Makefile into our current Makefile.am? I can try it and see what happens...
We have to do this manually unfortunately. There is no better way around this IMHO. A program would also only be able to just guess. Let's face the advantages: we do have some time for clean up ;)
Either clean up or try to patch 'xml2po', so that it can handle those problematic cases.
How do we want to proceed? Do the authors (please yell know ;) want to do a migration for themself, using a small script or should someone of us (Ulf or me probably) should do the migration?
If no one of the authors votes for step 1, than I propose a migration using step 2 the next weekend.
Meanwhile I have committed a simple shell script you can use to extract localized XML files, e.g.
mkdir -p tmp/xx bash tools/profile-xml.sh --srcdir oldsrc --destdir tmp/xx --lang xx
It's just a wrapper for 'xsltproc', and it may be buggy...
We do need a fixed date anyways, where we start the migration. That means, that after the date only the work and the cleanup on the po files continues.
After starting the migration we should keep the current multi-lang XML files up to date untill we definitely know everything works. Do you mean that with "the work"?
It's probably a good time to do another release before the migration (which would be 2.4.2),
Yes!! :-)
BTW, I think we should remove any entries before 2008-07-30 from the branches/xml2po-support/ChangeLog file. And maybe it's a good idea to split (and compress?) the trunk/ChangeLog file (1.4MB!).
Ulf
Current status on migration to gettext/po
Hi Ulf,
thanks for the work on adding the autoconf stuff to the Makefile.
On Tue, Aug 26, 2008 at 08:10:07PM +0200, Ulf-D. Ehlert wrote:
Roman Joost (Montag, 25. August 2008, 22:19):
We have to do this manually unfortunately. There is no better way around this IMHO. A program would also only be able to just guess. Let's face the advantages: we do have some time for clean up ;)
Either clean up or try to patch 'xml2po', so that it can handle those problematic cases.
I wonder what xml2po should do in those cases. Just include the strings of the next siblings seen from the current HTML node, in the translation? Could try to check if that works ...
If no one of the authors votes for step 1, than I propose a migration using step 2 the next weekend.
Meanwhile I have committed a simple shell script you can use to extract localized XML files, e.g.
mkdir -p tmp/xx bash tools/profile-xml.sh --srcdir oldsrc --destdir tmp/xx --lang xx
Nice! I'll try it...
We do need a fixed date anyways, where we start the migration. That means, that after the date only the work and the cleanup on the po files continues.
After starting the migration we should keep the current multi-lang XML files up to date untill we definitely know everything works. Do you mean that with "the work"?
No - I actually mean, that we only work on the po files and add new content by editing the XML based on our reference language (English that is).
Keeping track of two content holders sounds double work to me. If the content migration goes well, I don't see any reason why we should manage the content using gettext and besides also in the multilangual XML file. That would mean we have to make the migration probably a second or third time. How do we know, that 'definitly everything works?'.
That's why we do this on a branch, test it, do the migration and finally merge with the current TRUNK.
BTW, I think we should remove any entries before 2008-07-30 from the branches/xml2po-support/ChangeLog file.
Why do we want to do that? Doesn't make any sense to me...
And maybe it's a good idea to split (and compress?) the trunk/ChangeLog file (1.4MB!).
We could rename the ChangeLog to ChangeLog2.4 or something and start from scratch a new one. Sounds good to me.
Greetings,
Current status on migration to gettext/po
Roman Joost (Mittwoch, 27. August 2008, 10:55):
I wonder what xml2po should do in those cases. Just include the strings of the next siblings seen from the current HTML node, in the translation? Could try to check if that works ...
This is exactly the basic idea of a Python script I have started to write and that (hopefully) will split a multi-language XML file into single-language XML files of the same structure: Walk recursively through the XML tree and either take the node or take sibling nodes of the same type for the specified languages.
If it really works (I don't promise anything!) these files should work with xml2po's "--reuse" option.
So far it seems to work for a simple XML file (e.g. chrome.xml). I will need some days to clean up, then I may commit it. Then we can start to add features to handle the not so simple parts...
BTW, I think we should remove any entries before 2008-07-30 from the branches/xml2po-support/ChangeLog file.
Why do we want to do that? Doesn't make any sense to me...
E.g. because of
diff gimp-help-2/{branches/xml2po-support,trunk}/ChangeLog
Ulf
Current status on migration to gettext/po
Hi Ulf,
On Mon, Sep 01, 2008 at 09:16:39PM +0200, Ulf-D. Ehlert wrote:
Roman Joost (Mittwoch, 27. August 2008, 10:55):
I wonder what xml2po should do in those cases. Just include the strings of the next siblings seen from the current HTML node, in the translation? Could try to check if that works ...
This is exactly the basic idea of a Python script I have started to write and that (hopefully) will split a multi-language XML file into single-language XML files of the same structure: Walk recursively through the XML tree and either take the node or take sibling nodes of the same type for the specified languages.
Interesting - I still, wonder if we really want to do that. I figured, that there is sometimes even more English strings than translated strings.
Do you implement it as a special mode (like the default is the docbook mode)?
Anyways - I'd be interested in your results :)
If it really works (I don't promise anything!) these files should work with xml2po's "--reuse" option.
That's actually the thing. I figured today, if there is no 1:1 mapping between reference and translation language - the whole reuse option is b0rked. That's because of the sequential parsing of the files. First the translation, than the original XML is parsed (or vice versa). Then the translation strings are merged. If we have more translated strings than reference language strings... well - do something with it... And if the difference between the reference language and the translation already happens in the start of the article, the whole *.pot file is messed up.
So far it seems to work for a simple XML file (e.g. chrome.xml). I will need some days to clean up, then I may commit it. Then we can start to add features to handle the not so simple parts...
Try to test it with the src/dialogs/tools-dialog.xml for example...
BTW, I think we should remove any entries before 2008-07-30 from the branches/xml2po-support/ChangeLog file.
Why do we want to do that? Doesn't make any sense to me...
E.g. because of
diff gimp-help-2/{branches/xml2po-support,trunk}/ChangeLog
Why not merging it?
Greetings,
Current status on migration to gettext/po
Roman Joost (Montag, 1. September 2008, 21:58):
Interesting - I still, wonder if we really want to do that. I figured, that there is sometimes even more English strings than translated strings.
The script shall use English strings if necessary to produce files of same structure.
Do you implement it as a special mode (like the default is the docbook mode)?
No, but I use some list of tag names to decide whether or not these nodes (tags) are to be processes recursively. These tags are DocBook tags, of course.
If it really works (I don't promise anything!) these files should work with xml2po's "--reuse" option.
That's actually the thing. I figured today, if there is no 1:1 mapping between reference and translation language - the whole reuse option is b0rked. That's because of the sequential parsing of the files. First the translation, than the original XML is parsed (or vice versa). Then the translation strings are merged. If we have more translated strings than reference language strings... well - do something with it...
We'll have to throw them away - what else?
And if the difference between the reference language and the translation already happens in the start of the article, the whole *.pot file is messed up.
Yes, and revhistory and indexterm tags are at the top of the XML files, causing xml2po to produce garbage right from the beginning...
Try to test it with the src/dialogs/tools-dialog.xml for example...
Result looks good. Not perfect, but promising.
Why not merging it?
IMHO there is something wrong if we have to merge two ChangeLogs manually...
Ulf
Current status on migration to gettext/po
Hi Ulf,
On Tue, Sep 02, 2008 at 10:01:29PM +0200, Ulf-D. Ehlert wrote:
Roman Joost (Montag, 1. September 2008, 21:58):
Try to test it with the src/dialogs/tools-dialog.xml for example...
Result looks good. Not perfect, but promising.
Cool - I'd be eager to see the script :)
Why not merging it?
IMHO there is something wrong if we have to merge two ChangeLogs manually...
No .. that's why svn merge is for. We merge the branch back into TRUNK if the migration worked...
Greetings,
Current status on migration to gettext/po
Roman Joost (Mittwoch, 3. September 2008, 08:34):
Result looks good. Not perfect, but promising.
Cool - I'd be eager to see the script :)
Ok, I've just committed the script.
Remember that it's work in progress, don't expect it to work without any
problems.
We still have to find out when the script fails or when it produces
files that we can't use.
"split-xml-multi-lang.py --help" will give you a short help message.
No .. that's why svn merge is for. We merge the branch back into TRUNK if the migration worked...
Oh. :-)
Ulf
Current status on migration to gettext/po
On Wed, Sep 03, 2008 at 10:13:01PM +0200, Ulf-D. Ehlert wrote:
Roman Joost (Mittwoch, 3. September 2008, 08:34):
Result looks good. Not perfect, but promising.
Cool - I'd be eager to see the script :)
Ok, I've just committed the script.
I tested it. Awesome! Great work.
The intendation is a bit messed up after the script splitted the languages, but I think that it's something we don't have to care about. For the migration, it's probably the most important step.
The migration would look like this now:
1. Merge the TRUNK into the Branch, to be up-to-date with the latest
content.
2. Split the languages using Ulf's new python script.
3. Create the initial *.po files which already have English
original and translated strings.
^---- dry run -----^
4. Check if we can continue working from this point
5. If 4. is true, than backmerge the branch into TRUNK
6. Continue to fix the translations and work on the documentation
for 2.6
We can test if the points 1,2 and 3 work flawlessly and produce what we expect. If that works and we can use the migrated content (4.) we could merge our branch into TRUNK and continue using *.po files.
Did I missed something?
Greetings,
Current status on migration to gettext/po
Hi Roman,
Roman Joost (Freitag, 5. September 2008, 14:19):
The intendation is a bit messed up after the script splitted the languages,
Yes, I skipped the empty text nodes which are caused by whitespaces
between xml tags due to intendation and empty lines:
but I think that it's something we don't have to care about.
'xml2po' would delete these empty texts anyway.
For the migration, it's probably the most important step.
The migration would look like this now:
1. Merge the TRUNK into the Branch, to be up-to-date with the latest content.
2. Split the languages using Ulf's new python script. 3. Create the initial *.po files which already have English original and translated strings. ^---- dry run -----^
4. Check if we can continue working from this point 5. If 4. is true, than backmerge the branch into TRUNK 6. Continue to fix the translations and work on the documentation for 2.6We can test if the points 1,2 and 3 work flawlessly and produce what we expect. If that works and we can use the migrated content (4.) we could merge our branch into TRUNK and continue using *.po files.
Did I missed something?
Maybe:
(a) I've made some changes to handle more XML tags. But there's still
an essential part of the algorithm I had in mind but did not yet
implement. This final / non-final stuff was just meant to simplify
and speed up the script....
Did you made some test with with xml2po's "reuse" feature? Are the
results usable?
(b) There are still some problems with unusual files, especially the
glossary files. And what about the key-references?
(c) I did not start to convert the Makefile to Makefile.am. (No, I'm
not working 24h/day on gimp-help. ;-))
(d) I think you wanted to release a new version based on the current
(old) build system and the XML source in trunk. IMHO you should
definetely do this before any serious attempts to migrate!
(e) There is still a bug in xml2po (at least I considered it to be a
bug - see http://bugzilla.gnome.org/show_bug.cgi?id=545478) and
I'm using a patched version of xml2po. These two versions produce
different po files. I'm not sure what to do here.
Ulf