Skip to content

Rhaptos Software Development

Personal tools
You are here: Home » Developer Blog » Chuck's CnxBlog » Rhaptos Internationalization

Rhaptos Internationalization Rhaptos Internationalization

Document Actions
Submitted by cbearden. on 2006-03-15 15:13. Development
A few notes on starting to create translations of Rhaptos/Connexions site text.

We are now starting to collaborate with folks outside of Connexions to internationalize the text provided by our Rhaptos and Connexions templates. In particular, we are about to begin working with volunteers from OOPS and the Vietnam Education Foundation (VEF) to internationalize our software for Chinese and Vietnamese.

Plone internationalization in a nutshell

Internationalization of Plone products works like this:

  • Each element to be translated in a Zope page template in a product is identified by a message id or msgid. The msgid is specified as the value of the i18n:translate attribute of the element, like so:
    <div class="formHelp" i18n:translate="workgroup_name">
    Enter a name for the workgroup.
    </div>
    Here, workgroup_name is the msgid that identifies the string "Enter a name for the workgroup." and its translations.
  • When the i18n:translate attribute is present but has no value (and hence no explicit msgid, then the string itself in the default language (usually English) is used as the msgid.
  • Mappings of msgids to translated strings are in .po files, of which there is one per product per language into which the product is translated. .po files are found in the i18n subdirectory of a Plone product. For translations of Plone simpliciter we are using the PloneTranslations product, whose .po files override the translations in CMFPlone's own i18n subdirectory.
  • The name of a .po file follows this form: <productname>-<lang abbrev>.po. So the name of the .po file for the German translations of RhaptosSite might look like this: RhaptosSite-de.po
    There is some flexibility in the productname part, but not in the language code or the extension.
  • Visitors to a Plone site send a list of acceptable language codes to the server when requesting a page. Plone uses the first supported language and, if it's not the default, substitutes whatever translations there are from the language-specific .po file based on the msgids of the elements in the page templates.
  • .po files are generated from a master .pot file that should contain all msgids for translatable elements in the product in question.
  • .pot files are generated from a set of page templates with the i18ndude script--a script suite, really. This tool provides a wide array of i18n functionality for the Plone developer and translator.

Process issues

Some of the issues that arise in this system are obvious.

  • As the templates are developed, new elements will be added with new msgids, which must be merged into the master .pot file and thence into each .po file.
  • At present, very few explicit msgids have been assigned to elements in Rhaptos-specific Plone products. This has at least two consequences: (1) Connexions folks must assign meaningful msgids to each element in each template in each product; and (2) since this will not be done before we hand off .po files to translators, we will eventually have to merge early .po files with literal instead of explicit msgids into newer .po files with explicit msgids for their elements.
  • We shouldn't assume that all translatable elements have i18n:translate values--we may have to retrofit code with that attribute as well as assign useful msgids.
  • A fair number of our ZPT files are not valid XML, which causes i18ndude to fail.

Priorities

Here are some things we need to do in rough order of priority:

  • roll PloneTranslations 0.8 into production/trunk (Done!)
  • hand off .po files (mostly with literal msgids, unfortunately) to Alvin (OOPS) and Hung (VEF)
  • decide on conventions for creating msgids
  • Elizabeth & I choose a product to which we start adding msgids
  • fixing XML in ZPT files (Bug Day?) so that the i18n tools can work with them
  • identify elements lacking i18n:translate attributes in existing code, and add attributes and msgids to them
  • i18n Day for adding msgids to existing code? (wouldn't everybody love that?)
  • explore tools/techniques for moving from literals to explicit msgids?
  • look into meaning & usage of i18n:domain
  • create a dev server for i18n work
  • look into i18n tagging for Python code

Re: Rhaptos Internationalization

Posted by maxwell at 2006-03-20 12:30
Are msgids site-wide or just product-wide?

Re: Re: Rhaptos Internationalization

Posted by cbearden at 2006-06-05 14:36
They are domain-wide. At the top of a .po file you specify the i18n:domain (Plone uses "plone"). If we want to create our own domain so that our translations don't overwrite those of Plone, we would use something like 'rhaptos' as the i18n:domain. However, if we feel we can assume that the Plone translations of certain msgids are and will always be appropriate for our software, we could also use "plone".

Re: Re: Rhaptos Internationalization

Posted by jccooper at 2006-06-05 14:49
msgids are available anytime you're in their "i18n domain", which is essentially a namespace.

Generally, each template declares its domain with an i18n:domain attribute. Sometimes it'll be "plone", sometimes "archetypes", sometimes something else. The po file notes the domain it work in by the first part of the filename (like RhaptosSite).

Re: Rhaptos Internationalization

Posted by maxwell at 2006-06-05 14:20
Another question (hopefully w/ an answer this time), and it may sound ignorant, but humor me: what would be so wrong w/ just leaving the string literals in instead of msgids?

My assumption is that the msgids are unique, and that therefore if there are two strings out there that happen to have the same text but perhaps have different semantics depending on their contexts or one of whose text might change at some point, then using literal strings might not be very good in those instances.

But otherwise, it would seem to me that creating a bunch of msgids is a lot of work for 1) us, because we have to go through and add msgids for every piece of text, making sure they aren't repeated elsewhere (i.e. make sure some msgid like "enter_title" isn't used both for "Enter a descriptive title for the course" and "Enter the title of this module") and also making sure that whenever we create new text, we also update the msgid, so that we don't get cases like our "Spotlight" translating to "About" in Spanish (that's currently what happens), and 2) the translator, since I assume that they will have to first look at the msgid in the .po file, then go look for what the msgid stands for in another file before they can go back and translate it. Let me know if I'm wrong on this last point.

BTW, I looked at some of the .po files in the PloneTranslations product, and it seems like 1/4 of the texts there use string literals anyway.

Re: Re: Rhaptos Internationalization

Posted by jccooper at 2006-06-05 14:52
i18ndude will provide the context of a msgid, so the translator doesn't always need to go to the source.

Using auto-gen (literal) msgids is often bad because if the string in the original template changes then the translation is broken. This is especially true for long strings, like paragraphs, that are likely to change by a few words over time.

Re: Re: Re: Rhaptos Internationalization

Posted by maxwell at 2006-06-05 18:25
I still don't see how having a msgid would make the L2 translation any less incorrect if the string from L1 was later changed. Or is the problem that the literal msgids are only generated once in the .po file and don't automatically update if changed in the original template?

Re: Re: Re: Re: Rhaptos Internationalization

Posted by cbearden at 2006-06-06 16:13
Here's one way this might happen. Let's say your templates are in English and you are using literal msgids. You've got a string (and hence a msgid) "We specialize in widgets!", and someone provides a German language translation "Wir spezialisiern uns auf Widgets!". In a subsequent version of that product, you change the English string to the British spelling: "We specialise in widgets!". The msgids are extracted anew from the English templates and merged back in to the .po files for the other languages. A msgid and message for "We specialise in widgets!" will be added, with no translation, and the msgid "We specialize in widgets!" will be removed and placed in a list that is commented out. The translators can still go through the .po files and catch the errors, but if explicit msgids had been used, there would be no need.

Literal msgids are also not so good if the translated string is very long, like a paragraph. Then the msgid (the value of an attribute) is also a paragraph.

Re: Re: Re: Re: Re: Rhaptos Internationalization

Posted by maxwell at 2006-06-06 18:48
Ok, I get it now. Thanks!

Re: Rhaptos Internationalization

Posted by jccooper at 2006-06-05 14:54
There are other command attributes; i18n:attributes, for example, translates attributes. Also, i18n:name allows you to turn some data into a token which the translator can move around.