Internationalize a Python application

(Remove duplicated line)
(use <code>)
Line 1: Line 1:
This page explains step by step how to add support for translations in a Python application for Maemo.  
This page explains step by step how to add support for translations in a Python application for Maemo.  
-
There are already some [http://www.learningpython.com/2006/12/03/translating-your-pythonpygtk-application/ tutorials online] about how to internationalize (i18n) python applications, but i find them difficult to follow and they lack some needed code and final tricks to make everything work fine.  
+
There are already some [http://www.learningpython.com/2006/12/03/translating-your-pythonpygtk-application/ tutorials online] about how to internationalize (i18n) Python applications, but I find them difficult to follow and they lack some needed code and final tricks to make everything work fine.  
To support i18n we need to accomplish 5 tasks:
To support i18n we need to accomplish 5 tasks:
-
# Define correctly a function '_()' that translates strings
+
# Define correctly a function '<code>_()</code>' that translates strings
# Mark the strings in the code with that function
# Mark the strings in the code with that function
# Generate a template for the translators
# Generate a template for the translators
Line 12: Line 12:
== The overall process ==
== The overall process ==
-
From the source code, using a command line tool, we will generate a ".pot" file. It is plain text file containing all strings that need translation in the project. This .pot file only needs to be generated/recreated when the strings change in the code (and _not_ in every build!). There is only one .pot file per project.
+
From the source code, using a command line tool, we will generate a "<code>.pot</code>" file. It is plain text file containing all strings that need translation in the project. This <code>.pot</code> file only needs to be generated/recreated when the strings change in the code (and ''not'' in every build!). There is only one <code>.pot</code> file per project.
-
Then, from the .pot files and using another command line tool, we will generate the .po files for each language. There is one .po file per language, and it is the file that the translators need to complete. Both .pot and all .po files should be committed in the repository.
+
Then, from the <code>.pot</code> files and using another command line tool, we will generate the <code>.po</code> files for each language. There is one <code>.po</code> file per language, and it is the file that the translators need to complete. Both <code>.pot</code> and all <code>.po</code> files should be committed in the repository.
-
But once your program is running, it doesn't use directly .po files, but a binary (compiled) version of them: the .mo files. These .mo files must be re-created on build time (when creating the package) and installed in the right location in the system. These are generated files, so don't commit them in the repository.
+
But once your program is running, it doesn't use <code>.po</code> files directly, but a binary (compiled) version of them: the <code>.mo</code> files. These <code>.mo</code> files must be re-created on build time (when creating the package) and installed in the right location in the system. These are generated files, so don't commit them to the repository.
From the code point of view, you just need to call few functions to tell gettext where are the translation files, what language do you want to use (usually the locale of the environment), and then it will use its implementation to find the right string.
From the code point of view, you just need to call few functions to tell gettext where are the translation files, what language do you want to use (usually the locale of the environment), and then it will use its implementation to find the right string.
Line 33: Line 33:
== Mark strings for i18n ==
== Mark strings for i18n ==
-
This is one of the easiest parts. Browse the source code files that show something on the screen, and wrap the visible strings (window titles, dialog titles, notifications, labels, button labels and so on) with the '_()' function.
+
This is one of the easiest parts. Browse the source code files that show something on the screen, and wrap the visible strings (window titles, dialog titles, notifications, labels, button labels and so on) with the '<code>_()</code>' function.
For example:
For example:
Line 53: Line 53:
  xgettext --language=Python --keyword=_ --output=po/PROJECTNAME.pot `find . -name "*.py"`
  xgettext --language=Python --keyword=_ --output=po/PROJECTNAME.pot `find . -name "*.py"`
-
It will parse files, written in Python, looking for strings marked with the keyword (function name) '_' and saving the output in a file called 'po/PROJECTNAME.pot'. The list of files is the last argument. I like to use "find", but you can manually put a list of files there.
+
It will parse files, written in Python, looking for strings marked with the keyword (function name) '_' and saving the output in a file called '<code>po/PROJECTNAME.pot</code>'. The list of files is the last argument. I like to use "find", but you can manually put a list of files there.
-
That command will generate the .pot file. Easy, isn't it?
+
That command will generate the <code>.pot</code> file. Easy, isn't it?
== How to add translations ==
== How to add translations ==
-
Now you want to get a translation in an specific language. You need to generate a .po for it.
+
Now you want to get a translation in an specific language. You need to generate a <code>.po</code> for it.
-
Go to the po/ folder and run this command (again, inside/outside scratchbox doesn't matter):
+
Go to the <code>po/</code> folder and run this command (again, inside/outside scratchbox doesn't matter):
  msginit --input=PROJECTNAME.pot --locale=LOCALE
  msginit --input=PROJECTNAME.pot --locale=LOCALE
-
For instance, if i want a .po file for the spanish translation of my project Mussorgsky, i write:
+
For instance, if I want a <code>.po</code> file for the Spanish translation of my project Mussorgsky, I write:
  msginit --input=mussorgsky.pot --locale=es_ES
  msginit --input=mussorgsky.pot --locale=es_ES
-
It will generate a file "es.po" and translators can/must work on that file.  
+
It will generate a file "<code>es.po</code>" and translators can/must work on that file.  
-
'''IMPORTANT''': by default the generated .po file declares its charset as ASCII. This means that translations with non-ascii characters will provoke an error in the program. To solve this, set the charset to 'utf-8':
+
'''IMPORTANT''': by default the generated <code>.po</code> file declares its charset as ASCII. This means that translations with non-ascii characters will provoke an error in the program. To solve this, set the charset to '<code>utf-8</code>':
  "Language-Team: Spanish\n"
  "Language-Team: Spanish\n"
Line 81: Line 81:
== Include translations in your installation ==
== Include translations in your installation ==
-
This part is more tricky, because it depends on your build system. I'll explain how it goes with the common python 'distutils' (what makes the usual ''python setup.py install'' command work). In distutils all the magic happens in the ''setup.py'' file.
+
This part is more tricky, because it depends on your build system. I'll explain how it goes with the common python 'distutils' (what makes the usual ''<code>python setup.py install</code>'' command work). In distutils all the magic happens in the ''<code>setup.py</code>'' file.
-
First of all, if in the previous steps you have added the i18n.py file to your source tree, don't forget to include it in your data_files list!
+
First of all, if in the previous steps you have added the <code>i18n.py</code> file to your source tree, don't forget to include it in your <code>data_files</code> list!
Then, copy to the root folder of your project the file [https://garage.maemo.org/plugins/ggit/browse.php/?p=mussorgsky;a=blob;f=msgfmt.py;hb=HEAD msgfmt.py]. It is a program that translates the .po files into the binary .mo files that the application needs.
Then, copy to the root folder of your project the file [https://garage.maemo.org/plugins/ggit/browse.php/?p=mussorgsky;a=blob;f=msgfmt.py;hb=HEAD msgfmt.py]. It is a program that translates the .po files into the binary .mo files that the application needs.
-
In your setup.py file you need to add all this code. It "overloads" the default 'build' and 'install' instructions to include the translations in the process. Now in 'build' time it will generate the .mo files (using the msgfmt.py we added just now), and in 'install' time it will include the '.mo' files in the data_files list. The code is copied from this a [https://garage.maemo.org/plugins/ggit/browse.php/?p=mussorgsky;a=blob;f=setup.py;hb=HEAD setup.py] file.
+
In your <code>setup.py</code> file you need to add all this code. It "overloads" the default 'build' and 'install' instructions to include the translations in the process. Now in 'build' time it will generate the .mo files (using the <code>msgfmt.py</code> we added just now), and in 'install' time it will include the '<code>.mo</code>' files in the <code>data_files</code> list. The code is copied from this a [https://garage.maemo.org/plugins/ggit/browse.php/?p=mussorgsky;a=blob;f=setup.py;hb=HEAD setup.py] file.
<source lang="python">
<source lang="python">
Line 98: Line 98:
import os
import os
</source>
</source>
-
This command compiles every .po file under the po/ folder into a .mo (the .mo files will be under build/locale/)
+
This command compiles every <code>.po</code> file under the <code>po/</code> folder into a <code>.mo</code> (the <code>.mo</code> files will be under <code>build/locale/</code>)
<source lang="python">
<source lang="python">
class build_trans(cmd.Command):
class build_trans(cmd.Command):
Line 136: Line 136:
         _build.run(self)
         _build.run(self)
</source>
</source>
-
Installation time: put every .mo file under build/locale in the data_files list (the list of things to be installed) and call the default 'install' operation
+
Installation time: put every <code>.mo</code> file under <code>build/locale</code> in the <code>data_files</code> list (the list of things to be installed) and call the default 'install' operation
<source lang="python">
<source lang="python">
class install_data(_install_data):
class install_data(_install_data):
Line 155: Line 155:
}
}
</source>
</source>
-
And don't forget to add this in your setup function, in the setup.py
+
And don't forget to add this in your setup function, in the <code>setup.py</code>
<source lang="python">
<source lang="python">
setup(name        = 'your project',
setup(name        = 'your project',
Line 175: Line 175:
  ...
  ...
-
Usually dpkg-buildpackage uses python2.5 setup.py build and install, so everything should work out-of-the-box with your previous package configuration.
+
Usually <code>dpkg-buildpackage</code> uses <code>python2.5 setup.py build</code> and <code>install</code>, so everything should work out-of-the-box with your previous package configuration.
Feel free to complete/correct this wiki page with your own experience.
Feel free to complete/correct this wiki page with your own experience.
Line 182: Line 182:
[[Category:Development]]
[[Category:Development]]
[[Category:HowTo]]
[[Category:HowTo]]
 +
[[Category:Python]]

Revision as of 11:14, 21 October 2010

This page explains step by step how to add support for translations in a Python application for Maemo.

There are already some tutorials online about how to internationalize (i18n) Python applications, but I find them difficult to follow and they lack some needed code and final tricks to make everything work fine.

To support i18n we need to accomplish 5 tasks:

  1. Define correctly a function '_()' that translates strings
  2. Mark the strings in the code with that function
  3. Generate a template for the translators
  4. Add translations
  5. Include the translations in the installation

Contents

The overall process

From the source code, using a command line tool, we will generate a ".pot" file. It is plain text file containing all strings that need translation in the project. This .pot file only needs to be generated/recreated when the strings change in the code (and not in every build!). There is only one .pot file per project.

Then, from the .pot files and using another command line tool, we will generate the .po files for each language. There is one .po file per language, and it is the file that the translators need to complete. Both .pot and all .po files should be committed in the repository.

But once your program is running, it doesn't use .po files directly, but a binary (compiled) version of them: the .mo files. These .mo files must be re-created on build time (when creating the package) and installed in the right location in the system. These are generated files, so don't commit them to the repository.

From the code point of view, you just need to call few functions to tell gettext where are the translation files, what language do you want to use (usually the locale of the environment), and then it will use its implementation to find the right string.

Ok, this is what we need to do. Let's code.

Configure gettext Define the '_()' function

If you don't care too much about the details, just copy this i18n.py file in your src/ directory (change the APP_NAME variable there!), and in every .py file in your module that needs to translate any string, add these lines at the top:

import i18n
_ = i18n.language.gettext

Done. If you are curious about what is going on there, the file has plenty of comments explaining every line.

Mark strings for i18n

This is one of the easiest parts. Browse the source code files that show something on the screen, and wrap the visible strings (window titles, dialog titles, notifications, labels, button labels and so on) with the '_()' function.

For example:

# import the _() function!
import i18n
_ = i18n.language.gettext
 
class Example (hildon.StackableWindow):
 
   def __init__ (self):
       hildon.StackableWindow.__init__ (self)
       self.set_title ( _("All music") )   <------- ADD THIS!

Generate template for the translators

Create a po folder in your project (usually at the same level as src/) and run the following command from the top directory of your project. You can do this inside or outside Scratchbox, there is no difference.

xgettext --language=Python --keyword=_ --output=po/PROJECTNAME.pot `find . -name "*.py"`

It will parse files, written in Python, looking for strings marked with the keyword (function name) '_' and saving the output in a file called 'po/PROJECTNAME.pot'. The list of files is the last argument. I like to use "find", but you can manually put a list of files there.

That command will generate the .pot file. Easy, isn't it?

How to add translations

Now you want to get a translation in an specific language. You need to generate a .po for it.

Go to the po/ folder and run this command (again, inside/outside scratchbox doesn't matter):

msginit --input=PROJECTNAME.pot --locale=LOCALE

For instance, if I want a .po file for the Spanish translation of my project Mussorgsky, I write:

msginit --input=mussorgsky.pot --locale=es_ES

It will generate a file "es.po" and translators can/must work on that file.

IMPORTANT: by default the generated .po file declares its charset as ASCII. This means that translations with non-ascii characters will provoke an error in the program. To solve this, set the charset to 'utf-8':

"Language-Team: Spanish\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"  <--- THIS MUST BE UTF-8

If you don't know the locale of a language, can check this list.

Include translations in your installation

This part is more tricky, because it depends on your build system. I'll explain how it goes with the common python 'distutils' (what makes the usual python setup.py install command work). In distutils all the magic happens in the setup.py file.

First of all, if in the previous steps you have added the i18n.py file to your source tree, don't forget to include it in your data_files list!

Then, copy to the root folder of your project the file msgfmt.py. It is a program that translates the .po files into the binary .mo files that the application needs.

In your setup.py file you need to add all this code. It "overloads" the default 'build' and 'install' instructions to include the translations in the process. Now in 'build' time it will generate the .mo files (using the msgfmt.py we added just now), and in 'install' time it will include the '.mo' files in the data_files list. The code is copied from this a setup.py file.

from distutils.core import setup
from distutils import cmd
from distutils.command.install_data import install_data as _install_data
from distutils.command.build import build as _build
 
import msgfmt
import os

This command compiles every .po file under the po/ folder into a .mo (the .mo files will be under build/locale/)

class build_trans(cmd.Command):
    description = 'Compile .po files into .mo files'
    def initialize_options(self):
        pass
 
    def finalize_options(self):
        pass
 
    def run(self):
        po_dir = os.path.join(os.path.dirname(os.curdir), 'po')
        for path, names, filenames in os.walk(po_dir):
            for f in filenames:
                if f.endswith('.po'):
                    lang = f[:len(f) - 3]
                    src = os.path.join(path, f)
                    dest_path = os.path.join('build', 'locale', lang, 'LC_MESSAGES')
                    dest = os.path.join(dest_path, 'mussorgsky.mo')
                    if not os.path.exists(dest_path):
                        os.makedirs(dest_path)
                    if not os.path.exists(dest):
                        print 'Compiling %s' % src
                        msgfmt.make(src, dest)
                    else:
                        src_mtime = os.stat(src)[8]
                        dest_mtime = os.stat(dest)[8]
                        if src_mtime > dest_mtime:
                            print 'Compiling %s' % src
                            msgfmt.make(src, dest)

Now we append the previous command to the 'build' command.

class build(_build):
    sub_commands = _build.sub_commands + [('build_trans', None)]
    def run(self):
        _build.run(self)

Installation time: put every .mo file under build/locale in the data_files list (the list of things to be installed) and call the default 'install' operation

class install_data(_install_data):
 
    def run(self):
        for lang in os.listdir('build/locale/'):
            lang_dir = os.path.join('share', 'locale', lang, 'LC_MESSAGES')
            lang_file = os.path.join('build', 'locale', lang, 'LC_MESSAGES', 'mussorgsky.mo')
            self.data_files.append( (lang_dir, [lang_file]) )
        _install_data.run(self)

Finally, add the new commands in distutils...

cmdclass = {
    'build': build,
    'build_trans': build_trans,
    'install_data': install_data,
}

And don't forget to add this in your setup function, in the setup.py

setup(name         = 'your project',
      ...
     license      = 'GPL v2 or later',
     data_files   = DATA,
     scripts      = SCRIPTS,
     cmdclass     = cmdclass   <----- DON'T FORGET THIS
     )

Ok, now everything should be ready. If you run

python2.5 setup.py build

the terminal will print things like

compiling es.po
compiling de.po
...

Usually dpkg-buildpackage uses python2.5 setup.py build and install, so everything should work out-of-the-box with your previous package configuration.

Feel free to complete/correct this wiki page with your own experience.