Integrating Django i18n with Jinja2 and Vue.js
It is trivial to add translations to a Django project that uses built in Django templates, however if you're using other template engines like Jinja2 it might take some effort to make those two work together. Things can get tricky if you want to manage translations for both the frontend and the backend using the same Django tools. I'm not going to cover the basics of i18n in Django, as there is already a great deal of information at Django website, I'm going to talk about the stuff that is not integrated out of the box instead.
Jinja2
Jinja2 doesn't support Django templates i18n tags or any other i18n functionality out of the box. There is also a command line tool in Django that scans your project and produces a .po file with translation strings. Unfortunately, it only works with the default Django templates out of the box. Here's how to solve those problems.
Jinja2 has a i18n extension that adds a {% trans %} block and a _() function. The first one looks like a similar tag in Django templates, but those are two different things. You can read more about them here. Here's an example:
# Django template
{% trans 'hello world' %}
# Jinja2 template
{% trans %}
hello world
{% endtrans %}This extension is disabled by default, so we need to add some configuration first. Jinja2 is customized with the help of the Environment class. When Django is used with Jinja2 it uses a default environment, which needs to be overriden in order to enable the extension. I assume you already have a custom Jinja2 environment in your project, otherwise you can learn how to configure a custom environment here. Here's an example of a basic Jinja2 environment file with i18n support:
from django.utils.translation import gettext, ngettext
from jinja2 import Environment
def environment(**options):
# i18n extension
options['extensions'] = ['jinja2.ext.i18n']
env = Environment(**options)
# i18n template functions
env.install_gettext_callables(gettext=gettext, ngettext=ngettext,
newstyle=True)
return envAt this point Django will translate the i18n strings in Jinja2 templates if those strings are present in .po and .mo translation files. We can collect the strings manually but it takes time and is error-prone. Let's leverage Django makemessages command to do this for us. As I said, it doesn't work with Jinja2 templates out of the box and there is no clean way to enable the support, so we'll resort to a few hacks to make this work. As I searched for the solution I stumbled upon a django-jinja project which solved this problem, but instead of using the whole project I'm going to extract this single feature. Put the code inside the {project_root}/{app_name}/management/commands/makemessages.py file:
import re
from django import VERSION as DJANGO_VERSION
from django.core.management.commands import makemessages
from django.template.base import BLOCK_TAG_START, BLOCK_TAG_END
if DJANGO_VERSION[:2] < (1, 11):
from django.utils.translation import trans_real
else:
from django.utils.translation import template as trans_real
strip_whitespace_right = re.compile(r"(%s-?\s*(trans|pluralize).*?-%s)\s+" % (BLOCK_TAG_START, BLOCK_TAG_END), re.U)
strip_whitespace_left = re.compile(r"\s+(%s-\s*(endtrans|pluralize).*?-?%s)" % (BLOCK_TAG_START, BLOCK_TAG_END), re.U)
def strip_whitespaces(src):
src = strip_whitespace_left.sub(r'\1', src)
src = strip_whitespace_right.sub(r'\1', src)
return src
class Command(makemessages.Command):
def handle(self, *args, **options):
old_endblock_re = trans_real.endblock_re
old_block_re = trans_real.block_re
old_constant_re = trans_real.constant_re
old_templatize = trans_real.templatize
# Extend the regular expressions that are used to detect
# translation blocks with an "OR jinja-syntax" clause.
trans_real.endblock_re = re.compile(
trans_real.endblock_re.pattern + '|' + r"""^-?\s*endtrans\s*-?$""")
trans_real.block_re = re.compile(
trans_real.block_re.pattern + '|' + r"""^-?\s*trans(?:\s+(?!'|")(?=.*?=.*?)|\s*-?$)""")
trans_real.plural_re = re.compile(
trans_real.plural_re.pattern + '|' + r"""^-?\s*pluralize(?:\s+.+|-?$)""")
trans_real.constant_re = re.compile(r""".*?_\(((?:".*?(?<!\\)")|(?:'.*?(?<!\\)')).*?\)""")
def my_templatize(src, origin=None, **kwargs):
new_src = strip_whitespaces(src)
return old_templatize(new_src, origin, **kwargs)
trans_real.templatize = my_templatize
try:
super(Command, self).handle(*args, **options)
finally:
trans_real.endblock_re = old_endblock_re
trans_real.block_re = old_block_re
trans_real.templatize = old_templatize
trans_real.constant_re = old_constant_reThis code, basically, overrides the builtin makemessages command to add Jinja2 support and falls back to the default command when it deals with Django templates. In order to collect the translation strings inside a .po file, go to an app directory that you want to translate, make sure there is a locale directory inside this app ({app_name}/locale) and run django-admin makemessages -l {locale_name}, where {locale_name} is an actual locale name, like de. If you followed the steps, you will have a {app_name}/locale/{locale_name}/LC_MESSAGES/django.po translation file that contains the strings from Jinja2 templates that you'll need to translate. Having a .po file is not enough though, so once you've provided the translations, run django-admin compilemessages from the app directory – this command will produce a .mo file which is going to be used by Django at runtime. At this point, the Jinja2 18n functionality is pretty much on the same level as the Django templates. Awesome!
Vue.js
There is a myriad of i18n libraries on the client side, but with this approach we'll have to manage the translations for the frontend and the backend separately. Ideally, we'd like to have a single tool to do the work. Django docs has a whole section about working with translations on the client side. Basically, it provides a view that generates a JavaScript code. This code adds i18n functions to the client and an object that contains all the translations from the .po, .mo files. It also supports a collection of translation strings from the JavaScript files.
To enable this functionality, add this to urls.py in your project:
from django.views.i18n import JavaScriptCatalog
urlpatterns = [
path('jsi18n/', JavaScriptCatalog.as_view(), name='javascript-catalog'),
]And this line to your base.html:
<script type="text/javascript" src="{% url 'javascript-catalog' %}"></script>If you don't have url helpers in your Jinja2 configuration, set the src attribute manually to jsi18n/.
Now, when you load a page you have an access to gettext, interpolate and a bunch of other i18n functions. Adapt your strings to use those functions and when finished Django will be able to collect those strings into a .po file. Once again go to an app folder and run a django-admin makemessages -d djangojs -l {locale_name} command, where -d djangojs tells Django to scan the JavaScript files. Once you are finished with the translations you'll need to run a django-admin compilemessages command to produce a .mo file. At this stage, JavaScript code is internationalized and the translations for both the frontend and the backend are managed by a single tool – nice!
Bonus: automatic language detection & optimizations
The client side i18n integration works smoothly, but we still need to make an extra request to get the client side code. This is fine in most scenarios. However, we use djaongo-assets package which takes all the assets and minifies them into a single file to reduce the size and the number of requests to the server. We also leverage CDN services to distribute assets to datacenters around the globe. Out of the box JavaScriptCatalog view generates a code that contains translation strings for a single language only. We can't really cache this if we'd like to support automatic language detection on page load. We can solve those problems by moving the code into a Django management command. It would generate a static file with the i18n code that includes translations for every locale that we support. We can then minify the assets at a build time and serve them via CDN, while a client will be able to pick the right translations because they are all included in this file.
JavaScriptCatalog is a Django view which means it expects a HTTP request, however we'd like to call it as a function without providing a request object. Extracting functionality from this class to create a standalone function would be a tedious process so I settled on subclassing it and creating a method that leverages the JavaScriptCatalog methods to generate the JavaScript code. I duplicated the JavaScript code source in order to make modifications to support the automatic language detection. Finally, I adapted the resulting function into a management command. Here's what I ended up with:
"""Command to generate JavaScript file used for i18n on the frontend"""
import json, os
from django.conf import settings
from django.core.management.base import BaseCommand
from django.views.i18n import JavaScriptCatalog, get_formats
from django.template import Context, Engine
from django.utils.translation.trans_real import DjangoTranslation
js_catalog_template = r"""
{% autoescape off %}
(function(globals) {
var activeLang = navigator.language || navigator.userLanguage || 'en';
var django = globals.django || (globals.django = {});
var plural = {{ plural }};
django.pluralidx = function(n) {
var v = plural[activeLang];
if(v){
if (typeof(v) == 'boolean') {
return v ? 1 : 0;
} else {
return v;
}
} else {
return (n == 1) ? 0 : 1;
}
};
/* gettext library */
django.catalog = django.catalog || {};
{% if catalog_str %}
var newcatalog = {{ catalog_str }};
for (var ln in newcatalog) {
django.catalog[ln] = newcatalog[ln];
}
{% endif %}
if (!django.jsi18n_initialized) {
django.gettext = function(msgid) {
var lnCatalog = django.catalog[activeLang]
if(lnCatalog){
var value = lnCatalog[msgid];
if (typeof(value) != 'undefined') {
return (typeof(value) == 'string') ? value : value[0];
}
}
return msgid;
};
django.ngettext = function(singular, plural, count) {
var lnCatalog = django.catalog[activeLang]
if(lnCatalog){
var value = lnCatalog[singular];
if (typeof(value) != 'undefined') {
} else {
return value.constructor === Array ? value[django.pluralidx(count)] : value;
}
}
return (count == 1) ? singular : plural;
};
django.gettext_noop = function(msgid) { return msgid; };
django.pgettext = function(context, msgid) {
var value = django.gettext(context + '\x04' + msgid);
if (value.indexOf('\x04') != -1) {
value = msgid;
}
return value;
};
django.npgettext = function(context, singular, plural, count) {
var value = django.ngettext(context + '\x04' + singular, context + '\x04' + plural, count);
if (value.indexOf('\x04') != -1) {
value = django.ngettext(singular, plural, count);
}
return value;
};
django.interpolate = function(fmt, obj, named) {
if (named) {
return fmt.replace(/%\(\w+\)s/g, function(match){return String(obj[match.slice(2,-2)])});
} else {
return fmt.replace(/%s/g, function(match){return String(obj.shift())});
}
};
/* formatting library */
django.formats = {{ formats_str }};
django.get_format = function(format_type) {
var value = django.formats[format_type];
if (typeof(value) == 'undefined') {
return format_type;
} else {
return value;
}
};
/* add to global namespace */
globals.pluralidx = django.pluralidx;
globals.gettext = django.gettext;
globals.ngettext = django.ngettext;
globals.gettext_noop = django.gettext_noop;
globals.pgettext = django.pgettext;
globals.npgettext = django.npgettext;
globals.interpolate = django.interpolate;
globals.get_format = django.get_format;
django.jsi18n_initialized = true;
}
}(this));
{% endautoescape %}
"""
class Command(BaseCommand):
"""Generate JavaScript file for i18n purposes"""
help = 'Generate JavaScript file for i18n purposes'
def add_arguments(self, parser):
parser.add_argument('PATH', nargs=1, type=str)
def handle(self, *args, **options):
contents = self.generate_i18n_js()
path = os.path.join(settings.BASE_DIR, options['PATH'][0])
with open(path, 'w') as f:
f.write(contents)
self.stdout.write('wrote file into %s\n' % path)
def generate_i18n_js(self):
class InlineJavaScriptCatalog(JavaScriptCatalog):
def render_to_str(self):
# hardcoding locales as it is not trivial to
# get user apps and its locales, and including
# all django supported locales is not efficient
codes = ['en', 'de', 'ru', 'es', 'fr', 'pt']
catalog = {}
plural = {}
# this function is not i18n-enabled
formats = get_formats()
for code in codes:
self.translation = DjangoTranslation(code, domain=self.domain)
_catalog = self.get_catalog()
_plural = self.get_plural()
if _catalog:
catalog[code] = _catalog
if _plural:
plural[code] = _plural
template = Engine().from_string(js_catalog_template)
context = {
'catalog_str': json.dumps(catalog, sort_keys=True, indent=2),
'formats_str': json.dumps(formats, sort_keys=True, indent=2),
'plural': plural,
}
return template.render(Context(context))
return InlineJavaScriptCatalog().render_to_str()
You can test the command by calling ./manage.py generate_i18n_js {app/static/path} where {app/static/path} is the path where to put the generated file. Finally, you might want to include this file in your base.html template to see if it works:
...
<head>
...
<script src="{a_url_to_the_i18n_file}"</script>
...
</head>
...That's it! Besides having a convenient system to manage the translations, the code is fast and the automatic language detection still works.
Summary
Django has a powerful and a somewhat easy to use i18n library that works pretty well with the default template engine, but needs a bit of work to adapt it to other template engines. It is great that it also handles the client side part, but the downside is that it results in two separate .po files to maintain: one for the backend and one for the frontend. This is not critical, but could've been easier I suppose.