A library to use GNU Gettext as translation solution for clojure projects. This library is meant to extract translatable strings from your codebase (clj, cljs and cljc), generate a PO Template file and parsing PO (translation) files.
Gettext is an old but great solution for managing translatable codebases. In comparison to the i18n or similar ways:
- No need to invent unique keys anymore for every piece of text that needs translations. The string itself is the key.
- Codebase and translation files stay in sync via tooling, not developers. Since the string is the key, if the string changes, translations must follow.
- Defered development time vs translation time. When developing you don't want to have to manage localisation files. Gettext defers this process to a more appropriate time, like before releases for example. The developer just has to write text in the codebase in whatever language the team is comfortable with.
- Outsource translations easily to professional translators. PO and POT files are an industry standard for translation bureaus.
There are some libraries in the wild that do some parts of what Pottery tries to achieve. However most of them either had an incomplete API, or not a functional one. Pottery is a functional approach to using gettext, leaving the developer in the front seat when translating his or her application.
Add this dependency to your project:
[brightin/pottery "0.0.4"]And require it:
(require '[pottery.core :as pottery])This library is only meant to generate and read PO files. Translating in and of itself is up to you. There are plenty of great libraries out there for this with which Pottery has no intention to compete. There is a full example to demonstrate how this could work.
(defn tr [s & args]
(translate-string-to-current-language s args))Throughout your code this tr function will be used, and now Pottery will manage the gettext part.
Call the scan function from the REPL:
(pottery/scan-codebase!)This will scan the codebase for any translatable strings and write the PO template file to resources/gettext/template.pot. This is not limited to clojure only files, strings in .cljc and .cljs files are also extracted.
See Configuration / Extra features for more configurations of the scanner.
For every string in the codebase there needs to be a translation to other languages. These get written in <lang>.po files. There are various tools available to achieve this. The most popular and recomended one would the free GUI tool Poedit. Use Poedit to load the PO template file, generate the translation files and interactively translate all the strings.
Pottery parses the translation files from step 2 into a dictionary with the form: {"Original String" "Translated string"}.
(pottery/read-po-file (io/resource "gettext/es.po"))
=> {"Hello" "Hola"}These can act as dictionaries for your translation function in step 0.
Whenever strings change in the codebase, re-scan at will as in step 1. This will replace the old template file, ensuring changed strings are changed, and removed strings are removed. Next merge that new template file in the translation files. In Poedit you can achieve this by going to Catalog -> Update from POT file. After saving in Poedit you can also purge deleted strings from the po file via Catalog -> Purge Deleted Translations. You translation files are now in sync!
This library supports most features that gettext supports. More will be added at request, feel free to create an issue for them. The following examples use the default scanner:
In your code:
(tr "Hello %s" name)The output after parsing a PO file:
{"Hello %s" "Hoi %s")In your code:
(trn ["One horse" "%s horses"] horse-count)The output after parsing a PO file:
{["One horse" "%s horses"] ["Een paard" "%s paarden"]}Not implemented yet
Sometimes the same string can have multiple translations according to context. Gettext has support for this as a translation context. This has not been implemented yet, please submit an issue if the need arises.
Translator notes are great to provide context for the string to be translated. Since we defer development time and translation time, there might be confusion due to the lack of context. You can add notes to strings as metadata:
^{:notes "Abbreviation of horsepower, not healthpoints"}
(tr "Hp")These translations will be extracted and added to the template file. At translation time these will be visible in, for example, Poedit:
The scan function takes a few options:
| Key | Description | Default |
|---|---|---|
| :dir | The source directory which needs to be scanned | "src" |
| :template-file | The output template file | "resources/gettext/template.pot" |
| :extract-fn | The function which maps any expression found to a string, strings or nothing | pottery.scan/default-extractor |
A function which takes a clojure expression as data, and returns a string (single) or a vector of strings (plural).
A simple extractor would look like this:
(defn my-extract-fn [expr]
(when (and (list? expr)
(= 'tr (first expr))
(string? (second expr)))
(second expr)))
(my-extract-fn '(tr "Some string")) => "Some string"
(my-extract-fn '(inc 12)) => nilIt may get tedious to write a good function when you have multiple translation functions that have multiple arities. Pottery offers a shorthand using core.match to declare patterns in which translation functions are called.
(pottery/make-extractor
['tr (s :guard string?) & _] s
['tr [s & _]] s
['trn [s1 s2 & _]] [s1 s2]
...)It's a good idea to also warn when extraction did not pass any of the patterns, as a safe guard. As last pattern of the match sequence, you can provide:
(pottery/make-extractor
... patterns
[(:or 'tr 'trn) & _] (pottery.scan/extraction-warning
"Translation function called but no string could be extracted:"))When the expression starts with the familiar function call, but did not match any pattern the warning will be printed with the failing expression. It's a good idea to write patterns for common "mistakes".
The default extractor is defined as such:
(make-extractor
['tr (s :guard string?) & _] s
['trn [(s1 :guard string?) (s2 :guard string?) & _] _] [s1 s2]
[(:or 'tr 'trn) & _] (extraction-warning
"Could not extrapolate translation string for the form:"))It's a minimal extractor, which will match these clojure forms:
(tr "My string" & args) => "My string"
(trn ["Singular" "Plural" & args] count) => ["Singular" "Plural"]
(tr some-var) => nil ;; And a warning is printed
(trn ["String"]) => nil ;; And a warning is printedSee the full example for a simple but complete example of integrating Pottery in your application.
This next example shows an approach for compiling dictionaries into the clojurescript source. We use a macro to read the po file and compile the translation dictionary at compile time, which will result in the dictionary being hardcoded in the source.
;; src/app/tr.clj
(ns app.tr
(:require [pottery.core :as pottery]))
(defmacro inline-dict [filename]
(pottery/read-po-file filename))
;; src/app/frontend/tr.cljs
(ns app.frontend.tr
(:require-macros [app.tr :refer [inline-dict]]))
(def DICT
{:en (inline-dict "gettext/en.po")
:it (inline-dict "gettext/it.po")})If you're using shadow-cljs you can
leverage shadow.resource/slurp-resource to read the contents of the PO file
and also record the file as a recompilation dependency.
;; src/app/tr.clj
(ns app.tr
(:require [pottery.core :as pottery]
[shadow.resource :as res]))
(defmacro inline-dict [filename]
(pottery/read-po-str (res/slurp-resource &env filename)))With Pottery, translation is done as a function but the function call should be regarded as data. The scanner reads source code with the clojure reader to figure out which strings are to be translated. For example:
;; Bad
(let [msg (if available? "Available" "Unavailable")]
(tr msg))
;; Good
(if available?
(tr "Available")
(tr "Unavailable"))In the first case only "msg" or nothing would get extracted. In the second case all strings are extracted.
Thanks to Carlo Sciolla for coming up with the name of this project!
Copyright © 2024 Brightin
This project uses the Hippocratic License, and is thus freely available to use for purposes that do not infringe on the United Nations Universal Declaration of Human Rights.