Support for ICU format translation files

[wolfadex]

At work we’re starting our first Elm project and I’ve been tasked with figuring out our options for i18n. We need to support being able to change the language without having a network connection, so all languages must be pre-loaded into the app. The 2 options we’re looking at currently are one use vanilla Elm and have something of the form

translate : Language -> Key -> String
translate lang =
    case lang of
        EnUs -> enUsDictionary
        SpUs -> spUsDictionary
        ...

type Language = EnUs | EnSp | etc

type Key
    = HelloWorld
    | ItemCount Int


enUsDictionary : Key -> String
enUsDictionary key =
    case key of
        HelloWorld ->
            "Hello, World!"

        ItemCount count ->
            case count of
                0 -> "You have no items."
                1 -> "You have one item."
                _ -> "You have many items." 

spUsDictionary : Key -> String
...

or option two use ICU format files http://userguide.icu-project.org/.

I’ve done a little searching and haven’t found any current support for ICU format. There is https://github.com/kirchner/elm-icu though it hasn’t been updated to support 0.19. There’s also ChristophP/elm-i18next but we need to support offline first. There’s also elm-i18n-module-generator but it doesn’t look like it supports ICU and hasn’t been updated for 0.19 yet either.

Does anyone have any suggestions or relevant experience with these or other options that they could speak to or recommend?

[edkv]

Here’s what I’m using:

-- Locale.elm

type Locale
    = En
    | De
    | ...

-- Text.elm

type alias Text =
    { home : String
    , about : String
    , contact : String
    , ...
    }

get : Locale -> Text
get locale =
    case locale of
        Locale.En ->
            en

        Locale.De ->
            de

en : Text
en =
    { home = "Home"
    , about = "About"
    , contact = "Contact"
    , ...
    }

Use it like this: (Text.get locale).home.

You can also put functions that accept some useful arguments into this record, for example:

type alias Text =
{ ...
, emailSent : String -> String
, ...
}

en : Text
en =
    { ...
    , emailSent = 
        \email -> 
            "We have sent an email with a confirmation link to "  
                ++ email
                ++ ", please click on it to finish your registration."
    , ...
    }

This approach is really simple, but works well, and I’m very happy with it!

[allanderek]

You can still use

@wolfadex

ChristophP/elm-i18next

if you need to support offline. It will of course depend upon how you are initially serving the app, but you can for example embed the translations json in the HTML page and pass them in as a program flag.

You will also want to consider who will write the translations and how you are going to get them from there into the app. JSON might be good for this. But even if you want to actually store the translations within Elm code you can store the json as string literals.

However, one thing you give up over your Key custom type approach, would be that you have no compile-time guarantee that all translations have all keys, or even that the backup (say English) translation has all the keys that are used in the app.

[wolfadex]

The app is being installed on a TV. It has to go online at least once for activation, but could then go offline for months or more at a time and still has to run during that time. I don’t expect someone will change the language while it’s offline, but it’s possible. I could imagine a support tech going to the TV onsite and changing the language.

[allanderek]

Sorry, what I meant was you could embed all of the language JSONs as a flag to the program (or store them in the Elm source code as string literals). You could embed them as a json dictionary mapping language name (or id) to the set of translations:

{ "en_US": { "helloworld" : "Hello world" }, "sp_US" : { "helloworld" : "Hola Mundo" } }

Then you decode the flag as Decode.dict I18Next.translationsDecoder, with the result being a dictionary from the language id to set of translations.

That would mean the language could be changed offline, because all of the languages are embedded into the app.

[bitterjug]

We use ICU format messages to translate our large app at Featurespace. Here’s how:

Even before internationalising the app, we kept all our string literals separate from the pages, in modules called I18n.elm. We kept those modules but, instead of strings, they now export Translatable values. Every translatable has a key, and default message in English:

sorting : Translatable
sorting =
    "Sorting"
        |> i18n.key "sorting"
        |> i18n.translate
        

They may also include a maximum length and a comment to the translator, like this:

sortingSecondary : Translatable
sortingSecondary =
    "sec"
        |> i18n.key "sortingSecondary"
        |> i18n.comment "Very restricted space"
        |> i18n.maxLength 3
        |> i18n.translate

We keep key values unique within an I18n module. In the above examples, i18n is a record instantiated in the module like this:

module Page.Table.I18n exposing (sorting, sortingSecondary)

import Helper.Translate as Translate exposing (Translatable)

i18n : Translate.Functions
i18n =
    Translate.init "Page.Table"

The function i18n.key prepends the module scope identifier, so the full key value of the translatable sortingSecondary is “Page.Table.sortingSecondary”.

To include a translatable message in a page we need a dictionary from keys to messages in the required language which we fetch from the server at load time and keep in a session object together with the currently selected language.

secondaryTitle : Session -> Html msg
secondaryTitle session =
    I18n.sortingSecondary
        |> Translate.translate session 
        |> Html.text

This looks up the fully qualified key sortingSecondary in the currently selected dictionary and returns the message text. If it can’t find the key in the dictionary it will return the English message directly from the translatable. So we don’t need an English translation file: an empty dictionary is enough to develop and test the app.

Plurals

We write plurals in ICU message format, like this:

results : TranslatableWithPlural
results =
     "{count, plural, one {{count} result} other {{count} results}}"
         |> i18n.key "results"
         |> i18n.translatePlural

To use these we provide the value for the {count} variable (here resultCount):

Translate.translatePlural session I18n.results resultCount []

The [] at the end is an empty list of key value pairs to substitute for other {key] variables.

In addition to keys and translated messages, our language files include a rule to derive plural groups (‘one’, ‘two’, ‘few’,‘many’, ‘other’) from numbers. When we decode the language file we put the keys and messages into a dictionary and parse the plural rule to give an Elm function we can use to chose the right part of the message text.

Translating

When we’re ready to translate we run a script that compiles all the I18n modules and, in a similar way to the node elm-test runner, builds an elm module that imports them and extracts the keys, messages, comments and maximum length values. This module gets executed on Node and its output becomes the source files for our translation company. The translation company we’re using at the moment wants Gettext PO files, so we convert the ICU messages to PO!

When we receive the translated PO files we run another script that converts them to JSON with messages in ICU format, and integrates them with the language definitions (which include the plural rules). The resulting JSON files are the resources that our app fetches and decodes to get the language dictionaries to translate messages. We write an ICU message format parser but its not (yet) a separate package.