#1412 ✓resolved
Nebu

play.i18n.Lang does not resolve near matches for locales

Reported by Nebu | February 3rd, 2012 @ 08:01 PM | in 1.2.5 (closed)

Framework version: git commit dbb851d9b3e8c72ad0ab2356d74037468209a679
Platform you're using: Linux

Details:
The standard behaviour for Java locale resource files is such that when the user requests a locale "en_CA", the application will first try to use "messages.en_CA", then "messages.en" and finally the default "message" file. Play!'s behaviour is such that it will only try "messages.en_CA" and then immediately the default "message" file. This makes it more difficult to handle i18n support for country-specific dialects of a given language.

Comments and changes to this ticket

  • Matthias Kurz
  • Morten Kjetland

    Morten Kjetland February 12th, 2012 @ 02:43 PM

    • State changed from “new” to “inprogress”
    • Assigned user set to “Morten Kjetland”
    • Milestone set to 1.2.5
    • Milestone order changed from “1024” to “0”
  • Peter Hilton

    Peter Hilton March 28th, 2012 @ 07:55 PM

    This came up as an issue on today’s training course in Switzerland, where a messages.fr_CH might only contain a small number of exceptions for a big messages.fr file.

  • Theodore Nguyen-Cao

    Theodore Nguyen-Cao March 31st, 2012 @ 10:01 PM

    I'm not sure if this belongs here but here's another use case:

    java.util.Locale supports a 3rd optional attribute, variant, in addition to language and country.

    For example, en_US_CA would represent a locale specifying English in the US for California.

    I would expect Play look for message keys loaded from the following message files in order of precedence:

    messages.en_US_CA
    messages.en_US
    messages.en
    messages

  • Nebu

    Nebu April 5th, 2012 @ 05:28 PM

    Note that the changes in the pull request at https://github.com/playframework/play/pull/449 introduces a slight subtly to the API.

    The design in the pull request is such that "change" and "set" now have slightly different semantics. "set" will attempt an exact match only, and then return a boolean to indicate whether any match was found. In contrast, "change" performs the "nearest match" algorithm, and returns void, because it always succeeds.

    This exposes a new problem, which is that "change" assumes that Response.current() will return a non-null value (since the code immediately dereferences it). A use-case where this assumption is false: If you design a Job that sends out e-mails every week, and you want the e-mails to be in the language specified by the recipient's profile, there is no response object, and so the "change" method will always fail with a NullPointerException.

    The "set" method doesn't assume a Response, but it also doesn't perform the "nearest match" algorithm.

    Cleanest fix looks like it might be to add a null check in the "change" method.

  • Morten Kjetland

    Morten Kjetland April 13th, 2012 @ 01:13 PM

    • Milestone order changed from “19” to “0”

    I think there should be created a new ticket for this issue: "This came up as an issue on today’s training course in Switzerland, where a messages.fr_CH might only contain a small number of exceptions for a big messages.fr file.".

  • Peter Hilton

    Peter Hilton April 13th, 2012 @ 01:21 PM

    Morten, perhaps I misunderstood this ticket’s description, since I think this is exactly the same case: the user selects locale fr_CH, and wants Play to first look for a message in messages.fr_CH, then messages.fr, then messages.

    The idea is that a typical French/German application in Switzerland would have almost all of the user-interface messages in messages.fr and messages.de, with a small number of Swiss-French exceptions in messages.fr_CH and unlocalised messages (e.g. an admin interface) in messages.

  • Theodore Nguyen-Cao

    Theodore Nguyen-Cao April 13th, 2012 @ 03:20 PM

    I think we are trying to address this two different things in this ticket. Right now the patch attempts to fallback on the Lang being set on the user's browser. That is, if the Accept-Language header passes en_US, and Play only is configured to support en, we want Lang to fall back to setting Lang as en.

    The fallback for Messages is another story. When the Lang is set as en_US_CA, we want Messages to fallback on where it pulls the messages from:
    messages.en_US_CA
    messages.en_US
    messages.en
    messages

  • Matthias Kurz

    Matthias Kurz April 13th, 2012 @ 03:52 PM

    There is some work done for message-fallbacks in ticket #924 already, maybe it helps.

  • Nebu

    Nebu April 13th, 2012 @ 10:01 PM

    Theodore,

    I hadn't considered the case where the languages listed in application.conf might differ from messages file available, but after thinking about it some more, I can't think of a usecase where you would want these two lists to differ.

    If you have your files

    messages.en_US_CA
    messages.en_US
    messages.en
    messages
    messages.fr_CA_QC //I don't know if this really exists, I'm just making it up, but I hope it gets the point across.
    messages.fr_CA
    messages.fr
    messages

    then but your application.conf set to only en_US and fr_CA, then it seems like contents of messages.en_US_CA and messages.fr_CA_QC are never accessible?

    And if you have the reverse situation where you have application.conf set to only en_US_CA and fr_CA_QC, and the user sends an HTTP request for "fr_CA", surely we'd want to provide "fr_CA" if it's available, rather than falling back to the default, which is en_US_CA in this case.

  • Morten Kjetland

    Morten Kjetland April 17th, 2012 @ 08:31 PM

    I've spent quite some time now (over multiple days..) with the pull request.
    First I modified it to behave as before (change).

    Then I discovered that it breaks auto-test for just-unit-tests-project.

    It works fine when running in browser but fails when using the headless auto-test runner.

    After a lot of confusion, this is the problem:

    binding.test.html:

            <!-- Now unbind dates -->
            open('/databinding/changeLanguage/en/')
            open('@{Application.book(new java.util.Date(2879877856556))}')
            assertTextPresent('Booked at 04/04/61 !!')
    

    The confusing part: The link above is generate when the selenium template page/html is generated.
    The date generated in the url is using the current lang/locale WHEN the selenium test is generated/rendered.

    the changeLanguage page requests sets/changes the current language for all the requests to come - this selected lang/locale affects how the date-in-url generated before (when selenium is rendered) is parsed.

    When running with current play-master branch, the current lang (when rendering selenium page with headless auto-test) is en.

    when doing the same, using this modified pullrequest-fix, the current lang is fr.

    This breaks the test and is evidence that something strange is happening.

    It might be related to the fact that just-unit-tests-project uses play.pool=2 in test mode - and that something is happing at the same time, which then confuses the client current language cookie value..

    But the conclusion is that Something is not working correct... It should behave exactly the same as before for this test app.

  • Play Duck
  • Play Duck

    Play Duck April 18th, 2012 @ 08:13 PM

    (from [96f194093c5e21b7c8bce52cd448a79474620236]) [#1412] Modified 'Implemented resolving nearest matching locale.' to not break API + added tests Also had to fix an existing test because it was unpredictable - It tests the same thing after the modification but in a predictable way
    https://github.com/playframework/play/commit/96f194093c5e21b7c8bce5...

  • Play Duck
  • Play Duck

    Play Duck April 18th, 2012 @ 08:13 PM

    (from [b02e65d4999e1deb97a839e30f4a8b930c0abf81]) [#1412] Modified 'Implemented resolving nearest matching locale.' to not break API + added tests Also had to fix an existing test because it was unpredictable - It tests the same thing after the modification but in a predictable way
    https://github.com/playframework/play/commit/b02e65d4999e1deb97a839...

  • Morten Kjetland

    Morten Kjetland April 18th, 2012 @ 08:16 PM

    • State changed from “inprogress” to “resolved”

    Thanks for pullrequest.

    I had to modify it a little bit to not break the api compability.
    I've also added tests.

    Fixed in master and 1.2.x

  • Neeme Praks

    Neeme Praks May 10th, 2012 @ 12:42 PM

    This patch is broken.
    It assumes that "Accept-Language" header values have the same format as Java locale (e.g. "en_GB") while the real format uses a dash instead of an underscore (e.g. "en-GB").
    Fix: https://github.com/playframework/play/pull/509

  • Morten Kjetland

    Morten Kjetland May 10th, 2012 @ 12:51 PM

    • State changed from “resolved” to “inprogress”

    Will have another look

  • Neeme Praks

    Neeme Praks May 10th, 2012 @ 01:48 PM

    That Lang class is broken also in a bigger way - it assumes that whatever language the user wants to use, it is supported by the JVM (included in Locale.getAvailableLocales() locale list).

    See method Lang.getLocale(String lang), Lang.java#192
    Why doesn't it just return a new Locale instance?

    From Play perspective, it should only matter that particular Locale is configured in Play config (application.langs); JVM locales should not matter.
    And, on line #189 it returns Locale.getDefault() - this is also not desirable, Play should return it's own default.

    Should I open a new ticket for this?

  • Nicolas Leroux

    Nicolas Leroux May 10th, 2012 @ 06:38 PM

    How does this issue relate to [#924]?

  • Nicolas Leroux

    Nicolas Leroux May 10th, 2012 @ 07:29 PM

    Please open another issue for the Locale. Thanks again.

  • Nicolas Leroux

    Nicolas Leroux May 10th, 2012 @ 07:41 PM

    • State changed from “inprogress” to “resolved”
  • Matthias Kurz

    Matthias Kurz August 13th, 2013 @ 03:39 PM

    fyi: I opened a new ticket #1695 ("fallback when retrieving messages") including a pull request which addresses some comments here.

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

<h2>Play framework</h2>

Play makes it easier to build Web applications with Java. It is a clean alternative to bloated Enterprise Java stacks. It focuses on developer productivity and targets RESTful architectures. Learn more on the <a href="http://www.playframework.org">http://www.playframework.org</a> website.<br><br>

<h2>Source code is hosted on github</h2>Check out our repository at <a href="http://github.com/playframework/play">http://github.com/playframework/play</a><br><br>

<h2>Contributing, creating a patch</h2> Please read the <a href="http://play.lighthouseapp.com/projects/57987/contributor-guide">contributor guide</a><br><br>

<h2>Reporting Security Vulnerabilities</h2> Since all bug reports are public, please report any security vulnerability directly to <em>guillaume dot bort at gmail dot com</em>.<br><br>

<h2>Creating a bug report</h2> Bug reports are incredibly helpful, so take time to report bugs and request features in our ticket tracker. We’re always grateful for patches to Play’s code. Indeed, bug reports with attached patches will get fixed far quickly than those without any.<br><br>

Please include as much relevant information as possible including the exact framework version you're using and a code snippet that reproduces the problem.<br><br>

Don't have too much expectations. Unless the bug is really a serious "everything is broken" thing, you're creating a ticket to start a discussion. Having a patch (or a branch on Github we can pull from) is better, but then again we'll only pull high quality branches that make sense to be in the core of Play.

Tags

Referenced by

Pages