#2071 ✓resolved
Robert Rettig

precompile yaml templates for initial data usable in production

Reported by Robert Rettig | November 24th, 2016 @ 11:06 AM | in 1.4.4 (closed)

Framework version: 1.*.*
Platform you're using: windows|linux

Reproduction steps: create some yaml e.g. conf/initial-data.yml which use template tags, precompile your application for production usage.
While bootstrapping your application there might be a condition like 'is database empty' and you call Fixtures.loadModels("initial-data.yml");.
You will receive a error like

11:53:46,469 WARN  ~ Precompiled template /conf/initial-data.yml not found, trying to load it dynamically...
java.lang.RuntimeException: Cannot load precompiled template /conf/initial-data.yml
        at play.templates.BaseTemplate.loadPrecompiled(BaseTemplate.java:44)
        at play.templates.TemplateLoader.load(TemplateLoader.java:70)
        at play.test.Fixtures.loadModels(Fixtures.java:219)
        at play.test.Fixtures.loadModels(Fixtures.java:187)
        at jobs.Bootstrap.doJob(Bootstrap.java:51)
        at play.jobs.Job.doJobWithResult(Job.java:54)
        at play.jobs.Job$2.apply(Job.java:208)
        at play.db.jpa.JPA.withTransaction(JPA.java:258)
        at play.db.jpa.JPA.withinFilter(JPA.java:217)
        at play.db.jpa.JPAPlugin$TransactionalFilter.withinFilter(JPAPlugin.java:298)
        at play.jobs.Job.withinFilter(Job.java:185)
        at play.jobs.Job.call(Job.java:204)
        at Invocation.Job(Play!)
Caused by: play.exceptions.UnexpectedException: Unexpected Error
        at play.libs.IO.readContent(IO.java:111)
        at play.templates.BaseTemplate.loadPrecompiled(BaseTemplate.java:41)
        ... 12 more
Caused by: java.io.FileNotFoundException: File '{your applicationpath}\precompiled\templates\conf\initial-data.yml' does not exist
        at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:292)
        at org.apache.commons.io.FileUtils.readFileToByteArray(FileUtils.java:1815)
        at play.libs.IO.readContent(IO.java:109)
        ... 13 more

Details:
The problem is the compilation phase play precompile. play should somehow get additional templates for precompilation.
I would suggest to use a System environment variable like PLAY_YAML_TEMPLATES (or some name indicating 'optional' templates) and adding some code at the end of play.templates.TemplateLoader#getAllTemplate



        if (System.getenv ( "PLAY_YAML_TEMPLATES" ) != null) {
            for (String yamlTemplate : System.getenv ( "PLAY_YAML_TEMPLATES" ).split ( System.getProperty ( "os.name" ).startsWith ( "Windows" ) ? ";" : ":" )) {
                VirtualFile vf = VirtualFile.open ( Paths.get (yamlTemplate).toFile () );
                if (vf != null && vf.exists ()) {
                    Template template = load ( vf );
                    if (template != null) {
                        template.compile ();
                    }
                }
            }
        }

If you like i create a pull request but maybe you have a better idea where to place some similar logic. Thx

Comments and changes to this ticket

  • Robert Rettig

    Robert Rettig November 24th, 2016 @ 11:19 AM

    Maybe some usage hints here from a real application distribution (maybe as application used in docker).
    Play! 1 Framework might not be platform neutral as it would like to be. We use windows as development platform and we have several fixes to make that work.

    set "PLAY_YAML_TEMPLATES=d:\myapp\conf\contracttypes.yml;d:\myapp\conf\damageevents.yml;d:\myapp\conf\forecasttypes.yml;d:\myapp\conf\initial-accesscontrol.yml;d:\myapp\conf\initial-config.yml;d:\myapp\conf\processtrigger.yml;d:\myapp\conf\riskmap-immobilia.yml;d:\myapp\conf\riskmap-immobilia-testfonds.yml;d:\myapp\conf\sample-data.yml"
    play precompile
    

    Now we prepare to distribute. Maybe in a docker container.

    Somewhere out there someone has a empty database and we initial install/migrate/update the existing database with our liquibase plugin (in docker might be behind the scenes).

    play liquibase:install -Dprecompiled=true
    play liquibase:update -Dprecompiled=true
    

    As the normal application run command to get logging on the console in docker we use

    play run -Dprecompiled
    

    And here we need to initially populate some data if not available.

  • Robert Rettig

    Robert Rettig November 24th, 2016 @ 12:56 PM

    Better would be to support absolute and relative paths, resolved by applicationPath
    and module relative notation as well.

            if (System.getenv ( "PLAY_YAML_TEMPLATES" ) != null) {
                for (String yamlTemplate : System.getenv ( "PLAY_YAML_TEMPLATES" ).split ( System.getProperty ( "os.name" ).startsWith ( "Windows" ) ? ";" : ":" )) {
                    VirtualFile vf = null;
                    for (int retry = 0; ; retry++) {
                        if (retry == 0) {
                            try { vf = VirtualFile.open ( Play.applicationPath.toPath ().resolve ( Paths.get ( yamlTemplate ) ).toFile () );
                            } catch (InvalidPathException invalidPathException) { /*ignored*/}
                        } else if (retry == 1) {
                            vf = VirtualFile.fromRelativePath ( yamlTemplate );
                        } else {
                            vf = null;
                            break;
                        }
                        if (vf != null && vf.exists ()) {
                            Template template = load ( vf );
                            if (template != null) {
                                template.compile ();
                                break;
                            }
                        } else {
                            vf = null;
                        }
                    }
                    if (vf == null) {
                        Logger.warn ( "A template specified by system environment 'PLAY_YAML_TEMPLATES' does not exist or path is wrong. template: '%s'", yamlTemplate );
                    }
                }
            }
    
  • Play Duck
  • Play Duck

    Play Duck December 27th, 2016 @ 11:53 AM

    (from [b021c3c6428cc814c6c5d24a024e4d6ae4aaa1f4]) [#2071] precompile additional templates

    update configuration.textile with the new configuration parameter
    use '.' to separate word in config param name
    https://github.com/playframework/play1/commit/b021c3c6428cc814c6c5d...

  • Play Duck
  • Alex

    Alex January 4th, 2017 @ 07:50 AM

    • State changed from “new” to “resolved”
    • Tag set to fixtures, template, templates
    • Milestone set to 1.4.4

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

<h2>Play framework</h2>

Play makes it easier to build Web applications with Java. It is a clean alternative to bloated Enterprise Java stacks. It focuses on developer productivity and targets RESTful architectures. Learn more on the <a href="http://www.playframework.org">http://www.playframework.org</a> website.<br><br>

<h2>Source code is hosted on github</h2>Check out our repository at <a href="http://github.com/playframework/play">http://github.com/playframework/play</a><br><br>

<h2>Contributing, creating a patch</h2> Please read the <a href="http://play.lighthouseapp.com/projects/57987/contributor-guide">contributor guide</a><br><br>

<h2>Reporting Security Vulnerabilities</h2> Since all bug reports are public, please report any security vulnerability directly to <em>guillaume dot bort at gmail dot com</em>.<br><br>

<h2>Creating a bug report</h2> Bug reports are incredibly helpful, so take time to report bugs and request features in our ticket tracker. We’re always grateful for patches to Play’s code. Indeed, bug reports with attached patches will get fixed far quickly than those without any.<br><br>

Please include as much relevant information as possible including the exact framework version you're using and a code snippet that reproduces the problem.<br><br>

Don't have too much expectations. Unless the bug is really a serious "everything is broken" thing, you're creating a ticket to start a discussion. Having a patch (or a branch on Github we can pull from) is better, but then again we'll only pull high quality branches that make sense to be in the core of Play.

Pages