Back in 2003 I found myself at a sprint (a hackaton) to help build a web framework. But that's not what I'm talking about right now. For some reason I was messing around with JavaScript. This was before JavaScript was cool. Let's give an impression of this era. JavaScript was this abomination of a language used only by front-end designers. Horrible code was written in it. The term AJAX hadn't been coined yet - that was years into the future. We didn't really have the browsers for it anyway, at least on Linux. Netscape 4.5 was the state of the art; Firefox had barely been born and was still called Phoenix. It was that dark moment in net history where it looked like Internet Explorer might have won the browser wars for good. So I was messing around with JavaScript. Zope, which I was using, had this then fairly new template language called ZPT. I was wondering whether I could implement ZPT in JavaScript and do templates in the browser. I told other developers about it, and they all asked "why?" My answer was something like "I don't know man, it's just cool!". I remember I even worked on implementing unit tests for it in JavaScript. But I genuinely didn't know what I'd use it for; how was I even going to get data into it? JSON as a concept didn't exist yet either so I probably was thinking about using XML. The project went nowhere, of course. The idea was way ahead of its time; so far ahead it was in fact still a crazy idea. I'm sure many people had thought of it; even back in 2003 people had been doing crazy things with JavaScript for years. We flash forward to 2009. I was working on a project that involved a web application that needed a dynamic search interface. It involved JavaScript. I had noticed this client-side templating language called JSON-Template. I had already used its server-side Python implementation for another project to provide customizable email templates in an application. I figured we could write a little JavaScript framework to make it easy to use JSON-Template. So we did that. And we built the user interface, and it worked quite well, and the code for it was pretty easy to follow too, so that was good. The idea wasn't crazy anymore: client-side templating languages were now useful! Over the course of the last year and a half a bunch of us refined the ideas of this browser-side framework some more, and this resulted into the creation of Obviel (which does much more than just support client-side templates). Many client-side templating languages, JSON-Template included, are minimalistic. You combine a JSON object with a text file that has some templating instructions in it to render a template. JSON-Template doesn't even feature an expression language. JSON-Template is a push only template language. Many templating languages (but not JSON-Template) are pull: allow you to embed calls to bits of application code within the template: some_object.get_something(). But get_something could do anything; call the database, raise exceptions, take a long time to execute, etc. If you write templates that call into the application's API, you can easily create a fairly tight coupling between application and template. This means the template becomes harder to test and debug and change; the bits of the application used by the template need to be available as well during testing and debugging. It also means the application becomes harder to change, because who knows what methods are being called by templates? A push-only template language doesn't have such a problem. But
how do you use such a minimalistic template language for complex scenarios where all sorts of decisions need to be made and data needs to be massaged? You can't do it in the template. Instead, you do the work in advance, in JavaScript or Python or Ruby or whatever language you are using. This means the logic is written in a full-featured programming language which was designed to implement complicated logic, instead of in a template language. It also means this logic is easier to test and debug in isolation, without the template language being involved. Loose coupling is awesome that way! In fact, I believe many of the benefits of minimalistic push-only template languages exist on the server as well as the client, and would encourage people to write push-style templates also when writing server-side templates. Let's go through some benefits of minimalistic push-only templates: Running a template language on the client instead of on the server has some important benefits as well: I'll go into the last statement a bit more deeply. It's a bit of a paradox: while minimalistic push-only template languages can render templates very quickly, running them on the client means they don't really have to be that fast after all. On the server, if you have a thousand users, a thousand users are exercising your server-side template language engine. It can become a bottleneck. On the client, there's only ever one user: the user of the web browser. If the template language is fast enough for that user, you're done. Your server-side application may have a thousand users more, but the template language performance is no longer a bottleneck. (Your JSON serializer is. but JSON serializers can be very fast) So can we exploit that? Since our language doesn't need to be blazingly fast, can we do neat things that might take a bit more time? I think so. I've been thinking for a while now about a template language much like JSON-Template that would work with the HTML DOM as opposed to being text based like JSON-Template is. This would likely make it slower, but we can make it fast enough for a single user for sure. And then we can do "live" binding between the JSON object being rendered and the DOM tree rendered by the template. That would allow: I'm sure I wasn't the first person to have such an idea, and I'd like to hear about implementations of it if you know any. (in fact, I created a hack based on JSON-Template back in 2009 that could do the second bit). I hope to eventually integrate such a templating language into Obviel. If you're interested in talking more about this, I hope to talk to you in the comments or on the Obviel mailing list!
![[Comments]](/blog-content//themes/Default/comments.png "27 comments")
(27) Fri Nov 18 2011 00:36 The New Hot Thing in Web Development: Client-side Templating Languages:
- Comments:
Posted by Martin Aspeli at Fri Nov 18 2011 01:36
Hi,The 'live binding' idea sounds a bit like Knockout: http://knockoutjs.com/Martin
Posted by anonymous@84.24.127.246 at Fri Nov 18 2011 01:42
Martin, thanks for that Knockout reference; I remember looking at a bunch of these frameworks before and finding that, but I forgot which one. :) I think the binding can be done non-intrusively though - no need to annotate the HTML with explicit binding information but deduce it from the template's structure.
Posted by Bradley Wright at Fri Nov 18 2011 08:38
Mustache (and sibling Handlebars) are probably the most widely used client-side templates. The good news about Mustache is that it's also server-side (assuming you use one of the languages it's been ported to), which means you can share code with the server for rendering. This gives you even more power and flexibility.
Posted by Dirkjan Ochtman at Fri Nov 18 2011 08:54
I've been experimenting with compiling Jinja template code to JavaScript. I have most of Jinja's features working (or at least basic variants of them), enough to use this code on a real-world internal app at work. It's been a tremendous help in making our webapp more dynamic without having to rewrite templates.https://bitbucket.org/djc/jasinja
Posted by Rene at Fri Nov 18 2011 10:53
Hello,indeed it is getting popular. Good to hear of other people using it too.Have you seen my posts on using jQuery on the server side? Here is the post: http://renesd.blogspot.com/2011/10/server-side-jquery-ssjquery.htmlAs you mention, this lets you use the DOM insertion techniques. eg, $('img').attr('alt', 'big face')For when you are using jQuery already on the client side, it is really nice to be able to reuse that code on the server side.json + html + server side jquery => rendered htmlI use it now instead of templating languages, because there are over 1000 templating languages, but most people I work with know a little jQuery. Being able to debug in the browser with firebug and then move it onto server side later is really nice. No server side environment needed at all for the front end developers... just a json file.I also use it with python projects! The latest project I used it on is being released soon (I hope!) http://www.rugsonline.com.au/ It's a rugs shop, so luckily a lot of the content does not change frequently. So it is possible to cache the results. Since so far the server side jQuery is a bit slow, since the DOM is implmented in javascript itself, and even though V8 is fast with node.js, it is not as fast as a browser DOM. But is sure is nice to code!
Posted by Michael Kerrin at Fri Nov 18 2011 15:41
Hi,I have also written a Python Jinaj2 template -> Java Script compiler. It supports a good portion of the Jinja2 syntax but with some restrictions. It works by compiling all Jinja2 macros to a corresponding Java Script function that you can call from Java Script.It also supports a WSGI middleware that will serve a Jinja2 template as Java Script file, a command line interface to manually generate the Java Script and experimental integration of closure compiler with zc.buildout.It can be found here: https://github.com/mkerrin/pwt.jinja2js and there is a PyPi release here: http://pypi.python.org/pypi/pwt.jinja2js/, documentation: http://packages.python.org/pwt.jinja2js/
Posted by Martijn Faassen at Fri Nov 18 2011 15:59
Martin: that anonymous commenting before on you was actually me. :)Bradley: Mustache looks cool (and is also minimalistic). Looks like it (or Handlebars) could be integrated into Obviel, except that it relies on functions being passed into the template, not just a plain JSON objects. I wonder how mustache templates are commonly used with AJAX applications; how do people get the helper functions into an object returned by the server? Looks like Handlebars' helper function construct might be of use...
Dirkjan: doesn't Jinja let you write code that calls into arbitrary Python code? I guess if you only pass in basic objects into your templates it's possible to port the code to the client too. Jinja's expression language is quite extensive too, so it's going to be relatively expensive (though should be fast enough).Rene: Server side jQuery looks interesting, Rene, and the idea to enable client-side developers to write server side code is neat. Of course it doesn't have some of the benefits I listed for client-side templating, as the code needs to run on the server.Posted by Martijn Faassen at Fri Nov 18 2011 16:06
Michael: cool! I wonder what it would take to integrate a javascript jinja2 story into Obviel. Both jinja to JS compilers rely on server-side compilation technology in Python as opposed to client-side template compilation, which will provide an extra challenge for Obviel integration.
Posted by brad clements at Fri Nov 18 2011 16:22
client-side templates are great!I started using client-side templating (via xslt templates) back in 2004, downloading xml instead of json to the client. So back in 2005 I found a partial implementation of TAL for XSLT which I subsequently updated quite a bit and added METAL support, etc see https://bitbucket.org/bkclements/tal2xslt/overview This allows you to define your ZPT-like template in XHTML and get back an XSLT template. TALES is xpath..It also includes somewhat crusty WSGI middleware for automatically filtering xml through tal2xslt based on xsl-stylesheet pi (for server-side conversion of template to xslt or further converting the xml to html using the xslt template)
Then in 2007 I wrote a crude implementation of ZPT for
javascript. It doesn't support METAL, it does read the template from the DOM itself, it supports a form of TALES and is typically "PUSH" mode but you can pass anything in 'options', including javascript functions.I keep hoping to find time to clean up the implementation, add support for templates from strings, etc. But it works well for me. See https://bitbucket.org/bkclements/jstal/overviewPosted by Michael Kerrin at Fri Nov 18 2011 16:34
Martijin: I do have a tool to do the compilation by hand. So after changing the template you compile it to Java Script and save the JS to a file. This file can then be used like any other static Java Script file. Obviously if you forget to compile it you will end up serving stale template.So integration to obviel should be straight forward. You get your data and call the generated function in your obviel view object. Giving you a string to insert into the page.
Posted by Martijn Faassen at Fri Nov 18 2011 16:45
Brad: that's interesting, I'll look at jstal in particular for inspiration if I ever get around to the "live linking" version of client side templates.Michael: the thing is that Obviel currently assumes templates are available in source text form and compilation of the source text happens on the client side. I need to think about how to properly plug in JS files as templates - how could an Obviel view refer to one (spelling), and how would it load it up? There are some details to be worked out there.
Posted by mike bayer at Fri Nov 18 2011 16:59
"This means the template becomes harder to test and debug and change; the bits of the application used by the template need to be available as well during testing and debugging. It also means the application becomes harder to change, because who knows what methods are being called by templates?"replace "called by templates" with "called by other modules". What's the difference ? As always, Python is a dynamic language - you write unit tests. Template output should not be excluded from the testing procedure and with modern techniques there aren't any technical issues with this.
Posted by Martijn Faassen at Fri Nov 18 2011 17:16
Mike, I am not sure what you're disagreeing with. I was suggesting a template becomes more easy to test if there is looser coupling between it and the application, and you're saying template output should not be excluded from the testing procedure - how exactly are you disagreeing? I'm just saying it's a good thing templates can be unit tested instead of only integration tested. I wasn't talking about Python, but independently unit testable Python modules are often a good thing too, right?I'm also saying that a loose coupling between template and application (through a pipelining model such as push templates) can make it easier to change the application without breaking templates as the contract between templates and application is more explicit.By the way: I do think there are differences between templates and programming language modules; templates in some cases are edited by folks with less technical knowledge about the workings of the underlying software than modules. Customizable apps with such needs are quite common; take a CMS like Plone for instance.I would encourage people to try to write loosely coupled systems in general - in some cases a pipelining model also makes sense in reducing coupling between bits of software, including Python software.
Posted by Michael Kerrin at Fri Nov 18 2011 17:19
Martijin: Its Java Script so you load it up via the script tag. I have never used obviel but from reading the docs couldn't I not just doobviel.view({
iface: 'animal',
name: 'alternative',
render: function() { this.el.html(animalcolor(this.obj) }; });My template could be{% macro animalcolor(color) %}
Color of animal is {{ color }}
{% endmacro %}Posted by Michael Kerrin at Fri Nov 18 2011 17:21
For your reference the above template gets compiled to:animalcolor = function(opt_data, opt_sb, opt_caller) {
var output = '';
output += '\nColor of animal is ' + opt_data.color + '\n';
return output;
}Posted by Martijn Faassen at Fri Nov 18 2011 17:27
Michael: sure, you can do that, but I was thinking about referring to such templates from the view definition itself; making it a bit more declarative, something like this:obviel.view({
iface: 'animal',
template_url: 'http://example.com/animalcolor.js',
});but template_url expects to find a template text file with an extension identifying the template language, not a js file. So we have to look for a different way to express this declaratively. Possibly:obviel.view({
iface: 'animal',
template_func: animalcolor
});what do opt_sb and opt_caller do? I see you don't have to use them.Posted by Michael Kerrin at Fri Nov 18 2011 17:43
ok I get what you mean. I was just showing a simple way of doing it.opt_sb and opt_caller are there to support different features of the Jinja2 template syntax all relating to calling other templates from a template. My compiler is very simple, it was easier to include the arguments in by default. They should be just ignored. All the data should be passed in the first argument.Note that I did look into client side compilation ages ago but ran into the problem of trying to concat and minify my Java Script foot print. I could put the templates in the HTML body of the page but I just found this hard to manage.
Posted by Kevin Horn at Fri Nov 18 2011 18:16
backbone.js does (or at least can do) something like the live binding idea you mentioned.When combined with JQuery (for DOM manipulation) and mustache.js/handlebars.js you get something very like what you have described.
Posted by Karl G at Fri Nov 18 2011 20:10
Bindings are the new wave of JS development. Backbone is the best known, but Angular, Knockout, and Sproutcore 2.0 are also very well put together systems. Having tried them all, Sproutcore allows for the most declarative code and has a significant amount of effort put into only updating the part of the DOM that changes on object update (uses metamorph, it's a clever hack). I like Angular as well but that's more conceptually in line with the *sp family of languages and I tend to work with designers so I like harder separation.If you're looking for a non-text based templating system hij1nx/tmpvar's weld sounds the closest but there are other options like pure and chain. I don't think they've hooked it up to a set of bindings but I haven't looked in the last 6 months or so.The most important thing about templating in the browser isn't the raw templating perf but rather how much it touches the DOM. Doing a innerHTML has historically been more performant, hence the continuing preference for text-based templating. In the newer iterations of Firefox and Chrome, DOM templating is actually faster but only if you fragment template then swap.
Posted by Martijn Faassen at Fri Nov 18 2011 20:36
Kevin, any examples of backbone + jquery + handlebars?Karl G: thanks for the references to weld, I'll check that out. I looked at Angular and Knockout briefly before. I also just ran into jsRender and jsViews; the latter is a dynamic binding system for jsRender and is jQuery based and might match Obviel best.
Posted by mike bayer at Fri Nov 18 2011 21:30
Martijn -I guess what I'm getting at is that with a template, there's not much difference between "push" and "pull" provided a template that does "pull" is doing it in a reasonable way - or perhaps what defines "pull" is not totally clear.If you supply the namespaces "x" and "y" to a template, the template can then call "x.foo()" and "y.bar()", is that "push" or "pull"? The coupling here is still "loose", a test environment would provide mocks of "x" and "y". It just doesn't seem to be much of a distinction to me. The "non-programmers might write your templates" thing is also IMO a myth I frequently need to debunk as in http://techspot.zzzeek.org/2010/12/04/in-response-to-stupid-template-languages/ .All of that said, client side templating *is* interesting as it allows both full render flexibility (the full power of JS, for better or worse) *and* an enforced distinction between data and presentation.
Posted by Martijn Faassen at Fri Nov 18 2011 21:41
Mike: If the template can call any code anywhere in your application, I wouldn't call this push at all. But you're right in that a test environment could provide mocks. But you do introduce a tighter coupling nonetheless: the template writer now might get exceptions or slowdowns from the underlying codebase that with a pure push design he wouldn't. That doesn't mean those exceptions or slowdowns aren't there anymore, but it does mean they can be handled by someone else than the person writing the templates.I've worked with non-programmers writing templates in the past, so that's not quite a myth in my universe.
Posted by Leigh Klotz at Sat Nov 19 2011 01:28
Take a look at XForms, a W3C standard for MVC templating in the browser. There's many implementations, include server-side and client-only (JS) versions.https://docushare.xerox.com/dsdn/dsweb/Get/Document-9426/XRX-MVC.html
Posted by Rene at Sat Nov 19 2011 10:23
@Martijn Faassen: regarding server side jquery not being able to be run on the client side... It can be run on either the client side or server side. It is just jquery code after all. cheers!
Posted by Feneric at Sat Nov 19 2011 15:05
Don't forget the DojoX Django Template Language. If you're already familiar with DTL, you may be able to save yourself quite a bit of work messing around with something less familiar.
Posted by Robert at Sun Nov 20 2011 15:36
And like most new hot things, it's going to be massively overapplied for situations it is not appropriate for and a lot of stupid web pages will be the result.
Posted by Martijn Faassen at Mon Nov 21 2011 15:07
Rene: uh, yeah, of course, I'm dumb for not seeing that, and that's neat! :)Feneric, Leigh: thanks for more references. I had never considered looking at XForms in this light...Robert: I'll join the cynicism for a bit: given human nature, there's no avoiding lots of stupid web pages on the web. :)
