Wednesday, 20 August 2014

liberator-transit 0.3.0

I have released liberator-transit 0.3.0. The main new feature in this release is the option to set write handlers for Transit. To do so, just add them to the options using the :handlers key just as if you were calling transit/writer. For more details, check out the README.

Sunday, 10 August 2014

liberator-transit 0.2.0

It didn’t take long. I have released liberator-transit version 0.2.0, which fixes one bug and adds some configurability. Now, the response will not have a charset added to the Content-Type header. Furthermore, you can now disable verbose JSON output, make verbose JSON the default, and change the size of the initial buffer used in serialising the response. You can read the details in the README. Most importantly, this work paves the way for setting write handlers for the Transit output, currently planned for release 0.3.0.

Friday, 08 August 2014

Presenting liberator-transit

As part of my prepartion for the core.async workshop I’ll be giving at Strange Loop, I have come across the need for a RESTful API for a web application I am creating. Naturally, I thought this would be a good chance to try out Liberator and Transit. I was pleased to see how easy Liberator is to use. Moreover, it was nearly trivial to add Transit serialisation support to Liberator. Nonetheless, I thought it would be good to wrap it all up in a library ready for reuse, which I have unimaginatively called liberator-transit.

How to use it

Using liberator-transit is straightforward:

  1. Add it to your project as a dependency.
  2. Require the io.clojure.liberator-transit namespace
  3. Accept the application/transit+json and/or application/transit+msgpack media types.

For example, given the following project.clj:

(defproject liberator-transit-demo "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.6.0"]
                 [compojure "1.1.8"]
                 [com.cognitect/transit-clj "0.8.247"]
                 [liberator "0.12.0"]
                 [ring/ring-core "1.3.0"]
                 [io.clojure/liberator-transit "0.1.0"]]
  :plugins [[lein-ring "0.8.11"]]
  :ring {:handler liberator-transit-demo.core/app})

And the given liberator_transit_demo/core.clj:

(ns liberator-transit-demo.core
  (:require [compojure.core :refer [ANY defroutes]]
            [io.clojure.liberator-transit]
            [liberator.core :refer [defresource]]
            [ring.middleware.params :refer [wrap-params]]))

(defresource hello [name]
  :available-media-types ["text/plain"
                          "application/edn"
                          "application/json"
                          "application/transit+json"
                          "application/transit+msgpack"]
  :handle-ok {:time (System/currentTimeMillis)
  :greeting (str "Hello, " name \!)})

(defroutes routes
  (ANY "/:name" [name] (hello name)))

(def app (wrap-params routes))

You can run the server using lein-ring:

$ lein ring server-headless
2014-08-08 23:12:23.330:INFO:oejs.Server:jetty-7.6.8.v20121106
2014-08-08 23:12:23.349:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:3000
Started server on port 3000

Now, it is possible to see what the different encodings look like:

$ curl http://localhost:3000/text
time=1407557612822
greeting=Hello, text!
$ curl -H "Accept: application/edn" http://localhost:3000/edn
{:time 1407558307759, :greeting "Hello, edn!"}
$ curl -H "Accept: application/json" http://localhost:3000/json
{"time":1407558370116,"greeting":"Hello, json!"}
$ curl -H "Accept: application/transit+json" http://localhost:3000/transit-json
["^ ","~:time",1407558488590,"~:greeting","Hello, transit-json!"]
$ curl -H "Accept: application/transit+json;verbose" http://localhost:3000/transit-json-verbose
{"~:time":1407558554647,"~:greeting":"Hello, transit-json-verbose!"}
$ curl -s -H "Accept: application/transit+msgpack" http://localhost:3000/transit-msgpack | xxd
0000000: 82a6 7e3a 7469 6d65 cf00 0001 47b9 08d1  ..~:time....G...
0000010: 52aa 7e3a 6772 6565 7469 6e67 b748 656c  R.~:greeting.Hel
0000020: 6c6f 2c20 7472 616e 7369 742d 6d73 6770  lo, transit-msgp
0000030: 6163 6b21                                ack!

There’s just a couple of things to note:

  1. The default Transit/JSON encoding is the non-verbose format. By adding the “verbose” to the “Accept” header, liberator-transit emits the verbose JSON encoding.
  2. Since the MessagePack encoding is a binary format, I pipe the output through xxd. Otherwise, unprintable characters are output.

How it works

Liberator has a built-in mechanism for formatting output of sequences and maps automatically depending on the headers in the request, as seen above. Moreover, it’s possible to extend this easily by adding new methods to the render-map-generic and render-seq-generic multimethods in liberator.representation. These methods dispatch on the media type that has been negotiated and take two arguments: the data to render and the context.

Getting the verbose output to work was just a touch trickier. The parameters to the media type are stripped by Liberator. As a result, it is necessary to actually examine the request headers that were placed in the Ring map. Fortunately, this is easy to do as it is a part of the incoming context.

Here is an example:

(defmethod render-map-generic "application/transit+json"
  [data context]
  (let [accept-header (get-in context [:request :headers "accept"])]
    (if (pos? (.indexOf accept-header "verbose"))
      (render-as-transit data :json-verbose)
      (render-as-transit data :json))))

Further work

This library is really quite simple. I spent far more time creating test.check generators for the various Transit types than I did on the library itself. It mostly exists to provide an even easier way to add Transit to Liberator.

Nonetheless, if other people find a need for it, there is possibly room for improvement, such as:

  1. Is there a better way to handle the request for verbose JSON?
  2. Should the library be configurable? If so, what should be configured and what is the best way to do it?

Monday, 04 August 2014

Six tips for using Polymer with ClojureScript

Over the past month or so, I have started dabbling a bit with writing code for the web/front-end. In large part, I have done so in an effort to have a nice interface for the workshop on core.async that gave at Lambda Jam and will give again at Strange Loop. I am comfortable enough picking up what I need to know for the back end, but how can I create a site that doesn’t look like it came from 1997? I have used Twitter’s Bootstrap before, and it’s not a bad starting point. However, this time I decided to try using Polymer.

In this entry, I briefly introduce the motivating principle behind Polymer, and I share some of the lessons I learned in creating Polymer elements with ClojureScript. Finally, I answer the question: Why use Polymer and not something like Om/React?

A quick introduction to Polymer

Polymer is a technology developed by Google built atop Web Components, a group of related standards such as templates, HTML imports, custom elements, and shadow DOM. Ultimately, the end goal of this combine chunks of styling, markup, and logic into discrete reusable components. For example, embedding a Google map into a page is as simple as:

<google-map latitidute="…" longitude="…"></google-map>

It couldn’t get much simpler than that, right? Well, Polymer has a whole series of UI components designed to help create web applications implementing Google’s new material design. These looked nice, so I decided to put them to use.

Using Polymer with ClojureScript

Overall, using Polymer has been a relatively smooth process. For my workshop, I primarily just used the Polymer element libraries, but I did create a few. I primarily used JavaScript to implement the logic behind my elements. However, there is one component where I absolutely needed to use ClojureScript, as it demonstrates the different types of buffers for core.async channels. Rather than creating a simulation, I used core.async itself. Learning to use the two technologies together correctly took a bit of experimentation, but here are a few tips for you should you decide to go down the same path:

Tip 1: Beware of what you build in your prototype

The typical call to register a Polymer element with a few properties in ClojureScript will look something like:

(js/Polymer
  "my-element-name",
  #js {:foo false
       :bar 42
       :baz "ClojureScript rocks!"})

However, you may want to create an element with more complex properties such as JavaScript arrays or objects or even ClojureScript objects such as a core.async channel for managing callbacks. As such, you may be tempted to do something like:

(js/Polymer
  "my-element-name",
  #js {:anArray #js [1 2 3]
       :anObject #js {:foo 1 :bar 2}
       :channel (async/chan)})

However, as the documentation will warn you, this is not the right way to do it. The object created here is a prototype that will be used in each instance of your element. As such, any complex types will not get deep copies and essentially constitute global shared state. Instead, you need to instantiate these in the created callback, as in:

(js/Polymer
  "my-element-name",
  #js {:anArray nil
       :anObject nil
       :channel nil
       :created #(this-as me
                   (aset me "anArray" #js [1 2 3])
                   (aset me "anObject" #js {:foo 1 :bar 2})
                   (aset me "channel" (async/chan)))})
Tip 2: Pass your element reference around

If you don’t grab onto the this reference inside the scope of your prototype, it can be very hard to get it later. As such, when calling your own functions from a callback, be sure to pass the reference in. For example, the previous example could be rewritten as:

(defn on-create [element]
  (doto element
    (aset "anArray" #js [1 2 3])
    (aset "anObject" #js {:foo 1 :bar 2})
    (aset "channel" (async/chan))))

(js/Polymer
  "my-element-name",
  #js {:anArray nil
       :anObject nil
       :channel nil
       :created #(this-as me (on-create me))})
Tip 3: Use the right property accessors/mutators

This isn’t a Polymer-specific tip, but I got bit by it nonetheless. As you may know, ClojureScript supports two different syntaxes for accessing and mutating JavaScript object properties:

; Syntax one
(.-prop obj)
(set! (.-prop obj) value)

; Syntax two
(aget obj "prop")
(aset obj "prop" value)

When using ClojureScript’s advanced compilation mode, it will munge the property names if you use the first method. The result will be a broken element featuring lots of null or undefined values. Unless you are sure you will never use the advanced compilation mode, just use the second syntax.

Tip 4: Accept mutability

As a Clojure/ClojureScript programmer, it maybe very tempting to try to store your application state in an atom and deal with it a nice, sane way—just the way Om does it. Well, to do so in Polymer while supporting Polymer’s two-way data binding is just a pain. It can certainly be done, but I am not sure it is worth the effort. Let’s take a look at a simple example to see what I am talking about:

(js/Polymer
  "my-element",
  #js {:property "foo"
       :propertyAtom nil
       :propertyChanged #(this-as me
                           (reset! (aget me "propertyAtom") %2))
       :create #(this-as me
                  (let [property-atom (atom (aget me "property"))]
                    (add-watch property-atom ::to-polymer
                      (fn [_ _ _ new-val]
                        (when (not= new-val (aget me "property"))
                          (aset me "property" new-val))))
                    (aset me "propertyAtom" property-atom)))})

What does this code all do? Well, let’s break it down:

  1. On line 3, we set up the mutable cell that will be visible to Polymer.
  2. On line 4, we set up the immutable cell for our Clojure code.
  3. On line 5, we set up a handler so that we can update the Clojure atom when the property is changed from Polymer. For example, this may occur if the property was bound to the value of some user input.
  4. On line 8, we create our atom.
  5. On line 9, we set up a watcher on the atom so that if a change comes from the Clojure side, i.e. from a swap!, the change will be pushed into the mutable cell that Polymer will see.
  6. Finally, on line 13, we add the atom to the element’s state.

This is a lot of overhead for just one property. You can try to manage this more easily by just having one atom with all application state in some sort of map/object. This does mean you have fewer atoms to watch, but otherwise it doesn’t help too much.

The reasons for this are twofold:

  1. Now your template code has to reference a bunch of nested properties. What used to be {{property}} now looks like {{state.property}}.
  2. You must now use an observe block to get notifications of property changes, and you must specify each path through your state map.

In practice, this can look something like:

(def init-state
  {:foo 42
   :bar true
   :baz "Why am I not using Om/React?"})

(js/Polymer
  "my-element",
  #js {:state nil
       :stateAtom nil
       :create #(this-as me
                  (aset me "state" (clj->js init-state))
                  (let [state-atom (atom init-state)]
                    (add-watch property-atom ::to-polymer
                      (fn [_ _ _ _]
                        (let [clj (clj->js @state-atom)
                              js (aget me "state")]
                          (when (not= clj js)
                            (aset me "state" clj)))))
                    (aset me "stateAtom" state-atom)))
       :observe #js {:state.foo #(this-as me
                                   (swap! (aget "stateAtom" me) assoc :foo %2))
                     :state.bar #(this-as me
                                   (swap! (aget "stateAtom" me) assoc :bar %2))
                     :state.baz #(this-as me
                                   (swap! (aget "stateAtom" me) assoc :baz %2))}})

This is an awful lot of boilerplate. I suppose it may be possible to automate much of it using a macro, but the question that remains is what does this additional complexity buy you? In my opinion, though it may be a bitter pill to swallow, just accepting and using the mutability of Polymer element seems to be the most pragmatic route. Doing so allows things like Polymer's two-way data binding to just work.

Tip 5: Properly include the ClojureScript in development mode

There are three ways to register a Polymer element that has logic:

  1. Inside the element definition in the body of a script tag,
  2. Referencing an external script inside the element definition, and
  3. Loading the external script before the element definition.

With ClojureScript, we can choose from the latter two. Most of the time, the second form is the most convenient:

<polymer-element name="my-element">
  <template>…</template>
  <script type="text/javascript" src="my-element.js"></script>
</polymer-element>

However, this only works when one of the ClojueScript optimizations levels is used, i.e. whitespace, simple, or advanced. If you are in development mode, you would might be tempted to do something like:

<polymer-element name="my-element">
  <template>…</template>
  <script type="text/javascript" src="out/goog/base.js"></script>
  <script type="text/javascript" src="my-element.js"></script>
  <script type="text/javascript">
    goog.require('my_element');
  </script>
</polymer-element>

However, if you are importing your element, e.g. <link rel="import" href="my-element.html">, you will run into problems with the Google Closure library. It doesn’t like this since it will attempt to inject your script’s dependencies after the main page has loaded. Instead, while you are in development mode, place all of your script tags in your main HTML page.

Tip 6: Be wary of processing the component HTML

This isn’t ClojureScript-specific, but I thought I’d include it nonetheless. During development, I use some Ring middleware that injects script tags into the served HTML to connect to my browser REPL environment. This generally works well, but sometimes I saw some really bizarre behaviour. For example a paper-input component refused to take focus, or a core-submenu component hid its icon.

It turns out that Polymer uses a conditional attribute syntax:

<span hidden?="{{isHidden}}">You can hide me!</span>

If the expression is false, then the attribute will be omitted from the markup. Some HTML parsers such as Enlive or htmlmin cannot process that. At least in the case of htmlmin, it caused a mysterious hang. I received no warning at all from Enlive.

The bottom line is: don’t try to process a component’s HTML unless you are sure your processor can handle it.

Why bother with Polymer?

So given the existence of ClojureScript-friendly frameworks like Om/React, why bother trying to write Polymer elements in ClojureScript? That’s a great question. Here’s my take on it:

  1. The two are not mutually exclusive. You can easily use both Om/React components and Polymer elements on a page.
  2. However, nesting them in each other proves a lot more tricky. I don’t think there is any reason why you couldn’t embed Om/React component inside of a Polymer element. Unfortunately, I am not sure it’s possible to go the other way around. As best as I can tell, React expects to be React all the way down to the DOM.
  3. With the core and paper elements libraries, Polymer offers a compelling case for using it. As I am not a web front-end developer, the ability to easily use nicely-styled widgets and declaratively define animations is particularly nice. I am not an HTML/CSS/JavaScript wizard, and it would take me a long time to implement what Polymer provides. Using Polymer, I can instead spend my time working on my application.
  4. Frankly, I don’t think that ClojureScript is the ideal fit for Polymer. If you need to use ClojureScript, as in my case where I needed to use core.async, it’s certainly an option. However, if ClojureScript isn’t absolutely necessary, consider sticking to JavaScript. In the end, it’s all about choosing the right tool for the job.