boot-gae: Interactive Clojure Development on Google App Engine

[Third in a series of articles on boot-gae]

boot-gae supports interactive development of Clojure applications on GAE. It does not have a true REPL, but it's pretty close: edit the source, save your edits, refresh the page. Your changes will be loaded by the Clojure runtime on page refresh.

This is mildly tricky on GAE. GAE security constraints prevent the Clojure runtime from accessing the source tree, since it is not on the classpath. Nothing outside of the webapp's root directory tree can be on the classpath.

Part of the solution is obvious: monitor the source tree, and whenever it changes, copy the changed files to the output directory. The built-in watch task makes this easy; gae/monitor composes that task with some other logic to make it work.

The tricky bit here is to make sure the changed files get copied to the appropriate place in the output directory; for Clojure source files, that means

target/WEB-INF/classes    ;; for servlet apps

target/<servicename>/WEB-INF/classes    ;; for service apps

boot-gae tasks use configuration parameters to construct the path. The gae/monitor (and the gae/build) task uses the built-in sift task to move input from the source tree to the right place.

That's half a solution; we still need to get Clojure to reload the changed files. The trick here is to use a  Java filter to monitor the files in the webapp and reload them on change, just as the gae/monitor does with source files. A filter in a Java Servlet app dynamically intercepts requests and responses; by installing a filter, we can ensure that changed Clojure code can be reloaded whenever any page is loaded.  See The Essentials of Filters for more information.

The gae/reloader task generates and installs the appropriate filter. No configuration is necessary; the whole process is hidden, so all the programmer need do is run the gae/reloader task.

The reloader task generates a reloader "generator" file (named using gensym) whose contents look like this:

;; TRANSIENT FILTER GENERATOR
;; DO NOT EDIT - GENERATED BY reloader TASK
(ns reloadergen2244)

(gen-class :name reloader
           :implements [javax.servlet.Filter]
           :impl-ns reloader)

It saves this to a hidden location (this is easily done, since doing so one of the core features of boot) and then AOT-compiles it to produce the reloader.class file.

It also generates the Clojure file that implements the filter's doFilter method. Here's the content of that file:

;; RELOADER IMPLEMENTATION NS
;; DO NOT EDIT - GENERATED BY reloader TASK
(ns reloader
  (:import (javax.servlet Filter FilterChain FilterConfig
                          ServletRequest ServletResponse))
  (:require [ns-tracker.core :refer :all]))
(defn -init [^Filter this ^FilterConfig cfg])
(defn -destroy [^Filter this])
(def modified-namespaces (ns-tracker ["./"]))
(defn -doFilter
  [^Filter this
   ^ServletRequest rqst
   ^ServletResponse resp
   ^FilterChain chain]
  (doseq [ns-sym (modified-namespaces)]
    (println (str "reloading " ns-sym))
    (require ns-sym :reload))
  (.doFilter chain rqst resp))

This process results in some transient files, which are filtered out of the final result. The only files we need are reloader.class (which the servlet container needs) and reloader.clj (to which reloader.class will delegate calls to the filter methods, like doFilter).

If you want to inspect the transient files, you can retain them by passing the --keep (short: -k) flag to the gae/build task. Here's an example of what you will find in WEB-INF/classes in that case (other files omitted):

reloader.class

reloader.clj

reloadergen2244$fn__35.class

reloadergen2244$loading__5569__auto____33.class

reloadergen2244.clj

reloadergen2244__init.class

Since the reloadergen* files are not needed by the app, then are removed by default.

Deployment

This works find for local development; however, it's just a waste in an application deployed to the cloud. Before deploying (gae/deploy), be sure to omit the gae/reloader task from your build pipeline; is you're using gae/build, use the --prod (short: -p) flag.

Previous article: boot-gae: building and assembling service-based apps