In part 2 of this series we built upon the foundations we created in part 1 to generate a Connect descriptor. That descriptor specifies, among other things, the API that Bitbucket should call on key events in our repository and add-on lifecycle. In this installment we’re going to look at how to specify and serve this API, and how to convert JSON sent to us by Bitbucket into Clojure data-structures.
Serving our API
If we take a closer look at our descriptor we can see we define the followinhcg API:
/installed: Called on installation to an account and provides a shared key./uninstalled: Called on uninstall from an account./webhook: Called on repository events such as pushes./connect-example: Called to be inserted into a webpanel in the repository.
We’ll cover the webpanel component later on; for now we’ll look at the REST endpoints. The most important of these is /installed, which is called by Bitbucket when a user adds the repository to an account. This provides the add-on with some key information the it needs to store for future use, including a shared secret to verify future calls and authorise calls to the Bitbucket API, a unique per-installation key, and information about the installing user.
Obviously we need to store some of this information across restarts of our add-on, so we need some storage. In a full production environment we’d probably use a database of some sort (for example the Docker Hub add-on uses Amazon’s DynamoDB); however for this example we can just abuse Clojure’s ability to dump and read runtime data in its EDN format. Let’s create a new storage.clj file and add some save/load operations:
(ns hello-connect.storage
(:require [clojure.string :as string]
[clojure.edn :as edn]
[clojure.java.io :as io]
[clojure.tools.logging :as log]
[environ.core :refer [env]]))
(defonce addon-ctx (atom nil))
(defonce context-file (str (env :data-dir "/tmp") "/addon-context"))
(defn load-addon-context []
;; Load up shared key if we have one
(when (.exists (io/as-file context-file))
(reset! addon-ctx (edn/read-string (slurp context-file)))
(log/info "Loaded addon context of" @addon-ctx)))
(defn save-addon-context [ctx]
(reset! addon-ctx ctx)
(io/delete-file context-file {:silently true})
(spit context-file ctx))
(defn delete-addon-context []
(io/delete-file context-file {:silently true})
(reset! addon-ctx nil))
(defn shared-secret []
(@addon-ctx "sharedSecret"))
(defn client-key []
(@addon-ctx "clientKey"))
(Note: This is a very simplistic implementation and should not be used in a real add-on. In particular it’s not multi-tenanted as it doesn’t separate storage per installation via the clientKey field. But it will do for simple illustration purposes.)
Now we have something to do with the information sent to us on installation let’s handle the API call. Bitbucket sends the installation context as JSON in the body of a POST to the endpoint; we’d prefer that the information was converted into Clojure’s internal data structures. There are libraries to do this, but as it’s such a common operation Ring provides a handy wrapper that will do this for us. To enable this, just import wrap-json-body from ring.middleware.json and add it to our routing stack:
(def app
;; Disable anti-forgery as it interferes with Connect POSTs
(let [connect-defaults (-> site-defaults
(assoc-in [:security :anti-forgery] false)
(assoc-in [:security :frame-options] false)) ]
(-> app-routes
(wrap-defaults connect-defaults)
(wrap-json-body))))
This will check the Content-Type of incoming requests and parse any JSON body into Clojure.
Installation API
Now we can add the API endpoint to the routes; first add the following to handler.clj:
(defn process-installed [params body]
(log/info "Received /installed")
(storage/save-addon-context body)
{:status 204})
This will save the installation context and return an empty OK status. Then we add the endpoint to the defroutes section:
(POST "/installed" {params :query-params body :body}
(process-installed params body))
We should also process /uninstalled calls. While not a major issue in our toy implementation we generally would like to clear unneeded data out of our database, and it is generally good practice to not store user information unnecessarily. However we don’t want to allow just anybody to make an uninstall call; in fact, we’d like all our API to be authenticated as coming from Bitbucket from now on. Luckily that’s what the installation information above is for, in particular the sharedSecret entry in the context.
Obviously we’d like this context to be loaded on future runs of our add-on, so let’s load it on startup in handler.clj:
(defn init []
(log/info "Initialising application")
(storage/load-addon-context))
Installation data
The method Atlassian Connect uses to pass authentication to an add-on is JWT, a web-standard for encoding authorisation claims. There are already Clojure JWT libraries available, but Atlassian’s Connect defines an extension to the standard to add additional verification of query parameters. However as part of writing the Docker Hub Bitbucket add-on I produced a library to implement this extension, along with some helper operations. This allows to define a simple Ring-style wrapper that will authenticate a request and either deny access or forward onto another handler:
(defn wrap-jwt-auth [request handler]
(if (not (jwt/verify-jwt request (storage/shared-secret)))
{:status 401}
(handler request)))
Uninstalling
Now we can create a handler to do the uninstallation (which with our naive storage implementation is just a delete):
(defn process-uninstalled [request]
(log/info "Received /uninstalled")
(storage/delete-addon-context)
{:status 204})
Then we add a route for /uninstalled that calls the uninstaller wrapped in authentication:
(POST "/uninstalled" request
(wrap-jwt-auth request process-uninstalled))
We’ll use this method for the rest of our API.
Webhooks
The last REST endpoint we need to create is /webhook. The webhook is called for any key events in the repository, for example on a push event. We’re not going to actually do anything with this information for our plugin, but it’s useful to see what we receive from Bitbucket. So we’ll create a function to pretty-print the received data:
(defn process-webhook [request]
(log/info "Received /webhook of:n" (clojure.pprint/pprint (request :body)))
{:status 204})
And again we just define and authenticate the endpoint:
(POST "/webhook" request
(wrap-jwt-auth request process-webhook))
The code
The code for this part of the tutorial series is available in this tag in the accompanying Bitbucket repository. There will also code appearing there for the later parts as I work on them if you want to skip ahead.
Next time
Now we’ve handled the Bitbucket to add-on aspects of the API we need to address the more complex issue of retrieving information from Bitbucket and creating user-visible content from it. In the next part we’ll look at how to do that using both the Bitbucket REST API and a pure-Javascript implementation. We’ll use both of these techniques to display some key information in an embedded component on the Bitbucket repository page. Tune in next-time for more Clojure goodness!