Arjen Wiersma

clojure

When I tell people that I like to code in {{< backlink “clojure” “Clojure”>}} the common response is “wut?”. Clojure is not known as a programming language in which you create big systems. As all Clojure people know, this is not true. There are many systems written in Clojure. Let me show you some that are very actively maintained.

First there is Lipas, a Finnish platform that shows you information about sports clubs. The structure and techniques used in this code base I use as a reference implementation for my own ClojureScript + Clojure systems. A screenshot of the application is shown here:

Lipas

Next, there is Metabase, a business intelligence platform. The below gif shows you some of the features it has.

Metabase

There is a great talk at Conj 2024 about supporting 50000 users on Metabase. You can watch it over on YouTube.

Finally, also found on the Conj 2024 streams, there is Cisco Threat Intelligence API. This a full threat intelligence service and data model that is built using Clojure. Link to the repository. The talk about the project can be seen on YouTube.

There are plenty of other projects using Clojure, if you know of more that I should add to my list, do let me know!

#clojure #web #programming

Observability in cloud-native applications is crucial for managing complex systems and ensuring reliability (Chakraborty & Kundan, 2021; Kosińska et al., 2023). It enables continuous generation of actionable insights based on system signals, helping teams deliver excellent customer experiences despite underlying complexities (Hausenblas, 2023; Chakraborty & Kundan, 2021). In essence, adding proper observability to your system allows you to find and diagnose issues without having to dig through tons of unstructured log files.

The running project

In {{< backlink “20250107-clojure-reitit-server” “my previous post on reitit”>}} we built a simple endpoint using {{< backlink “clojure” “Clojure”>}} and reitit. The complete code for the small project was:

(ns core
  (:require
   [reitit.ring :as ring]
   [ring.adapter.jetty :as jetty]))

(defn handler [request]
  {:status 200
   :body (str "Hello world!")})

(def router (ring/router
             ["/hello" {:get #'handler}]))

(def app (ring/ring-handler router
                            (ring/create-default-handler)))

Nice and easy eh? That simplicity is what I truly love about {{< backlink “clojure” “Clojure”>}}. That, and the fact that there is an awesome interoperability with the Java ecosystem of libraries.

Adding observability

In {{< backlink “clojure” “Clojure”>}} it is possible to add observability through the wonderful clj-otel library by Steffan Westcott. It implements the OpenTelemetry standard which makes it integrate nicely in products such as HoneyComb.io and Jaeger.

The library has a great tutorial that you can follow here. Applying the knowledge from this tutorial to our reitit application is also trivial. To show the power of observability a JDBC connection will be added to the application. It is not necessary to mess with any tables or such, it will just leverage a connection to a Postgres database and a value will be queried from it.

First, lets see the updated deps.edn file.

{:deps {ring/ring-jetty-adapter {:mvn/version "1.13.0"}
        metosin/reitit {:mvn/version "0.7.2"}

        ;; Observability
        com.github.steffan-westcott/clj-otel-api {:mvn/version "0.2.7"}
        
        ;; Database access
        com.github.seancorfield/next.jdbc {:mvn/version "1.3.981"}
        org.postgresql/postgresql {:mvn/version "42.7.4"}
        com.zaxxer/HikariCP {:mvn/version "6.2.1"}}

 :aliases {:otel {:jvm-opts ["-javaagent:opentelemetry-javaagent.jar"
                             "-Dotel.resource.attributes=service.name=blog-service"
                             "-Dotel.metrics.exporter=none"
                             ]}}}

You will notice some new dependencies, as well as an alias that you can use to start the repl with. If you, like me, use Emacs you can codify this into a .dir-locals.el file for your project.

((nil . ((cider-clojure-cli-aliases . ":otel"))))

Now, whenever cider creates a new repl it will use the otel alias as well.

The agent that is listed as javaagent can be downloaded from the OpenTelemetry Java Instrumentation page. This will immediately bring in a slew of default instrumentations to the project. Give it a try with the starter project, you will notice that all the jetty requests will show up in your jaeger instance (you did look at the tutorial, right?).

Finally, here is the update project for you to play with.

(ns core
  (:require
   [next.jdbc :as jdbc]
   [reitit.ring :as ring]
   [ring.adapter.jetty :as jetty]
   [ring.util.response :as response]
   [steffan-westcott.clj-otel.api.trace.http :as trace-http]
   [steffan-westcott.clj-otel.api.trace.span :as span]))

(def counter (atom 0))

;; add your database configuration here
(def db {:jdbcUrl "jdbc:postgresql://localhost:5432/db-name?user=db-user&password=db-pass"})

(def ds (jdbc/get-datasource db))

(defn wrap-db
  [handler db]
  (fn [req]
    (handler (assoc req :db db))))

(defn wrap-exception [handler]
  (fn [request]
    (try
      (handler request)
      (catch Throwable e
        (span/add-exception! e {:escaping? false})
        (let [resp (response/response (ex-message e))]
          (response/status resp 500))))))

(defn db->value [db]
  (let [current @counter]
    (span/with-span! "Incrementing counter"
      (span/add-span-data! {:attributes {:service.counter/count current}})
      (swap! counter inc))
    (:value (first (jdbc/execute! db [(str "select " current " as value")])))))

(defn handler [request]
  (let [db (:db request)
        dbval (db->value db)]
    (span/add-span-data! {:attributes {:service.counter/count dbval}})
    {:status 200
     :body (str "Hello world: " dbval)}))

(def router (ring/router
             ["/hello" {:get (-> #'handler
                                 (wrap-db ds)
                                 wrap-exception
                                 trace-http/wrap-server-span)}]))
                                 
(def app (ring/ring-handler router
                            (ring/create-default-handler)))

(def server (jetty/run-jetty #'app {:port 3000, :join? false}))
;; (.stop server)

There are several interesting bits to be aware of. First the handler is wrapped in several middleware functions, one to pass the database connection, the other to wrap the exceptions (such as in the tutorial) and finally the middleware to wrap a server request. The db->value creates its own span to keep track of its activity.

After making several requests you will see that Jaeger contains the same amount of traces. A normal trace will show 3 bars, each of which you can expand and explore.

A trace in Jaeger

If you take the database offline (that is why we used Postgres), you will notice that the exception is neatly logged.

Exceptions in Jaeger

Observability allows you to get a great insight into how you application is running in production. With the clj-otel library it is a breeze to enhance your own application.

#clojure #web #observability #programming

In {{< backlink “a-new-theme” “my previous post”>}} I highlighted that I set myself the goal of creating a self hosted comic book collection tool. Before that, in {{< backlink “choose-your-tools” “a post about tooling”>}}, I reiterated my ❤️ for {{< backlink “clojure” “Clojure”>}} as a language. So, this is the start of a series of articles detailing how the development is going, and also as an introduction to the various parts of the tech stack.

Clojure is special to me in that there are hardly any big frameworks in the ecosystem. Clojure is more like Lego, there are countless building blocks of various shapes and sizes. It is up to you as the developer to stick the blocks together to get something usefull. You might guess that I also ❤️ Lego.

{{< admonition type=“tip” >}} On youtube you will find various series that detail the creation of Clojure apps. Check out:

If you would like to be added to this list, send me a message: @credmp@fosstodon.org {{< /admonition >}}

So, today I am starting with the first component of my techstack: Metosin's Reitit.

What is reitit?

Metosin's Reitit is a highly performant and extensible routing library for Clojure and ClojureScript applications. It provides a declarative way to define routes for web servers. Reitit integrates seamlessly with Ring, enabling middleware and handler chaining, and offers robust features like path parameters, route coercion, and schema validation.

It is easy to get started with, but is flexible enough to provide everything we need in any type of API. In this post I am going to show you the essentials to get a workflow up and running.

{{< admonition type=“tip” >}} The reitit documentation is extensive and very valuable, read it here. {{< /admonition >}}

A very simple API

There are many ways to start building an API, and pretty much everything is ok. I like to start from the handler and then work my way down all the way to the http server.

A handler

A handler is the code that, well, handles the request. Let's create a Hello World handler, its only task is to return a map which has a :status key and a :body key.

The :status represents the HTTP status code that should be returned, in this case 200 – all is good. The :body will be a string for now. In a later post it will become JSON, but to get started a string is fine.

(defn handler [request]
  (println "Handling" request)
  {:status 200
   :body "hello world!"})

That was quite easy, right? The handler is a function, so it can be called in the repl. As you would expect, it returns a map with the data.

(handler {})
;; => {:status 200, :body "hello world!"}

In the application the handler has to be connected to a URL endpoint, a so-called route.

The router

The router connects routes to handlers. The routes are defined using vectors ([]). The handler that was defined earlier is a greeting, an endpoint for such a thing might be /hello (or /greet, but it is always /greet...). The endpoint becomes a route when it is combined with a method to get there.

In HTTP there are several methods: POST, GET, PUT, DELETE, and a bunch more. These methods are the way HTTP tells the server to create, read, update and delete something on the server.

In this case the handler is only asked to return some information, so a GET method is the right choice here.

(ns blogpost
  (:require
   ;; add these
   [reitit.ring :as ring]
   [reitit.core :as r]))
   
(def router (ring/router
             ["/hello" {:get #'handler}]
             ))

{{< admonition type=“note” >}} I am using #'handler here, which is the same as (var handler) to refer to the var named handler. It is used to reference the var itself instead of its value.

During development this means that the var's value can be updated and the result will immediately be available in the web server, with no need to restart the server. This helps greatly in the development experience. {{< /admonition >}}

With the router created it can be queried to ensure everything is as expected. This is a good way to check what kind of middleware or interceptors are applied to the routes. Currently there is none of that magic going on, but later-on it might be necessary to confirm that the configuration is correct.

An interesting fact, when a route is created or a get, reitit will also create an options route. This is to satisfy browsers and frontend tooling that will request some metadata (options) before calling a potentially expensive, in time, method.

;; return all routes in the router
(r/routes router)
;; => [["/hello" {:get {:handler #'core/handler}}]]

;; retrieve the path within the router
(r/match-by-path router "/hello")
;; => {:template "/hello",
;;     :data {:get {:handler #'core/handler}},
;;     :result
;;     {:get
;;      {:data {:handler #'core/handler},
;;       :handler #'core/handler,
;;       :path "/hello",
;;       :method :get,
;;       :middleware []},
;;      :head nil,
;;      :post nil,
;;      :put nil,
;;      :delete nil,
;;      :connect nil,
;;      :options
;;      {:data
;;       {:no-doc true,
;;        :handler #function[reitit.ring/fn--14482/fn--14491]},
;;       :handler #function[reitit.ring/fn--14482/fn--14491],
;;       :path "/hello",
;;       :method :options,
;;       :middleware []},
;;      :trace nil,
;;      :patch nil},
;;     :path-params {},
;;     :path "/hello"}

With a router defined, the ring handler can be constructed. It is confusing that there are multiple handlers now, so lets refer to the ring handler as the app (or application handler), basically a fully wired up application that can process requests.

The application handler

Constructing the app makes it possible to take a request map, the thing the webserver will receive from a client, and route it to the handler. The handler will then process the request and will return a result. The app will return the result to the client.

For now the ring-handler can be constructed with the router that was created earlier and the ring/create-default-handler. The default handler ensures more correct error responses are created. It differentiates :not-found (no route matched), :method-not-allowed (no method matched) and :not-acceptable (handler returned nil).

(def app 
  (ring/ring-handler 
    router 
    (ring/create-default-handler)))

The ring/ring-handler returns a function. That function can be called with a request map to test it out. Passing a request to the app for an endpoint that does not exist should return a 404, HTTP's way of saying “I have no idea what you want from me”.

(app {:request-method :get, :uri "/clojure"})
;; => {:status 404, :body "", :headers {}}

But calling the route that was defined ealier should yield a very welcoming message.

(app {:request-method :get, :uri "/hello"})
;; => {:status 200, :body "hello world!"}

It works as expected! The final step is to actually connect it to a webserver.

Making it available as a service

The Jetty server is a battle tested http server. It is very easy to use through the ring adapter. By calling run-jetty, and passing in our app (again as a var reference for easy development), the endpoint will finally become available online (on our system).

There are 2 important parameters that are passed to jetty; :port and :join?. The port tells jetty on which port the server should bind, anything about 1024 is good here.

:join? tells jetty to run in its own thread and allows the repl to accept other commands. If it was not passed the repl would have to be restarted to stop the server. The result of run-jetty is stored in server.

;; add a require
[ring.adapter.jetty :as jetty]

(def server 
  (jetty/run-jetty #'app 
                   {:port 3000, :join? false}))

Using a tool such as curl it is now possible to query the API. You can also use the browser of course!

$ curl -v localhost:3000/hello
* Host localhost:3000 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:3000...
* Connected to localhost (::1) port 3000
> GET /hello HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< Date: Tue, 07 Jan 2025 21:26:36 GMT
< Transfer-Encoding: chunked
< Server: Jetty(11.0.24)
<
* Connection #0 to host localhost left intact
hello world!%

From the result (which is verbose due to -v) it is clear that the Jetty server is responding (note the Jetty(11.0.24) line in the headers). Also, there is the very welcoming hello world message at the bottom.

In the repl it is possible to make changes to the handler. After evaluation the API should immediately return the updated message.

To stop the webserver either close the repl, or call .stop on the server var.

(.stop server)

This is a first small step to a new API. Reitit has many things to offer, I would recommend checking out the docs and the examples.

#clojure #web #programming

{{< admonition type=“note” >}} Originally posted on 2024-09-30 (Monday). It was updated in January of 2025. {{< /admonition >}}

I ❤️ to build software. I sadly do not have a lot of time next to my daily work to spend on my side projects, so I have to be disciplined in where I invest time. I wish I could spend endless amounts of time on exploring new technologies, but sadly I simply do not have that time. In writing this is sometimes referred to as “to kill your darlings”.

Sir Arthur Quiller-Couch wrote in his 1916 book On the Art of Writing: “If you here require a practical rule of me, I will present you with this: ‘Whenever you feel an impulse to perpetrate a piece of exceptionally fine writing, obey it—whole-heartedly—and delete it before sending your manuscript to press. Murder your darlings.’”

Luckily for me, I just finished my latest round of education, so I now do have time to spend on building some of the ideas that have been floating around in my head for the last 3 years. And I did start out writing stuff. Some in {{< backlink “rust” “Rust”>}}, some in Go and others in Clojure.

Like many programmers I love to explore new languages, I think you always learn something new from them. As Clojure really taught me about functional programming when all I knew was imperative languages. In the end, after having a summer of not working on my studies I have 0 projects completed, but I do have 4 versions of them.

So, I decided to step back and evaluate. I decided to kill my darlings of different programming languages and focus solely on Clojure again. Development in Clojure conforms to Rule 6 for me. While working out the problem I love the interactive build method. I actually like the parentheses, I know... weirdo me 🤗.

update 2025: during the holiday season I got the book Zero 2 Prod, which is a book about making Rust project production worthy. Experience I already have in Java and Clojure. This sparked rule 6 for me for the {{< backlink “rust” “Rust”>}} language again. The experience following the book has been quite smooth, but the real proof is, of course, creating something yourself. I know, I am like a {{< sidenote “puppy” >}}I love puppies!{{< /sidenote >}} puppy chasing his tail... Let's see where this goes.

From reading the book I already see lots of improvement for my Hed tool.

You might even remember that I used to do a live-streaming series in Clojure. I still don't have a lot of time to continue that one, but who knows... I might drop some videos later again.

Since the summer I have been somewhat involved in Biff, a rapid prototyping web framework in Clojure. It provides a set of sensible defaults to just get started, and it allows you to easily change all its bits. I have been building my latest project on top of it, which, with a bit of luck, might even make it to production.

#clojure #development #rust #emacs

This is my first article in a series called Rock Solid Software. In it I explore different dimensions of software that does not simply break. You can write good software in any programming language, although some are more suited to a disciplined practice then others, Clojure is definitely in the relaxed space of discipline here.

Today I am exploring the use of Selmer templates in Clojure. If you have explored Biff at all you will know that all the UI logic works by sending Hiccup through a handler, which will turn into HTML through rum (specifically the wrap-render-rum middleware). If you provide a vector as a result for an endpoint, it will be converted to HTML.

;; You provide this...
[:h1 "test"]
;; => [:h1 "test"]

;; It will then be converted to HTML
(rum/render-static-markup
  [:h1 "test"])
;; => "<h1>test</h1>"

This is absolutely great for rapid prototyping, however it becomes quite tedious if you want to test it. The idea of testing a function is to provide it some inputs and to validate if the outputs match the expectation. Verifying if HTML matches an expectation, or a vector of hiccup for that matter, is quite difficult.

To increase testability I added selmer to my project. This separates presentation from data by having selmer render templates with a map of data. Selmer is based on Django templates, which means that it has a rich set of features, such as extending base templates, defining blocks and providing control structures such as if and for loops. A very simple template looks like this:

{% extends "_layout.html" %}

{% block content %}
<article>
Hallo <b>{{name}}</b> from simple template
</article>
{% endblock %}

As the template extends _layout.html, lets take a look at that as well. I have stripped it down to the bare minimum here, but you might expect scripts, css, nav bars and many other things in the base template. The important thing here is that the block has the name of content, and our snippet above also has a block called content, the above article will be put inside the main below.

<!doctype html>
<html class="no-js" lang="">
  <head>
    <title>{{title}}</title>
  </head>
  <body>
    <main id="main">
      {% block content %}
      {% endblock %}
    </main>
  </body>
</html>

All that is left is to provide a middleware that will handle the selmer return type from an endpoint. In this case a map with a :template and :content key. If both keys are inside the response, the given template will be rendered using the content map.

(defn wrap-render-selmer [handler]
  (fn [ctx]
    (let [response (handler ctx)]
      (if (and (map? response) (every? response [:template :content]))
        (let [res (selmer/render-file (:template response) (:content response))]
          {:status 200
           :headers {"content-type" "text/html"}
           :body res})
        response))))

My new authentication function has become quite simple, provide a login page using the auth/login.html template. Of course it requires a whole bunch of different attributes in order to render the whole page, but another wrapper adds all the required metadata known to the application already, such as css and script files, application settings and even theme information. All the endpoint has to take into account is its own required information.

(defn login [{:keys [params] :as ctx}]
  (let [err (:error params)]
    (ui/page ctx {:template "auth/login.html"
                  :content (merge {} (when err {:errors [err]}))})))

This is all great and all, but it has nothing to do with testability, right? Well, a map is easier tested then an unstructured vector, right? In other languages, such as Rust, you can get compile time validation of templates, which is great! Sadly selmer does not have that, however we can just simply render a template file and see if there are any missing values.

The below snippet takes the “missing” value and replaces it with a placeholder. So, given a template and an endpoint function we can easily check if all required entries are provided in the map. The below function renders the template, provides a csrf token which is not available inside testing, and verifies that the template does not have any missing values.

(defn missing-value-fn [tag _context-map]
  (str "<Missing value: " (or (:tag-value tag) (:tag-name tag)) ">"))

(selmer.util/set-missing-value-formatter! missing-value-fn)

;; Ensure all the page's required fields are present.
(deftest selmer-validation
  (let [s (sut/page {} {:template "_layout.html" :content {}})
        ;; CSRF is not set during testing...
        res (selmer.parser/render-file (:template s) (assoc (:content s) :csrf "csrf"))]
    (is (not (str/blank? res)))
    (is (not (str/includes? res "<Missing value:")))))

Another step into building rock solid software.

#clojure #development