Game development in Clojure : Alchemy 7DRL post-mortem

Posted: March 21, 2013 in Uncategorized

Last week I joined the “7 Day Roguelike Challenge” and wrote the game Alchemy. The goals I set myself were.

  • Complete a playable roguelike game in 7 days
  • Learn more about Clojure!
  • Share some of my experiences

Having successfully completed the first two goals, this is my attempt to deliver on the last point. In this (somewhat extended) post I’m going to describe my experiences using Clojure for the 7DRL challenge – with discussion how to get the most out of Clojure for game development.

Overall design considerations

I started out with a rough sketch of what I wanted to do, and thought through the implications from a coding perspective.

  • The goal was to make a traditional roguelike – i.e. ASCII graphics, no fancy animations, focus on dungeon crawling, turn-based gameplay, lots of interesting objects and interactions. On average, this simplifies engineering problems (no animation! low performance requirements!) but complicates architecture (how can you manage the code to create lots of interesting, special-case object behaviours without the code becoming a mess?). Clojure seemed a good fit for this.
  • I wanted to write the game in a purely functional style. Functional programming is one of my favourite features in Clojure, and games are traditionally coded with more of an imperative / OOP style so this seemed like a fun challenge. This introduced my first major design constraint: my game state must be immutable. I knew that persistent data structures with structural sharing would be key to make this work, but game data structures are quite large and performance sensitive, so this is venturing a little into uncharted territory. The only place I would allow mutability would be in the GUI (where it is somewhat necessary…)
  • I wanted to use a prototype based object system (or at least a data-driven functional variant of this…). This approach worked extremely well in my old roguelike Tyrant , which allowed you to define new object types in just a few lines of code (even in Java!) and avoided many of the pitfalls of inheritance-based object systems. So I was keen to try out this style in Clojure.
  • My theme was Alchemy, inspired by the fact that the multi-coloured, mysterious potions were my favourite item type in the game that started it all, the original game Rogue. I wanted to have potions that created a wide variety of different effects on the player, which suggested the need to generalise this into an effect system. This would need to provide the ability to define effects, integrate them into various items, and apply them to the player or monsters.
  • I wanted to be 100% cross platform. Fortunately running on Clojure . the JVM solves most of this problem for you straight away. I was left with a choice of what graphics frameworks to use. I could have used lanterna for a Java-based text console, and even considered LibGDX which looks extremely promising. But in the end I decided to play it safe and use good old Swing and my swing-console component. The nice thing about Swing is that it is well tested, guaranteed to be part of the Java runtime environment, and offers “good enough” performance for graphically simple games.

Planning and project structure

With a seven day time constraint, there was little time to waste.

In particular, I knew I couldn’t afford to do any big refactorings or engine redesigns mid-way through. One of Clojure’s few weaknesses compared to Java relates to the difficulty of refactoring (see: Something I still love about Java). There are no static types to help you. You need to refactor tests. Tool support for refactoring is limited. Lack of circular dependencies restricts your code layout. etc. etc. Anyway, suffice it to say I wanted to avoid having to go through any pain here.

So I made a rough plan:

  • Ensure I had a good tool setup. Always task No.1!
  • Code the data structures first. Everything else would depend on the data structures – so it was critical to get this right first.
  • Define the object library and prototype based item definition. I need some test objects to populate my data structures, after all…..
  • Next, build a GUI with map visualisation – doing this early is critical to test out the data structures if nothing else: there is no better way to debug than to use our amazing visual perception system to understand the data. Also, you only start to feel like you are getting somewhere once you have a little “@” walking around a map
  • Then work on game logic / state updates. I have this big immutable data structure, so need code to create new updated versions whenever something happens in the game. All the game logic will depend on this, so it is critical to get this working.
  • Then work on the dungeon generation. Important to get good randomised dungeons in a roguelike, also a good opportunity to create new items (in the object library) and new game logic as the world gets created and refined.
  • Finally spend some time fine tuning and balancing – needed to get a fun game with an appropriate difficulty level. Lots of ideas to play with. I decided I would go for a REPL-based environment so I could experiment with different ideas on the fly

In hindsight, this was a pretty good plan. Though it caused me to spend a bit too much time on the engine up-front: I only really got into gameplay / content creation on days 4-7 which was a bit too late for my liking.

The rest of this post tackles these areas in order.

Tool setup

My tool setup was as follows:

  • IDE is Eclispe with the excellent Counterclockwise plugin
  • Maven for the builds / dependency management
  • GitHub for the SCM
  • Clojure 1.5.1 (because it came out just in time for the 7DRL!)

Nothing too fancy here. Some people may find my choice of Maven over Leiningen a bit unusual, but I prefer it for working in Eclipse (better integration, better support for polyglot Java+Clojure projects etc.).

The only thing I tested at this stage was that I could build a runnable uberjar with all dependencies packaged. I knew I would need this to distribute my finished game, and didn’t want to be figuring it out at the last minute.

Fortunately it’s easy with the maven-assembly-plugin. After a bit of setup in the pom.xml it is just a one-liner to make the build work:

mvn install assembly:single

Et voila: you now have an alchemy-0.0.1-SNAPSHOT-jar-with-dependencies.jar file in your /target subdirectory that you can rename and distribute as you see fit.

Game data structures

Everything in the game state was going to be immuatble. So I needed to define a data structure that would contain all information about the game state. Adopting  standard Clojure techniques, I decided to use a defrecord for the top level data structure:

(defrecord Game [^PersistentTreeGrid world    ;; grid of terrain
                 ^PersistentTreeGrid things   ;; grid of things (contains vectors)
                 thing-map                    ;; map of id -> thing

The PersistentTreeGrid fields are instances of mikera.engine.PersistentTreeGrid - a Java class that I created for a previous project that stores a 3D grid of data as an immutable, persistent data structure. It’s like a Clojure vector, but with (x,y,z) indexing. Just what we need! A nice thing about immutability is that any immutable objects compose together nicely, plus Clojure has very easy Java interop, so we are able to use this handy Java class unmodified.

The world is going to contain our world map tiles (floors, walls etc.). Only one tile is allowed in each grid location. The things are any items on the map (the player, monsters, items, doors etc.). We need to allow for the potential of multiple items in each tile, so things field is a grid of vectors of things (most of these will be null == empty).

The thing-map is not strictly necessary, but I had a hunch that performing lookup by thing ID was going to be such a frequent operation that it would need an index of (ID => Thing). Without this, finding a Thing by ID would require walking the entire grid of things. As it turned out, this was essential for performance.

Things themselves are just a very simple Clojure defrecord:

(defrecord Thing [])

I could have used plain maps here. I don’t think it would have made any difference in this case, but I wanted to have a clearly defined concrete type for Thing. This could help with debugging, or enable Thing to implement protocols if required (I never actually did this in Alchemy, but it could conceivably be useful).

I also defined a Location data type to represent (x,y,z) co-ordinates:

(deftype Location [^int x ^int y ^int z]
    (nth [loc i]
      (let [li (long i)]
        (case li
          0 (.x loc)
          1 (.y loc)
          2 (.z loc)
          (throw (IndexOutOfBoundsException.
                     (str "index: " i))))))
    (nth [loc i not-found]
      (let [li (long i)]
        (case li
          0 (.x loc)
          1 (.y loc)
          2 (.z loc)

In this case I used deftype rather than defrecord. I wanted Locations to be as lightweight and efficient as possible: most Things would have a location, and I would want to pass around a lot of Locations as parameters to functions or manipulate them in various algorithms.

For convenience, I made Location implement a few interfaces such as clojure.lang.Indexed, clojure.lang.IFn etc. The main  reason for this is that it enabled me to support some nice notation tricks:

(let [a (loc 1 2 3)
      [x y z] a      ;; destructuring on locations!
  (println (a 2))    ;; IFn: access z co-ord i.e. 3

Everything else in the data structures would just be properties – indexed with keywords and assoc’d into the relevant defrecords. So the following would be a valid definition of a Thing:

(map->Thing {:name "Bob" :age 100})

See how using regular Clojure associative maps gives us a lot of flexibility? There is no need to pre-define all the properties or fields we want to use in our game in a rigid class definition. This is a big win – it made it much easier to implement a large number of custom properties and item types later.

The hard part of the data structure definition was in defining all the functions to create, read and update the data structures. This was hard: some of the tricky things I had to deal with:

  • Things needed to be stored twice: once in the thing-map field (map of ID -> Thing) and once in the things grid (a vector of things in each map location). These needed to be kept identical: if they got out of sync then “bad things happen”. Consistency problems are a nightmare to debug.
  • Hierarchical storage of Things (the hero is on the map, a potion is in the hero’s inventory). This required handling two cases for most updates (thing is on map vs. thing is contained within another thing). Because the game state is immutable, if you update a child you have to update the parent as well…..
  • Implementing modifiers – i.e. the ability for child Things to modify the properties of their parents. This was an important feature for the effects system (e.g. potions of weakness reducing the strength of whoever is affected by them). This meant that I had to write a custom property reading macro (? thing :property) rather than just doing the simple (:property thing) which would return the unmodified property.

The most useful coding trick I used here was encapsulating the complexity behind a small number of functions. Every other part of the code base would call this mini-API to make game state updates, so I could keep the complexity of handling the data structures in one place. The main functions in this API were:

  • ? : query a property of a thing (taking modifiers into account)
  • ! : modify a property of a thing
  • get-thing : get the latest version of a thing within the game (indexed by :id)
  • add-thing : add a thing to a map location or another thing
  • remove-thing : remove a thing from the game (wherever it is)
  • update-thing : update a thing in its entirety
  • merge-thing  : a variant of update-thing, takes a map of new properties to merge in

Together these were enough to implement nearly all the game logic required, and succeeded in encapsulating most of the complexity of game state update. Note that any functions which cause modification / update return an entire new Game data structure.

One pitfall I fell into several times: multiple versions of your game data can structures exist, and it is easy to use the wrong version by mistake. An example would be modifying the game, but then passing the old version of the game on to the next function. Whoops – it is as if your modification never happened. This problem hit me several times, and wasn’t fun to deal with: the errors are hard to detect, and there isn’t an easy way to see where you made the mistake. I think nearly all of the nasty bugs that I had to deal with during the week were in this category.

My overall take on this phase of data structure work:

  • Making everything immutable in Clojure is harder than it would have been in an OOP language like Java where everything can be encapsulated in mutable classes. In particular, the state update functions are tricky to make both correct and performant.
  • The payoff is big however: in terms of the simplicity and effectiveness later on, and in the conceptual clarity being able to treat the entire game state as an immutable value
  • It is absolutely critical to encapsulate the complexity of state updates behind a small number of functions. I think implementing Alchemy in 7 days would have been impossible if every part of the code base had to deal with the complexity of the data strcutures. Choosing the right API / level of granularity for these functions is important if you want to save some headaches.

Overall this probably took 2-3 days out of my week! Maybe the best part of a day at the start, then many more hours debugging / adding more features throughout the week. A huge time investment, but I think it was worth it to get a solid foundation for the rest of the game.

Object library

The purpose here is to create an effective way to define the library of objects that can be created in the game. Ideally we want a DSL that makes it easy to define and maintain a large number of objects. Here’s an example of the kind of code we are aiming for:

    (proclaim "cobra" "base snake"
                   {:level 5
                    :SK 4 :ST 3 :AG 10 :TG 7 :IN 4 :WP 9 :CH 8 :CR 3
                    :char \c
                    :hps 15
                    :attack (merge ATT_POISON_BITE {:damage-effect "poisoned"
                                                    :damage-effect-chance 40})
                    :colour-fg (colour 0xD0A060)})

i.e., we want to be able to easily define creatures with an arbitrary set of properties.

Conceptually, I decided that the object library should be stored within the Game data structure. It could have been separate: however I did it this way for two reasons:

  • I liked the idea of having some random generation within the library itself, so it made sense to generate a different object library for each Game. 
  • It removed the need to manage another element of state outside the Game data structure
  • At times, it makes sense to modify the library during game progress (e.g. you identify a particular item, and want to mark that all items of the same class should be considered identified from now on)

The actual library generation was initiated with the following code:

(defn build-lib []
  (as-> {:objects {} ;; map of object name to properties
    (define-objects lib)
    (assoc lib :objects (post-process (:objects lib)))
    (assoc lib :objects (assign-potion-ingredients (:objects lib)))
    (or lib (error "Lib creation failed: null result!?!"))))

(defn setup
  "Sets up the object library for a given game"
  (assoc game :lib (build-lib)))

As you can see, we are just assoc’ing the item library into the game in the “setup” function. The more interesting part is the sequential steps to build the library, which I perform with the “as->” macro (which is my favourite feature in Clojure 1.5, by the way!). This successively rebinds “lib” to the result of several steps.

  • It starts out as just {:objects {}} ( no-objects defined)
  • We then define all the objects with (define-objects lib) which returns a new library map populated with all the objects.
  • We then perform a couple of post-processing steps on the whole library (I won’t go into details, but it is stuff like algorithically assigning random colours and ingredients to potion definitions, filling in default value etc.)

The definition of objects happens heirarchically: no particular reason other than it helps me to structure the generation of items into different item types so that I can navigate and maintain the code easier. We are doing this in a functional style, so we use the threading macro “->” to make successive additions to the library:

(defn define-objects [lib]
  (-> lib

The lower level definition functions update the library via successive “proclaim” calls, e.g.

(defn define-tiles [lib]
  (-> lib
    (proclaim "base tile" "base object"
              {:is-tile true
               :z-order 25})
    (proclaim "base wall" "base tile"
              {:is-blocking true
               :is-view-blocking true
               :colour-fg (colour 0x808080)
               :colour-bg (colour 0x808000)
               :char (char 0x2591)
               :z-order 50})

And the proclaim function itself is just a convenience function to build an item from the properties of a parent item and add it to the :objects map of the library:

(defn add-object
  "Adds an object definition to the library. Updates any indexes necessary"
  ([lib obj]
    (assoc lib :objects
           (assoc (:objects lib)
                  (or (:name obj) (error "Trying to add unnamed object to lib!: " obj))

(defn proclaim
  "Adds an object definition to the library, deriving from an existing named object"
  ([lib name parent-name obj]
    (let [objects (:objects lib)
          parent (or
                   (objects parent-name)
                   (error "Parent not found: " parent-name))
          obj (merge parent obj)
          object (merge obj {:name name
                             :parent-name parent-name})]
      (add-object lib object))))

Finally, you need some code to generate things form the library. This is pretty simple: just look up the library definition for a specific name, then create a new Thing with the library properties. There are a couple of nice extra features I added to this:

  • The ability to run an :on-create function to customise an item when it is created. This can be used for stuff like adding items to the inventory of a monster, or randomising some properties on a per-new-instance basis
  • Random creation according to flags. You can do create “[:is-reptile]” to create a random creature that has the :is-reptile flag (so you can create a pit full of various snakes etc.). Relative frequency of each item is controlled by the :freq property, and the level of object creation by the :min-level and :max-level properties (so that you don’t generate dragons or troll kings on Level 1…..)

Hopefully that gives a good overview of how the library generation works. I’m pretty pleased with the design, key points are:

  • It’s fully functional, and the library becomes an intrinsic part of the game state.
  • You can have any kind of properties you like: including functions like :on-create and :on-consume and :on-death to define custom responses to game events. This makes it very easy to add customised behaviour in one place (the library definition) rather than scattering special-case code throughout the code base

GUI / main loop / event handlers

Obviously, the key purpose of a game it to have the player interact with it! Also GUI interactions are inherently stateful, so it was an interesting design challenge to determine how to make the GUI interactions work together with our immutable game state.

Key elements of the design I went for:

  • All of the GUI / state handling went in a single namespace: main.clj
  • Support for multiple instances of the game (no global state!)
  • Ability to run/restart game from within the REPL or as a separate application

At the heart of everything was just two functions that initiate the game and launch the GUI in a new JFrame:

(defn new-state
  "Create a brand new game state."
    (let [game (world/new-game)
          state {:game (atom game)
                 :console (new-console)
                 :frame (new-frame)
                 :event-handler (atom nil)}]

(defn launch
  "Launch the game with an initial game state. Can be called from REPL."
    (def s (new-state))
    (launch s))
    (let [^JFrame frame (:frame state)
          ^JConsole jc (:console state)]
      (setup-input jc state)
      (.add (.getContentPane frame) jc)
      (.pack frame)
      (.setVisible frame true)
      (main-handler state)

Dissecting this:

  • We have a “state” object that encapsulates all the mutable GUI state
  • State object contains a :game atom (which contains an immutable Game). We will use this atom to swap in updated game states when changes occur. The initial game state is provided by a function “world/new-game” which you can safely assume does all the necessary game setup.
  • The state object also contains references to the JFrame and JConsole gui components
  • Launching the game consists of populating the state object, performing some setup and displaying the GUI compoennts to the user.
  • The state itself isn’t global: so if you run (launch) several times you get independent frames with independent game state
  • The only global piece is (def s ….) which stores the most recently created state in a top level var in the mikera.alchemy.main namespace. Why? Just convenience, so that we can access this state easily from the REPL…..

Now this is a roguelike game, so we don’t have to deal with a time-based game loop. Every action gets driven by a keyboard event.  The (setup-input jc state) does the boring details of registering key bindings for all the keys we are interested in.

What is more interesting is what gets called when the key get pressed: which is provided by the the mysterious (main-handler state) function.

(defn main-handler
  "Sets up the main handler"
    (redraw-screen state)
    (reset! (:event-handler state) (make-main-handler state))))

This is deceptively simple. It redraws the screen (so you can see the current world!) and sets up the event handler in the state to be ready for the next keypress. And that keypress will be handled by whatever is returned by (make-main-handler state). SImple right? Here is make-main-handler:

(defn make-main-handler
  "Create main keypress handler, for general game position"
    (fn [k]
      (let [game @(:game state)
            k (map-synonyms k)
            game-over? (:game-over game)]
          (= "i" k) (show-inventory state)
          (= "l" k) (do-look state)
          (= "m" k) (show-messages state)
          game-over? :handled  ;; stop here if game is over

          (.contains "12346789<>" k)
              (swap! (:game state) world/handle-move (or (move-dir-map k)
                                                         (error "direction not recognised [" k "]")))
              (redraw-screen state))
          (= "a" k) (choose-analyse state)
          (= "c" k) (choose-alchemy state)
          (= "d" k) (choose-drop state)
          (= "q" k) (choose-quaff state)

Now this is clever:

  • This is a higher order function, that returns a newly created  event handler.
  • The event handler captures the state in a closure . So the event handler doesn’t need to be passed a copy of the state to work on: it has been generated specifically to work with a specific state object. Hence we can have muliple instances of the event handler working with different state instances. We’ve used a closure to avoid global state.
  • The event handler’s main purpose is to examine the keypress and perform the correct action.
  • In the most basic case it handles the action directly: handling a single movement direction. In this case it calls world/handle-move to update the game, swaps in the result to the :game atom, and redraws the screen (ready for the next keypress)

The more interesting case is when a command takes us down a different command route: if the player decides to drink a potion with “q” for example. Here the handling needs to be quite different:

  • We want to show a list of potions rather than the regular screen
  • We need to capture and handle the next keypress in a different way (selecting a potion to drink)

2013-04-14 Inventory Screen

So we need to do a completely different kind of handler setup. This is what the “choose-quaff” function does:

(defn choose-quaff [state]
  (let [game @(:game state)
        hero (engine/hero game)
        inv (vec (filter :is-potion (contents hero)))]
    (item-select-handler state "Quaff a potion:"
                      (vec (map (partial engine/base-name game) inv))
                      (select-item-pos inv)
                      (fn [n]
                        (swap! (:game state) world/handle-consume (inv n))
                        (main-handler state)))))

This does the same kind of thing  “main-handler” but delegates to “item-select-handler” to allow the user to select a potion. Things to note:

  • There is logic to define the set of items to consider. Clojure makes this easy: In this case we just filter on :is-potion to ensure that the player isn’t invited to quaff other kinds of objects (plate armour?), which wouldn’t make much sense.
  • item-select-handler is a higher order function: it takes a function that describes what to do once the item is selected (in this case, consume it and return to the main handler to wait for the next action)
  • again, the state parameter is passed through to ensure we always work on the same state object. This lets us access the latest Game object via an atom lookup with @(:game state)
  • world/handle-consume is a pure function that returns a new Game object for us to update the :game atom. This is exactly what we want: we have converted the messiness of stateful event handling into a pure function call into the game logic.

item-select-handler looks like this:

(defn item-select-handler [state msg items pos action]
  (reset! last-item-pos pos)
  (let [c (count items)]
    (redraw-item-select-screen state msg items pos)
    (reset! (:event-handler state)
          (fn [^String k]
            (let [sel (.indexOf "abcdefghijklmnopqrstuvwxyz" k)]
                (and (>= sel 0) (< (+ pos sel) c))
                  (action (+ sel pos))
                (and (> pos 0) (.contains "784" k))
                   (item-select-handler state msg items (- pos 26) action)
                (and (< (+ pos 26) c) (.contains "623" k))
                   (item-select-handler state msg items (+ pos 26) action)
                (= "Q" k)
                  (main-handler state)

Again things to note:

  • It does a screen redraw, but in this case draws the item selection screen rather than the main screen
  • It also installs a key event handler, but this one responds to a different set of key presses that correspond to the item selection screen logic.
  • The event handler calls item-select-handler again if it needs to redraw the inventory screen (e.g. scrolling up and down pages of items). This isn’t quite recursion (it is more like a trampoline, since the event handler is getting called later when the next key press occurs…). I think of the item-select-handler state as being like an immutable instance: you need to recreate a new version if you want to change the viewing position.
  • Again, the state is captured in a closure, and passed on / accessed as required.
  • item-select-handler is generic: it has nothing to do with quaffing potions! All the customisation necessary to allow it to handle quaffing potions is done through parameters (what items to list, what to do when an item is selected). This is a big win: the same code can be used to handle dropping items, throwing items, crafting new potions etc.

Hopefully this is a good overview of how the GUI / event handling in Alchemy works. It seems to work pretty well. There is some complexity (switch different event handlers all the time, continually passing around a reference to the mutable state object), but I think to a certain extent this is unavoidable when you are dealing with GUI interactions. And the core code ends up looking pretty clean

Most importantly, I think it does the job of handling the messy parts of state handling and keeping it out of the game logic itself, which I will talk about in the next section.

Handling game logic updates

We’ve now got to the happy place where our GUI is taken care of, and we just have to write a pure function that updates the game according to a player action. To continue with the example above, here’s the world/handle-consume function:

(defn handle-consume
  "Handles consuming an item (eating or quaffing)"
  [game item]
  (let [h (engine/hero game)]
    (as-> game game
      (engine/clear-messages game)
      (engine/try-consume game h item)
      (end-turn game))))

The logic here pretty simple:

  • First we clear and existing message. I should probably have factored this out into a “start-turn” function but this was the only thing that needs to be done….. maybe some other time
  • We delegate the actual action to the engine/try-consume function. Why a separate function? Well, handle-consume needs to perform actions related to the fact that it is a player turn, whereas try-consume is just the game logic for item consumption, which could be called by other actors (e.g. monsters drinking potions….). I like to keep the layers of responsibility clearly separate.
  • Finally we call an end-turn function. Among other things this does all the subsequent AI actions: moving all monsters on the map after the player has moved.

We’re using my favourite “as->” macro to handle successive operations. Did I say that I think this is the best feature in Clojure 1.5 ? I much prefer this to “->” for the following reasons:

  • The function calls look like normal: you aren’t inserting a “hidden” first parameter which can cause confusion. 
  • You can use “game” in different positions in each function call (and even more than once) so it is much more flexible than “->” or “->>”
  • Steps can have more flexible forms, e.g. an if statement like (if (some-update-needed? game) (do-update game) game) – handy if you only want to change the game in certain circumstances. Just remember the unchanged “game” at the end or you risk setting you whole game to nil…..

The engine/try-consume function deals with the actual consumption, and any results:

(defn try-consume [game actor item]
  (as-> game game
    (if-let [con-fn (:on-consume item)]
      (con-fn game item actor)
      (message game actor (str "You don't know how to consume " (the-name game item))))
    (use-aps game actor 100)))

Clever and concise…  here we just check if the item has an :on-consume property and delegate to this. This is important from a game engine architecture perspective: we want to define the results of consumption within the items themselves (via the item library), rather than putting a lot of special case logic in our engine code. Otherwise we would need to write a horrendous amount of complex code: eating food vs. potions, what to do if it is poisoned, what to do if the potion has any special effects etc. Much better to define this in the items themselves, rather than scattering this logic all over the code base.

The only thing we do in common is remove some aps (action points) to represent the time taken by the actor to consume the item. This is going to be true of any consumption, so it is fine to leave it here (otherwise every item :on-consume handler would have to remember to to this….. i.e. this is the DRY principle in action)

For completeness, here’s a potion definition with a :on-consume handler defined:

   (proclaim "health gain potion" "base potion"
              {:level 2
               :on-consume (consume-function [game item actor]
                             (let [boost (Rand/d (long* 0.5 (:TG actor)))]
                               (as-> game game
                                 (!+ game actor :hps boost)
                                 (!+ game actor :hps-max boost)
                                 (engine/message game actor
                                   "You feel amazingly good..."))))})

And thus we have a nice custom potion that boosts :hps and :hps-max . Defining new objects in Alchemy really is that easy. There is no other code anywhere related to the “health gain potion”. Finally, we are seeing the pay-off of all the hard work of encapsulating our data structure objects behind clearly defined update functions and structuring the code around prototype objects and an item library. Again, we see “as->” in action :-)

Dungeon generation

Roguelike dungeon generation algorithms are a big topic in their own right so I won’t go into too much detail on my approach here. The outline is:

  • Define the Dungeon as a large 3D block from (-35, -25, -11)  to (35 , 25 , -1). I used negative z-co-ordinates because I like to think of negative z as “going down”.
  • Recursively subdivide the dungeon, BSP style
  • At each sub-division, remember “connection points” between the divided areas
  • At the lowest level, generate individual rooms, tunnels and corridors
  • At the end perform some customisation (decorate rooms, add The Philosopher’s Stone on level -10, etc.)

If you are interested you can examine the dungeon generation code here.

The only interesting point from a Clojure coding perspective is this: sometimes parts of dungeon generation fail – e.g. inability to find a suitable location to add a specific item, or inability to find a way of linking two levels. In some cases the failure can be ignored (just make no change, e.g. it isn’t a big loss if some random decoration can’t be placed) but in other cases it is a problem (if it is impossible to connect the levels with stairs, then the game can’t be completed… whoops!)

I developed a quite nice strategy to deal with this:

  • Normally each function takes a Game and returns a new Game with a changed dungeon (in the case of no change or a non-important failure, the same Game may be returned). In this sense, we are doing regular functional programming updates.
  • In the case of a potentially fatal failure, nil is returned. Then it is the responsibility of the calling function to decide what to do (maybe retry with a different parameter configuration? maybe also return nil and let the level above handle it?)

This approach necessitated a couple of extra utitily functions.

The first was “and-as->” which is just like “as->” but bails out early and returns nil if any of the individual steps return nil. It’s just a simple macro:

(defmacro and-as-> [expr sym & body]
  `(as-> ~expr ~sym
     ~@(map (fn [b] `(and ~sym ~b)) (butlast body))
     ~(last body)))

But is very useful in lots of circumstances… e.g. in the decoration of a lab we want to fail the whoel function if for some reason we are unable to place the alchemy apparatus:

(defn decorate-lab [game room]
  (let [lmin (:lmin room) lmax (:lmax room)]
    (and-as-> game game
      (maybe-place-thing game (loc-add lmin 1 1 0) (loc-add lmax -1 -1 0)
                         (lib/create game "[:is-apparatus]"))
      (mm/scatter-things game lmin lmax (Rand/d 4) #(lib/create game "[:is-potion]")))))

In the code above, decorate-lab will fail and return nil if the “maybe-place-thing” returns nil, without trying to call the “mm/scatter-things” function (which would throw a null pointer exception if we passed a nil game parameter).

The second utility function was the “or-loop” macro, which performs a loop (up to a given number of tries) and returns the first non-nil value. This can be used to retry a specific operation until it succeeds, e.g. the generate-level function calls generate-zone a up to five times:

(defn generate-level [game ^mikera.orculje.engine.Location lmin
                            ^mikera.orculje.engine.Location lmax]
  (let [[x1 y1 z] lmin
        [x2 y2 z] lmax]
    (as-> game game
      (generate-tunnel game lmin lmax (rand-loc lmin lmax) (rand-loc lmin lmax) "underground stream")
      (or-loop [5]
        (generate-zone game lmin lmax [])))))

Finally I’d like to emphasise a really nice advantage of all the immutable data structures: it is easy to roll back failed generation! You can modify the dungeon to your heart’s content, and as long as you keep a reference to an earlier version (typically in the call stack as a parameter to one of the nested dungeon generation function calls), it is easy to roll back an continue from an earlier state. This is a big advantage over algorithms that mutate the state as they build it: they need to do a lot of extra work if they want to do rollbacks.

Fine tuning / exploiting the REPL

Towards the end of the 7DRL challenge, I set aside some time for fine tuning.

My strategy was to use the REPL intensively, to interactively code and test new features / items. I created a set of useful macros / tools to make REPL-based hacking on Alchemy simple. Here are a few examples:

;; launch a new game instance in a new window, binding the state to "s"

;; get the current Game state
@(:game s)

;; perform an action in the current state (update the current game)
;; e.g. summon a random level 10 item next to the player
(go (engine/summon game hero (lib/create game "[:is-item]" 10)))

;; do some code with "game" bound to the current game
;; e.g. count the number of Things in the game
(dog (count (all-things game)))
=> 1041

(dog (count (filter :is-creature (all-things game))))
=> 117

Creating and testing a new potion, for example, would only take a few minutes and would typically involve:

  • Adding a few lines to define the potion in lib.clj. Reload this at the REPL.
  • Do a “(launch)” to get a fresh game with the updated library.
  • Summon a few copies of the potion, try quaffing it etc. Maybe summon a monster to throw it at to see what happens.

Other tools I hacked together that proved useful included a set of debug options that could be dynamically rebound for abilities like walking through walls, having vision of the entire level, giving the player immortality etc. This made it much easier to test the dungeon generation. It helps enormously if you can get around quickly, see where you are going, and don’t get killed in the middle of testing some feature.

Basically, I managed to do most of my late-stage content creation and fine tuning in the REPL, without ever needing to relaunch the application. This was a huge productivity win – in fact I’m not sure I would have been able to complete the game without it.


Well this turned out to be a longer post that I expected, but I hope it proved an interesting tour of some of the challenges and opportunities for using Clojure in roguelike game development.

For anyone wanting to try out the game, you can download the latest version here:

  • Alchemy (my final 7DRL version)
  • Alchemy (latest release with extras / bug fixes)

And all the source code is on GitHub:

Some parting thoughts:

  • My faith in Clojure as a highly productive language is re-affirmed. I doubt I would have been as productive, or had as much fun, in any other language.
  • Immutability of game state is feasible for semi-serious game development. It introduces some complexity (especially around state update, which must be carefully encapsulated) and requires some planning around performance (especially indexing, intelligent use of persistent data structures), but the benefits are pretty big in terms of simplicity, expressiveness etc. I expect we will see this approach used more in the future.
  • Prototype based object models are awesome. Even in Clojure, where there seems to a cultural tendency to avoid anything that looks like an “Object”. It’s really “object behaviour definitions as immutable data”.
  • I still think Clojure needs more typing. A game is a complex beast and it’s a pain to have to pick up type errors at runtime that the compiler should have detected. I ended up having to write some type validation functions to run in tests – not a good use of human effort! I have high hopes for Typed Clojure in this regard.
  • Performance was not an issue. Perhaps surprisingly (since I was doing everything in FP style with and immutable game state), or perhaps not (since roguelikes aren’t really very demanding on performance on modern machines). Anyway, I was certainly very happy with Clojure in this regard.
  • I’d strongly encourage anyone interested in Clojure to try writing a game with it. I’m a big fan of game development as a learning exercise: It’s a great way to learn new techniques, it offers a huge variety of interesting problems to solve, it requires you to work towards a concrete goal and most importantly it’s a lot of fun.


About these ads
  1. embee says:

    Amazing stuff! I’m just learning Clojure and I’m sure this is a post I will go back to often. Thank you!

  2. Excellent overview! I especially like the order in which you described how the application was build, starting from the internal data structures and how they were used.
    There are a lot of blogs about various Clojure frameworks, libraries, and APIs. There are very few about Clojure applications.

    • mikera7 says:


      I guess it’s still early days with Clojure applications: I’m not sure if people have really worked out all the best practices yet. Certainly, I feel like I have been discovering a lot of good techniques by trial and error.

  3. georgek says:

    Great post mikera. I’m curious about a minor detail, the thing-map. It sounds like you noticed a performance hit without it, can you discuss that more?

    • mikera7 says:

      Sure – it turns out that nearly anything that happens requires thing lookups by ID. This is because the Game state is a value, and you often need to know the latest state of a Thing. If you have an old version of a Thing, you have it’s ID but any of the properties may be invalid so you need to do a fresh lookup. The thing-map makes these lookups O(1). If you didn’t have the thing-map, the lookups would be horrendous: approx O(N) where N is the number of grid squares in the world (i.e. a biggish number). I didn’t take exact benchmarks, but my guess is that the game would suffer a 10x or worse overall slowdown if thing-map didn’t exist.

  4. isd says:

    I was looking for such a good “how to make a roguelike in Clojure” tutorial for a long time!

  5. [...] Anderson: Game development in Clojure : Alchemy 7DRL post-mortem (and the previous 7 daily updates, Alchemy @ GitHub) – an interesting report about game [...]

  6. Alex Dowad says:

    Interesting post, thanks. What you said about using prototypal inheritance intrigued me, and I wished there were more details… perhaps you might consider doing a blog post specifically on that topic in the future?

  7. Thank you very much for sharing your experience in that detail!

    The code base of Alchemy relates to Tyrant by 3.5k to 110k LOC Clojure vs. Java. Is it possible to give a fair estimate of the relation of functionality between both?

    • mikera7 says:

      Engine wise I think Alchemy is a much better core engine, though it doesn’t yet have quite as much functionality. It’s probably 50% of what Tyrant has.

      Tyrant has a *lot* more content (maps, monster and item types etc.). Alchemy has perhaps about 15% of the content of Tyrant.

      Based on the stats above, I guess that remaking Tyrant in Clojure would need maybe 20k lines of Clojure? i.e. it’s probably about 5 times as concise overall.

  8. Pierre Masci says:

    Amazing article, thank you! I’m pretty new to Clojure so my question will be naive.

    Because the `game` is immutable, whatever happens anywhere in the “world” you have to update the `game`. I’m guessing that’s the reason why you pass it everywhere: `(as-> game game`.
    Is that right?

    I understand that it might be the best solution overall, however it seems a bit cumbersome. For example here:
    `try-consume [game actor item]`
    the code would be more readable if it was just an item being consumed by an actor, without needing to pass the game around. The game is not “consuming” an item, it’s not being consumed either, so what is it doing there? Some kind of omnipresent god-object role? Do you think there is a reasonable alternative solution, without resorting to mutable state?

    I have started to implement a (much simpler) game myself as a way to learn Clojure. The solution that I’ve come up with is exactly the same as you, because I want to keep my `board` immutable. I found your article by looking for alternative solutions. In another article with a similar solution, a commenter said: “you don’t need to pass your whole `world` around, just pass the details that are useful”. But how can we do that, as we must return a whole new game at every step that affects it?

    Any insight?

    Again, thanks a lot for this very interesting article =o)

    • alexdowad says:

      Pierre Masci: that is the “big question” which you must come to grips with when moving from imperative to functional programming. A couple possible solutions:

      1. Your “world” object is likely a nested data structure (a map of maps of …). In your “try-consume” function, it is possible you might need only one of those nested maps. In other words, you might need only one “piece” of the “world” object. If so, just pass in the piece you need, and return an updated version. Then, higher up in the call stack, update the world object.

      2. Rather than passing in the “old” world object, and returning an updated version, return a value which represents “what happened”. For example, maybe there are 3 possible outcomes of “try-consume”. Rather than returning a “new” world object, just return a symbol (or vector, map, etc) which indicates which of those 3 outcomes actually occurred. Then (again) higher up in the call stack, update the world object.

      Does anyone have another possible solution?

      • Pierre Masci says:

        Yes, I’d like to read other solutions too :-)

        I’m trying to understand the alternative solutions that you propose. I’ve jotted a few lines on a piece of paper and came our with this simple example:

        (as-> game G
        (while-not (end-of-game? G)
        (play-a-turn G player)))

        where (play-a-turn) must return an updated game. Now let’s be wild and replace (play-a-turn) by its insides:

        (as-> game G
        (while-not (end-of-game? G)
        (->> (prompt-for-move player)
        (move-actor G player)))

        The key here, is that at every step we must return a game. (move-actor) returns a game. And if the player drinks a potion, the (try-consume) function will take in a game, and have to return an updated game. It’s convenient in a way, because most functions manipulate and return the same value. But does it mean that functions have to know the exact data structure of the game, and are tied to it? That could make any change in this data structure pretty painful.

        As a solution, the alternative 1 that you just presented would look like this:

        (as-> game G
        (while-not (end-of-game? G)
        (->> (prompt-for-move player)
        (move-actor (:board G) player)
        (assoc G :board)))

        Is that right?
        In this case the (move-actor) function only knows about the game board, and returns a new game board. Each function returns an updated version of what they take in, but they only need to take in the relevant bit of data. (drink-potion) might only need to know about the (actor) and (potion) for instance.
        There’s an extra line of code to update the game, but the insides of (move-actor) are less coupled to the data structure of game.

        With your alternative 2, the function does not update the world. It processes some data and returns an outcome or a decision. Then it’s up to some other function to use this info to update the game appropriately. It doesn’t work so well with (move-actor), but I can see that being helpful pretty often.

        Wow, I’ve started Clojure 3 weeks ago and it’s a lot to take in, but it’s very interesting.

  9. Pierre Masci says:

    Without indentation it’s much less nice to read this code, sorry (^c^)

    • alexdowad says:

      Pierre Masci: you got both ideas exactly. It will be interesting to see if mikera weighs in with some other thoughts. By the way, while not specific to Clojure, the blog “Programming in the 21st Century” also has some useful ideas regarding functional programming.

  10. Pierre Masci says:

    Cheers, I’ll check out this blog.

    One more question in the same vein. Because we are dealing with chains of transformation on some `game` data, how to you deal with functions that can’t return a game? For instance if the UI is on the command line:

    (as-> game G

    (println “some message”) ; => returns nil, so we’ve lost our game data there! The chain is broken.

    I could fix it like this but it doesn’t look good:

    (as-> game G

    (do (println “some message”) G) ; => returns the game, all is good :-)

    One solution is to write a (say) function which will do this for us:

    (as-> game G

    (ui/say “some message”) ; => does the same as before, under the hood

    Is that the best solution in this case, or is there a workaround?

    • Pierre Masci says:

      Is there perhaps some kind of 0> macro which simply returns what is passed in? That would be a nice way to cause side effects in the middle of a chain. (io/say) is bad, because it would require to write a function for each type of side effect.

      (as-> game G

      (0> println “some message”)

      would be better… if it exists. Any better solution?

      • alexdowad says:

        Pierre Masci: I don’t know if mikera wants this whole discussion to appear in his blog. Feel free to contact me by privmail (GMail account “alexinbeijing”) if you want to discuss further. Your “0>” macro reminds me of “prog1″ in Common Lisp. I don’t think Clojure supplies such a thing “out of the box” but you can easily write it for yourself (and should do so if you feel it makes your code better).

  11. Pierre Masci says:

    Ok :)
    PS: mikera, FYI never get email notifications when I get an answer on this blog.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s