Skip to main content

Multimethods in Clojure

Posted by manning_pubs on February 20, 2013 at 12:12 PM PST



Multimethods in Clojure

by Amit Rathore, author of Clojure in Action, Second Edition

Clojure multimethods support not only multiple dispatch but much more. Indeed, once you look past multiple dispatch, a commonly asked question is whether a language can dispatch on things other than the types of values. With Clojure’s multimethods, methods can be made to dispatch based on any arbitrary rule. In this article, based on chapter 4 of Clojure in Action, Second Edition, the author explains how Clojure multimethods work.

A multimethod is defined using defmulti. A multimethod by itself isn’t useful; it needs candidate implementations from which to choose when it’s called. The defmethod macro is used to define implementations. Let’s take a look.

Without multimethods

Consider the situation where our expense-tracking service has become popular. We’ve started an affiliate program where we pay referrers if they get users to sign up for our service. Different affiliates have different fees. Let’s begin with the case where we have two main ones: mint.com and the universal google.com.

We’d like to create a function that calculates the fee we pay to the affiliate. For the sake of illustration, let’s decide we’ll pay our affiliates a percentage of the annual salary the user makes. We’ll pay Google 0.01%, we’ll pay Mint 0.03%, and everyone else gets 0.02%. Let’s write this without multimethods first:

(defn fee-amount [percentage user]
  (float (* 0.01 percentage (:salary user))))
(defn affiliate-fee-cond [user]
  (cond
    (= :google.com (:referrer user)) (fee-amount 0.01 user)
    (= :mint.com (:referrer user)) (fee-amount 0.03 user)
    :default (fee-amount 0.02 user)))

The trouble with this way of writing this function is that it’s painful to add new rules about affiliates and percentages. The cond form will soon get messy. We’ll now rewrite it using Clojure’s multimethods.

Using multimethods

Before we implement the same functionality using multimethods, let’s take a moment to understand how they work. As mentioned earlier, multimethods are declared using the defmulti macro. Here’s a simplified general form of this macro:

(defmulti name dispatch-fn & options)

The dispatch-fn function is a regular Clojure function that accepts the same arguments that are passed in when the multimethod is called. The return value of dispatch-fn is what is called a dispatching value. Before moving on, let’s look at the previously mentioned defmethod macro:

(defmethod multifn dispatch-value & fn-tail)

This creates a concrete implementation for a previously defined multimethod. The multifn identifier should match the name in the previous call to defmulti. The dispatch-value will be compared with the return value of the dispatch-fn from earlier in order to determine which method will execute. This will be clearer through an example, so let’s rewrite the affiliate-fee function using multimethods:

(defmulti affiliate-fee :referrer)

(defmethod affiliate-fee :mint.com [user]
  (fee-amount 0.03 user))

(defmethod affiliate-fee :google.com [user]
  (fee-amount 0.01 user))

(defmethod affiliate-fee :default [user]
  (fee-amount 0.02 user))

That looks a lot cleaner than the cond form, because it separates each case into its own method (which looks somewhat similar to a plain function). Let’s set up some test data, so you can try it out:

(def user-1 {:login "rob" :referrer :mint.com :salary 100000})
(def user-2 {:login "kyle" :referrer :google.com :salary 90000})
(def user-3 {:login "celeste" :referrer :yahoo.com :salary 70000})

And now try a few calls to affiliate-fee:

(affiliate-fee user-1)
30.0

(affiliate-fee user-2)
9.0

(affiliate-fee user-3)
14.0

This works as expected. When a call is made to affiliate-fee, the multimethod first calls the dispatch-fn. In this case, we used :referrer as the dispatch-fn. The arguments to the dispatch-fn are the same as the arguments to the multimethod, which in this case is the user hash map. Calling :referrer on the user object returns something like :google.com and is considered the dispatch-value. The various methods are then checked over, and when a match is found, it's executed.

If a match isn't found, then the default case is picked. The default value for this catchall case is :default, but you can specify anything you want when calling def-multi, as so:

(defmulti affiliate-fee :referrer :default :else)

After you change the default value with such a call, the default case method would need to use the same value:

(defmethod affiliate-fee :else [user]
  (fee-amount 0.02 user))

It’s that simple! Now, to add new cases, you add new methods, which is far cleaner than ending up with a long-winded cond form. Now let’s try to stretch this example a bit by expanding the capabilities of our system.

Multiple dispatch

Imagine that our service is even more successful than before and that the affiliate program is working great. So great, in fact, that we’d like to pay more profitable users a higher fee. This would be a win-win situation for the affiliate network and our service, so let’s get started by quantifying a user’s level of profitability. Consider the following dummy implementation of this functionality:

(defn profit-rating [user]
  (let [ratings [::bronze ::silver ::gold ::platinum]]
    (nth ratings (rand-int (count ratings)))))

This is quite the dummy implementation, as you can see; it doesn’t even use the user parameter. It serves our purpose nicely, because it demonstrates that this function could be doing anything (number crunching, database access, web-service calls, whatever you like). In this case, it returns one of the four possible ratings out of ::bronze, ::silver, ::gold, or ::platinum. The extra colon resolves each of these keywords to the namespace it’s used in. As you’ll see shortly, creating ad hoc hierarchies in Clojure requires the use of namespace-qualified keywords if they’re used as dispatch values.

Now let’s consider the business rules shown in table 1.

Table 1 Affiliate fee business rules

Affiliate Profit rating Fee (% of salary)
mint.com Bronze 0.03
mint.com Silver 0.04
mint.com Gold/platinum 0.05
google.com Gold/platinum 0.03

From the rules it’s clear that there are two values based on which fee percentage is calculated: the referrer and the profit rating. In a sense, the combination of these two values is the affiliate fee type. Because we’d like to dispatch on this virtual type, comprising two values, let’s create a function that computes the pair:

(defn fee-category [user]
  [(:referrer user) (profit-rating user)])

This returns a vector containing two values, for instance, [:mint.com ::bronze], [:google.com ::gold], and [:mint.com ::platinum]. We can use these as dispatch values in our methods, as follows:

(defmulti profit-based-affiliate-fee fee-category)
(defmethod profit-based-affiliate-fee [:mint.com ::bronze] [user]
  (fee-amount 0.03 user))
(defmethod profit-based-affiliate-fee [:mint.com ::silver] [user]
  (fee-amount 0.04 user))
(defmethod profit-based-affiliate-fee [:mint.com ::gold] [user]
  (fee-amount 0.05 user))
(defmethod profit-based-affiliate-fee [:mint.com ::platinum] [user]
  (fee-amount 0.05 user))
(defmethod profit-based-affiliate-fee [:google.com ::gold] [user]
  (fee-amount 0.03 user))
(defmethod profit-based-affiliate-fee [:google.com ::platinum] [user]
  (fee-amount 0.03 user))
(defmethod profit-based-affiliate-fee :default [user]
  (fee-amount 0.02 user))

This reads a lot like our table with business rules, and adding new rules is still quite easy and doesn’t involve modifying existing code.

On a separate note, this is a form of double dispatch because we’re dispatching on two values. The affiliate-id and profit-level, even though they aren’t types in the class-based sense of the word, behave as types in this domain. This Clojure code performs a double dispatch based on types in our domain. There’s nothing to stop you from dispatching on any number of such types if you wanted to; the mechanism is straightforward.

Simulating single dispatch with multimethods

Incidentally, it’s trivial to simulate single dispatch in Clojure—all that’s needed is a dispatch function that ignores all but the first argument. If the first argument is a data type (or a class), you can even simulate Java-style single-dispatch.

Although we’ve shown polymorphic behavior in these examples, we’ve not mentioned inheritance at all. This shows that inheritance isn't a necessary condition for polymorphism. But inheritance can be quite convenient, as you’ll see in the next section.

Ad hoc hierarchies

Now that we have the profit-rating infrastructure in place, we might want to expose the ratings to our members in a form similar to airlines’ membership status. We could treat bronze as an entry-level status, and members could graduate to silver and higher levels. We could also treat gold and platinum as premier statuses, affording such members a higher class of service. This classification might look like that shown in figure 1.
We can codify this hierarchy using Clojure’s derive function, as follows:

(derive ::bronze ::basic)
(derive ::silver ::basic)
(derive ::gold ::premier)
(derive ::platinum ::premier)

Figure 1 Our domain demands a hierarchy of membership statuses. We can use this external-facing classification to simplify our code by defining an ad hoc hierarchy reflecting this structure.

This is why we namespace-qualified the keywords; we can now use them in hierarchies as shown here. Having done so, we can programmatically ensure it works as expected using the isa? function:

(isa? ::platinum ::premier)
true
(isa? ::bronze ::premier)
false

Because we now have a hierarchy, we can redefine the multimethod to calculate the affiliate fee as follows:

(defmulti affiliate-fee-for-hierarchy fee-category)
(defmethod affiliate-fee-for-hierarchy [:mint.com ::bronze] [user]
  (fee-amount 0.03 user))
(defmethod affiliate-fee-for-hierarchy [:mint.com ::silver] [user]
  (fee-amount 0.04 user))
(defmethod affiliate-fee-for-hierarchy [:mint.com ::premier] [user]
  (fee-amount 0.05 user))
(defmethod affiliate-fee-for-hierarchy [:google.com ::premier] [user]
  (fee-amount 0.03 user))
(defmethod affiliate-fee-for-hierarchy :default [user]
  (fee-amount 0.02 user))

This is even more succinct; it also captures the intent that acquiring a premier member from an affiliate partner is more expensive.

Java class hierarchy

Although we've shown that it’s easy enough to create ad hoc hierarchies, Clojure goes one step further to make things simpler by providing support for Java classes out of the box. When Java classes are used as dispatch values, inheritance relationships are automatically respected, relieving the need for the hierarchy to be created again via calls to derive.

This ensures that both javax.swing.JComboBox and javax.swing.JFileChooser automatically treat javax.swing.JComponent as a parent. A method that uses the JComponent as a dispatch value will match if the dispatching function returns either a JFileChooser or a JComboBox. The programmer doesn’t have to do anything special for this to work.

The visitor pattern revisited

Now that you know how multimethods solve the dispatch problem, let’s rewrite the AST program using multimethods. Imagine we represent a couple of syntax nodes as so:

(def aNode {:type :assignment :expr "assignment"})
(def vNode {:type :variable-ref :expr "variableref"})

We’ll use the appropriately named :type value of the hash map to behave as the type on which we’ll dispatch our multimethod. Here’s the implementation of checkValidity:

(defmulti checkValidity :type)
(defmethod checkValidity :assignment [node]
  (println "checking :assignment, expression is" (:expr node)))
(defmethod checkValidity :variable-ref [node]
  (println "checking :variable-ref, expression is" (:expr node)))

Similarly, here’s the multimethod for generateASM:

(defmulti generateASM :type)
(defmethod generateASM :assignment [node]
  (println "gen ASM for :assignment, expr is" (:expr node)))
(defmethod generateASM :variable-ref [node]
  (println "gen ASM for :variable-ref, expr is" (:expr node)))

This is much simpler than creating the whole double-dispatch mechanism in a language like Java or C++ that doesn’t support it. In order to add new types of operations on the AST, you create a new multimethod.

We’ve covered creation and usage of multimethods. Before moving on, let’s look at the situation where more than one dispatching value matches, which confuses the multimethod.

Resolving conflicts

Sometimes, a hierarchy may involve what looks like multiple inheritance. Consider the situation where a programmer can be both an employee and a geek, as shown in figure 2.

Figure 2 A hierarchy could lead to multiple inheritance. Clojure doesn’t do anything to prevent this, but it provides an elegant way to break a tie if multiple dispatch values match.

To reflect this, you’d call derive with the following:

(derive ::programmer ::employee)
(derive ::programmer ::geek)

Such multiple-inheritance relationships are perfectly valid in several domains, and Clojure doesn’t stop the programmer from creating or using them. But when methods are created with dispatch values of ::employee and ::geek, and the dispatch function returns ::programmer, the multimethod doesn’t know which one to pick because they’re both valid.

This multiple-inheritance problem is by no means unique to Clojure. Languages like Java avoid it by disallowing classes from being derived from more than one parent class. In Clojure, you can break the tie by specifying your order of preference using the prefer-method function. Here’s how you’d specify that being a geek trumps being employed:

(prefer-method multimethod-name ::geek ::employee)

Read this as a preference of ::geek over ::employee.

We’ve covered the mechanics of multimethods and shown how they’re a superior way of achieving polymorphism to what languages limited to single dispatch can ever provide. Having said that, multimethods aren’t used much, mostly because the functional programming style makes this kind of polymorphism less needed. When there’s a need, they can be an elegant solution. In the next section, we’ll examine an example of where multimethods are used to great effect.

Redis-clojure

Redis is a key-value database. It is fast and persistent. Redis-clojure is a Clojure client for the Redis server, originally written by Ragnar Dahlen. Even though the project is no longer maintained, it is a great example of multimethods. The library is open source and the code is still hosted on github.com. The multimethod we’re interested in is used to parse responses sent by the server as the client communicates with it.

When the server sends a response, it prefaces the data being asked for with a single character. These control characters are shown in table 2. Ragnar has created a multimethod called parse-reply, which processes the responses from the server.

(defmulti parse-reply reply-type :default :unknown)

From what we showed earlier, the default dispatch value has been set to :unknown.

Table 2 Control characters sent by the Redis server

Character Meaning
An error
+ Single-line reply
$ Bulk data
* Multi-bulk data
: Integer number

There are six implementations (methods) of this multimethod—one for each of the five control characters and one for the default case. The following listing shows relevant parts of the code (only the multimethod portion is listed). The choice of a multimethod here keeps the code in the next listing clean, easy to read, and easy to modify.

Listing 1 Implementing the Redis protocol using multimethods

(defmulti parse-reply reply-type :default :unknown)
(defmethod parse-reply :unknown
  [#^BufferedReader reader]
  (throw (Exception. (str "Unknown reply type:"))))
(defmethod parse-reply \-
  [#^BufferedReader reader]
  (let [error (read-line-crlf reader)]
    (throw (Exception. (str "Server error: " error)))))
(defmethod parse-reply \+
  [#^BufferedReader reader]
  (read-line-crlf reader))
(defmethod parse-reply \$
  [#^BufferedReader reader]
  (let [line (read-line-crlf reader)
        length (parse-int line)]
    (if (< length 0)
      nil
      (let [#^chars cbuf (char-array length)]
        (do
          (do-read reader cbuf 0 length)
          (read-crlf reader) ;; CRLF
          (String. cbuf))))))
(defmethod parse-reply \*
  [#^BufferedReader reader]
  (let [line (read-line-crlf reader)
        count (parse-int line)]
    (if (< count 0)
      nil
      (loop [i count
             replies []]
        (if (zero? i)
          replies
          (recur (dec i) (conj replies (read-reply reader))))))))
(defmethod parse-reply \:
  [#^BufferedReader reader]
  (let [line (trim (read-line-crlf reader))
        int (parse-int line)]
    int))

Summary

In Clojure, a multimethod doesn’t belong to anything. The concrete methods instead belong to the multimethod, and they can be dispatched based off any number of types. In fact, the dispatch function is an ordinary function written by the programmer, and it can do anything the programmer wants. It’s by no means limited to only the type of the arguments, opening up possibilities that can't even be dreamed of in other languages. After all, if you think only in a singular dispatched language, you can’t imagine a world of multimethods.


Here are some other Manning titles you might be interested in:

The Real-World Functional Programming

The Real-World Functional Programming
Tomas Petricek with Jon Skeet

Scala in Action

Scala in Action
Nilanjan Raychaudhuri

Play for Scala

Play for Scala
Peter Hilton, Erik Bakker, and Francisco Canedo


AttachmentSize
cia2001.png15.44 KB
cia2002.png9.03 KB
cia2003.jpg22.24 KB
cia2004.jpg21.28 KB
cia2005.jpg10.25 KB
Related Topics >>