John D. Mitchell's Blog

DSLs feelin' groovy (or, graduating from elementary school)

Ben Galbraith has posted the first of a series of blog entries about How I Learned to Love Domain-Specific Languages. It's great that more and more people are starting to see the value of explicit, focused languages over ridiculously inhumane "formats" like XML. Hopefully, we're finally reaching a tipping point.

Explicit DSLs feel weird to a lot of programmers because there's been so little mainstream focus on them. I.e., as shown by one of the comments, developers have been herded and otherwise sucked in by shiny-looking tools (by poor education, management, laziness, peer-pressure, ignorance, lack of training, marketing hype, etc.) and haven't (consciously) realized the power of domain languages. It's amazingly odd to me how little energy has been applied to languages among mainstream developers given how much programmer time is spent arguing about the minutia of programming languages and tools.

The fact is that we're already surrounded by and are constantly implementing "DSLs". Look at the "language" of printf and friends, the declarative "specification" of makefiles, the myriad "protocols" that we deal with everyday like HTTP, SMTP, SSH, and FTP, the "APIs" of code libraries, the "design patterns" embodied in frameworks, the analogies and "metaphors" we use to described software architectures, the implicit languages that we create each time we define a class, the jargon we use to talk with each other, etc.

A big part of the problem that I see happening right now is that too much of the discussion around "DSLs" is being framed as some sort of "either/or" / "black/white" conflict when it's really just a more conscious and explicit approach to things that we've already been doing. Whether it's the hype juggernaut of Ruby on Rails or the Java is old, boring, bloated, etc. ideas exemplified by Beyond Java or the "IDE" wars between Eclipse, NetBeans, IntelliJ IDEA, and Emacs, or whatever, the biggest issue with this "us/them" thinking, IMHO, is that people are fighting the wrong fights. The leverage that matters most is the ability of developers to think and communicate clearly with themselves, each other, systems, business folks, and users. Biologically and sociologically, human are built to be linguistic.

That is, languages are fundamental to how we work internally and with each other. Sure, we have various tools to help us communicate but isn't it clear that e.g., PowerPoint isn't the point, it's just a tool — and, alas, a tool that usually induces poor communication rather than enriching conversations). On the other hand, look at the "modern" killer apps and how they are all about helping us (manage our) communicating: email, web, blogs, P2P, wifi, cell phones, faxes, VoIP, agile/XP, open source, etc. I.e., we've graduated from the elementary school building blocks (word processing, spreadsheets, databases, Belief of Control, etc.) to the middle school of communication. Now, we just need to learn and develop languages and tools built around this new level of understanding and put aside our old, comfortable, but ultimately dead-end habits.

Comments

• Here are some old links that approach the subject from various angles:

  Why Humans Should Not Have to Grok XML
  ITLS!
  Conversational Programming Languages
  Architecture: Abstract or Manifest?
  Design vs. Architecture
  The Poetry of Programming
  

  Posted by: johnm on November 17, 2005 at 11:15 AM

• I think that I agree with you John (on the promise of DSLs), but I am concerned with losing the debugging support that is provided for Java, C++, etc.

  I don't have the gift of being able to read code and spot the bug... I'm a debugger cripple, and make heavy use of breakpoints and the ability to step through the code one line at a time.
  Note that my concern is not in debugging the DSL itself (although that could prove to be a maintenance problem over time), my concern is with the logic that I am expressing with the DSL (although admittedly the code should be a lot easier to read).

  In the Meta-language FORTH your DSL was just an extension of the existing language. All the debugging tools that were available to FORTH were available to whatever DSL you defined with FORTH.

  I miss that.

  --JohnR
  

  Posted by: johnreynolds on November 17, 2005 at 12:25 PM

• Spot on about the black/white and us/them thinking. Is that a basic human characteristic in an effort to simplify the world or is it just self-validation?

  Posted by: tobega on November 17, 2005 at 01:57 PM

• @johnreynolds:
  Take a look at this talk by Jim Weirich on the topic of DSLs in Ruby:
  http://onestepback.org/articles/lingo/index.html

  (The audio for this talk is also available at ODEO: http://odeo.com/channel/34396/view )

  Basically, many DSLs can easily be implemented in Ruby (or JRuby if you want), but the benefit is that this DSL code is then simply Ruby code, which means you could just set breakpoints in the DSL handling code or DSL code to see what's happening.

  Posted by: murphee on November 21, 2005 at 08:30 AM

• Spot on about the black/white and us/them thinking. Is that a basic human characteristic in an effort to simplify the world or is it just self-validation?

  Yes. :-)

  
  Seriously, there's a lot of factors contributing to how we perceive, think, decide, and act. Examing the language(s) we use is a way to start exploring that because when you boil it all down, programming is all about communication.

  Posted by: johnm on November 23, 2005 at 02:21 PM

• Re: Debugging

  
  Yes, that can be an issue. It really depends -- and on a lot of different factors. As you mention, so-called "fully embedded DSLs", such as macros in Common Lisp, are really just short-hand for a slew of code written in the base language. Completely standalone DSLs tend to have their own development toolsets. IME, the real problem cases are the weird, hacked up "solutions" that are somewhere in the middle.

  
  In practice, I find much less need for traditional "debugging" with DSLs. Partly it's the fact that by taking a more formal, linguistic approach to problem makes it easy to write lots of really good tests. Partly it's because focusing the language on the needs of a particular domain makes it very succint, clean, clear, comprehensible, etc. so it doesn't take nearly as much effort to follow what is (should be) going on. Partly it's the fact that adding "debugging" information is easy to do because you can hook it straight into the translation/interpretation. That is, since you have the high-level context and intent (from the DSL source level) you can get more (useful) leverage out of the traces/logs/etc. I.e., I find that people find the DSL level errors faster and more easily.
  

  Posted by: johnm on November 23, 2005 at 02:43 PM

• I was always a great believer in DSLs. There is just something so sexy about a language tailored to the specific set of tasks involved ... no general purpose language could have that sort of specificity. I see several comments on debugging (which in the comment of johnreynolds I see a great solution) but none on the learning curve.

  I once worked on a project where one long-since-gone programmer used JavaCC to design a language. While it did what it did very well, it fell into disrepair for lack of other programmers knowing JavaCC, and the language itself was designed by a single person with what appeared to be limited "humanity" in his thoughts on language design. So, I guess I am for DSLs if they are well designed, but not really interested in learning 20 new languages at every new company. That's why we have general languages.
  Main stream DLSs, good. Letting one wild man create DSLs just internal to one company or project, probably not so much.

  Posted by: mzsanford on December 09, 2005 at 07:21 AM

• mzsanford, yes, one of the biggest arguments for the support and creation of "in-language" DSLs in general purpose programming languages (such as macros in Lisp) is precisely because there's a relatively seamless integration of the DSL with the base language. Rake is a fresh example of doing a modern "make" in Ruby.

  
  In terms of language proliferation, for relatively formal languages, yes, too many people create crappy little languages on their own rather than using any of the myriad of embeddable existing languages that are well made, supported, etc. such as Tcl, Lua, etc.

  Of course, as I've noted in some of the pieces linked to above, we're always creating "languages" including the APIs, protocols, format strings, etc. So, IMHO, the real value is that we take a more conscious approach to that fact.
  

  Posted by: johnm on December 09, 2005 at 09:37 AM

This work is licensed under a Creative Commons License.