David Herron's Blog

Syntactic sugar and hybrid dynamic languages

I just listened to Software Engineering Radio Episode 14: Interview Ted Neward and boy that's got a lot of interesting food for thought.

The main focus was to discuss the LINQ addition to C#.

The first idea that came to mind is he, Ted Neward, described LINQ as little more than "syntactic sugar". What LINQ does is integrate database queries with the C# language in what sounds like interesting ways. And, he says that under the covers in the generated bytecodes, it's clear that this is implemented through compiler tricks. Or, as he calls it, syntactic sugar.

Now, that brings to mind the new Java language features which came with Java 5 (Tiger), and how some have described them as syntactic sugar. This seems to me to be a way to dismiss the language features as less valid, because they're just syntactic sugar.

But, you know what, I think every high level language structure is just syntactic sugar. That every high level language structure ultimately is translated down to low level instructions. Even the "for loop" from C is really just syntactic sugar for an "if" statement and a set of "goto" statements.

That's what computer languages do, after all, is build high level abstractions that allow us to more easily describe to the computer what it is we want the computer to do.

The other thing I take from the interview is the potential value of marrying dynamic and static programming language features. Languages don't have to be either static or dynamic, they could have features of both. That's an interesting thought to ponder. And to give one even more food for thought, he sketches out a method where a suitable Java compiler could implement hybrid static/dynamic language features.

My last thought is it sounds highly attractive to be able to weave multiple languages together into one program.

It's always seemed very clunky to me to use JDBC to make SQL calls. I hate doing that so much, that one time I spend 3 weeks writing a persistence layer which would handle the JDBC for me.

What the LINQ thing does, I gather from the interview, is to make SQL statements be something you can write in the middle of a C# program. You can put C# variables into the middle of the SQL statement, and the compiler interpolates the values for you into the SQL call. And the result set gets automatically interpolated to an appropriately pseudo-dynamic object. Sounds cool.

To me the general principle is that each niche of activity has tools suitable to that activity. For example the tool for a painter is a brush, the tool for a writer is a pen, the tool for a carpenter is a saw or hammer, etc. The tools are shaped and designed to make certain jobs easier. And it's easy to fall into the trap of "if all you have is a hammer, then all the world begins to look like a nail".

To computerise our human activities it's helpful to have domain specific tools. A painter who wants to paint on a computer shouldn't be writing code in C that draws their painting. Instead the painter should have something like a Wacom tablet and be using something like Photoshop.

Hence, a database programmer should have a language that looks like SQL and integrates SQL nicely with the rest of the software system.

Hence, an XML programmer should have a language that looks like XML and integrates XML nicely with the rest of the system.

Comments

• Perhaps the most interesting thing about LINQ is that the query expressions are meaningful for the appserver and the dbserver. This opens a way for very interesting caching and optimization techniques that go beyond statistical analysis. This could have been done with the preprocessor-like SQLJ, too but at the price of emulating a db server. When strongly separating application and persistence logic you intentionally rule out synergies between them. This had been a trend for two decades now , because we hadn't been able to control (engineering-wise) the effects on consistency/scalabilty. The point is that for many applications persistence is not just an 'aspect' - it is in the core of the business logic. Syntactic sugar: The only 0 calories lang I know is Lisp - and this is the main reason why it is disliked by so many. As computers are still programmed by humans we need some redundance in syntax

  Posted by: csar on May 13, 2006 at 01:09 AM

• Apologies for hijacking this topic, but I can't let this go...

  Syntactic sugar: The only 0 calories lang I know is Lisp - and this is the main reason why it is disliked by so many. As computers are still programmed by humans we need some redundance in syntax

  Lisp is the Jolt Cola of Syntactic Sugar! Through the vast array of abstractions that Lisp provides, Lisp makes Captain Crunch cereal downright bitter.

  The fundamental functionality is provided by DEFMACRO, which essentially lets you write code that runs at compile time and operates on anything that Lisp can read. Out of the box, that's S-expressions and the fundamental datatypes.

  Most folks stop there as there is rarely a need to go farther.

  But the other concept most folks miss is that in order for DEFMACRO to do its job, it must work on a result provided by the Lisp READER, and the Lisp READER is also extensible to read pretty much anything you want.

  For example:

  (select [Researcher] :from '([TreeData] [SpeciesList])
                        :where [and [= [TreeData SpeciesID]
                                       [SpeciesList SpeciesID]]
                                    [like [LocalName] "v%"]]
                        :distinct t :flatp t)
  

  This is from COMMON-SQL, and SQL facility for Common Lisp. 'select' is a Lisp function, the brackets are part of a custom Lisp Reader Macro that converts their contents in to the sub components of a SQL expression. :from and :where, etc. are standard Lisp function keywords.

  Most of the magic here is done with the [ reader macro. It converts the text it's enclosing in to higher level Lisp forms that the 'select' function then uses to create SQL statements and execute them. The 'select' function is essentially ignorant of the reader macro, just as the macro is ignorant of the 'select' function.

  Next:

  (loop for columns being the records of
                      [select [*] :from [SpeciesList]
                              :where aardvark]
                      do (print columns))
  

  This combines the reader macro along with an extended Lisp LOOP macro, as well as a generic Lisp form (print columns).

  All the things combined in this one statement are pretty mind blowing from a sugary point of view.

  One, you have the Lisp Reader macro triggered by the '[' character, which means that you have user wrtten code almost down to the lexer level of Lisp.

  Next, you have the extended LOOP macro. Lisp already comes with a LOOP macro which uses this "sorta english" style of declaring iterations. But, COMMON-SQL took that and extended it. They added more funcationality to an inplace capability, yet it still works for the generic native cases (and generates the same code for the native cases). Macros give you access to the code stream at the Parser level of Lisp.

  Finally, you have the (print columns), which is simply generic everyday Lisp code, and could be anything (includiing another LOOP macro or any other arbitrarily complex Lisp statement).

  So, while MS has to create a new C# compiler to add LINQ, something like COMMON-SQL was written using Lisps native extensibility, and could be done by any programmer. MS needs to update the entire C# grammar and runtime to add in LINQ, whereas COMMON-SQL is simply another of the thin veneers that makes Lisp more and more powerful without having to start anew.

  No other language exposes their underlying mechanisms like Lisp does (FORTH sort of comes close). Just because folks choose to continue to use the S-expr idiom doesn't mean it's a requirment, it just happens to be easier for Lisp people.

  So while you listen to the "shook shook shook" sound of MS shaking on a light frosting of sugar adding in LINQ, you should also be listening for the loud "BEEP BEEP BEEP" of the sugar laded Lisp dump truck backing up ready to make your code as sweet as you like.

  Note, I'll be the first to admit Lisps other short comings and I understand and realize it has not yet taken over the programming world, and for many reasons, but inability to sugar coat its syntax is not one of them.

  Posted by: whartung on May 15, 2006 at 05:54 PM

This work is licensed under a Creative Commons License.