William C. Wake's Blog

June 2005 Archives

Fit code, part 8 of 8 - RowFixture

Data and Abstract Methods

The fixture holds three bits of data: an array containing the results of the query, a list of surplus items, and a list of missing items. From showing usages, I see that the list of surplus items is a list of domain objects; the missing list is a list of Parses.

The first abstract method is query(). It is responsible for producing an array of the "actual" results. The second abstract method is getTargetClass(). It returns the class object representing the type of the row. It's abstract for an interesting reason: the parent class ColumnFixture defines that method to return the type of the fixture itself. That would just lead to weird errors. By making it abstract, it forces the user to override it.

This is an interesting twist - usually my abstract methods are at the top of the hierarchy, and may get filled in along the way down. In this case, the method is becoming abstract in the middle.

In a sense, that happens because RowFixture and ColumnFixture have a slightly strained relationship. Maybe I'm just not getting why the latter is an example of the former; it feels like the inheritance is more for implementation than anything.

doRows() - The Overall Algorithm

Along the way, this method calls two overloaded list() methods: one for making a list of Parses, the other for making a list of objects. This parallel structure (methods for each of the two main data structures) continues across the class.

Method doRows() calls buildRows() to add in new rows for the surplus values. This method works by building up a "fake" head of a parse chain, then adding each item to the last one in the list. In the end, it throws away the head and returns the interesting part of the list, which gets attached to the end of the table. This seems like a little pattern worth remembering if I ever need to add rows to a fixture.

match() - The Heart

Since this algorithm looks complicated, I'm going to start by just looking around. First, what are the places that call it? By doRows() certainly, since that's how we got here. Then it's called recursively at two places inside match() itself. The good news is we don't have some sort of pair of mutually recursive methods.

Recursive algorithms have a base case and a recursion case. The recursive case here is just incrementing column, and passing along lists. The column is always incrementing, and the first if says that if we've exceeded the number of columns, we should just do a check on the lists. That makes it look like we'll always terminate: we either increment column, in which case we'll eventually stop, or we do something that doesn't recurse, which will stop as well. (Or rather, if it doesn't, it won't be because of the recursion here.)

The other thing to look at on these recursive calls is the lists. We know the column gets bigger - do the lists get smaller? One case passes on the originals, so we know it's no bigger. The other is trickier - I see things that seem to indicate that the lists will shrink (tests for 0 or 1 item), but it's not obvious that it must be so without a little digging.

So, from the top of match(): the first case we mentioned before - if we're past the number of columns, do a check() on the currents lists. (We'll come back to that method later.) The second case says that if the current column binding is null, move on to the next column. The third and final case is where the meat is: we're in a column in the middle, trying to match.

So, we build up two maps: one for the expected, one for the computed ("actual"). Each map has a list of items that have the given value at the chosen column. We pull out the keys in either list, and work our way through them. Here, there are four interesting cases:

1. The expected list is empty - we have a value found only in the computed list, so add it to the surplus list.
2. The computed list is empty - we have a value found only in the expected list, so add it to the missing list.
3. There is only one value in each list, and they have the same key (by how we got here) - check this row (actual vs. computed).
4. Finally, both lists have more than one item with the same key value - recurse, but only on the list of items with this same key value. (Now I can see that I'm recursing on a list that's no bigger. It could be the same size if all the keys are the same.)

eSort() and cSort()

Each routine produces a Map, from key values to a List of Objects (either domain objects or Parses). The bin() method takes care of putting items in the map. That method expands an Array into a List; the RowFixtureTest mentions a bug in that neighborhood and I suspect this is to address that.

The sort() methods handle exceptions and rows with no value in a particular cell.

Back to the Algorithm

check()

Leftovers and Learnings

The other big thing for me to wonder is how I'd have done a similar fixture. I think I'd have expected a simple set. The problem is, that's fine for the query() side, but not so good for the "expected" side: how would you get from those values to construct the objects to compare as sets? (Knowing the contents of an object's fields and return values from its methods doesn't tell you how to construct it.)

Another alternative would be to get the query values, and match each one up against the rows. The naive algorithm for this is a little slow (n^2). It might be a bit simpler. I suspect its report wouldn't be as nice.

The current algorithm is able to take advantage of partial matches - if enough data cells make it a unique match, it can then know it has the "right" element even though some of the fields/methods are wrong.

Closing Out...

Fit code, part 7 - ColumnFixture

doRows() - Capture the Header Row

If the cell ends in "()", it's a method call, and set via bindMethod(). Peeking inside there, it camel cases the name (so "shirt size()" becomes "shirtSize()") and uses the TypeAdapter.on() method to create the adapter.

Otherwise, the cell is assumed to be a field name, set via bindField(). That helper method also camel-cases the name and uses the "field" version of TypeAdapter.on().

Any problem in the header parsing marks the cell with an exception.

doRow() - Handling the Basics

Execute() is to be called before processing the first column that represents a method name: you can have a bunch of inputs, use execute() to make things happen, then check the outputs. If you don't override execute(), then you either have to know which column is first and make it kick things off (which is brittle), or you let each output column compute "from scratch". (Reading it here makes me wonder if I've been diligent about this in all my ColumnFixtures:)

If there are any unhandled exceptions, they're attached to the first leaf cell of the table.

doCell() - Per-Cell Handling

I'm going to write a test for it, but it took a couple minutes to figure out how to do so. I think what I'll do is create a table with a default value for x and y, leave x blank, and have execute() print the value of y. If it's called when x is processed, it'll print the default value rather than the one that was set.

(Pause)

OK - I'm back, and it does act like I expected from the code - execute() is called for a blank cell. I guess that makes sense to do if the blank cell were a call cell (like plus()); I'm not clear on the value for an input cell.

Back to doCell(): if the TypeAdapter is null, it ignores the cell. If it's a field, it parses the text and sets the field. If it's a method, it calls check().

check() - Calling execute()

What I've learned

• Don't forget about reset() and execute()
• Execute() is a little word in the face of blank input cells.

Martin Fowler's article on Language Workbench

National Games Week

JUnit 4 for JDK 1.5

Fit code, part 6 - TypeAdapter

C# Fit

TypeAdapter

This gives the unification of fields and methods. In Fit, you can have a ColumnFixture with a field, and it has an implicit setter ("name") or getter ("name()"). Or you can have a method (also "name()"). For most purposes, we don't care what it is, we just want to treat it as a setter or getter.

The TypeAdapter has five fields:

• target - the fixture this adapter is "on" (set for a field or method, but not a type)
• fixture - the fixture this adapter is "on" (always set)
• field - the Field (only set for field references)
• method - the Method (only set for method references)
• type - the type, always set

Methods

The set() method does a field set. (Could we extend this to call "setter methods" with a signature like setFoo(Foo value)?)

The invoke() method assumes that a method is set, and calls it with no parameters.

The parse() method asks the fixture to parse the string according to type. In C#, parse() is something each type (even primitives) define. I'm sure that simplifies some of this code.

So let's say we have a ColumnFixture where phone() is of type PhoneNumber. How do we make that get parsed naturally? It looks like it works its way back to Fixture, which has a parse() method. So the ColumnFixture overrides it, and checks for an attempt to parse a class it knows about.

It seems like we could do some fanciness here, too, pushing an attempt to parse onto the domain classes. (So let Fixture.parse() take a look for "type.getMethod("parse")"; if we don't want that we could subclass and override to avoid it.)

Primitives and Their Classes

The last one is the only exception: Arrays. There, the parse() method tokenizes it by looking for commas. Like so many other places in Fit, it trims spaces. Each element of the array is given its own TypeAdapter. The toString() method puts the commas in when printing it out.

Reflection

One-button games

Just think of the possibilites if you have two buttons! :)

Fit code, part 5 - ActionFixture

Fields

The class has three fields:

• cells - a Parse
• actor - a Fixture
• empty - an array of Class

Actor holds the object created by a start() action. Notice that it is static - that is what lets separate ActionFixture tables keep working with the same object without repeating start in every table.

Empty is the easiest - it's just an empty list so that things that want a list of argument types can have one. I marked it final, since it's never changed.

Methods: doCells()

The fixture invokes the method on itself by "getClass().getMethod()" - looking for the method on itself. This is a place where the Java version is nicer than the C# one. The C# version hard-coded that line to the equivalent of "ActionFixture.class.getMethod()". That meant that a subclass of ActionFixture would only have access to the four methods ("start" etc.) originally planned. The Java version lets you extend this vocabulary easily.

Another thing to notice is that the fixture calls getMethod() on cells.text(), not camel(cells.text()). That's a pity - my extended vocabulary has to be spelled exactly. (I don't think the rules for camel casing are consistent throughout. I'm probably getting hung up from Fitnesse experience - I think it may have slightly different rules.)

Methods: Actions

Enter() looks on the actor for a one-argument method it can use as a setter. It creates a TypeAdapter, which knows how to parse objects, passing it the cell text. Then it invokes the setter.

Press() invokes the named 0-argument method on the actor.

Check() assumes its 0-argument method is a getter, fetches the result, and passes it to Fixture's check() routine, which does the comparison and cell coloring.

Methods: method()

Next time, I'll take up TypeAdapter.

Fit code, part 4 - Fixture

Fixture: Fields and Two Helper Classes

There's a field counts that has counts of tests passed, failed, and exceptions/errors. The Counts class is just a data bag for these things. When a fixture calls wrong(), for example, the count is incremented.

The last field is args, which has the arguments from the first row of the fixture. The method getArgs() returns a String[] and lets a fixture use them. I don't think this is in the C# version yet but we definitely use that sort of thing there.

There's an internal class RunTime. It takes a snapshot of the current time. Right now, the only use of this is to put it in the summary, under the key "run elapsed time". Presumably some fixtures pull the RunTime object back out, and use toString() to display the elapsed time. But nothing in the standard distribution appears to use it directly. (Fit.Summary will display the elapsed time when it dumps the summary table.)

Starting Fixtures

Method getLinkedFixtureWithArgs() tries to load the fixture named in header.text(), then it sets up the arguments (for getArgs().

The method loadFixture() takes the name of the fixture, and attempts to "new up" the named fixture via reflection. Between the last method and this, I'm worried by what I don't see: what routine uses the camel method? That suggests a test: let's load "fit.ActionFixture", "fit.Action Fixture", and "fit.Action fixture" and see what happens. From what I understood going in, all three should be ok. From what I'm seeing here, I don't see what would make that work.

Why did I expect this? Because ColumnFixture does it for column names. It turns out that's not a good enough reason. The test shows that only "fit.ActionFixture" loads.

Up to interpretTables() again. It does getArgsForTable() - again. There's even a comment to that effect. I don't see why it should be necessary, though. Actually - it's all a little subtle, and I'd say the comment is misleading. The comment says, "// get them again for the new fixture object". But really, that's what we did in getLinkedFixtureWithArgs(). Now we're getting the arguments for the original fixture.

It works like this: when FileRunner starts, it runs doTables() on a new Fixture object. That's the object that tries to pull fixtures from tables and run them. When the first table is seen, its arguments are pulled out and given to the corresponding fixture. But then they're also copied back to the initial fixture as well. I imagine they're actually rarely needed there.

At any rate, interpretTables() then calls doTable(), which does a straightforward job of working its way into doCell(). Finally, it calls interpretFollowingTables().

InterpretFollowingTables()

All this work seems a little off - it seems the Fixture class is paying for interpretation that a particular table wants. I'm a long way off from looking at DoFixture, but if that's the table that should be first, it seems to me like it should pay for this complexity. I know I'm second-guessing here...

Check()

Up next...

Fit code, part 3 - Parse and Fixture

Parse

// This test shows offset isn't applied the way I expected
   public void testOffset() throws Exception {
        int offsetToData = 2;
        Parse p = new Parse(
                "xx
data
",
                Parse.tags, 
                0, 
                offsetToData);
        assertEquals("", p.leader);
    }

// This test shows cells with embedded tables "go away"
   public void testInnerTables() throws Exception {
        Parse p1 = new Parse(
"
stuff plus
");
        Parse p2 = new Parse(
"
stuff " + 
"
 inner
plus
");
        
        Parse cell = p1.at(0,0,0);
        assertFalse(cell.body.equals(""));
        assertEquals("stuff plus", cell.text());
        
        cell = p2.at(0,0,0);
        assertFalse(cell.body.equals(""));
        assertEquals("stuff inner plus", cell.text());
     }

FixtureTest

  assertEquals("     ", Fixture.escape("     "));

But there's a little more going on in Fixture...

Fixture

The core of this is: doTables() calls getLinkedFixtureWithArgs() and interpretTables(), which (eventually) calls doTable(), which calls doRows(), which calls doRow() once for each row, which calls doCells(), which calls doCell() once for each cell. In Fixture, doCell() calls ignore(), which marks the cell gray. ColumnFixture(), for example, overrides doCell() to do something interesting (like look for expected results).

I'm going to skip over how the first table gets loaded and interpreted - it looks interesting (i.e., tricky:) Instead, I'll peek down to a section labeled "Annotation". This area contains methods that can mark and color cells: right(), wrong(), info(), ignore(), error(), and exception(). I've seen these called in several of the standard fixtures before.

I see where exception() puts the stack trace into the cell. That gets SO ugly when something goes wrong. (It makes the cell huge, full of scary content useful only to programmers.) Maybe someday I'll take a whack at a more readable version.

Utilities

assertEquals("twoWords", Fixture.camel("two words"));
assertEquals("MiXedCAsE", Fixture.camel("MiXed cAsE"));
assertEquals("aFewWordsTogether", Fixture.camel("aFewWordsTogether"));
assertEquals(
    "acronymsLikeHTMLStillUppercase", 
    Fixture.camel("acronyms like HTML still uppercase"));

There's a parse() method that appears to handle only Strings, Dates, and ScientificDoubles. I know that C# works a little differently, since parse() is more integrated.

There's a check() method that looks too complicated to understand in 30 seconds. It uses a TypeAdapter, which is another class I want to look at soon.

Finally, there's a method to get the arguments from a Fixture. This is new - it used to be that the first row had the fixture only. (Rather, you had to parse any arguments out yourself.) Now that's built in, accessed via getArgs(), which returns a String array.

That's it for today. Next time, I want to dig into how fixtures get started (since this has changed some), and into the check() method.

Fit code, part 2

Inside Parse

The top of the class reveals strings for leader, tag, body, end, and trailer, as expected. There are also parts and more, which are Parses. A skim through the class, looking for big routines, shows that the constructor, findMatchingEndTag(), removeNonBreakTags(), print(), and footnote() methods seem to be the biggest and most complex.

Footnote? What's that? The tests didn't mention it! Looks odd - it's not referenced inside fit anywhere; rather, it's used by some clients, typically after a call to wrong(). It appears to create a file Reports/footnote/n.html, and prints the parse to it.

My strategy today is to chew off the routines that are small and/or simple, then go back and figure out the big routines. I have two things I'm trying to understand: "What happens with nested tables?" and "How do I insert stuff into the middle of a Parse?" (I need the latter for fixtures that want to report a little more nicely.) I guess I have a third question too - "how are spaces handled?" This arises because I saw a note on the mailing list that says there are differences in the various fit implementations.

Small Fry

There's a little helper routine addToBody() that just appends text to the body. That doesn't sound like much - and it's a one line routine, basically "body = body + text", but a search for usages shows that this is what fixtures use to get their info into the output. (If a fixture wants to show a cell's expected value, it uses this method to append some HTML text to the cell's Parse.) That answers one of my questions. I'll have to play with it to learn it better.

The print() routine is longer than these one-line methods, but looks straightforward. It writes the Parse out: leader, tag, then either body or parts, the end tag, and either the more or the trailer. I knew body and parts were mutually exclusive; I hadn't realized that more and trailer are exclusive as well. I wonder if body and trailer appear together, and parts and more appear together? If so, I wonder about splitting Parse up so subclasses can deal with that difference. It's not a huge class; may not be worth it.

Constructors

One of the ones that didn't is in fat.Table It is using this constructor to copy an existing Parse. That looks misplaced - if we need to copy these, then we can put a method on Parse to do so. A second place is fat.FixtureNameFixture. The GenerateRowParses() method passes in a string for "body" and a Parse for "more". (So we have an example where "parts" and "more" don't go together.) I can't tell why it does this on a quick look. The final place is eg.AllFiles.td(), which also uses "body" and "more" together.

The first constructor passes in all the pieces separately. Then there are a few constructors that default tags and so on, to the main constructor that actually parses some HTML. That fixture looks for several key positions in the input: the start of the target tag, the end of the target tag, the start and end of the corresponding end tag, and the start of the rest of the text.

I see that the first search starts at the beginning of the string, rather than at "offset". That seems odd.

We'll have to double-check how findMatchingEndTag() works, but the rest of the constructor looks straightforward: if there are more tag levels, turn the body into a new Parse (and set body to null). If there's a nested table, parse the table and set the body to "". (That seems odd also, like it's throwing away any non-table stuff. I'm not sure what the "" body accomplishes either.) Finally, if there are more tags at the current level, null out the trailer and parse the remaining tags into "more".

FindMatchingEndTag() looks like an implementation of the parenthesis-balancing rule - add 1 every time you see a left parenthesis, subtract 1 every time you see a right parenthesis. If you're balanced, you'll have a net of 0.

So I have an answer about nested tables: it's trying to handle them. I'm seeing a little weirdness that makes it look like a nested table is the only thing retained inside a cell. But at least I know it's trying. I'll make some tests to fill in what I'm seeing. I only have a few minutes left, so I want to move on to the htmlToText() part of the code.

Html to Text

Removing non-break tags is a little tricky-looking, but it basically squeezes out tags other than the normalized break tags we just produced. The method "looks forward" to see an end-of-tag; if it's there, it trims out the tag and looks at the rest of the string.

Condensing white space applies the rule: convert multiple blanks to a single blank, convert a "160" to a space, and convert   to a space. I assume 160 is the code for a non-breaking space in Word's font.

Unescaping is simple too: br tags are converted to newlines, standard entities such as <lt; are converted to their simple character, and smart quotes are converted to " or ' as appropriate.

The result of all this is that text() produces the Parse in straight text form - no tags. This is what fixtures will want when they compare expected values.

Summary

• What happens with nested tables? They are apparently handled, althrough it looks like only the nested table is retained, not anything surrounding it.
• How do I put stuff inside a Parse? Use the addToBody() method.
• What happens with spaces? Multiple spaces get converted into one, and non-breaking spaces get converted into one space each.

This work is licensed under a Creative Commons License.