William C. Wake's Blog

Extreme Programming Archives

ScrumGathering '07

On Tuesday, Mike Cohn and I taught a course centered around a series of case studies. Wednesday and Thursday were Open Space sessions.

The overall site for the Gathering is here". This is the result of a session on Games for Scrum, along with a particular planning game, AgileTripTik.

I saw three themes:

• How can coaches/ScrumMasters effectively intervene?
• Leadership/vision "versus" self-organization
• "Beyond iterations" - moves outside generic Scrum approaches

Stories for SideReel.com

Schools of Software Testing

CFP - Tabletop 2007

The Second IEEE International Workshop on Horizontal Interactive Human-Computer System (Tabletop2007)

Newport, Rhode Island, USA, October 10-12, 2007. Held in conjunction with ACM UIST 2007

http://www.ieeetabletop2007.org/

Supported by IEEE

The use of the tabletop as an input/output device is an exciting and emerging research area. This cross-disciplinary domain brings together experts in projector based display systems, augmented reality, user interface technologies, multi-modal interaction, input and sensing technologies, CSCW, and information visualization.

The purpose of the workshop is to bring together leading researchers in the field so that they can present and exchange current results of ongoing investigations.

We encourage researchers and developers to present in the following areas as they relate to interactive tabletops:

• Applications
• Gesture-Based Interfaces
• Multi-Modal Interfaces
• Speech Interfaces
• Tangible Interfaces
• User Interface Technologies
• Computer Supported Collaboration Systems
• Middleware and Network Support
• Augmented Reality
• Social Protocols
• Haptic Rendering
• Information Visualization
• Horizontal Display Hardware
• Sensing and Input Technologies

Paper Submission:

We invite submissions of two kinds - regular research papers of up to 8 pages and short papers of up to 4 pages describing original research on investigations into horizontal oriented user interface based (tabletop) systems.

Each paper must be submitted as a single PDF file in IEEE Computer Science Press format (described at ftp://pubftp.computer.org/press/outgoing/proceedings/instruct.pdf).

Each paper will be peer reviewed by no less than three experts in the field. Accepted research papers will be included in the IEEE proceedings and presented in the paper sessions.

Submission details will be available at: http://www.ieeetabletop2007.org/

Important Dates:
June 8th, 2007: Paper titles and abstracts due
June 15th, 2007: Paper submissions due
July 12th, 2007: Notification of acceptance sent to authors
October 10-12, 2007: Tabletop2007 workshop, Newport, RI.

Chairs:

General Co-Chairs:
Andy Wilson (Microsoft Research) awilson [at] microsoft [dot] com Clifton Forlines (Mitsubishi Electric Research Labs) forlines [at] merl [dot] com

Program Chair: Kathy Ryall (Mitsubishi Electric Research Labs) ryall [at] merl [dot] com

Program Co-Chair: Stacey Scott (Massachusetts Institute of Technology) sdscott [at] mit [dot] edu

Project portfolios and agile

Leadership Reading List

Using actuals with estimates - an experiment

We also track actuals. Whenever it's convenient, but definitely at the end of each day, the pair adds its tick marks onto the board.

We've used a couple approaches to coming up with "the" estimate. One was to have the longest-standing team member propose a consensus number => not consistently right. Then we tried taking the maximum estimate => usually too high.

Now, we're keeping track of who made what estimate. We compare estimates to actuals and decide who was the "winner" - the one with the most accurate estimates that week. When it's time to do new estimates, we let that person hear all the estimates, and they make the overall estimate we record.

It's too early to say how well this works. The same winner showed up twice in the first month, which may be an early indicator.

My estimates? I'm almost always too optimistic. So I'm going to pay more attention to how the winner thinks about our tasks.

Set-Based Concurrent Engineering

ScrumGathering '06, Open Space

Product Owner - I hosted a session on challenges of being a product owner. We described various problems, and broke into subgroups to look at it from the perspectives of "patterns of product owners," "traps," and a "values/principles/practices framework."

Planning a Long Project, hosted by Kelly Weyrauch. We defined a long project as lasting more than a year, and considered a variety of issues that affect planning. Two key challenges are "dealing with changes in scope" and "being absolutely clear about how much is left to do."

Being a Manager, hosted by Jens Ostergaard. Looked at things managers do: remove impediments, set goals, coach/help with individual development, encourage innovation, provide resources/environment, block for the team. But the most important thing is to provide vision.

What is business value?. Hosted by Joe Little. Value to the customer versus benefit to the business. The Kano model - mandatory, linear, delighters. Value vs. cost.

Write a product backlog, hosted by Mike Cohn. We took a couple examples and broke them into stories. Mike talked about using the expected implementation cost as a driver for whether to make stories bigger or smaller. He also described using the back of the card to record conditions of satisfaction, high-level acceptance criteria.

Scrum for non-software applications. We identified a number of areas that use or can use Scrum. Then we explored some challenges and potential solutions for problems they might face.

In the afternoon, we did some closure exercises, to identify important areas going forward and to turn the energy into accomplishment.

I joined the Leadership group. We decided to survey scrum masters and trainers about how leadership is presented as a topic in the courses, to create a resource list of books and articles, and to gather success stories of people on Scrum teams who demonstrated leadership.

I had a fun couple of days, and took away some inspiration, some ideas to try, and some ideas for new articles. It's always feels good to re-connect to the community.

Alistair Cockburn article on improving bottlenecks

A bottleneck limits the rate of production of a system. Clearly we want to improve its performance. But what should we do about non-bottlenecks? Alistair discusses several strategies, including the non-obvious one that the non-bottleneck may be better off deliberately spend extra time exploring alternatives before feeding into a bottleneck. He points out that there's no single strategy that always works; it's context-dependent.

IEEE Software issue on Test-Driven Development

Papers are due by December 1, 2006; the issue will appear in May/June 2007.

Latest Communications of the ACM

This issue has a few articles on global software development. Some touch on agile softare. One of the articles was a little odd, implying you needed duplicate role positions on both sides (which I can somewhat believe), and that cost is 150% of nominal (which seems unintuitive). (I'll admit I just did a quick skim of these articles, so perhaps I'm misinterpreting conclusions.)

There's also an article, "Building Objects Out of Plato: Applying Philosophy, Symbolism, and Analogy to Software Design." If you're interested in the XP idea of metaphor, this may give you another take on it.

Trends in Agile

Extreme Test Makeover at Agile '06

• Watij tests in Fit are too long/confusing to read for customer.
  • You could write it in JUnit instead of Fit
  • Break them up into small focused tests
• Neat new delegate syntax (with .Net 2.0)
• Descriptive variable names are good [even] for short term variables.
• Keep tests focused on one purpose - If a test needs 3 things to work, create 3 tests
• Generic isn't always useful.

What I hope to learn at Agile '06

• Extreme Test Makeover - Monday, 11 AM - 5 PM, Mirage room. I hope to see a variety of types of tests, and a variety of styles that experts use in helping improve them.
• Example-Based Specifications - Tuesday, 1:30 PM - 5 PM, Nicolette D1. What makes a good "spec" test, what types of examples are good, how much to trade off exhaustiveness for conciseness, what's the state of the art, how can we improve things?
• Challenges of the customer / product-owner role. Where are people struggling? What would help?
• Agile methods in non-software domains - When do agile methods apply in domains without software? I especially want to hear about projects involving teams in a commercial or volunteer domain.
• Technology - What's new? What's better?

Example-Based Specifications - workshop at Agile 2006

We think this session will really be enhanced if it includes product owners/customers who need to help specify systems. If yours will be at the conference, we'd love to see them at our session!

You might want to check out Martin Fowler's blog entry on Specification by Example for more thinking about this topic.

Here's our conference blurb:

Examples as Specifications

Overview: Some tests are thinly-disguised walkthroughs of a user interface. Other tests are so voluminous that you suspect that even their author didn't read them. But some tests seem designed to facilitate conversations about the key objects and processes in a domain.

How can we foster the latter style, to create specifications useful to both machines and people—including end users, product owners, testers, and programmers?

Length: 3 hours

Audience:

• People already writing or automating tests in an example-based style, and
• Real product managers, "customers," or product owners who have a domain to talk about (whether they’ve used this style or not).

Goal:

• Explore the consequences of an example-based, domain-oriented testing style.
• Develop experience and guidelines useful to others writing such tests.

Process:
Try it (1 hour): Break into small groups, including a customer (pseudo-customer if necessary). Have the customer repeatedly describe a story, and have the group turn it into tests.

Review (1/2 hour): Meet back as a larger group to compare tests and discuss the experience.

Break

Address specific topics (1 hour):

• Identify two to four areas of interest.
• Divide into groups to address those areas.
• Each group will produce a poster summarizing their results.

Next steps (1/2 hour): Meet again as a large group, comparing results and identifying next steps.

Deliverables:

• Example stories and tests, demonstrating the example-based style.
• Posters from each of the breakout groups, on a particular aspect of testing.
• Identification of future steps.
• Summary for the conference wiki.

Smell to Refactoring Cheat Sheet

Agile adoption

It's always a little hard to know how to interpret such statistics. I'm pretty sure agile methods aren't being used in 14 to 33% of all projects. And it's hard to know exactly people mean by "we do agile." I'd feel better if the analyst were somebody "anti-agile." But, they did spell it right and that's worth something in Hollywood:)

Origins of the cubicle

A couple interesting quotes:

Robert Propst invented nothing so destructive. Yet before he died in 2000, he lamented his unwitting contribution to what he called "monolithic insanity."

As Steelcase, Knoll, and Haworth brought their versions to market, they figured out that what businesses wanted wasn't to give employees a holistic experience. The customers wanted a cheap way to pack workers in.

Guidelines for Example-Based Specifications

I've tried to create guidelines capturing what I do to improve Fit tests.

http://xp123.com/xplor/xp0601/index.shtml

Agile 2006 submission deadline

Continue Reading...

Calling your shot

Agile methods build in this same "call-your-shot" dynamic. Each iteration, we make a prediction about what features will be present, and put them in. Every day, if we do Scrum-style standups, we'll say what we as an individual commit to do, and what commitments we kept from the day before.

Just as in pool, we can't hit every shot. Once in a while we'll mess up. But overall, we're transparent about how we want to make and keep commitments that others can rely on.

Secrets of Agile Teamwork

"The Secrets of Agile Teamwork: Beyond Technical Skills"
Dec. 6-8, 2005, Portland OR USA
http://www.futureworksconsulting.com/events.html

Roots of Lean - Kaizen

Tom Poppendieck pointed this out on one of the lists. It's an article on how a WWII effort called "Training Within Industry" was the basis for the "kaizen" continuous improvement approach in the Toyota Production System.

Schwerpunkt = focal point

"But there's one word, a German word, that we haven't yet stolen that should be high on our list of targets: schwerpunkt. It means 'focal point' or 'concentration of effort point' or 'central point of attack.' It's a beautiful word because it expresses an idea that we just don't have in English: the notion that, in any effort, you may have many necessary tasks, but there is one central task that must take first place in your considerations."

Crawford gives an example of the army: the cook is important, but the soldier (and fighting) is the shwerpunkt. In games, he says, interactivity is the schwerpunkt. It leads me to ask, what is the schwerpunkt for what I'm doing?

Agile '05 conference, part 5 (last) - Invited talks, TDD workshop

Random notes from Open Space

Is XP Sustainable? Some things people do:

• Daily: variety
• Medium-term: cleanup projects
• Gold cards (explicitly scheduled 'free time')
• Slack

Rick Mugridge, Domain-Driven Design Patterns and Fit. Examples: queries vs. operations, entities vs. value objects, aggregates (key root objects), repository (turns into query or iterator). Can use a different test harness with the same fixture and same code, but control whether to connect to a "real" database via configuration.

A test writing pattern: create a setup, do some action, repeat the setup but show what has changed.

Norm K erth: The Secrets of Leading from a Position of No Power

Some principles: transparency, persistence, courage. An unjust law: don't resist, but don't comply either.

How did he do it? "He decided soemthing needed change--something important, and worthy of his effort." He let go of assumed authority.

Inauthentic power = anointed power - ("I'm powerful because I'm a manager.") - a weak form of power since it goes away if the position does. Authentic power - inside yourself.

Gandhi accepted the cost of punishment. "Better to lose your job for doing something, or for doing nothing?" He was respectful, sought commonality. He knew his support system: the law. He started small, finding people of like mind.

Characteristics of change agents:

1. Ability to articulate a vision of where you are going (though it can change)
2. Persistence
3. Confidence - inner energy for the right thing
4. Optimism - energy that you lend to someone else so they can gain confidence.

"The most effective coach is sitting next to you, involved from the beginning to the end."

Gandhi: it paid to advertise. Peaceful revolutions are the lasting ones - they're really an evolution. Pick your battles - you don't need to do everything yourself. Know your real mission.

Joshua Kerievsky: Commoditizing Agility

He had statistics from a project showing productivity increases by a factor of 2.5, achieved in one year with an XP team.

"The agile community is thriving, but transitions are too slow and expensive." Why? We're not agile enough in transitions. Problem: serialized knowledge transfer. The shift is repetitive and exhausting.

A couple models: 16 days coaching for a 15-person community. Or, inhouse workshop with 3-6 month full-time coaching for a 20-30 person community. Then try to stretch to other communities, transferring experts out and future experts in. The Problem: people are trained technically but not so much in coaching.

The basics are taught manually. This gives inconsistent content and it doesn't scale. Books don't go far enough. Basic knowledge is fragmented. Books require dedicated study.

This marginalizes coaching time: the basics take away from the hard stuff. It burns out the coaches. So we need to commoditize the basics. Experts are in short supply. How many coaches have 3+ years experience? Internal experts are "too indispensable" to share.

Commoditization

Ian Murdoch: "Open source and the commoditization of software": Commoditization happens eventually. It's unstoppable, but it's good for all. There's a cycle: Innovation leads to standardization [accessible] leads to commoditization (maybe) [affordable] leads to innovation. Innovation builds on a custom platform.

Commoditization can go two ways: Decommoditization is an incompatible innovation, or customization. But be careful with decommoditization - it can leave you out of the innovation loop.

What can be commoditized: basics, stories, planning, chartering, retrospectives, ... What can't: advanced practices, specialized things, "living agile values", people issues.

Josh showed an example of Husqvarna's sales training - a sort of animated comic book, with a model of the sales process built in. He showed a demo he made, using a screen capture tool.

Bottom line: Tomorrow we can have parallel processing for the basics, leaving more quality time for advanced topics. We can get quality, speedy, consistent knowledge transfer.

Workshop: TDD and Beyond, Astels and Marick

Rick Mugridge: We need to move toward declarative tests. "Procedural tests are a smell."

Questions: "Do you have to have normal (JUnit) TDD in place before STDD?" "Do you need a strong customer to make it work?"

Roadblocks: Fit is a challenge because you need programmers and testers to make it work.

Fit Unification Summit

End

Agile '05 conference, part 4

Jeff Sutherland on Advanced Scrum

Scrum is out of development and into the whole company. This approach is for experienced ScrumMasters/developers only. See Lawrence Leach: "Eight Secrets to Supercharge Project Performance", Adv. Project, Inc.

Productivity is measured as "features/$100K"

How can we (agile?) win?

One study shows outsourcing cuts costs by only 20% - significant, but not quite the same as cutting 80% or more of costs as some have promised.

The new approach: anticipatory, requires accurate analysis of process and automatic monitoring.

Type A, B, and C Scrum - A: isolated cycles (breaks between sprints), B: overlapping iterations (no handoffs), C: all at once. Consider the sprint level. Need to do pre-staging work for the next iteration inside this one (or else we'll be tempted to fall back to type A).

Advanced scrum has multiple simultaneous concurrent sprints. All sprints release live software.

Changes:

• MetaScrum for release planning
• Variable-length sprints
• Overlapping sprints for one team
• Pre-stage product backlog
• Daily scrum of scrums
• Integrate product backlog and sprint backlog
• Paperless project management and realtime reporting
• Administrative overhead of 60 seconds/day/developer and 10 minutes/day/ScrumMaster.

One of the big challenges is having items in the backlog be ready. In their case, that means "intuitive for a doctor" - usable within one hour. This means stories need enough detail, and must have been prototyped with doctors. It forces the product owners to work with the developers to prototype things.

MetaScrum: weekly meeting of stakeholders. Led by the Product Owner. Use the three questions. All product decisions are made here. Release status. All decisions are communicated that day. Damage control plans are executed in the same day.

Iterations: three-month - major product releases (eliminate bugs from portfolio); 1-month sprint for new customer/upgrades (eliminate high-priority bugs); one-week sprint for critical issues. This gives them 45 production releases a year. "Assumes good practices. They're using one code base, working at the tip.

Pre-staging: Overlap sprints, with no break between them. Work only on product backlog that is ready to go the sprint. Pre-staging can double throughput in sprints.

Prioritizing sprints. Constraint theory tells us we must have slack. This lets you optimize multiple overlapping sprints. The backlog is fully automated, with priority levels for the different length sprints (week, month, quarter). Developers focus on one task at a time in priority order. Completing weekly/monthly tasks let them get to the "fun" quarterly tasks.

Sprint backlog: Set to embrace change. Every sprint releases. Customer satisfaction is the top priority. Can restart - a MetaScrum decision.

Automated backlog: Via a bug tracking tool. For each task, the developer is asked: 1. For tasks touched today, how much time is invested in the task? 2. What percent is done? This provides enough data for management "micro-accounting".

Agile '05 conference, part 2

Delivering APIs in an Agile Context, John Major

Tensions:

• Stable APIs vs. agile change
• General features vs. "Do the simplest thing that could possibly work"
• Frameworks vs. global refactoring
• Framework design vs. features

Unique practices:

• Internal support: old and new features. Used the idea of "office hours" for support. Provided training. Tried having platform developers work on applications, but it didn't work so well.
• Lightweight code reviews. Rather than pairing, they required at least a 20-minute weekly code review with an office-mate.
• Agile database schemas. Problem: RDBMS is rigid, but schemas evolve. Solution: build agile schema features and tools to manage change.

Lessons learned: Good:

• Agile practices help keep technical debt low.
• Build tools to support RDBMS and large codebase.
• Pull in senior people to help design and adoption.

Bad:

• Cost of absorbing API changes is paid by the application teams, but benefits accrue to the business.
• It's hard to get feature design right (to balance flexibility and focus).
• The business had trouble understanding the costs of release management. (Branches made the whole thing even crazier; he described a "London subway" map they created to show all the paths.)

Ugly:

• People issues - don't go into denial.
• Weren't able to tell QA what the test suites did, so there was overlap between the automated tests and manual testing.
• Be humble - the platform needs the app and vice versa.

Summary:

• Build reusable platform technology
• Use agile practices to cope with change
• Work with "eyes wide open"

Agile '05 conference, part 1

Brian Marick and Bob Martin Keynote: Where were we, where are we, where are we going?

Brian invited Jim Highsmith up to announce the formation of the Agile Project Leadership Network (APLN). Its focus is on applying agile principles to all projects, not just software. Relative to the Agile Alliance, this organization has consistent principles and intends to collaborate; it just has a different focus.

Bob Martin: "Value the flow of value."

The software industry: was either no process or waterfall. Now we're recognizing the failure of waterfall and the need to measure in-process work. It's time to do "software at home." Where we're going is to resolve this challenge, with short cycles, feedback, testing, craftsmanship.

The agile community: The Scrum paper in '95, XP in '99: resonated with the community. But we were fractured into many agile methods; branding. Now, industry is learning to pull and mix practices; we're at the moment of convergence. Future: head away from brands, strong focus on project management. At the end, we'll have a definition of agile.

Users: Were small teams with a programming focus; limited acceptance tests, project managers "lost." We are a community that has grown. We see 80-people teams often mixing Scrum and XP. 300-person teams exist. A company of thousands doing transitions. Future: we'll grow. Agile project management will happen. Automated acceptance tests will be regarded as the measure of progress.

Brian Marick: There was a group of English cyberneticians. They made discoveries of performance, adaptation, surprise. But oops... no (single) institutional home, no acknowledged base body of techniques, no grooming of successors. Their tradition is gone. To avoid this, "we gotta take over a university department." He announced the "Gordon Pask award", to honor people whose recent contributions demonstrate their potential to be leaders of the field.

Open Space

• Breath mints
• Haiku written on card before starting a task
• Hacky sack at standup
• Traffic light for builds
• Darts when a pair finishes a task
• "Story starting" gong
• Story completion frog (noise-maker) or bell
• Daily progress chart - two thermometers, showing days used and points locked in
• Automatic break program
• Smiley stickers for story cards

Metaphors, by Dave West

Metaphor has a lifecycle from poetry through use. Lakoff and Johnson say all thought is based on metaphor. Dave argues:

• If you don't choose consciously, you get one unconsciously.
• Metaphor selection has a real effect on software design (architecture).
• Metaphor is a skill you can develop.
• Therefore, it should be an agile practice.

We have many default metaphors: machine, org chart, entity, dualism, computer science, software engineering, product manufacturing. There are alternatives: e.g., object as person.

To use metaphors well, learn: reflect, experiment, read widely, liberal arts, point of view.

Tim Lister: XP Denver Invited Talk

• Managers are lonely - they don't have a collegial environment.
• QA is a bottleneck, and no wonder: it's done late and in serial rather than in parallel.
• Getting agreement on requirements is not neat. "Gathering requirements" sounds like they're Easter eggs in the yard. But really, most requirements have to be invented.
• Problem: people believe the customer.
• It's messy, so model and prototype. Don't prototype solutions - prototype wrong things on purpose (so they can laugh at it).
• Estimating is overwhelmed by expectations: skew (never early). Separate goals from estimates.
• Software in general is uninteresting. "One right way" is anti-intellectual.
• "Never lose the satisfying feeling of making something valuable, together with your colleagues."
  
  

  Brief review - Fit for Developing Software

  Posted by wwake on July 08, 2005 at 05:53 AM | Permalink | Comments (0)
  
  Fit for Developing Software, by Rick Mugridge and Ward Cunningham.

  [My bias disclosure - I know both Rick & Ward, I was a reviewer, and I've written for their publisher myself. This review is substantially as posted on the agile-testing group.]

  Fit (see http://fit.c2.com) is a testing framework that Ward Cunningham developed. A test author writes tests as tables in a document that can be converted to HTML (e.g., Word, Excel, text editor, etc.). The programmers develop fixtures that connect to the system under test. Fit mediates the tests and the fixtures to run the tests, and captures the results. It colors in the document using red/yellow/green to show what happened.

  The book is in two halves. The first half is targeted at test authors: from the user perspective, how does Fit work? It covers the basics of tables, fixtures, error handling, and so on. Then it goes into an extended example (several chapters) following a team developing rental software. The first half closes with advice about designing better tests.

  The second half is targeted at programmers. While programmers should really read and understand the first half of the book, test authors will probably at most skim this half. This half starts by explaining how to implement various types of fixtures. Then it continues the earlier rental software example by showing the fixture code that would be developed. Finally, this half closes with some advanced topics: Fit's architecture, custom fixtures and runners, and model- based test generation.

  The authors have done a good job explaining Fit from both the test- writing and programming perspectives. The text is clearly written, using plenty of examples, frequent breaks for Questions and Answers along the way, and exercises at many chapter ends.

  This book is unique. While you can find information about Fit and fixtures on the web, what's on the web is much less readable than what this book provides. The book also gives you an extended example and helpful advice from two experts.

  If you are considering Fit, or just want to understand its philosophy, this book provides the clearest explanation I've seen. For test authors, the first part of the book justifies the whole price. For programmers who need to understand how and why fixtures work, it's even more of a bargain.

  Fit for Developing Software, by Rick Mugridge and Ward Cunningham, with a foreword by Brian Marick. Prentice Hall, 2005. ISBN 0-321-26934-9.
  
  

  Fit code, part 8 of 8 - RowFixture

  Posted by wwake on June 25, 2005 at 09:08 AM | Permalink | Comments (0)
  
  A RowFixture is used to test that a set of items is as expected. The fixture flags surplus or missing items. They look like this:

  MyRowFixture
  first	last	status()
  Alexander	the Great	ok
  Alexander	the Mediocre	unknown
  Winnie	the Pooh	lagging

  Each row represents a domain object of some sort. The columns have inputs and outputs, as for ColumnFixtures.

  Data and Abstract Methods

  First, I'll note that this fixture extends ColumnFixture. This lets it pick up bind() and check(). The former handles the "header" row; the latter makes sure execute() is called. But due to the way the overrides happen, that method is called under different circumstances than for ColumnFixture. I don't see anything that will call reset() on a per-row basis.

  The fixture holds three bits of data: an array containing the results of the query, a list of surplus items, and a list of missing items. From showing usages, I see that the list of surplus items is a list of domain objects; the missing list is a list of Parses.

  The first abstract method is query(). It is responsible for producing an array of the "actual" results. The second abstract method is getTargetClass(). It returns the class object representing the type of the row. It's abstract for an interesting reason: the parent class ColumnFixture defines that method to return the type of the fixture itself. That would just lead to weird errors. By making it abstract, it forces the user to override it.

  This is an interesting twist - usually my abstract methods are at the top of the hierarchy, and may get filled in along the way down. In this case, the method is becoming abstract in the middle.

  In a sense, that happens because RowFixture and ColumnFixture have a slightly strained relationship. Maybe I'm just not getting why the latter is an example of the former; it feels like the inheritance is more for implementation than anything.

  doRows() - The Overall Algorithm

  The main algorithm is in doRows(): bind the columns (ala ColumnFixture), run the query() to get a list of actuals, run a match(), then add rows for any surplus or missing items.

  Along the way, this method calls two overloaded list() methods: one for making a list of Parses, the other for making a list of objects. This parallel structure (methods for each of the two main data structures) continues across the class.

  Method doRows() calls buildRows() to add in new rows for the surplus values. This method works by building up a "fake" head of a parse chain, then adding each item to the last one in the list. In the end, it throws away the head and returns the interesting part of the list, which gets attached to the end of the table. This seems like a little pattern worth remembering if I ever need to add rows to a fixture.

  match() - The Heart

  The match() routine is a recursive algorithm. Given the list of expected items, the list of computed items, and a column to start looking in, it figures out what matches and what's missing or surplus.

  Since this algorithm looks complicated, I'm going to start by just looking around. First, what are the places that call it? By doRows() certainly, since that's how we got here. Then it's called recursively at two places inside match() itself. The good news is we don't have some sort of pair of mutually recursive methods.

  Recursive algorithms have a base case and a recursion case. The recursive case here is just incrementing column, and passing along lists. The column is always incrementing, and the first if says that if we've exceeded the number of columns, we should just do a check on the lists. That makes it look like we'll always terminate: we either increment column, in which case we'll eventually stop, or we do something that doesn't recurse, which will stop as well. (Or rather, if it doesn't, it won't be because of the recursion here.)

  The other thing to look at on these recursive calls is the lists. We know the column gets bigger - do the lists get smaller? One case passes on the originals, so we know it's no bigger. The other is trickier - I see things that seem to indicate that the lists will shrink (tests for 0 or 1 item), but it's not obvious that it must be so without a little digging.

  So, from the top of match(): the first case we mentioned before - if we're past the number of columns, do a check() on the currents lists. (We'll come back to that method later.) The second case says that if the current column binding is null, move on to the next column. The third and final case is where the meat is: we're in a column in the middle, trying to match.

  So, we build up two maps: one for the expected, one for the computed ("actual"). Each map has a list of items that have the given value at the chosen column. We pull out the keys in either list, and work our way through them. Here, there are four interesting cases:

  1. The expected list is empty - we have a value found only in the computed list, so add it to the surplus list.
  2. The computed list is empty - we have a value found only in the expected list, so add it to the missing list.
  3. There is only one value in each list, and they have the same key (by how we got here) - check this row (actual vs. computed).
  4. Finally, both lists have more than one item with the same key value - recurse, but only on the list of items with this same key value. (Now I can see that I'm recursing on a list that's no bigger. It could be the same size if all the keys are the same.)
  I'm left to wonder - does this work as a set or a multi-set? That is, can we have comletely duplicate rows if the "same" object is in the list twice? I'll come back to this.

  eSort() and cSort()

  There are two "sort" routines, one for eexpected and one for computed. They're pretty similar, so I'll describe them together. (I don't get why they're a sort of any sort, though.)

  Each routine produces a Map, from key values to a List of Objects (either domain objects or Parses). The bin() method takes care of putting items in the map. That method expands an Array into a List; the RowFixtureTest mentions a bug in that neighborhood and I suspect this is to address that.

  The sort() methods handle exceptions and rows with no value in a particular cell.

  Back to the Algorithm

  I think I understand what's going on enough to put it into words now. To make a match, we start in a given column. Each list gets divided into buckets, based on the value of the cell at that column. If buckets have 0 or 1 item, then we have a good enough match. Otherwise, we'll look at more columns for those items. Eventually, we'll run out of items or columns.

  check()

  The last interesting bit is around the check() routine. It goes through and checks the columns one at a time, using the normal TypeAdapter facilities. The routine is recursive: it peels off the front of each list, recursing until one or both lists is empty.

  Leftovers and Learnings

  I had a question about whether it acted like a multi-set or a set. It looks like it's basically multi-set-like, from a simple test with a list of integers.

  The other big thing for me to wonder is how I'd have done a similar fixture. I think I'd have expected a simple set. The problem is, that's fine for the query() side, but not so good for the "expected" side: how would you get from those values to construct the objects to compare as sets? (Knowing the contents of an object's fields and return values from its methods doesn't tell you how to construct it.)

  Another alternative would be to get the query values, and match each one up against the rows. The naive algorithm for this is a little slow (n^2). It might be a bit simpler. I suspect its report wouldn't be as nice.

  The current algorithm is able to take advantage of partial matches - if enough data cells make it a unique match, it can then know it has the "right" element even though some of the fields/methods are wrong.

  Closing Out...

  That concludes my tour of Fit. I focused on the main fit package, skipping a couple more minor classes. The code reading was a good exercise for me - I have a better sense of some of the tradeoffs in the code, and of the dynamics in the Fit community.
  
  

  Fit code, part 7 - ColumnFixture

  Posted by wwake on June 24, 2005 at 12:28 PM | Permalink | Comments (0)
  
  ColumnFixture is an easy fixture to understand from the user's point of view: each row is a test case, with some columns being inputs, and others being outputs:

  MyCalculatorFixture
  x	y	plus()
  0	2	2
  1	1	2

  doRows() - Capture the Header Row

  Method doRows() calls bind() to peel off the header row, then processes the rest of the table. Bind() creates an array of TypeAdapters, one per column in the table. If the column header cell is empty, the ColumnBinding is set to to null. (An opportunity for a Null Object? Later, we check for null.)

  If the cell ends in "()", it's a method call, and set via bindMethod(). Peeking inside there, it camel cases the name (so "shirt size()" becomes "shirtSize()") and uses the TypeAdapter.on() method to create the adapter.

  Otherwise, the cell is assumed to be a field name, set via bindField(). That helper method also camel-cases the name and uses the "field" version of TypeAdapter.on().

  Any problem in the header parsing marks the cell with an exception.

  doRow() - Handling the Basics

  DoRow() is fairly simple. It calls a stubbed-out reset() method before handling anything in the row. This lets a fixture reset anything on a per-test basis. It lets the normal row-processing take place, then it checks if execute() has been called; if not, it calls it.

  Execute() is to be called before processing the first column that represents a method name: you can have a bunch of inputs, use execute() to make things happen, then check the outputs. If you don't override execute(), then you either have to know which column is first and make it kick things off (which is brittle), or you let each output column compute "from scratch". (Reading it here makes me wonder if I've been diligent about this in all my ColumnFixtures:)

  If there are any unhandled exceptions, they're attached to the first leaf cell of the table.

  doCell() - Per-Cell Handling

  This boils down to four cases: empty text, null TypeAdapter (mentioned earlier), field, and method. For empty text, we call execute() and move to the superclass' handling of the cell, which marks the cell as "info". This seems a little odd to me - why should a missing value trigger that?

  I'm going to write a test for it, but it took a couple minutes to figure out how to do so. I think what I'll do is create a table with a default value for x and y, leave x blank, and have execute() print the value of y. If it's called when x is processed, it'll print the default value rather than the one that was set.

  (Pause)

  OK - I'm back, and it does act like I expected from the code - execute() is called for a blank cell. I guess that makes sense to do if the blank cell were a call cell (like plus()); I'm not clear on the value for an input cell.

  Back to doCell(): if the TypeAdapter is null, it ignores the cell. If it's a field, it parses the text and sets the field. If it's a method, it calls check().

  check() - Calling execute()

  Method calls go through the (overridden) check() routine. This is really here to make sure the execute() method gets called if it hasn't yet. Then it just defers to the superclass version, which calls the method and compares the result.

  What I've learned

  • Don't forget about reset() and execute()
  • Execute() is a little word in the face of blank input cells.
  
  
  

  JUnit 4 for JDK 1.5

  Posted by wwake on June 15, 2005 at 04:23 AM | Permalink | Comments (2)
  
  JUnit 4 is out for JDK 1.5. Gunjan Doshi summarizes the changes here. It uses the JDK 1.5 "attribute" feature, so you label tests with "@Test" rather than following the convention of naming them "testSomething()".
  
  

  Fit code, part 6 - TypeAdapter

  Posted by wwake on June 14, 2005 at 07:38 PM | Permalink | Comments (0)
  
  

  C# Fit

  I've gotten some mail letting me know that the C# Fit has forked a bit - there's a newer version that's the regular Fit distribution, and an older/modified version that's part of Fitnesse. I was having trouble extending the Fitnesse version. There's an effort to do some unification work this summer; that should help.

  TypeAdapter

  TypeAdapter exists to give a common interface to types, so they can all have setters, getters, and parsers. There are three factory methods, all named on(): one takes a fixture and a class, another a fixture and a field, and a third takes a fixture and a method.

  This gives the unification of fields and methods. In Fit, you can have a ColumnFixture with a field, and it has an implicit setter ("name") or getter ("name()"). Or you can have a method (also "name()"). For most purposes, we don't care what it is, we just want to treat it as a setter or getter.

  The TypeAdapter has five fields:

  • target - the fixture this adapter is "on" (set for a field or method, but not a type)
  • fixture - the fixture this adapter is "on" (always set)
  • field - the Field (only set for field references)
  • method - the Method (only set for method references)
  • type - the type, always set
  I'm struck by the combinations of "this field set / that one not" - would a couple helper classes be an improvement?

  Methods

  The get() method tries to do a field access if the adapter is a field, or a method invocation if it is a method.

  The set() method does a field set. (Could we extend this to call "setter methods" with a signature like setFoo(Foo value)?)

  The invoke() method assumes that a method is set, and calls it with no parameters.

  The parse() method asks the fixture to parse the string according to type. In C#, parse() is something each type (even primitives) define. I'm sure that simplifies some of this code.

  So let's say we have a ColumnFixture where phone() is of type PhoneNumber. How do we make that get parsed naturally? It looks like it works its way back to Fixture, which has a parse() method. So the ColumnFixture overrides it, and checks for an attempt to parse a class it knows about.

  It seems like we could do some fanciness here, too, pushing an attempt to parse onto the domain classes. (So let Fixture.parse() take a look for "type.getMethod("parse")"; if we don't want that we could subclass and override to avoid it.)

  Primitives and Their Classes

  The rest of this file is a whole bunch of subclasses of TypeAdapter: one for each primitive type, and one for each corresponding Class. Most of these are the same: the primitive type's adapter is a subclass of the Class one. The primitive's defines set(), and the class one defines parse()

  The last one is the only exception: Arrays. There, the parse() method tokenizes it by looking for commas. Like so many other places in Fit, it trims spaces. Each element of the array is given its own TypeAdapter. The toString() method puts the commas in when printing it out.

  Reflection

  The big surprise here is the idea of unifying methods and fields. I'm not sure how I'd have come to the realization that they're the same at a level we care about. (That insight is of a piece with the whole framework - I've understood reflection for years, but I've used it more for plugin-style work than anything like Fit that uses test data to drive the reflection.)
  
  

  Fit code, part 5 - ActionFixture

  Posted by wwake on June 13, 2005 at 05:47 AM | Permalink | Comments (2)
  
  Wow - this one is a lot cleaner than I expected. I had tried overriding the C# version and had all kinds of grief. This version is straightforward and extensible.

  Fields

  The class has three fields:

  • cells - a Parse
  • actor - a Fixture
  • empty - an array of Class
  Cells holds the list of cells for this row. It's used by the action methods (such as enter()) to pull out data from the row.

  Actor holds the object created by a start() action. Notice that it is static - that is what lets separate ActionFixture tables keep working with the same object without repeating start in every table.

  Empty is the easiest - it's just an empty list so that things that want a list of argument types can have one. I marked it final, since it's never changed.

  Methods: doCells()

  The first method is doCells(). It saves off the cells, so other methods have access to this row's Parse. Then it looks up the method in the first cell, and invokes it. (This method will be one of "start," "press," "enter," or "check.")

  The fixture invokes the method on itself by "getClass().getMethod()" - looking for the method on itself. This is a place where the Java version is nicer than the C# one. The C# version hard-coded that line to the equivalent of "ActionFixture.class.getMethod()". That meant that a subclass of ActionFixture would only have access to the four methods ("start" etc.) originally planned. The Java version lets you extend this vocabulary easily.

  Another thing to notice is that the fixture calls getMethod() on cells.text(), not camel(cells.text()). That's a pity - my extended vocabulary has to be spelled exactly. (I don't think the rules for camel casing are consistent throughout. I'm probably getting hung up from Fitnesse experience - I think it may have slightly different rules.)

  Methods: Actions

  Start() is straightforward. It creates an object of the named type, and stashes it in the actor field. I note that it doesn't camel-case its argument, so "start MyFrameObject" is different from "start my frame object". (The latter won't work.)

  Enter() looks on the actor for a one-argument method it can use as a setter. It creates a TypeAdapter, which knows how to parse objects, passing it the cell text. Then it invokes the setter.

  Press() invokes the named 0-argument method on the actor.

  Check() assumes its 0-argument method is a getter, fetches the result, and passes it to Fixture's check() routine, which does the comparison and cell coloring.

  Methods: method()

  The two variants of method() try to find an n-argument method on the actor The simple form camel-cases, so the fields on the start object can have the more user-friendly form. ("start MyFrameObject // press the rightmost button".) It double-checks that there is only one possible method. (So, if "firstName(int)" and "firstName(String)" both exist, it will report that it doesn't know which to use.)

  Next time, I'll take up TypeAdapter.
  
  

  Fit code, part 4 - Fixture

  Posted by wwake on June 03, 2005 at 02:02 PM | Permalink | Comments (0)
  
  

  Fixture: Fields and Two Helper Classes

  There's a Map summary that accumulates things like the "run date." I don't know why the top-level Fixture has this, but it does. The fixture fit.Summary walks through this table and gives summary statistics.

  There's a field counts that has counts of tests passed, failed, and exceptions/errors. The Counts class is just a data bag for these things. When a fixture calls wrong(), for example, the count is incremented.

  The last field is args, which has the arguments from the first row of the fixture. The method getArgs() returns a String[] and lets a fixture use them. I don't think this is in the C# version yet but we definitely use that sort of thing there.

  There's an internal class RunTime. It takes a snapshot of the current time. Right now, the only use of this is to put it in the summary, under the key "run elapsed time". Presumably some fixtures pull the RunTime object back out, and use toString() to display the elapsed time. But nothing in the standard distribution appears to use it directly. (Fit.Summary will display the elapsed time when it dumps the summary table.)

  Starting Fixtures

  Now we come to doTables(), the top-level method. (It's called by FileRunner, passing in a Parse for each table.) This method first looks at the name in the first cell of the first row of the table. Then it tries to create the fixture, then use it via interpretTables(). Along the way I note that this routine is using a couple null checks; I wonder if those are necessary? If the first table's fixture fails to be created and run, it runs the remaining fixtures via interpretFollowingTables().

  Method getLinkedFixtureWithArgs() tries to load the fixture named in header.text(), then it sets up the arguments (for getArgs().

  The method loadFixture() takes the name of the fixture, and attempts to "new up" the named fixture via reflection. Between the last method and this, I'm worried by what I don't see: what routine uses the camel method? That suggests a test: let's load "fit.ActionFixture", "fit.Action Fixture", and "fit.Action fixture" and see what happens. From what I understood going in, all three should be ok. From what I'm seeing here, I don't see what would make that work.

  Why did I expect this? Because ColumnFixture does it for column names. It turns out that's not a good enough reason. The test shows that only "fit.ActionFixture" loads.

  Up to interpretTables() again. It does getArgsForTable() - again. There's even a comment to that effect. I don't see why it should be necessary, though. Actually - it's all a little subtle, and I'd say the comment is misleading. The comment says, "// get them again for the new fixture object". But really, that's what we did in getLinkedFixtureWithArgs(). Now we're getting the arguments for the original fixture.

  It works like this: when FileRunner starts, it runs doTables() on a new Fixture object. That's the object that tries to pull fixtures from tables and run them. When the first table is seen, its arguments are pulled out and given to the corresponding fixture. But then they're also copied back to the initial fixture as well. I imagine they're actually rarely needed there.

  At any rate, interpretTables() then calls doTable(), which does a straightforward job of working its way into doCell(). Finally, it calls interpretFollowingTables().

  InterpretFollowingTables()

  By the time we're here, we run through a loop, looking up fixtures and then interpreting them with doTable(). For these, we don't change the arguments on the fixture that started it all. Why not? I can only guess it has to do something with the way DoFixture wants to work - treating the first table special.

  All this work seems a little off - it seems the Fixture class is paying for interpretation that a particular table wants. I'm a long way off from looking at DoFixture, but if that's the table that should be first, it seems to me like it should pay for this complexity. I know I'm second-guessing here...

  Check()

  The other routines are either straightforward, or I've looked at them already. The exception is a largish routine at the bottom: check(). This is a helper method, used by some subclasses. It deals with blank cells, null adapters, "error" expected (to deal with expected exceptions), and text that should match. In each case, this method puts the output in the cell, colored appropriately.

  Up next...

  I think I want to look into ActionFixture next. I had an unhappy session trying to extend the C# version (which appears to be older). Then I want to dig into how TypeAdapters work.
  
  

  Fit code, part 3 - Parse and Fixture

  Posted by wwake on June 02, 2005 at 07:32 AM | Permalink | Comments (0)
  
  

  Parse

  First I want to chase down a couple oddities in what I saw last time. It boils down to these two tests:

  // This test shows offset isn't applied the way I expected
     public void testOffset() throws Exception {
          int offsetToData = 2;
          Parse p = new Parse(
                  "xx
  data
  ",
                  Parse.tags, 
                  0, 
                  offsetToData);
          assertEquals("", p.leader);
      }
  

  and:

  // This test shows cells with embedded tables "go away"
     public void testInnerTables() throws Exception {
          Parse p1 = new Parse(
  "
  stuff plus
  ");
          Parse p2 = new Parse(
  "
  stuff " + 
  "
   inner
  plus
  ");
          
          Parse cell = p1.at(0,0,0);
          assertFalse(cell.body.equals(""));
          assertEquals("stuff plus", cell.text());
          
          cell = p2.at(0,0,0);
          assertFalse(cell.body.equals(""));
          assertEquals("stuff inner plus", cell.text());
       }
  

  (I sent email to the maintainers; these may just be demonstrating my ignorance of how it's intended to work.)

  FixtureTest

  It won't take long to look at this: it only has one test!

    assertEquals("     ", Fixture.escape("     "));
  

  The method basically handles converting plain text to have HTML entities.

  But there's a little more going on in Fixture...

  Fixture

  I've only got a few minutes, so this is a quick overview. From the FileRunner, I can see that doTables() is an important entry point. Last time I looked in the Java version, it was simpler than it is now. There are comments showing code added for DoFixture and for fitnesse, and they've made it all a little trickier. This is all to make an improvement at the user level.

  The core of this is: doTables() calls getLinkedFixtureWithArgs() and interpretTables(), which (eventually) calls doTable(), which calls doRows(), which calls doRow() once for each row, which calls doCells(), which calls doCell() once for each cell. In Fixture, doCell() calls ignore(), which marks the cell gray. ColumnFixture(), for example, overrides doCell() to do something interesting (like look for expected results).

  I'm going to skip over how the first table gets loaded and interpreted - it looks interesting (i.e., tricky:) Instead, I'll peek down to a section labeled "Annotation". This area contains methods that can mark and color cells: right(), wrong(), info(), ignore(), error(), and exception(). I've seen these called in several of the standard fixtures before.

  I see where exception() puts the stack trace into the cell. That gets SO ugly when something goes wrong. (It makes the cell huge, full of scary content useful only to programmers.) Maybe someday I'll take a whack at a more readable version.

  Utilities

  The final section is Utilities. The lightly tested escape() method is there. There's also a method to put words into camel case. I'll add a test:

  assertEquals("twoWords", Fixture.camel("two words"));
  assertEquals("MiXedCAsE", Fixture.camel("MiXed cAsE"));
  assertEquals("aFewWordsTogether", Fixture.camel("aFewWordsTogether"));
  assertEquals(
      "acronymsLikeHTMLStillUppercase", 
      Fixture.camel("acronyms like HTML still uppercase"));
  

  Hmm. This is perhaps not the pattern I'd have chosen. I have the impression some of the other languages do it differently.

  There's a parse() method that appears to handle only Strings, Dates, and ScientificDoubles. I know that C# works a little differently, since parse() is more integrated.

  There's a check() method that looks too complicated to understand in 30 seconds. It uses a TypeAdapter, which is another class I want to look at soon.

  Finally, there's a method to get the arguments from a Fixture. This is new - it used to be that the first row had the fixture only. (Rather, you had to parse any arguments out yourself.) Now that's built in, accessed via getArgs(), which returns a String array.

  That's it for today. Next time, I want to dig into how fixtures get started (since this has changed some), and into the check() method.
  
  

  Fit code, part 2

  Posted by wwake on June 01, 2005 at 06:24 AM | Permalink | Comments (0)
  
  

  Inside Parse

  I spent last time on tests only - this time I want to go inside the Parse class.

  The top of the class reveals strings for leader, tag, body, end, and trailer, as expected. There are also parts and more, which are Parses. A skim through the class, looking for big routines, shows that the constructor, findMatchingEndTag(), removeNonBreakTags(), print(), and footnote() methods seem to be the biggest and most complex.

  Footnote? What's that? The tests didn't mention it! Looks odd - it's not referenced inside fit anywhere; rather, it's used by some clients, typically after a call to wrong(). It appears to create a file Reports/footnote/n.html, and prints the parse to it.

  My strategy today is to chew off the routines that are small and/or simple, then go back and figure out the big routines. I have two things I'm trying to understand: "What happens with nested tables?" and "How do I insert stuff into the middle of a Parse?" (I need the latter for fixtures that want to report a little more nicely.) I guess I have a third question too - "how are spaces handled?" This arises because I saw a note on the mailing list that says there are differences in the various fit implementations.

  Small Fry

  There are some small and simple recursive routines: size(), last(), leaf(), and at(). There are a bunch of little routines for escaping characters and dealing with HTML; I'll come back to those.

  There's a little helper routine addToBody() that just appends text to the body. That doesn't sound like much - and it's a one line routine, basically "body = body + text", but a search for usages shows that this is what fixtures use to get their info into the output. (If a fixture wants to show a cell's expected value, it uses this method to append some HTML text to the cell's Parse.) That answers one of my questions. I'll have to play with it to learn it better.

  The print() routine is longer than these one-line methods, but looks straightforward. It writes the Parse out: leader, tag, then either body or parts, the end tag, and either the more or the trailer. I knew body and parts were mutually exclusive; I hadn't realized that more and trailer are exclusive as well. I wonder if body and trailer appear together, and parts and more appear together? If so, I wonder about splitting Parse up so subclasses can deal with that difference. It's not a huge class; may not be worth it.

  Constructors

  That leads me up to the first constructor - Parse(String tag, String body, Parse parts, Parse more). Note that it has parameters for both body and parts. So much for my theory of a paragraph ago. But it's close - I did a search and found 15 places that called this constructor. All but three used either body or parts exclusively.

  One of the ones that didn't is in fat.Table It is using this constructor to copy an existing Parse. That looks misplaced - if we need to copy these, then we can put a method on Parse to do so. A second place is fat.FixtureNameFixture. The GenerateRowParses() method passes in a string for "body" and a Parse for "more". (So we have an example where "parts" and "more" don't go together.) I can't tell why it does this on a quick look. The final place is eg.AllFiles.td(), which also uses "body" and "more" together.

  The first constructor passes in all the pieces separately. Then there are a few constructors that default tags and so on, to the main constructor that actually parses some HTML. That fixture looks for several key positions in the input: the start of the target tag, the end of the target tag, the start and end of the corresponding end tag, and the start of the rest of the text.

  I see that the first search starts at the beginning of the string, rather than at "offset". That seems odd.

  We'll have to double-check how findMatchingEndTag() works, but the rest of the constructor looks straightforward: if there are more tag levels, turn the body into a new Parse (and set body to null). If there's a nested table, parse the table and set the body to "". (That seems odd also, like it's throwing away any non-table stuff. I'm not sure what the "" body accomplishes either.) Finally, if there are more tags at the current level, null out the trailer and parse the remaining tags into "more".

  FindMatchingEndTag() looks like an implementation of the parenthesis-balancing rule - add 1 every time you see a left parenthesis, subtract 1 every time you see a right parenthesis. If you're balanced, you'll have a net of 0.

  So I have an answer about nested tables: it's trying to handle them. I'm seeing a little weirdness that makes it look like a nested table is the only thing retained inside a cell. But at least I know it's trying. I'll make some tests to fill in what I'm seeing. I only have a few minutes left, so I want to move on to the htmlToText() part of the code.

  Html to Text

  The htmlToText() routine has four steps: normalize line breaks, remove non-break tags, condense white space, and unescape. Normalizing line breaks turns <br> and strings of <p> tags into <br />

  Removing non-break tags is a little tricky-looking, but it basically squeezes out tags other than the normalized break tags we just produced. The method "looks forward" to see an end-of-tag; if it's there, it trims out the tag and looks at the rest of the string.

  Condensing white space applies the rule: convert multiple blanks to a single blank, convert a "160" to a space, and convert &nbsp; to a space. I assume 160 is the code for a non-breaking space in Word's font.

  Unescaping is simple too: br tags are converted to newlines, standard entities such as <lt; are converted to their simple character, and smart quotes are converted to " or ' as appropriate.

  The result of all this is that text() produces the Parse in straight text form - no tags. This is what fixtures will want when they compare expected values.

  Summary

  I had three questions:
  • What happens with nested tables? They are apparently handled, althrough it looks like only the nested table is retained, not anything surrounding it.
  • How do I put stuff inside a Parse? Use the addToBody() method.
  • What happens with spaces? Multiple spaces get converted into one, and non-breaking spaces get converted into one space each.
  I'm left with a little bit of question in my mind about why the Parse constructor doesn't use the offset when it's looking for the first tag, and about the details of nested tables. But that's ok; I learned a lot today.
  
  

  Fit Reading (1 of n)

  Posted by wwake on May 30, 2005 at 07:33 AM | Permalink | Comments (1)
  
  "Fit" is Ward Cunningham's "Framework for Integrated Tests". You can pick up a copy by starting from fit.c2.com.

  I've used it for a while, and looked at some of its code along the way, but hadn't sat down and really studied it systematically. My plan is to spend an hour at a time, just digging in to what I find and sharing my notes.

  1. Download and Compile

  I download a copy and pull it in to my IDE. I see there are some yellow flags (syntax warnings). One of them looks like a style difference:
  if (x > y) return x+y; else return x*y;
  This is legal but it complains about the fact that there's an "else" clause following a "then" with a return. I'll just change things along the way even though I don't plan to feed this back to Ward.

  I also put in an AllTests suite. Method fit.FrameworkTest.testRuns() fails, showing that I haven't moved the examples directory to the right place. This has happened at least partly because I moved stuff around when I put it in Eclipse. I copy the examples folder in, and create an output/test folder. The test tries to run, but fails the actual test (not just from a bad folder name). But time is ticking, so I'm moving on.

  I take a quick look around at the files that are there - a lot of them are fixtures, which I've at least subclassed before.

  2. Parse

  FileRunner and WikiRunner are the two executables I see. WikiRunner says it's deprecated, so I delete it. FileRunner does three things: read its arguments, create a parse of the document, and run the fixture on the parse.

  I know already that Parse is some sort of tree-structure representation of the document (like DOM but simpler). I'll start by looking at ParseTest. The first test shows that it divides a string into 4 pieces: a leader, a tag, the body (inside the tag), and a trailer. The test just used "body"; I wonder what happens if you nest tags, so I try that (with a new test). Turns out it just accepts the new tags as part of the body. (That suggests how tags fit doesn't care about are handled - they're just left in the body.)

  The next test shows what happens when tr and td tags are thrown into the mix: parsing is different. Parse has a parameter that tells which tags it's interested in. It looks like we have a choice: when there are no interior tags, the information is put in "body"; when interior tags exist, they form a new level, inside "parts". "Parts" is another parse, so the whole structure is recursive.

  The next tests show that we can navigate to "sibling" nodes using "more". To move deeper into the parse, we use "parts"; to move to a sibling node at a given level, we use "more". I saw mentioned somewhere that Parse has a Lisp-like structure, and this is the place that makes that happen. (Lisp uses "car" to get to the head of a list, "cadr" to get to the second element, "caddr" to get to the third, etc. In the tests, we see code like p.parts.parts.more.body to get to the second cell of a table. My Lisp is rusty, but this gives me something to tie it to.)

  The Parse has an abbreviation, "at(i,j,k)". I really think of this as "at(table-index, tr-index, td-index)". The test for this shows that if an index is too big, you get the last element in the list (at that level). Also, there are helper methods to get the first leaf and the last node in a list.

  The next test shows that the parser throws an exception if a tag isn't found where it's expected. (So a table that is missing its td tag will complain.)

  The Parse has a text() method that prints the human-readable form of an HTML text. It handles & references, blanks, etc.

  Finally, there are a pair of tests for helper methods that deal with escape characters and white space.

  3. Where are we?

  My hour is over. Admittedly, some amount of it went into writing these notes. I've added a few tests that clarified some of what is going on. I have a little better understanding of the Parse structure (body versus parts, for example). I still don't know what it does in the face of nested tables. (This used to be a problem; I think they may have fixed it, but there's nothing in the tests to say one way or the other.)

  I had hoped to get into the code for Parse itself, and that's my plan for next time.
  
  

  XP Explained, 2/e

  Posted by wwake on March 01, 2005 at 05:16 AM | Permalink | Comments (0)
  
  [Trimmed and crossposted from XPlorations]

  I don't think the new edition of Extreme Programming Explained has had as much exposure as it deserves. I've summarized its key points, and added a bit of review.
  http://xp123.com/xplor/xp0502/index.shtml

  ---

  And a few other reviews at http://xp123.com/books/index.htm:

  • Crystal Clear, Alistair Cockburn. Addison-Wesley, 2004.
  • Office Kaizen: Transforming Office Operations into a Strategic Competitive Advantage, William Lareau. ASQ, 2003.
  • Mapping Inner Space, 2/e, Nancy Margulies with Nusa Maal.
  
  
  

  Kent Beck's "Programming Intensive" Workshop

  Posted by wwake on February 07, 2005 at 07:48 PM | Permalink | Comments (0)
  
  Kent Beck's workshop was a chance to spend a few days programming and thinking about programming.

  We used "games" as the vehicle. That worked well enough - you can get the feel of a game without having to develop it all the way out. In the evenings, we were on our own to do some writing and exercises in thinking about software.

  The first couple days, we focused on creating a crossword puzzle helper. The idea is that you'd populate some parts of a grid, and then the tool would fill in the remaining words. We got far enough into it to show it worked on moderate-sized examples, and to start optimizing.

  The next program we worked on was a partially developed tic-tac-toe program. We completed some screen hookup and looked at improving the code. We spent a good bit of time contemplating a couple approaches to part of the problem, and how you'd choose between them.

  Finally, we took a stab at something closer in spirit to an arcade game; a relative of the commercial games Pong or Breakout. We got as far as having a paddle and a ball bouncing around. We gave it a twist - you controlled the velocity of the paddle, not its direct position. As primitive as it was, Kent's children gave it a thumbs up.

  The week was good for me. I got to learn some things about Eclipse, I got to learn some things about design. We had a couple stretches where we had the "flow" feeling of losing time in the zone. One of the puzzles I've been working through is how to take a thin slice of a system; I had some time to think about that.

  Would I recommend it? Yes: I learned a lot, and found it great to spend some "renewal time" with others who just wanted to program.
  
  

  Agile 2005 conference call for participation

  Posted by wwake on February 06, 2005 at 04:18 AM | Permalink | Comments (0)
  
  See www.agile2005.org. The due date for tutorials and workshops is March 1; for Experience reports, research papers, and the educators' symposium is March 15.

  Call for Participation - Agile 2005
  July 24-29, 2005. Marriott Hotel, Denver, Colorado, USA.
  http://www.agile2005.org

  March 1: Submissions due for Tutorials and Workshops
  March 15: Submissions due for Research Papers, Experience Reports, and Educators' Symposium

  Agile 2005 integrates the best features of the Agile Development Conference and XP Agile Universe to create an exciting conference about techniques and technologies, attitudes and policies, research and experience, and the management and development sides of agile software development. The agile approach focuses on delivering business value early in the project lifetime and being able to incorporate late-breaking requirements changes. It accentuates the use of rich, informal communication channels and frequent delivery of running, tested systems, while attending to the human component of software development.

  Agile 2005 gives attendees access to the latest thinking in this domain, and bridges communities that rarely get a proper chance to exchange ideas and thoughts. It brings together researchers from labs and academia with executives, managers, and developers in the trenches of software development. Agile 2005 is not about a single methodology or approach, but rather provides a forum for the exchange of information regarding all agile development technologies.

  We invite submissions for the following:

  • Research Papers
  • Experience Reports
  • Tutorials
  • Workshops / Peer to Peer Sessions
  • Educators' Symposium
  Other conference activities include:
  • Introduction to Agile (for Agile "newbies")
  • Executive Summit
  • Open Space
  FOR MORE INFORMATION: http://www.agile2005.org
  
  

  Agile Immersion

  Posted by wwake on February 05, 2005 at 06:20 PM | Permalink | Comments (0)
  
  Object Mentor's reviving their XP courses with "Agile/XP Immersion 2"; see http://www.objectmentor.com/Immersion2. March 21-25.

  I attended the first one of the original series, and really enjoyed it.
  
  

  Retrospectives class

  Posted by wwake on January 17, 2005 at 05:00 PM | Permalink | Comments (0)
  
  Reflecting on your process and how to improve it is an important part of agile methods.

  Later this month, Diana Larsen is leading a class on retrospectives that may be of some interest: "Project Retrospectives and Reviews: A Facilitator's Toolkit," January 24-25, 2005, at Oregon Graduate Institute; http://www.cpd.ogi.edu/coursespecific.asp?pam=1573

  She's also announcing another class in April: "The Secrets of Agile Teamwork: Beyond Technical Skills," presented by Diana Larsen, Esther Derby, and Ken Schwaber. April 5-7, 2004. See www.futureworksconsulting.com .
  
  

  Purple Crayon Navigation

  Posted by wwake on January 06, 2005 at 07:08 PM | Permalink | Comments (9)
  
  

  "Harold and the Purple Crayon" is a children's book where Harold uses his crayon to draw whatever he needs, and then it's real enough to use.

  Sometimes you want to navigate to a class. One way is to go find it in the list of classes or out on disk. If you have a one-keystroke "jump to definition", one way is to just type the class name into the file you're in, and navigate on that. Yes, it may leave an extra word where you were, but it can be a quick way to bounce over to a class you want a peek at.

  
  
  

  Team rooms

  Posted by wwake on December 24, 2004 at 08:55 AM | Permalink | Comments (0)
  
  

  William Pietri has posted a picture of his team room, at http://www.scissor.com/resources/teamroom/

  I'm maintaining a gallery of team rooms, too, at http://xp123.com/xplor/room-gallery/index.shtml. I'd love to have pictures of your team room or charts that you use.
  
  

  XP2005 CFP

  Posted by wwake on December 09, 2004 at 09:27 PM | Permalink | Comments (0)
  
  XP2005 has their call for participation out. They'll be at Sheffield University (UK), June 18-23, 2005. See http://www.xp2005.org for more information.

  IMPORTANT DATES
  

  • Paper submissions: January 16, 2005
  • Acceptance notification for full papers: March 6, 2005
  • Submissions for tutorials, workshops, panels and activities, PhD Symposium, and posters: March 1, 2005
  • Acceptance notification for submissions other than full papers: March 20, 2005
  • All final manuscripts: April 3, 2005
  • Pre-registration ends: May 7, 2005

  Conference Chair: Michele Marchesi, University of Cagliari, Italy
  Local Chair: Mike Holcombe, University of Sheffield, UK
  Program Chair: Hubert Baumeister, University of Munich (LMU), Germany

  For any additional information please contact Hubert Baumeister at: program_chair@xp2005.org
  
  

  Scrum Gathering Oct '04

  Posted by wwake on December 04, 2004 at 09:36 AM | Permalink | Comments (0)
  
  

  The Scrum gathering was a workshop gathered for a couple days in Denver, this past October. We worked in three groups: metrics, process, and facilitation. (Scrum is an agile process. I think of it as approximately the project management part of XP, though that's of course not fair to either one:)

  I participated in the facilitation group. (The others were metrics and process.) We spent a lot of our time trying out a variety of simulations and other means to help teams understand what it means to do Scrum. I contributed two exercises: Push Line/Pull Line (a demonstration of lean manufacturing), and Second Agenda (a role-play of a standup meeting).

  Esther Derby shared a debriefing framework she uses: What/Gut/So What/Now What? "What" asks for objective data about what happened. "Gut" asks for your emotional reaction. "So what" asks for your interpretation. And "Now what?" asks what you'll do next. This framework helps us avoid jumping to interpretation too early.

  The process group identified challenges around cross-functional teams, the role of testing, and the challenges of leading or lagging behind (e.g., the analysts want to be an iteration ahead). The metrics group identified metrics as a means of making management comfortable, and created and highlighted various means of helping with that.

  I enjoyed this workshop a lot. As always, it's good to meet people and put names with faces. It really helped me feel a sense of community.

  On my way out of town, I passed a quotation on a building: "Stay firmly in your own path and dare; be wild two hours a day!" - Paul Gauguin.
  
  

  Refactoring Thumbnails

  Posted by wwake on August 09, 2004 at 10:45 PM | Permalink | Comments (0)
  
  

  Sven Gorts has introduced what he calls Refactoring Thumbnails. These are UML-like diagrams augmented with some flows, and used to summarize refactorings. (For example, the UML might have no words, but rather squiggles to represent identical text in two different classes.)

  In addition to summarizing the transformation involved in simple refactorings, he uses these to show how large refactorings can be created out of smaller ones. A nice example is Break Module Dependencies With Adapter. He shows that you can break package dependencies by doing Separate Interface from Implementation, and then Introduce Indirect Class, or by doing these in the opposite order. For Evolving to the Proxy / Decorator Pattern, he shows several approaches that end up in the same place.

  I really like these summaries, and I'll use this approach to help manage my focus on large refactorings.

  
  
  

  Refactorings require new tests

  Posted by wwake on May 31, 2004 at 05:19 AM | Permalink | Comments (0)
  
  

  Someone asked on the XP egroup about getting access to private methods for testing purposes. Others suggested a number of ways to get this effect, but it got me thinking about refactoring.

  Refactoring is often thought of as a pure, safe transformation: convert a program into another program with the same semantics but a better design. From the standpoint of a refactoring tool, the "same semantics" part is crucial.

  But refactoring also has a psychological side: a better design, but also a different design. A different design may induce people to act differently (indeed, that's why we do it!). In particular, a different design may give people different expectations about code.

  Following are some examples. In each case, I'll assume the code was created by test-driven development, and adequately tested before the refactoring.

  • Extract Method - the code worked as part of another method (and still does). But now, the reader's going to assume they can call this method from other parts of the class. Is the extracted method tested sufficiently on its own terms?
  • Expose Method (private becomes protected) - now, subclasses expect to be able to call this method (either directly or via a call to super()). We'll need to create testing subclasses to verify that it works in that context.
  • Expose Method (to ) - Other objects are free to call it. The original object no longer has control over the order in which this method is called. (We may have had a method that was only to be called if another method was called first; when it was private, we were ok; if we expose that method, it's hard to enforce this obligation.)
  • Extract Class - The object now stands alone. Is there a test class testing this object by itself? You may need to extract a new test class, but you also may find you need new tests to cover everything.
  
  
  

  Scratch Refactoring

  Posted by wwake on May 12, 2004 at 03:54 AM | Permalink | Comments (5)
  
  

  I recently had a chance to do some refactoring of some Visual Basic code. I hadn't worked with it in several years. In particular, I hadn't worked with the object support that's in VB.Net. It's very striking how much it's like C# with different keywords.

  My task was to convert some code from using web services (which were too slow) to just straight object code. Several factors came together:

  • My unfamiliarity with VB.Net and web services.
  • The fact that I was heading out of town for a week and didn't want to risk leaving problems.
  • The team's use of source control which locks checked-out files. (I'm lobbying to change this.)
  • A desire for extra care as the product has almost no automated tests.

  This led me to do a scratch refactoring: refactoring with the intent of throwing it away and re-doing it. I refactored away two sets of web services, writing down each of the changes I intended to make (along with the name of the affected file.)

  I found several benefits:

  • I learned some little tricks for how to get the compiler to tell me what needed doing.
  • Knowing I would throw the result away let the scratch refactoring go more quickly.
  • I learned a couple tricks for the IDE.
  • When I came back after my trip (to do the real thing), I felt like I was just flying through the changes.

  I think it was Brooks who said something like, "It's faster to make a 6-inch mirror and a 10-inch mirror than it is to make a 10-inch mirror." I found that true in this case.

  
  
  

  Test-Driven Development and Teaching to Test

  Posted by wwake on April 18, 2004 at 06:46 AM | Permalink | Comments (3)
  
  

  Example-Driven Development

  Test-Driven Development is a style that says "write a test for a small bit of functionality, write code to make it pass, refactor, and repeat."

  In a way, the "test" part of the name is misleading. TDD does produce tests in the sense that they are written to verify whether something works, that an expected answer is defined in advance, and so on. But they're not tests in the way a tester would seek - they're written for the programmer's purposes. That they are mostly somewhat useful as tests is in a sense a happy side effect.

  This became most clear to me in a group discussion including James Bach, one of the founders of the context-driven testing school. Someone described a good test that a TDD programmer might write, and James said, "I'd never write that test - it's too small." That really drove home to me how different the purposes of the tests are. For testers, the goal may be things like "efficiently acquire information about the status of the program" or "see if this area works" (and other things); for developers, the goal of the test is "drive me to create the next part of the design."

  Brian Marick has talked about how TDD produces checked examples. Maybe Example-Driven Development would have been a better name.

  Teaching to Test

  Let me shift gears to the education system: For a variety of reasons, states have moved to using tests to assess students, teachers, and schools. This has its own challenges and controversies. (It's hard to interpret what you see without more information - if students perform poorly, do you have wonderful students crushed by poor teaching, or heroic teaching that can't overcome other problems?)

  But on the ground as a parent, one of my concerns has been "teaching to test." There are two approaches I've seen:

  • The teacher covers an area thoroughly, and the test acts as a random sample of what students should know in this area.
  • The teacher tries to out-guess "what will the test questions be" and "how good are students at test-taking", covering nothing beyond what is necessary.

  In a sense, both care about the test. But the latter form is the pejorative sense of "teaching to test." It's like the students who ask, "Will this be on the test?" rather than "Do I understand this?".

  The first approach comes from a position of abundance, the other from a position of scarcity.

  The same thing can happen in Test-Driven Development. Early tests drive the design, and later tests tend more to verify it. At some point, my expectation has become "No matter what test I'd write, this code should pass."

  But note what happens if I stop "teaching" too soon. I may have covered the simple cases - I've passed the tests at hand - but I may have left out the design complexity needed to handle the full requirements. Once the system is in the real world, it has to pass the "real" test of use.

  Some of the TDD techniques, such as Fake It 'Til You Make It, explicitly create code that wasn't directly tested - you generalize your code from a small set of examples. This leaves a window where you may generalize incorrectly. If you stop testing too soon, you may not realize your mistake until later.

  I try to take a sense of scarcity as a sign - if I feel pressed to rush through some task, I take that as a flag that I may not have tried enough examples.

  
  
  

  TDD: Tension, Release, and Generalization

  Posted by wwake on March 03, 2004 at 09:46 PM | Permalink | Comments (4)
  
  

  Test-driven development uses a tight cycle of "test, code, refactor" to develop software.

  Tension and Release
  I use the analogy of a stoplight: you start with a green light (all tests passing). Then you write a test; often you're referring to classes or methods that don't exist yet, and get a compiler error (yellow light). You fix this error by writing stubs, and when you run the test it fails (red light). Then you add just enough code to make the tests pass (green light).

  Part of this style that may not be obvious is that most of the time you have a "green light": just like driving, you prefer not to spend a lot of time with a yellow or red light.

  The most obvious use of this principle is in a technique Kent Beck calls "Fake It ('Til You Make It)": you get rid of the red light by making the code return exactly the answer the test expects. For this test:

  public void testLength() {
assertEquals(3, buffer.size("abc"));
}

  you might write this code:

  public int size(String contents) {
return 3;
}

  The first time you see this approach, you're likely to think "that's cheating!" We're supposed to be writing a program to do some complicated thing, and we come back with a trivial answer like that.

  When you have tests that don't pass, you should feel tension: you have tests, and you have code, and they don't agree about how the code should work. It's the kind of tension from holding your breath, or trying to thread a needle, or watching a movie where people don't realize the danger they're in (depending how much tension you like, I guess:) It's unstable: what will happen?

  The solution is to get out of the unstable state quickly, and release that dramatic tension.

  This is important, because most of our tools work best when the system is working:

  • If our system works for the tests we have, but not for some tests we don't yet have, we can add those tests.
  • If other people have made changes, and checked in code with all tests passing, we can integrate their changes and know that if tests fail it's our problem.
  • If our system isn't as well-designed as it could be, we can refactor it. But refactoring is best applied when all tests pass.

  Generalization
  Once we've done the "fake it" part, we can use generalizationto do the "make it" part. Generalization is a relative of refactoring: where refactoring tries to preserve behavior, generalization tries to, well, generalize it.

  In our example, we created a fake response "return 3" for a reason: 3 is the right answer, determined by looking at the length of the string. We can generalize our answer from "return 3" to "return contents.length()".

  Generalization goes beyond the tests we already have, to the ones we haven't written yet. (Refactoring, in its pure form, preserves behavior for tests we have written, as well as the ones we could write.) Generalization thus makes a guess about the tests we would write. Sometimes this guess can be wrong, so we may write other tests that we think will pass, just to verify it.

  I think of test cases as the shadows of behavior. We write code that we think will project that shadow. The "Fake It" part is like using a cutout to generate the shadow - it's not a true 3-d object, but it's good enough to show the right shadow. Then generalization can be used to "inflate" the code into a full object.
  
  

  Fisheye Lens Project Management

  Posted by wwake on February 03, 2004 at 04:56 PM | Permalink | Comments (2)
  
  

  A traditional project plan has an ethos, "Plan the work, work the plan." The planning process will work out all the expected tasks, estimate them all, and assign workers to the tasks. This has several problems: the sheer mass of data makes it hard to see what's really important, the plan is vulnerable to changes in direction, and it's hard to keep up to date.

  XP and Scrum (and some but not all other agile methods) take a different approach: the set of features currently under development is tracked at a high level of detail (with an iteration plan or sprint backlog), the set of features planned for the next release are tracked at less detail (release plan or product backlog), and features further out are tracked at even less detail.

  There's a type of lens known as a fisheye lens. (You can see a sample picture at here). This lens gets a 180-degree field of view, so it shows a much wider area than a typical lens. But it does it in a way that appears distorted: things in the center appear mostly normal, but as you get closer to the edge, it looks more distorted.

  I see the agile methods described above as acting like that: lots of detail in the center (the now), and less detail in the periphery (the future), but still enough detail to let you make sense of the big picture. When you consider that agile methods focus on high business value first (slicing by feature rather than by technology), they truly focus on what's important.

  (For more on XP, see www.xp123.com, www.xprogramming.org, or www.extremeprogramming.org; for information on Scrum, see www.controlchaos.com.)

  
  
  

  Tools - especially JUnit and Fit

  Posted by wwake on December 25, 2003 at 03:22 AM | Permalink | Comments (2)
  
  

  I'm reflecting on the most important tools I've been using this past year for my Java projects.

  • IntelliJ Idea - A fine IDE. My current default.
  • Eclipse - I've used it some, and found it a little clunkier than IntelliJ's. But I plan to move toward it more this coming year.
  • P4 - Perforce source control system. It's free for a single user, and does a nice job.
  
  

  I've used two primary testing tools:

  • JUnit - for unit testing.
  • Fit - for system/acceptance testing.

  JUnit is a s