Skip to main content

Maven & multiple source trees

Posted by michael_n on May 9, 2005 at 2:43 AM PDT

I had heard recently from someone, yet again, that "contorting" maven to allow for multiple source directories was inconvenient, if not an outright chore.

Unfortunately, when people ask if maven "supports" multiple source dirs, the mavenites take the question sometimes too literally (not realizing that one who asks rarely knows exactly what to ask for -- as many an instructor would attest) and inadvertently give an answer that is less than helpful, and not really even in the spirit of what maven is trying to accomplish (which is, in essence, simplicity). Also, it doesn't help that most of the "Intro's to Maven", as well as the Maven "genapp" plugin, only show the 'degenerate' maven project: a single source-tree project. Although it's possible just to add in another src tree, which would then contribute to the existing generated jar/war/etc, it's 99% certain that isn't what you want to do.

So, the next time someone asks about Maven and "multiple source trees", please direct the person towards the Maven "multiproject" plugin. (This is probably obvious & old-hat to many; but, since I had once again heard the the issue broached recently, I thought I'd once again bring up the solution as well.)

Quick note on plugins & Maven: Maven implements just about all of it's functionality via plugins; so just because something is called a "plugin", don't think you really have to do anything special to get it to work. Also, w.r.t. "multiprojects", this functionality comes built-in for Maven2; so, the same basic ideas and project structure apply.

You will want to use a 'multiproject' for just about anything beyond a "hello world" project. The basic forces at work are:

  • you have (or will have) a lot of source,
  • you want to minimize dependencies between the subsystems / components
    of your project,
  • you want to (or at least have the option to) build individual jar files for each of the subsystems / components of your project

Other things to consider:

  • it's easier to have separate source trees now for separate components than trying to extract out components later from a mega-source directory tree (yes, you have packages, but what are your dependencies between packages? Do you know?)
  • you want to avoid cyclic dependencies between components - that's easier to avoid during development, than to rectify after the fact
  • it's easier to test components in isolation if the component and it's dependencies can be easily, well, isolated. (Or, "Those who build together, end up staying together.")

To start with (and to use as a reference point), here's what your basic single-source-tree (i.e., non-multiproject) Maven project directory structure looks like

+- myproject
    +- README.txt
    +- LICENSE.txt
    +- project.xml
    +- build.properties.sample
    +- target/
    |   +- ....
    |
    +- src/
        +- main/
        |  +- java/
        |  |   +- org/myorg/myproj/main...
        |  |
        |  +- resources/
        |      +- foo.properties
        |      +- ...
        |
        +- test/
        |  +- java/
        |  |   +- org/myorg/myproj/test/main...
        |  |
        |  +- resources/
        |      +- ...
        |
        +- site/
           +- xdoc/
               +- faq.fml
               +- ...

(note: the target dirs are for generated files; they aren't necessarily under version control)

The only change to create a "multiproject" is to add a directory, under myproject, such as "modules", which contains all of your additional "subprojects" (my term). Each subproject (or module) has it's own abbreviated project.xml (these are shorter, since they "inherit" most the meat from the top-level project.xml). Each "module" has its own src tree, exactly as shown above, with it's own test and it's own (optional) "site" directories (the site dirs contain, e.g., XML documentation to be automatically converted to HTML by the xdoc plugin).

Next, let maven know where your modules are, by setting the following in your project.properties file (a few other commented out props are added, fyi):

# where the subprojects are 
maven.multiproject.basedir=${basedir}/modules/

### etc:

## subprojects to ignore (relative maven.multiproject.basedir)
#maven.multiproject.excludes=sandbox/project.xml

# what type of artifact (deliverable) to build; should have this
#  in each module's dir, in a project.properties file (if different than jar)
#maven.multiproject.type=jar

# list of goal(s) when running multiproject:site
#maven.multiproject.goals=site

# location of xdocs
maven.docs.src=${basedir}/src/site/xdocs

If you like, you can leave in the top-level myproject/src tree, into which you could put your "main"-type functionality of your application (or ejb, war, jar, etc). For sanity's sake, the subprojects should not depend on the "main" (top-level) src; rather, the top-level src should depend on the subprojects. (In other words: the top-level src code "uses" the subprojects, but the subprojects should in no way ever use the top-level src. Otherwise, you're just asking for circular dependencies. Managing dependencies in this way & avoiding circular dependencies between artifacts just keeps life simpler for the future.)

Here's the multi-project, then:

+- myproject
    +- README.txt
    +- LICENSE.txt
    +- project.xml
    +- build.properties.sample
    +- target/
    |   +- ....
    |
    +- src/
    |   +- main/
    |   |  +- java/
        |      +- org/myorg/myproj/main...
    |   |  +- resources/
    |   +- test/
    |   |  +- java/
        |      +- org/myorg/myproj/test/main...
    |   |  +- resources/
    |   +- site/
    |      +- xdoc/
    |
    +- modules/
        +- core/
        |    +- project.xml
        |    +- target/
        |    +- src/
        |        +- main/
        |        |  +- java/
        |        |  |   +- org/myorg/myproj/core...
        |        |  +- resources/
        |        |
        |        +- test/
        |        |  +- java/
        |        |  |   +- org/myorg/myproj/test/core...
        |        |  +- resources/
        |        |
        |        +- site/
        |           +- xdoc/
        +- gui/
        |    +- project.xml
        |    +- target/
        |    +- src/
        |        +- main/
        |        |  +- java/
        |        |  |   +- org/myorg/myproj/gui...
        |        |  +- resources/
        |        |
        |        +- test/
        |        |  +- java/
        |        |  |   +- org/myorg/myproj/test/gui...
        |        |  +- resources/
        |        |
        |        +- site/
        |           +- xdoc/
        |
        +- module-xyz/
        |    +- project.xml
        |    +- src/
        |    +- target/
        |
        +- ...  (more modules as necessary)

Building the completed project is trivial: the commands below do the following, respectively: compile/test/build the jars; "install" the jars into the local repository; generate the site docs; "clean" the project:

  $ maven multiproject:jar
  $ maven multiproject:install
  $ maven multiproject:site
  $ maven multiproject:clean

Despite already being rather straight-forward as-is, I often create more readable "aliases" for these targets (to show in my "usage" goal) in the maven.xml file (note: other than this, you don't really need a maven.xml file):

  <goal name="myproj-clean" description="Clean project and subprojects">
      <attainGoal name="multiproject:clean" />
   </goal>

Note that you can run normal (non-multiproject) maven targets from within the modules; just go into the "modules/core" directory, and run "maven jar" and "maven install" to build the jar, and copy the jar into your local repository.

That's basically it. There are other configurations which could be introduced, as needed, such as:

  • Reporting: There's a list of reports that are generated by default; but, as soon as you list any of your own reports in project.xml, the defaults aren't run anymore. You have to list them explicitly in the top-level project.xml to get them to run again. Reports are not "inherited" to subprojects' project.xml files, either.
  • Artifacts: That which is generated by each module is called an "artifact" (e.g., the jar, war, ear, etc.) The type of artifact to generate should be explicitly denoted in the subproject's properties file if it's anything other than a jar.
  • The top-level src: When invoking the "multiproject:" targets, the top-level "src" isn't considered; you have to run it's targets separately (ie, "maven jar", "maven site", etc). You could also customize this via the maven.xml file.
  • Building the modules: you can build these from the top-level, or from within each of the subprojects. You may want to work just within a submodule to speed up the compile/test cycle.
  • Dependencies: Remember, any maven project draws it's "dependencies" (and "completes" its classpath, so to speak) from a repository; so, you will be "installing" your artifacts from your subprojects into your local repository in order to use them from other projects. You'll be labeling your generated artifact's version as a "snapshot" during development, in order to publish & use the most recent ones.
  • Non-java source: for example, other source files could be added under src/main/antlr if you have (for example) Antlr grammar definition files (see other maven conventions)

For more info, see:

Happy Maven'ing,
-Michael

Related Topics >>