Bob Lee's Blog

Package Scoping And Unit Testing

Scopes are one of the first things we learn as Java developers. Public, private and even protected scopes are easy enough to grok, but package-private scope alludes many Java newbies. I remember confusion when I first read about package-private scope; "Why would I ever need that?" I learn by understanding, not memorization. Missing the necessary context, my brain failed to retain package-private scope the first time through: in one ear, out the other.

Even when developers do have the background necessary to understand package-private scope, I think its inconsistency with the other scopes leads to further confusion. If you want a public method, prefix the method signature with public. For a private method, use the private keyword. Protected, protected. If you want a package-private method... omit the keyword altogether. For example, use String foo() instead of private String foo(). Adding fuel to the fire, Java documentation often interchanges the terms "package-private," "package," and "default." I'm embarrassed to admit I once tried to use package-private as a keyword.

Many programmers don't know (or think) they need package scoping. They try to work around its absence. Some messily expose implementation classes as part of their public API. This clutters the Javadocs and increases the library author's maintenance burden; whether the author likes it or not, they should support clients who depend on these implementation classes. Using package scope prevents clients from creating dependencies on internal implementation classes in the first place.

A few programmers like to put their public API in one package and their private implementation in another. They attempt to hide the "private" package from the client by not publishing its API. From a security viewpoint, exposing your private package to your public package inherently exposes it to everyone else (including the client). This may fly for application code, but you must hold reusable libraries to a higher standard. Java provides no way to say, "make foo public, but only for this one other package." In addition, splitting your public API and implementation into two different packages confines you to class granularity, whereas package scoping enables you to mix public and package-private members in the same class.

Others use protected scope when they should use package scope. My skin crawls when I see Javadocs dusted with protected members (Struts anyone?). This solution suffers the same fate as making your internal implementation classes public. Unlike package scoped members, protected members are still part of the public API; clients can access them, and the fields and methods show up in the Javadocs.

Package scoping particularly shines during unit testing. Some programmers argue that you should only test through the public API. Don't be silly. Limiting your tests to the public API contradicts the spirit of unit testing and subjects you to unnecessary dependency pain. I prefer to isolate and limit the amount of code I test at one time, and test as close to the code as possible.

To expose a class member to a test without impacting the public API, put the test class in the same package as the code it tests. Make the member you wish to expose to the test package-private instead of private (i.e. delete the private keyword from the declaration). I've actually grown accustomed to using package-private scope instead of private by default. Less typing, less clutter. To prevent clients from putting their classes in the same package as yours and accessing your package-private members, Java supports package sealing.

Comments

• I have always found it odd that the default level of access is more restrictive than "protected". For a newbie you'd assume that if you "protect" something you are making is less visible... but you're not, you're making it more visible.

  BTW, anyone remember the little "private protected" experiment from Java 1.0?

  Posted by: pat on July 19, 2005 at 05:52 PM

• Ha ha. I do remember that confusing me, too. I'm not familiar with the "private protected" experiment. Was this an alernative to using no keyword?

  Posted by: crazybob on July 19, 2005 at 06:19 PM

• Personally I've grown to hate package private stuff because of the huge amount of careless usage it's gotten throughout the javax.swing.text package. A huge amount of stuff is left package private simply because it's too much effort to get the addition to the public API approved - never mind the fact that the method is essential to being able to work with the API. There are even cases of documented functions (ie: in how to work with this package documentation) being left package private.

  So make sure you don't leave things package private by default, consider each and every scoping decision very carefully.

  Posted by: ajsutton on July 19, 2005 at 06:32 PM

• I agree with ajsutton; using package-private methods and instance variables should be a last resort, not a best practice. Code that uses this technique is often difficult to read and understand. It encourages tight coupling among classes within a package, and discourages solidly thought out APIs with clear extension points.
  I also strongly disagree with the idea that putting the "public API in one package and the private implementation in another" is a bad practice; it's a good one, and a great one for library developers, especially if you package the public and private APIs in different JARs, so the private API can be removed from the classpath when compiling.
  My bottom line when designing code: everything should be scoped as tightly as possible; details should be exposed as public, protected, and yes, even package-private APIs only when there is a need to do so, and when the API has been considered. This isn't easy, but good programming isn't supposed to be.

  Posted by: adamwiner on July 19, 2005 at 07:44 PM

• FYI "private protected" was like protected, except that is only gave access to subclasses, not other classes in the same package. It seemed like a good idea to me, while it lasted.

  Posted by: herbyderby on July 20, 2005 at 12:11 AM

• I think one of the fundamental problems that some coders have with scoping declarations actually stems from poor planning of their package hierarchy. In fact, conceptualizing a package system as a hierarchy seems like the start of the problem, because "child" packages have the same relationship to their "parent" packages that any other package has -- in other words, it's not a hierarchy in anything more than name. Classes in com.example.ui.util don't have any special privileged access to classes in com.example.ui that any other class in any other package might have. But because the package naming conventions look like a hierarchy, it can be tempting to just start creating a hierarchy of folders/packages and start throwing classes into them based on where they "feel" like they belong.

  Here's a better way of looking at it (I think this is from a book by Bob Martin): A package should be the fundamental element of distribution. If you make a change to even just one interface or implementation in a package, you should redistribute that entire package to any client of that package. For that reason, you should design your packages so that implementation changes would only affect other classes in that package. (Obviously an API change is a horse of a different color altogether, since you could have any number of clients using that API.)

  But the trap that many fall into is "categorizing" their code with packages. Let's use a somewhat dated, but traditional, Struts architecture as an example, using the separation of forms, business objects, and DTOs. By putting all of their Struts forms in one package, putting all their business objects into one package, putting all their DTOs in one package, etc. This destroys the package as a unit of distribution. Because now, if you make a change to some implementation of business logic, that might change the DTO that you're using (you need to add or change a property), which means that now the form needs to change to set a different property as well. That's at least three packages that had to be modified because of one tiny implementation detail. However, if the package hierarchy were organized such that all of the forms, DTOs, and business objects for a specific set of functionality are all in the same package, then that change would have only affected that one package. Of course, code that is shared across different sets of functionality, such as utilities, must be extracted out to a package appropriate for their functionality as well.

  Once a team is using a more sensible package design, they will find package-private members incredibly useful. The package-private method will essentially be as flexible as a public method would be in the other type of package "hierarchy", because just about any code that would need to use that method will be within that same package. If the developer suddenly finds him or herself needing to call one of those package-private methods from outside the package, it will alert the developer that s/he needs to think about the API and whether it accommodates the needs of client code. The developer might not expose that particular package-private method, but instead refactor by adding a method to or otherwise changing the public API which then delegates to the package-private implementations to do the work.

  Sorry if this came out as rambling.

  Posted by: erikprice on July 20, 2005 at 05:54 AM

• ajsutton, Ha ha. I suggested making private classes and members package-private, not public ones! ;)

  Posted by: crazybob on July 20, 2005 at 07:53 AM

• 
  adamwiner, First, if a programmer doesn't understand scoping enough to scope everything tightly and not couple their classes, package-private scoping won't be their problem, making too much stuff public will be.

  Second, you can use scoping to control your client's access, but it doesn't make much sense to try to use it to protect you from yourself. You can always change the scope afterall. You should know how your classes are going to talk to each other and what parts you want to expose before you decide on a scope.

  Third, speaking of class coupling, I notice a lot of people clutter their code with inner classes when a package-private class would more cleanly suffice (and maybe reduce the danger of coupling the classes too tightly which should make you happy).

  Fourth, back to separating your public API and implementation into multiple packages, what if the public portions of String and StringBuffer were in one package and the actual implementations were in another. How would you secure this?

  Last, you refer to package-private as if it's exposed to the client. It's not. Package-private classes and members don't even show up in the Javadocs. They might as well be private except for the fact that "private" keywords would clutter your code and get in the way of testing. If you call methods or access fields between classes when you shouldn't, shame on you, but you have bigger problems that aren't caused by package-private.

  Posted by: crazybob on July 20, 2005 at 08:16 AM

• erikprice, Excellent advice.

  Posted by: crazybob on July 20, 2005 at 08:25 AM

• It's actually very hard to find good references to how packages should be organised, especially refactoring guidelines. Are there any good reference sources available on this on the Web? I'm curious as to the recommended separation of interfaces and implementations, and the overhead, if any, of factory classes.

  Posted by: eeiabdy on July 20, 2005 at 05:32 PM

• "Scopes are one of the first things we learn as Java developers."

  This is not completly true, we only learn the half of how scopes work, we learn that a package is unique by its name but that is not true, there are unique by its name within a ClassLoader.

  Just try to load two classes with the same package declaration (for example org.somepackage ) by different classloaders, then let one class call a package private method on the other class, this will result in a IllegalAccessException.

  Posted by: carmello on July 21, 2005 at 07:18 AM

• I personally find it difficult not to be able to handle packages as semantic entitiy within my code. I agree with erikprice that putting everything into one package allows for tightest access. But I would prefer having the possibility of structuring packages and not having them too large. I personally admit that I grew for that reason to one of the blamed "inner class" guys which allows me even restricting access within the package. What I would like to have is some "friends" mechanism which allows to declare other packages within some "to-be-defined" scope as it has been announced by kgh in these blogs somewhat earlier.

  Posted by: rbirenheide on July 21, 2005 at 07:43 AM

• eeiabdy, I'm not sure to tell you the truth, and I'm not sure how much advice I could give you. That's like asking, "how many and which methods should be in a class?" The best pointers I can give: 1) Try to think of packages as mini sub projects within your project. 2) Try to avoid circular dependencies between packages. This helps refactoring a lot. Our build actually checks for it. 3) Resist the temptation to organize packages by class type, i.e don't have one package for DAOs, one package for Struts actions, one package for DTOs. Have a package for the web view, a package for the web services "view," and a third package that contains the view logic common between these two.

  Posted by: crazybob on July 21, 2005 at 07:53 AM

• carmello, You're absolutely correct. I should have been more specific, but I wasn't sure what to call public/private/package-private/protected scopes. Speaking of the package instance per class loader issue, I've run into this with tools like BeanShell and cglib. Tool authors have a choice, either create a new class loader and sacrifice package-private access (the "right way"), or turn of accessibility checking and reflectivley inject your classes directly into an existing class loader (ewwww).

  Posted by: crazybob on July 21, 2005 at 07:56 AM

• Sun uses the "undocumented package" in some cases, instead of package-scoped classes (i.e., the sun.* packages).

  This technique also allows access to the internals for developers who really need that access, while sending a message to everyone else not to go there, because they won't be supported in future versions. Of course, whether having that core of unsupported developers is a good thing is open for debate.

  Posted by: afishionado on July 21, 2005 at 11:13 AM

• afishionado, If you go this route, scoping can't help with security. You have to use SecurityManager to lock down your code.

  Posted by: crazybob on July 21, 2005 at 11:27 AM

This work is licensed under a Creative Commons License.