Skip to main content

Two problems with generics in Java

Posted by carcassi on April 9, 2010 at 5:21 AM PDT

Since generics were out and I started using them, there were always a few cases in which I couldn't make them do what I wanted. I always thought it was my problem, and that I didn't understand what was going on... Turns out: it's not. There are at least two things that are implemented in a way that break what I thought were very safe expectations.

Inference

Before Generics, you would write something like:

    List list = Collections.EMPTY_LIST;

Adding generics you get a warning:

    List<String> list1 = Collections.EMPTY_LIST;

GenericsGotchas.java:[19,40] [unchecked] unchecked conversion
found   : java.util.List
required: java.util.List<java.lang.String>

The problem is that we have a raw type on the right, and a generic type on the other. We could suppress the warning every single time, but there is a better option: using a generic method and let the compiler infer and match the types

        List<String> list2 = Collections.emptyList();

This compiles without a problem. But this does not:

        List<String> list3 = new ArrayList<String>(Collections.emptyList());

GenericsGotchas.java:[21,29] cannot find symbol
symbol  : constructor ArrayList(java.util.List<java.lang.Object>)
location: class java.util.ArrayList<java.lang.String

You would expect that it would, right? But it doesn't and you get an ERROR, not a warning. If you really want to put in one line, you can use (as suggest by the comments) the following

        List<String> list3 = new ArrayList<String>(Collections.<String>emptyList());

The problem here is that the language is behaving in a way I was not expecting. My expectation is that whetever you have on the right side can be used directly as a parameter to a function. Generics break this expectation. That the follwoing refactoring:

    a = expression;
    anotherExpression(a);
->
    anotherExpression(expression);

is always valid. Unfortunately, it's not true in this corner case. So, when I am refactoring my code, or decorating an expression, I have now one more thing to remember and to pay attention to.

Casting generics type

Casting of generic types is inherently unsafe because of erasure. We all know that. But sometimes we have to do it, we expect the compiler to give us a warning and we explicitly ignore it together with a comment explaining why. There is one case in which this does not quite work. And it comes up if you are using class tokens, so I'll use those as example. A simple class object:

    Class<List> class1 = List.class;

For a List of String you get:

    Class<List<String>> class2 = List.class;
GenericsGotchas.java:[24,41] incompatible types
found   : java.lang.Class<java.util.List>
required: java.lang.Class<java.util.List<java.lang.String>>

And this is an error, rightly so. So you may expect to simply add a cast, as you normally do in these cases for non generic types:

    Class<List<String>> class3 = (Class<List<String>>) List.class;
GenericsGotchas.java:[25,63] inconvertible types
found   : java.lang.Class<java.util.List>
required: java.lang.Class<java.util.List<java.lang.String>>

It does not work either. I understand that the compiler can't make the test, but the problem here is that I can't tell it that it's ok to make the conversion: it's an error, not a warning! So, how do you do it?

    @SuppressWarnings("unchecked")
    Class<List<String>> class4 = (Class<List<String>>) (Class) List.class;

You cast it first to an Class, "forgetting" all of the parameter type, then you are free to cast it to whatever you want. You get the warning, which you can suppress. Yes, this is extermely poor style, but at least it's concise. The expectation is that I can cast one type into the other, and I get a warning if the cast can in theory be done but can't guaranteed at runtime. If your type parameter is generic (a List of a Map of something), this expectation is not met.

Conclusion

I am surprised that I haven't been able to find them discussed anywhere, so I felt compelled to write about it. At least you know how to work around them... I think that all the problems I have been having with generics are one way or another caused by some combination of these two. The rest (erasure, bounds, ...) they all make sense to me. These do not, they feel like bugs... Maybe there is something I am missing, but I don't see why these behaviours are needed.

 

Related Topics >>

Comments

Problem #1 is fixed in Java7

You might be happy to find that the first problem you reported is actually fixed in Java7. That being said, I still really hate the Generics implementation in Java. It is extremely extremely broken. I would much rather see Java7 fix Generics (add type inference, find ways to clean up the language) than add any new features like Closures. The language is becoming a freaking mess. I don't have a problem with people advocating adding closures into the language so long as they fix what is broken before adding more stuff into the box.

The point of type inference

The point of type inference is to use information from the context. Type inference in Java comes in the form of generic methods, such as Collections.emptyList(). It infers its return type from the context. Unfortunately, the argument position of 'new ArrayList(..)' basically has no context available, so the type of Collections.emptyList() in that location is List, and that's no good for an ArrayList. You can use the EMPTY_LIST field instead:
List list3 = new ArrayList(Collections.EMPTY_LIST);
So your statement should be corrected to "My expectation is that whatever you have on the right side can be used directly as a parameter to a function. Generic methods break this expectation." Of course generic methods change their type when used in a different context - that's the whole point.

You had convinced me... but there was more... :-/

I got excited when I read this explanation... I got a "Ah!" moment. But after I did some experimentation I don't think it holds. That is: even if you give it the same context it does not work.

    ...
    List&lt;String&gt; strings = testMe(Collections.emptyList());
    ...

    public static List&lt;String&gt; testMe(List&lt;String&gt; test) {
        return new ArrayList&lt;String&gt;(test);
    }

TestGenerics.java:[21,74] inconvertible types
found   : java.util.List&lt;java.lang.Object&gt;
required: java.util.List&lt;java.lang.String&gt;

I'd think that the context of the testMe argument is exactly the same: it requires a List<String>. This gave me the idea to try this:

        List&lt;String&gt; objects1 = Collections.emptyList();
        List&lt;String&gt; objects2 = (List&lt;String&gt;) Collections.emptyList();

TestGenerics.java:[19,68] inconvertible types
found   : java.util.List&lt;java.lang.Object&gt;
required: java.util.List&lt;java.lang.String&gt;

The first compiles, while the second one does not!?! Now I would be _really_ surprised if anybody can give me a good explanation for that... :-)

Java is trying to infer the

Java is trying to infer the return type of Collections.emptyList(). Inference uses information from the arguments to the method - none in the case of emptyList() - and information from the context - none in the case of testMe(Collections.emptyList()). Why none? Because the only context that inference understands is the assignment context. That's why the first line compiles. Putting emptyList() as an argument means it's in the method invocation context; no good. Putting emptyList() behind a cast means it's no longer in the assignment context; no good. As you can see, generic methods (specifically, inference for generic methods) are complicated.
If you would like to extend inference for generic methods to handle other contexts, please write an ECOOP paper then submit a patch to the OpenJDK Compiler project. You will have to handle the difficult fact that a called method (testMe) might itself be a generic method that tries to infer its parameter type from the type of its argument (emptyList()), but the argument type isn't known yet since the argument is a call to a generic method.

First: I _really_ appreciate

First: I _really_ appreciate you spending the time to make these things clear to me!

>Because the only context that inference understands is the assignment context.

Ok, so this comfirms my feeling that the inference was specific to the assignment. Thanks! I am not seeing things then... :-)

>If you would like to extend inference for generic methods to handle other contexts...

Ah, so this _could_ in theory be addressed. Right now I am shooting for more "immediate relief": collecting all of the instances where the expectation is different and have workarounds. I feel a more pressing need for this than for adding a whole new set of features to the compiler, which may have yet another set of side effects... :-/

One more proof how the generics are broken in Java

I think that it was a major mistake to introduce generics with type erasure. Sun should have done this properly, instead we have to fight with these gotchas everyday. :(

Pure Affirmation

I absolutly feel the same on what you are describing here. This is definitly a weakness of the compiler. I use Generics a lot in my own frameworks and the two cases you described happen very often to me. The inference issue that makes up a difference between method parameters and R-values is solvalbe with a generic parameter as you showed explicitly in your text. It forces the actual unneccessary but understandable generic parameter. This is not nice but bearable because the f**king @SuppessWarning is not needed. The other case is worse. Any time where I add the confusing casts and the@SupressWarnings to finally get rid off the warning I am worring about this. And generally where ever necessary casts from raw types are made the warning appears. Since my days when I started casting in the world of C, I was used to the fact that a cast is in the responsibility of the coder itself as long as there is a potential that it could be working at all. Only in the case that the compiler can know at compile time that it can never be working there is an issue the compiler should bring up. Now in the days of Java Generics the compiler forces me to make strange casts for some situations when dealing with Generics and then it forces me again to underline my repsonsibility with an annotation. This is odd in a double sense.

There are more odd things to the generics I experience often. Here is one example for which I do not really understand the compilers complaining:

List<Map<String, ?>> map = new ArrayList<Map<String, String>>();

Type mismatch: cannot convert from ArrayList<Map<String,String>> to List<Map<String,?>>

Can anyone explain it?

Explanation for the type mismatch

If B is a subtype of A, List<B> is not a subtype of List<A>. It is a subtype of List<? extends A>.
So, Map<String, String> is a subtype of Map<String, ?> but ArrayList<Map<String, String>> is not a subtype of List<Map<String, ?>>.
List<? extends Map<String, ?>> map = new ArrayList<Map<String, String>>(); would be alright.

Yes

Yes. This is because in Java, all type constructors are invariant. Let's look at what variance means: If a type constructor T is covariant then if P is a subclass/subinterface of Q, then T

is a subtype of T. A good candidate for this would be java.util.LinkedList. If a type constructor T is contravariant then if P is a superclass/superinterface of Q, then T

is a subtype of T. A good candidate for this would be java.util.Comparator. Unfortunately, Java disallows this kind of thing, specifically variance. There is an exception to this rule: arrays. The reason this rule exists is because you can run into issues with imperative programming constructors (i.e. destructive updates). However, this doesn't mean that the only solution to this is enforced invariance of type constructors. For example, Scala and C# 4.0 both have uncontrolled imperative programming constructs, but allow variance annotations; there is just a limitation on how they are used (I recommend Scala as the example). Some languages also solve this by not having nominal subtyping (i.e. traditional inheritance), but alternative solutions to the benefits that nominal subtyping typically provides. One such example is Haskell, which provides type-classes. Since all Java type constructors are invariant, your two types are not assignable. I hope this helps.

Re: Yes

I understand that a List<String> can't be cast to a List<Object>. I don't have a problem with that. And I do not think the problem is variance per se. I think it's the lack of variance on the raw types. Consider the following:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; @SuppressWarnings(&quot;unchecked&quot;)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ArrayList&lt;String&gt; list = new ArrayList();

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; @SuppressWarnings(&quot;unchecked&quot;)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ArrayList&lt;ArrayList&lt;String&gt;&gt; list2 = new ArrayList();

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; @SuppressWarnings(&quot;unchecked&quot;)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ArrayList&lt;ArrayList&lt;String&gt;&gt; list3 = new ArrayList&lt;ArrayList&gt;();

The first two are fine, the third isn't. And there is no actual variance in the sense that all that it is remains. I am just making things better specified. That is, when I am casting List<List> to List<List<String>> in my mind I am not "changing type" like I would when I convert from List<Obect> to List<String>. I am simply adding information. Now, you may say: you are thinking it wrong, it's just a subcase of variance, casting from List<List> to List<List<String>> is like casting from List<String> to List<Object> (I am changing the type parameter). I understand that logic, but that's my point: that logic should be something else. Because it gets in the way... :-/ It's part of what makes generics hard to use... and maybe it can be fixed.

Forgot to mention

There is an additional problem with the @SuppressWarnings("unchecked") which is needed to get a warning free compiler result. With the annotation on a method all other potential generic type issues of real interest are hidden now. This is now a real problem which is the responsibility of the compiler and not of the coder. What about that?

Re: Forgot to mention

What I do is always put the suppress warning right in front of a variable decleration, almost never on the method. Like this:

&nbsp;&nbsp;&nbsp; ...&nbsp;&nbsp;&nbsp; 
    @SuppressWarnings(&quot;unchecked&quot;)
&nbsp;&nbsp;&nbsp; List&lt;String&gt; list2 = new ArrayList();
    ... 

And sometimes I add extra variable declarations just to have somewhere to put the annotation. I hear that in Java 7 we can put annotations anywhere, even on castings, so you may be able to write something like:

&nbsp;&nbsp;&nbsp; ...&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; List&lt;String&gt; list2 = (@SuppressWarnings(&quot;unchecked&quot;) List&lt;String&gt;) new ArrayList();
    ... 

I haven't tried yet, so I have no idea whether it's actually legal or if it works... :-/

This is not a problem with

This is not a problem with generics, but a problem with Java in general. It doesn't do any type inference requiring you to type annotate.

Misleading.....

1) correct way to use generics and collection is List<String> list2 = Collections.<String>emptyList(); Please google it before making a big deal. 2) There are way to achieve the right class type. For example look at Spring 3 GenericTypeResolver and how they resolve the type. carcassi you just have to look for solution. Yes there is some problem, but with any new introduction of construct there is little time it takes to mature it. Regards, Pratik

Re: Misleading

Hi Patrik,

1 does not help me. I tried to clarify that in the text and in the previous comment. 2. Sorry, I am not familiar with the GenericTypeResolver, if I understand correctly it allows to do introspection on generic types... but this is not what I am trying to do (I can do all the introspection that I want fine). I am simply trying to cast A<B> to A<B<C>>, and I can't find any valid way except going through A in the middle... I don't see any example of that in their source either (in fact, it's all mainly using raw types... or <?>).

Re: Misleading

Hello Gabriele,
Analogy: You don't need to be familiar with cows to know that going to the moon was impossible before rockets were invented.
So... going to the moon = what you would like Java to do but it doesn't, rockets = an event that did not happen yet in Java, cow = pparikh/pratik/patrik lied and what he said sounded like: Are you familiar with cows ? cause I know they can fly to the moon.
anyway... sorry for seing this nice post a year later.
Pptarik... you really are an inquisitor :). I'm sure you own a CallCenter or two :P and that's why. Just kidding...
It is impossible for the thing that you speak of to determine generic type arguments at runtime :)))). I am fully aware of the confusion that you made (preety lame one.. btw).
Generic Type Arguments do not exist at runtime in Java because Java has no generics in fact.
List<Integer>.class == List<Double>.class == .. etc
Mind teaser:
Do this in Java:
ArrayList<Integer> firstReference = new ArrayList<Integer>();
firstReference.add(23);
ArrayList<Double> secondReference = (ArrayList<Double>)(object)firstReference;
secondReference.add(3.14159);
// quiz: why does the next line cause a typecast exception ?
double x = secondReference.get(0);
// in case you understand (for a change) why ? wha happened ? :) read the next comment line:
// I guess it turns out it is You who should think, live, breath, be curious, hungry for knowledge and humble, before you make a big deal, Prakash :), Please !

First problem

In the first problem, you can enforce the Collections.emptyList() return type by: List<String> list3 = new ArrayList<String>(Collections.<String>emptyList());

Does not help... :-/

Thanks for pointing that out (included in text) but it does not help... my problem is not that I can't make a one liner. It's that I can't cut the expression to the right and simply paste it in another expression. That's the thing that bothers me, that I can't always make that refactoring. It's one more thing I have to pay attention to and gets in the way. Not that anybody will care in the same way... ;-)