Skip to main content

Request for Deprecation of Raw Types and Reification of Generic Types

Posted by jcnarahari on June 2, 2010 at 5:47 AM PDT




by
Jaisimha Narahari
Date: 02/June/2010

 

    In the month of June 2009, I did an initial post in the openjdk.java.net-jdk7-dev

    mailing list with a request for what I feel are two very important changes

    required in the Java Platform.This resulted in a thread of discussions on

    the Reification of Generic Types, but lead to no conclusions.Truly, the

    time is really very much ripe for Reification of Generic Types.The Java

    Language and Platform Implementers should not really ignore it much longer.


   
Contents


    1. Two Most Important Changes Required in Java

      1.1 The Requests for Change

      1.2 Reason for the Requests
          
    2. Why Reification of Generics was not taken up

      2.1 Design Constraints in Section 2.5 of JSR14

    3. Hindsight Wisdom: Solutions that could have been

      3.1 Dual APIs: Generic and Non Generic Versions

      3.2 Why the Dual API solution was not adopted

      3.3 Generics right from Java 1.2

    4. The Finally Adopted Solution in Java 5.0

    5. Why Resurrect the Controversy

      5.1 "NO-WIN" situation with No Reification

      5.2 Reversal of "NO-WIN" situation with Reification

    6. Conclusion

      6.1 Realizing Reification of Generics

      6.2 Work involved in the Implementation

      6.3 Sign Off Plea


    1. Two Most Important Changes Required in Java


        In the month of June 2009, I did an initial post in the

        openjdk.java.net-jdk7-dev mailing list -June 2009

        with a request for what I feel are two very important changes required in the Java
        Platform. This resulted in a thread of discussions on the Reification of Generic
        Types, but lead to no conclusions.Truly, the time is really very much ripe
        for Reification of Generic Types.The Java Language and Platform Implementers
        should not really ignore it much longer.

    1.1 The Requests for Change

        My main request for changes/inclusion in Java 7 were as follows:

        From Java Version 7.0, going forward:



                    Remove the provision for Backward Compatibility(See Section 2.1 below).

                    In other words:



                          1. "Deprecate" Raw Types


   
                           2. Adopt "Reification" in the place of "Erasure" for Generic Types.



                              The consequence of this will render the "contrived" and "constricting" rules

                              for the usage of Generic Types in Java 5.0 (given in Section 1.2 below)

                              unnecessary.



                              As a result, all types provided by the language will behave uniformly, most

                              especially without violating OOP principles, in what is essentially an OOP

                              Language.
   


        Legacy code can still make use of JDK up to Java version 6.0 for
        inter-operability, with support removed from Java 7.0.

    1.2 Reason for the Requests

        The three contrived rules mentioned above introduced as restrictions in the usage
        of Generics in Java 5.0, as a consequence of Erasure, are as follows:

         (A) For the sake of determining Type Hierarchy of Generic Types, the Type
             Parameters are not to be considered. That is,code such as the line below is
             rejected by the compiler:

                List<Object> lo = new ArrayList<Integer>

             even as pure OOPS principles allow an Integer to be recognized as an Object.

             That is, in Java 5.0 Generics, List<Object>and List<Integer> are
             treated as totally unrelated types by the above rule, not only violating the
             purity of OOPS but also sacrificing type system robustness.

         (B) Any kind of Array Creation involving Generic Types,such as List<E> [],
             List<String> [], E[], etc also get forbidden, on account of the fact
             that Arrays are "Reified" types, implying that they carry over their full
             type info into the Runtime, and are therefore aware of the type of their
             contained elements at runtime, in direct contrast to Generic Types, which,
             because of Erasure,discard the knowledge of the type of their contained
             elements (i.e.,Type Parameters) and are therefore  "non-reifiable" types.

             [However, the expression E[] can be used, (without "new" used for its
             creation), to specify an array of ANY REGULAR TYPE, not Generic Type. 
  
             This syntax therefore is not really one to be associated with Generic Types,
             even as E[] in itself represents a Generic Type whose Formal Type Parameter
             'E' can only have Regular, Non-Generic Types as its Actual Type Parameter
             instantiations].

         (C) Exceptions cannot be Parameterized Types.

        See Referenced Java Text

    2. Why Reification of Generics was not taken up

        As it became evident to me in the course of the thread of discussions at the

        openjdk.java.net-jdk7-dev mailing list - June 2009,

        as well as via the contents of a couple of blogs that I was able to home in on:

        (Thanks to Neal Gafter, Sun MicroSystems for the blogs)

        Neal Gafter's Blog/2004/09
                                            
                      and
 
        Neal Gafter's Blog/2006/11,

        the summary of reasons of why my requests #1 and #2 in Section 1.1 above could not
        be immediately accepted is as given in Section 2.1 below:   


    2.1 Design Constraints in Section 2.5 of JSR14

        (a) In JSR 14, which introduced Generics to Java, there were special

                Design Constraints in Section 2.5

            that sought to provide support for "Raw Types" in the Java Collections API.

            Reason One for these constraints was Backward Compatibility.

            This meant that existing legacy code that made use of the Java Collections API
            without specifying Type Parameters (Type Parameters are part of the newly
            introduced Generics version of the API) could continue to do so.

            That is, legacy code could continue to treat the Collection Classes as regular
            types rather than as Generic Types. The classes so used are referred to as
            "Raw Types".

            Reason Two for the above constraints was Migration Compatibility.

            Backward Compatibility as a facility also allowed Third Party API Library
            providers who directly used the old Java API to continue to do so, without
            any changes forced on the Third Party API Library product that they delivered
            to the market.

            By extension, this made it also easy for other Third Party API Library
            providers who in turn used the API provided by the above "first line" of Third
            party API suppliers: No changes were forced, also, to their "second line"
            product that they delivered to the market.

            For any chain of such Third Party API providers that existed, delivering in
            effect, an "API Stack" to the market, and also for other product vendors who
            used APIs from this "stack" at any level, Backward Compatibility ensured that
            the introduction of Java Generics did not force any such suppliers of products
            up the chain to change their products to accommodate Generics.

            Migration Compatibility is the provision that ensured that those users who
            wanted to migrate to the new Java APIs with Generics could do so, by using
            the Generic classes with their Type Parameters specified, rather than as
            "Raw Types" (without specifying Type Parameters, as used by the chain of
            product suppliers mentioned above).

            This, seemingly, is two problems solved with one stroke. The provision of
            "dual" usage of one and the same Java API - one usage as "Raw Types", and the
            other usage as "Generic Types", for two sets of users with differing
            requirements.

            The reasons given above look perfect, as a case AGAINST
            "Deprecating Raw Types",my request #1 cited in Section 1.1.
          
            Because NOT DEPRECATING Raw Types allows old users to continue as before,
            without facing "disruptive" changes to their product, even as migrating users
            who want to use the new features could also do so.

            But alas, there are problems with this approach, as evidenced by the (by now
            well-documented), contrived rules in Generics Usage in Java 5.0., as given in
            Section 1.2 above. These rules practically restrict the proper usage of
            Generics by the migrating users who want to take advantage of the new
            features.

        (b) However, the issue could not be considered closed. Reification could not
            simply be abandoned, with Generic Types receiving "slipshod" treatment
            forever.

            So it is that one finds "Reification of Generics" under:

            Bug ID 5098163 in Sun's Bug database.

            (Thanks to Alex Buckley, Sun Microsystems for the URL).

            Rather than treat this as a Bug, the entry labels it "RFE", standing for
            "Request for Enhancement".                           

            Against "Evaluation", it is marked as being "Under discussion as a potential
            feature in Java SE 7."

            And against "Priority" it is marked "4-Low".

            I could not find any other clause in this Bug database entry that indicated
            further action taken. Nor could I find any "current status" clause, maybe on
            account of the assigned low priority, as seen above.            
  
    3. Hindsight Wisdom: Solutions that "could have been"

    3.1 Dual APIs: Generic and Non Generic Versions

        In hindsight, one alternative solution that could have been adopted at the time
        of Java 5.0 release, with Generics, to overcome these restrictions, would have
        been to provide two versions of the all Java APIs into which Generics was
        introduced :

        One with Generics, and one without (the old API as it existed before introducing
        Generics).

        The APIs with Generics would then not have been required to support their usage as
        "Raw Types", and as a result the classes in those new APIs would have been
        "Reified" Types rather than types, each of whose type info was erased at runtime
        because of "Erasure".

        Java 5.0 would then have not been required to specify any restrictions in the
        usage of Generics for the migrating users, which is of course the way things
        should really be.

        Those users who were dependent on the old API could simply continue to use the
        "undisturbed" old API, without any disruptions.
  
        While this solution looks more elegant, it was anyway NOT ADOPTED.

        It has the drawback that Java would end up possessing a pair of APIs for
        performing the same function, and that might be quite a number of extra APIs.
         
        But if this solution had been adopted, the older version of each such API could
        have been deprecated in course of time, as and when all of the old non-migrating
        users could safely be deemed to have also migrated to the new API.

    3.2 Why the Dual API solution was not adopted

        The logic behind NOT ADOPTING this solution [of two versions for the same API],
        as I gathered, is as follows:
           
          - Doing so would result in a versioning nightmare of APIs, as one progressed
            up the product/API chain mentioned in Section 2.1 (a) above, starting with
            two versions in the Java API, and geometrically multiplying up the
           "API stack", provided by Third Party vendors.

          - This would happen because of a permutation of API combinations possible to
            get delivered by vendors of API products up the stack, for other vendors
            dependent on their APIs, by mixing and matching Generic plus Non-Generic
            versions of their individual API Libraries.

        I am not sure if a complete survey was conducted with vendors of such nature to
        arrive at the above conclusions, but on the face of it the logic above looks a
        bit stretched, and possibly, just possibly, may not reflect ground realities
        accurately.
 
        The very concept of stack upon stack of Third Party API Libraries existing in the
        market, a higher one in the stack dependent on a lower one, seems a bit
        far-fetched.

        Even granting that such stacks do exist, the instances of such situations are
        bound to be more an exception rather than the rule.

        In any case, Java 5.0 with Generics got released with one and the same API for
        each API that provided Generic Types, allowing the use of the Generic Classes as
        "Raw Types" as well as use of the same classes as Generic Types.

        So it is a moot point now whether a one-API solution or a dual-API solution would
        have been the better one.         

        Therefore
        the Design Constraints in Section 2.5 of JSR 14

        used the need for Backward Compatibility and Migration Compatibility as detailed
        in Section 2.1 (a) above as their basis, and made support for "Raw Types"
        mandatory as a result.

    3.3 Generics right from Java 1.2

        Also in hindsight, this whole problem of Generics and Backward and Migration
        Compatibility, could have been completely avoided if at the time of the release of
        the Collection classes in Java, that occurred with Java 1.2, the API had been
        released only with Generics rather than without, to start with.

        After all, if we are talking of Lists, Queues, etc, it makes a lot of sense to be
        able to distinguish between a List/Queue of "foo"s and a List/Queue of "bar"s.
        But this is in hindsight, and again, a moot point.

        By the time of release of Java 5.0, every piece of code written using these
        Non-Generic Collection API classes had become "legacy" code that used the classes
        as "Raw Types", code that did not care if a List was comprised of only fairies,
        only ogres, or a mixture of both. This is definitely an unfortunate turn of events,
        as it has turned out.

        ALL OF THE FOREGOING POINTS THAT I HAVE MADE IS ONLY TO PUT THINGS IN PERSPECTIVE,
        AND TO ESTABLISH THE CONTEXT FOR THIS BLOG.

        THEY MUST NOT BE MISCONSTRUED AS A FAULT FINDING MISSION AGAINST SUN MICROSYSTEMS
        (NOW PART OF ORACLE) OR ANY OTHER STAKEHOLDER.
           
    4. The Finally Adopted Solution in Java 5.0

        To the credit of the people involved in the development and release of Java 5.0
        Generics, as it came to light in the thread of discussions at the

        openjdk.java.net-jdk7-dev mailing list -June 2009,

        the experts at Sun Microsystems had tried to find the best-fit solution for
        simultaneously satisfying the needs of both the migrating users (those wanting to
        use Generics) and non-migrating users (those who wanted to stick to the old API
        usage) by considering as many workarounds as possible and finally settled for what
        actually got released as the Java 5.0 Generics APIs.

        The Design Constraints in Section 2.5 of JSR 14

        had to be satisfied, because Backward Compatibility could not be sacrificed,
        overnight breaking legacy code, even if the reason behind not releasing a
        separate set of Generic-only APIs,for "proper" Migration Compatibility,
        that of a versioning nightmare in a stack of interdependent APIs, is questionable.

        It just so happens that, FUNDAMENTALLY, the needs of the non-migrating legacy
        code users (read usage of Generic Types as "Raw Types") and the needs of the
        migrating users (which is ideally the usage of Generic Types without twists and
        quirks, achievable only without their simultaneous usage as "Raw Types") are
        MUTUALLY EXCLUSIVE requirements, even as only a SINGLE API for a single function
        became  available.

        If you satisfy the one, you cannot satisfy the other, with just one and the same
        API.

        That is, there can either be Erasure or there can be Reification, but not both
        together for Generic Classes of one and the same API.

        As result of the mutual exclusivity involved, the bias that became inevitable in
        arriving at a solution had to tilt towards the legacy users. Because, as mentioned
        above, support for millions of lines of legacy code cannot be dumped overnight.

        So Erasure won the day, Reification had to be discarded, but at very good cost for
        the migrating users.

        To be fair, Java 5.0 also provides a "patch up" API in the classes provided by
        java.util.Collections.checkedXXX classes, which try to reverse the effects of
        Erasure by carrying over the type info of Type Parameters of Generic Types also
        into the runtime, by separately providing reflection (".class") types of the
        Type Parameters as part of the checkedXXX classes.

        While this provision re-ensures type safety (to compensate for the loss of type
        safety due to erasure), there is no compensation concerning the three "makeshift"
        rules of Section 1.2 above that end up gravely restricting Java from having a very
        robust, elegant Type System, wherein Parameterized Types could enjoy the same
        status as any other type, without getting quirks and twists in their usage
        imposed upon them.

        So, in the status quo, all decisions concerning Java Generics as a feature have
        been explained to satisfaction as seen above, specifically the controversial one
        of choosing Erasure over Reification for Generic Types.

        Having got all the reasons outlined above, as the reasons for why my requests #1
        and #2 in Section 1.1 could not be immediately accepted, from the thread of
        discussions ensuing from my initial post at

        openjdk.java.net-jdk7-dev mailing list -June 2009,

        and the contents of these blogs:

        Neal Gafter's Blog/2004/09
                                            
                      and
 
        Neal Gafter's Blog/2006/11,

        the hopes of getting the requests incorporated in Java 7 had to be abandoned.

    5. Why Resurrect the Controversy

        So why am I re-posting the same requests in this java.net forum?
   
        The reason is:

           Changed circumstances of today, vis a vis the circumstance at the time of
           JSR 14 development and the release of Generics with Java 5.0, as well as
           a couple of more arguments, as given below.

As outlined in Section 3 above, the evolution of the Java Language has
        undergone two possibly avoidable hitches in its evolution, concerning
        Generics:

          (a). Not introducing Collections API only with Generics in the Java 1.2
               release

  (b). Not opting for two mutually exclusive API Libraries with Java 5.0
               (one with Generics, and one without), providing for Backward
               Compatibility and Migration Compatibility, but ignoring any
               possibility of a versioning nightmare of APIs in presumed API
               stacks in the market, treating this last problem as not affecting
               too many users.

Now, the consequence of the second "oversight" above is currently underway in
        Java's evolution, in the form of having to live with Erasure, with Reification
        of Generic Types pushed to the background on low priority.
  
Also, opting for two different APIs NOW is out of the question (as the stage
        for that is now past), and having only a single API for one set of features as
        a result makes one of the two (MUTUALLY EXCLUSIVE by nature) options - Erasure
        and Reification - edge out the other.

        In the event, Erasure has edged out Reification.

Erasure was certainly the concept to opt for, at the time of Java 5.0 release,
        also as seen above.

But that was in 2004, the year of release of Java 5.0.Now it is 2010, and
        nearly SIX FULL YEARS have elapsed, with Java 7 release expected in 2010.

Since the idea behind adopting Erasure was Backward Compatibility (not
        breaking legacy code) and Migration Compatibility (allowing users to migrate
        to Generics over a time period), and since in these days of Internet Time,
        SIX YEARS is pretty considerable, the time to re-assess the need to handhold
        legacy code users has arrived.

If Erasure had been a pretty harmless concept, Reification could be postponed
        indefinitely, without greatly affecting the migrating users.

Alas, the reality is that Erasure continues to play havoc with the Type System
        of the Language.

It has forced severe restrictions in the usage of Generic Types, as seen in
        Section 1.2, even as the whole idea of introducing Generics in the first place
        was Type System Robustness and its attendant advantages.


        So, Reification must now be adopted, since all users have now been given

       enough time to migrate.


       Legacy code must not any longer enjoy support in future versions of Java.

       And the way to ensure this is to DEPRECATE RAW TYPES.


Since Reification and Erasure are MUTUALLY EXCLUSIVE, introducing Reification
        will mean getting rid of Erasure. This will mean that Raw Types cannot be
        supported any more.

In other words, the time has really come for the two requests in Section 1.1
        to get incorporated into Java, thereby at one stroke getting rid of both the
        "mishaps" that the Java language has been subjected to in its evolution.

So I re-emphasize the issue by reproducing those two requests of Section 1.1
        here again:



                        Remove the provision for Backward Compatibility.(See Section 2.1 above).

                        In other words:



                            1. "Deprecate" Raw Types.



                            2. Adopt "Reification" in the place of "Erasure" for Generic Types.



                               The consequence of this will render the "contrived" and "constricting" rules for

                               the usage of Generic Types in Java 5.0 (given in Section 1.2 above)

                               unnecessary.



                               As a result, all types provided by the language will behave uniformly, most

                               especially without violating OOP principles, in what is essentially an OOP Language.
                   


    5.1 "NO-WIN" situation with No Reification

        If the above two requirements are put on hold, consider what Java users live
        with:

          (a)  Legacy Code users continue to prefer NOT to distinguish a List of Elves
               from a List of Goblins, which is of course, very, very bad programming
               practice, in an era in which Best Practices are sought to be adopted in
               most endeavors in the industry.

               In cases where Legacy Code has been careful to take care of such
               distinctions, it could have been achieved only by burdening the
               programmer, who must take care to code diligently and make updates in
               code maintenance even more diligently.

       Features added to the Java Language, JSE, and JEE in their respective
               evolutions have always tried to make the programmer's task easier, not
               tougher. Where possible, Java has tried to eliminate scope for human
               error.

               So Java should maintain this tradition and not lose it.

               Not breaking Legacy Code is really a NO-WIN situation for Legacy Code
               users, for the reason that it fails to bring them in line with better
               programming practices. They are forced to count on human infallibility
               to achieve robust programming.          

          (b)  Migrating users get Generics, but with the problems outlined. Type
               Parameters of Generics are really full-fledged Types in their own
               right.

               What is supposed to provide Type System Robustness, in fact what has
               been introduced for that very purpose, is made to lose its
               "raison d'être" immediately, upon its very release.

       Not being able to use the full type information for determining class
               hierarchy in an OOP Language, and not being able to create the most
               fundamental secondary type of them all, the Array, of Generic Types,
               is, without any doubt, NO-WIN situation also for the migrating users.
  
        In CONTINUING TO TRY TO SIMULTANEOUSLY MEET THE NEEDS OF BOTH THE LEGACY USERS
        AND THE MIGRATING USERS with one and the same API for one set of features,
        unfortunately there is a NO_WIN/NO_WIN situation for either category of user,
        as seen above.

    5.2 Reversal of "NO-WIN" situation with Reification

        If the two requests for change given above are incorporated, the situation is
        directly reversed:

          (a) Legacy Code users, even if their code is broken, adopt practices that are
              appropriate, do not depend on programmers to take care of a basic fault
              in Language Evolution that has unfortunately crept in, and therefore face
              a WIN situation. 

          (b) Migrating Users get exactly what they should expect to get: Generic Types,
              after all, are meant to make life easier, and by behaving like any other
              regular type, without "constricting, strange" rules constraining their
              usage, they WILL make life  easier, with Reification in place. A total WIN
              situation for Migrating users.

        In honoring the two requests for change, at this APT JUNCTURE in Java's
        Evolution, a NO-WIN/NO-WIN situation for either category of user can be
        converted to a WIN/WIN situation.

    6. Conclusion

        Pragmatically, since even at this juncture, the Deprecation of Raw Types
        cannot be done also "overnight", Sun Microsystems must obviously take steps
        to give appropriate prior warning about the deprecation to all users, using
        the usual channels it has used earlier for same purpose.

    6.1 Realizing Reification of Generics

        THE CONSTRAINTS (C1/C2/C4) IN SECTION 2.5 OF JSR 14

        MUST NOW BE TREATED AS NO LONGER RELEVANT.

        THE PRIORITY IN Bug ID 5098163 in Sun's Bug database

        FOR REIFICATION OF GENERICS MUST BE MADE "HIGH", from "LOW". 

        EVEN IF THIS CANNOT BE DONE IN JAVA 7, JAVA 8 MIGHT BE A TAD LATE IN THE DAY
        TO REMOVE THESE QUIRKS IN JAVA, W.R.T. GENERICS.

        HENCE AN INTERMEDIATE RELEASE, SAY JAVA 7.5, ONLY FOR SETTING RIGHT THE
        CONTROVERSIAL GENERICS PROBLEM, BY HONORING THE TWO SPECIFIED REQUESTS FOR
        CHANGE WOULD CERTAINLY BE A STEP IN THE RIGHT DIRECTION.

    6.2 Work involved in the Implementation

        THE CHANGES INVOLVED IN JAVAC, THE JVM AND THE API IMPLEMENTATIONS CANNOT BE
        REALLY MONUMENTAL, ONCE RAW TYPES ARE DEPRECATED. (See Below).

        A few of the major code changes that would be required would be as follows:
 
        For Generic Types, Actual Type Parameters (Instantiated Formal Type
        Parameters) must get treated as full-fledged Types in their own right.

        All the code for the treatment of Types already exists in the code base, for
        the Base Types of Generic Types and for Non-Generic Types. Actual Type
        Parameters must now get similar treatment in Type Checks, Generic Type
        Inheritance, etc.

        Reification replacing Erasure will of course mean that the full type info of
        a Generic Type (the Base Type plus all the Type Parameters, nested to any
        level) will now get carried over to the runtime, and not exist only at compile
        time.

        This will then mean that the Actual Type Parameters of a Parameterized Type
        will also be considered in determining Type Hierarchy, that arrays of Generic
        Types can now get created, and if necessary, Exceptions could also be
        Parameterized Types.

        All three currently existing major restrictions in the usage of Generics will
        disappear, as they rightly must.
  
        Generic Type Inheritance will additionally require that the framed rules for it
        be followed, this time with the Actual Type Parameters also used in
        determining Type Hierarchy.

        Since Formal Type Parameters of Generic Types can also be Nested, all
        new/modified code that has these types in context will be required  to make
        use of Recursion to handle Type Identity at each level of nesting.

        Since all types encountered in Java will now be considered part of any Type
        Hierarchy that exists in context, and java.lang.Object is at the root of all
        class hierarchies, any "unknown" types can also get represented by
        java.lang.Object.

        This will make wildcard usage, such as the type List<?> redundant.

        Even so, the Wildcard (<?>) (alone) will have to be retained for
        Backward Compatibility with Java <= 7.0. So, the syntax or semantics of wildcards
        need not get impacted in anyway.

        The above are just the major issues involved that I can think of.
      
        I don't make a claim that they in anyway represent the complete spec in summary
        for the changes requested. But as seen above, most of what is needed for the new
        treatment of Actual Type Parameters ALREADY EXISTS in the code base, which is why
        the effort involved will not be a massive one.

    6.3 Sign Off Plea

        IN THE INTEREST OF THE MULTITUDE OF JAVA USERS, AND THE RIGHT PATH IN THE
        EVOLUTION OF THE JAVA LANGUAGE, SUN MICROSYSTEMS CANNOT AFFORD TO OVERLOOK THIS
        AND PUT THE GENERICS ISSUE ON THE BACK BURNER.

Referenced Java Text

  • "Java In A Nutshell" - David Flanagan - 5th Edition -O'Reilly - © March 2005
    Indian Reprint Jan 2007 - Chapter 4 - pp.160-178

back to top

Jaisimha Narahari is based in India and has 11 years of experience in business
domain software development. Contact Author

Related Topics >>