In the past, the design work on modularity was split between two expert groups: JSR 294 was responsible for defining the language extension to support modularity in the Java programming language, while JSR 277 was responsible for defining the deployment aspect of the module system. The original rationale was that each expert group would have members with the most appropriate expertise to design what's best in each area for the Java platform. As it turned out, there were situations when experts needed to work at both the language and deployment levels, and a single expert group would have been more effective. To help make the discussions of modularity in Java more effective, Alex Buckley (the spec lead of JSR 294) has joined me to become a co-spec lead in JSR 277. In case you don't know Alex, he is *THE* language guy responsible for the Java Language Specification and the Java VM Specification, and I am very excited about this change. Our plan is to have all Java modularity discussions to occur in the 277 expert group from now on, and we hope this simplification will benefit everyone.
Alex and I are currently in the middle of producing the JSR 277 early draft review 2 specification for the expert group to review and discuss. This specification will be quite different from the original JSR 277 early draft in the following ways:
1. New language support based on Alex's latest proposal. If you haven't been following the language discussion around modularity, the most significant change is that there will be a new module keyword in the Java programming language:
module org.foo.app;
package org.foo.app;
module class Main {
...
}
Defining a Java module will be as easy as declaring a module name using the new keyword. But ... what if you want to create a deployment module with metadata that could be used in the module system? Well, it will be as simple as defining a Java module with metadata annotations:
//
// org/foo/app/module-info.java
//
@Version("1.0")
@ImportModules({
@ImportModule(name="org.foo.library", version="1.0+")
})
@MainClass("org.foo.app.Main")
module org.foo.app;
I won't dive into all the details here, but hope the example gives you some idea.
2. Incorporates inputs from EDR feedbacks and EG discussions on existing design and new features.
Since the original early draft was published, we have received many constructive feedbacks and most of them will be addressed in the upcoming specification. In addition, the specification will include details about resource modules (for internationalization), service/service-provider modules, etc. that have been defined by the expert group.
3. Reorganizes the specification to make the separation between the abstract module system framework and concrete module system implementation explicitly clear.
The original early draft actually has two separate parts: a framework for abstract module system, and a concrete module system implementation based on JAM (JAva Module). However, the distinction in that early draft was not clear because the focus of the original early draft was to gather feedbacks on the overall design rather than individual pieces. Bryan Atsatt, a very valuable contributor in the EG, recently suggested that a clear distinction between the two would help reduce a lot of confusion, and we agreed. This reorganization will happen and it will be one of the most obvious changes in the upcoming specification.
4. Validates the specification with the reference implementation.
We have been building the reference implementation (RI) through the Modules project in OpenJDK for quite some time now. The current RI has already implemented most of the functionalities as described in the upcoming EDR2, and we hope to bring the Modules project on par with the EDR2 around the same time when the specification is published. Stay tuned for more details.
Mandy Chung and I have been working on a draft specification for supporting OSGi bundles in the Java Module System, and we have just made it available for the expert group to review and discuss. We understand that people who have invested heavily in OSGi would like to preserve their existing investment going forwards, and this is what we hope the support for OSGi bundles will enable. Check out Mandy's blog if you want to learn more.
JavaOne is around the corner again. This year, Alex and I will be presenting a session and a BOF on modularity:
TS-6185: Modularity in Java™ Platform
Wednesday
5/7/2008
10:50 AM -11:50 AM
BOF-5032: Modularity in the Java™ Platform
Wednesday
5/7/2008
8:30 PM -9:20 PM
We'll go over the Java Module System from the language and deployment perspectives in details, and this will be a very technical and interesting talk. If you are attending JavaOne next week, we hope to see you in the session and BOF!
- Stanley
P.S. *random personal note* In the past year, I haven't been blogging much as I was on frequent leaves to deal with some family emergencies. The situation has been improving and I hope to blog more along with Alex and Mandy about all the exciting/interesting stuffs going on in modularity going forwards.
This snapshot currently covers the core features (see Andreas's blog) described in the updated specification in JSR 277, and the reflective APIs defined in JSR 294. If you are wondering how the module system works internally and want to take an early look or even contribute, I encourage you to check out the Modules project.
Besides the snapshot, we have also made other goodies available:
Obviously, this is just the beginning. You should expect the snapshot will be updated periodically and new goodies will be posted going forwards, as we evolve the implementation and add many functionalities in the coming months.
If you are interested in discussing the implementation in the Modules project, we have made two discussion lists available for the Java community to join:
If you want to discuss the Modules project with its developers, or how to use JSR 277 and JSR 294, be a subscriber today!
I have been planning to make the JSR 277: Java Module System expert group mailing list publicly readable for quite some time now, but the infrastructure in jcp.org has been pretty unstable in the last two months to make this possible and usable. Now the OpenJDK web site is available and offers better infrastructure, I have made the observer mailing list available from there today. This mailing list is read-only, and you will be able to observe the EG traffic going forwards. If you are interested, be a subscriber today!
Another exciting news is that we have also created a Modules project on OpenJDK. This project will cover the reference implementation of the modularity specifications defined by JSR 277 and JSR 294 as well as the related work in the JDK. Although there aren't much information on this page at the point of this writing, the intention is to make the source code, discussion list, and other implementation-related documentations available soon.
Finally, I'll be presenting a session (TS-2318) on JSR 277 at JavaOne this Thursday 4:10pm-5:10pm. Alex and Andreas will also be presenting a session (TS-2401) on JSR 294 this Thursday 9:35am-10:35am. If you want to learn more about the latest development of JSR 277 and JSR 294, come to the sessions! I hope to see you all there.
Now, are you ready? Here we go:
Many existing systems (e.g. .NET, OSGi, NetBeans, Maven, Ivy, etc.) offer reasonable solutions in their problem space, but they are not trying to solve the same problem as JSR-277. Because JSR-277 will become part of the core Java platform, it has several distinct advantages that allows us to solve problems in a different (and we think more elegant) ways than existing systems, for example,
* It can, in theory at least, define a better solution since it has the ability to mandate deep changes to the underlying architecture of the SE platform, and in particular class loading, which is something that no previous module system could do.
* It can integrate with JSR-294, which will move module definitions from crufty XML files into the Java language itself. Kind of like replacing Java EE 1.4 deployment descriptors with in-code annotations in Java EE 5.
* It is extensible. That should make it possible for other systems to interact with the Java Module System and even make it possible for them to offer new features made possible by the integration of JSR-277 into the Java platform. We are still working out many of the details and would appreciate feedback on this topic.
To learn from the experience of existing systems, JSR-277 has a large (20 members) and very diverse EG to accommodate a variety of viewpoints (e.g. a few members are .NET experts, two are OSGi experts who co-authored the R4 modularity spec, one is Maven's creator and one is Maven expert, one is Ivy's creator, etc.).
Yes. After each module instance is connected to all of its imports, shallow validation is performed on the module instance. One of the checks shallow validation will go through is to ensure that there is no overlap in the exported types from imported modules. Let's look at an example,
A import B v1.0, C v1.0
B import D v1.1
C import D v1.0
Suppose type T is in D. If type T is neither re-exported from B or C, then T is not visible to A at all and there is no type inconsistency. If type T is re-exported only from B or only from C, there is still no type inconsistency. Only when type T is re-exported from both B and C, there is a type consistency issue, but it will be caught by shallow validation automatically and A will not be fully initialized.
It is true that if B returns an object of type T to A via a "cast" to a superclass such as java.lang.Object, and B passes that object to C with tries to cast it to T, then that will fail. However, this is a rare scenario in real applications and we expect that the deep validation would catch this kind of inconsistency.
It is still fair to say that type consistency checking is not explicitly spell out in the early draft, and it probably requires lots of refinements in the next few months. If you have any good suggestion to contribute in this area, I'm sure the EG would appreciate your inputs.
Yes, modules can be loaded or unloaded at runtime.
New modules can be installed into a repository at runtime for the module system to use and load them. Existing modules can also be uninstalled from the repository at runtime so they will not be used when other modules request them subsequently.
When a new repository is created at runtime, modules from the new repository will become available to the module system dynamically. When a repository is shutdown, all modules will be torn down so they can be unloaded eventually. (See Section C.10 in the early draft - how to tear down a module is a future work item for the EG). In fact, we expect module-based Java applets and JNLP applications will be supported by using the URLRepository class, so web-based modules can be loaded and unloaded from a codebase dynamically in a JVM.
In any case, JVM restart is NOT required to load or unload modules.
Actually, I consider this degree of integration between JSR-277 and JSR-294 a big advantage. This allow us to move module definitions from crufty XML files into the Java language itself. This is kind of like replacing Java EE 1.4 deployment descriptors with in-code annotations in Java EE 5. Also, by specifying the module information directly in the source file, the compiler can validate the module information to ensure its correctness (How many hours did you spend on debugging manually written metadata?); it'll also be possible for developers to compile a module and generate a JAM file for deployment in a single step - without going through a separate step for writing separate metadata and packaging like what we have been doing for years.
The format for the module metadata will be defined in JSR-294, and the actual format has not yet been decided by the JSR-294 EG. What is shown in the JSR-277 early draft is just a logical representation of the information in the module metadata file and should not be mistaken for the physical file format. Make no mistake, XML is a reasonable representation format for typical metadata. On the other hand, because the module metadata will be used directly by the JVM to enforce module membership and module-level access control, I won't be surprised if the JSR-294 EG decides to use a more compact representation format.
Split package refers to the issue that classes from the same package can come from different modules through the import. We currently do not plan to support split packages in JSR-277. It will be spelled out more explicitly in the JSR-277 specification in the future.
Not necessarily. Designing declarative metadata is typically based on a set of requirements. However, as soon as there are new requirements arise that the existing design can no longer satisfy these requirements, the declarative metadata will need to be extended. Imagine if you want to use a module with metadata v2 in a module system that only recognizes metadata v1, you'll likely run into some problems. Import policy has no such problem because developers can express any logic they want in the code as long as the programming language and the class libraries support it, and the import policy can be executed and supported by all versions of the module system.
It's a valid concern that some import policies may not be written properly thus may cause side effect to the module system. However, it is not much different from encountering problems while executing code in the static initializer or instance initializer today. Even if the import policy results in exception during execution, this is really not an issue as long as the JVM can recover gracefully from it.
It's also a valid concern that expressing the import policy in Java code may be difficult for tools and management systems to analyze module dependencies, and this will be addressed in the next question.
It is not uncommon for the module system to have many repositories at runtime. Contents in the repositories are opaque, and they could be stored in different media (for instance, file system, database, HTTP server); the Repository APIs would effectively hides that from the application/tool. In addition, new repositories could be created and existing repositories could be removed at runtime. Also, it is possible that some implementations of the module system could provide mechanism to override/block/redirect module dependencies based on some deployment policies. Given there are so many potential variations in deployment, if tools and management systems attempts to analyze the module dependencies by accessing the physical metadata from the modules in the repositories, the task would be very complicated. Perhaps asking a different question would help:
Does it really make sense for tools and management systems to parse all the module metadata from the repositories and analyze module dependencies on their own, given all the capabilities in accessing the modules from the repositories and in supporting any potential override mechanism are already built into the module system?
Obviously, the answer is no. In fact, if the module system can provide APIs to expose dependency information it already has for a specific module, tools and management systems can simply make use of the dependency information provided by the module system. Hence, it is not necessary for module metadata to be declarative in order for tools and management systems to analyze module dependencies. As mentioned in Section C.5 in the early draft, missing dependency prediction is one of the future work items for the EG.
The early draft specification does not spell out any lifecycle support in details, but it is on the EG's radar. As mentioned in Section C.6 in the early draft, more explicit lifecycle is one of the future work items. If you have any requirements on this feature, I'm sure the EG would like to hear them.
Optional import dependency is currently not supported in the early draft. However, the EG may consider such feature if there are reasonable use cases and sufficient demands. If you have any requirements, I'm sure the EG would appreciate your feedbacks.
Yes, but for good reasons. One of the strengths of the Java Module System is its extensibility. It makes it possible for 3rd parties to plug in custom repository implementations to address complex or unforeseeable cases when these cases arise. Hence, we can keep the built-in repository model very simple, and it doesn't have to deal with every possible use cases.
The language aspect and compilation aspect of a module will be defined in JSR-294, instead of JSR-277. Having said that, we will likely provide more explicit instructions in the future for module-aware compilers to generate versioning constraints properly when compiling JSR-294 module in the context of JSR-277.
JSR-277 (Java Module System) is an architecture for the development and deployment of module-based applications and libraries. Applications and libraries written as Java modules are easier to develop and deploy, with first-class modularity, packaging, and deployment support in the Java SE platform.
]]> This architecture defines the following components:JSR-277 is currently targeted for Java SE 7.
Java Archives (JARs) are widely used as both the distribution format and the execution format for Java applications. The JAR file format dates back to the mid-1990s, and it has not scaled particularly well in either of these roles. JAR files are hard to distribute, hard to version, and hard to reference in general.
Distributing a simple Java application is considered to be a complicated task by many developers because it often involves creating a native installer to package multiple JAR files into a distribution unit, and it sometimes involves converting the application into a Java applet or JNLP (Java Network Launching Protocol) application for web-based deployment.
The versioning support in the JAR file format is for expressing dependencies on Java extensions (a.k.a optional packages), but not the version of the JAR file itself. There is also no reliable mechanism for expressing, resolving, and enforcing the dependency of one JAR file upon another. Referencing a JAR file, moreover, involves specifying it in the classpath. Since the path of a JAR file may change during deployment, developers are forced to fix up all the references to the deployed JAR files as part of the deployment process.
Developers also find it quite difficult to deploy installed Java extensions because they can easily run into issues like versioning conflict and namespace collision. It is also basically impossible to arrange for multiple versions of installed extension to be shared across multiple JRE installations.
These issues have been collectively known as the Classpath Hell, the JAR Hell, and the Extension Hell for years. The overall goal of the JSR-277 is to define first-class modularity, packaging, and deployment support in the SE platform that are capable to address these issues once and for all.
This is definitely one of the most frequently asked questions. The complete answer would never fit into a blog, but there are several points I want to highlight:
We deliberately made the design simpler by following this principle - make simple things simple, and make complex things possible. Because JSR-277 is targeted for Java SE 7, we have flexibility to update the JVM and classes libraries in the SE platform to better support JSR-277, thus reducing the overall design complexities. Also, the JSR-277 architecture is very extensible - 3rd parties may plug custom implementations to address complex or unforeseeable cases when these cases arise, so the core architecture can be kept simple.
Note: It is also fair to say that the module system described in the early draft is relatively simple because the early draft is by no means complete.
Repository is a mechanism for discovering, storing, and retrieving modules, and multiple versions of modules can be deployed side-by-side in a repository. Typically, the module system has one or more repositories at runtime, and the repository architecture is very extensible. Let's look at some implications:
Key abstractions in JSR-277, e.g. module definition, module instance, repository, etc., are all reified through the reflective APIs. This enables the module system to expose functionalities to everyone in a very consistent manner. Suppose the Java launcher needs to access the classloader object of a specific version of a module from the repository and an application needs to access the same thing - the Java launcher and the application simply invoke the reflective APIs to get what they want from the module system.
Each module defines exports that defines what classes and resources that are visible to other modules. At runtime, module-level access control is enforced at the VM level, thus keeping the Java platform secure by disallowing other modules to access module-private/non-exported classes in another module.
In typical module development, developers would start from writing a development module (defined in JSR-294), then compile and package the artifacts into a deployment module (defined in JSR-277). Basically, the development module is the source form of the metadata in the deployment module, and there are several implications:
In addition, exports in the deployment module are declared in the development module. If a class in a module attempts to access module-private/non-exported classes in another module, the error would be caught early on by the compiler during development rather than causing failure at runtime.
A trademark of many Java platform APIs is extensibility and allows 3rd parties to deliver innovations that plug into the JRE. For example, ClassLoaders, service providers, etc. JSR-277 is not only a module system by itself, but it is also an architecture that is extensible by other module systems (e.g. OSGi, NetBeans, etc.). Although the details still need to be worked out, the goal is to allow some degrees of interoperability between JSR-277 and other module systems. By providing custom module and repository implementations, other module systems may expose their modules through the repository for other JSR-277 modules to import. Similarly, through the powerful JSR-277 reflective APIs, other module systems may also allow their modules to import JSR-277 modules. A more extreme case would be for two or more other module systems to expose their modules as JSR-277 modules through the repository, and the framework in JSR-277 would enable modules from these module systems to interoperate with each others.
JSR-277 defines a distribution format for module called JAM (JAva Module), and it is based on the JAR file format. JSR-200 (Pack200) defines a hyper-compression scheme for the JAR file format. Hence, developers may use Pack200 to further compress modules for distribution. The module system supports packed module archives directly; a packed module archive is automatically unpacked if it is provided to the module system. Thus, JSR-277 module is typically packed for distribution, and its small physical size makes it very convenience to distribute, especially for download.
I hope you now have more ideas of what JSR-277 is about, and I encourage you to learn more by reading the early draft specification. The EG definitely would like to gather constructive feedback on the directions and the open issues indicated in the early draft. If you have comments, please send them to jsr-277-comments@jcp.org.
For those who have problems seeing the comments, this is a problem with the blogging software. Please visit both http://weblogs.java.net/blog/stanleyh/archive/2006/10/jsr277_early_dr.html
http://weblogs.java.net/blog/stanleyh/archive/2006/10/jsr277_early_dr_1.html
More information on the valid certificate:
Hopefully, it will significantly improve the deployment user experience of your applications very soon.
- Stanley
P.S. Thanks Andy Herrick in the Java Deployment team for backporting this feature; the codebases between Mustang and Tiger are quite different, so this backport is actually more difficult than it seems.
For more details, please check out JSR 277: Java Module System.
- Stanley
]]>Java Web Start and Java Plug-in will also support certificate revocation list (CRL). Basically, CRL is a list of certificates that have been revoked by the Certificate Authority. If a certificate contains a CRLDistributionPoints extension, Java Web Start and Java Plug-in will download the CRL from the URI specified in the CRLDistributionPoints extension for certificate validation. There will be a new deployment configuration option to let companies setup their own company-wide CRL to control the certificate validation policy. In addition, Java Web Start and Java Plug-in will support online certificate status protocol (OCSP) if the certificate provides an OCSP server URL in its AIA extension. There will also be a deployment configuration option to let companies to use their OCSP servers to control the certificate validation policy in the Intranet. Both CRL and OCSP support could be enabled/disabled independently from the Java Control Panel.
These improvements should be transparent to most developers and most users. However, if your certificate is not compliant with the RFC standards, especially in the case of self-generated certificate or certificate generated by internal PKI, your signed applications may break. Therefore, it is important to test your applications with the Mustang snapshot, so you could identify any potential certificate problem early.
Screenshot of login dialog box with save password option during Integrated Windows authentication on Windows 2000:
Screenshot of keystore password dialog box on Windows XP:
- Stanley
Note: Thanks Dennis Gu and Ashley Woodsom in the Java Deployment team for making these enhancements happen. Dennis is main contributor for the "Certificate validation enhancements", "Password protected keystore", and "Automatic certificate selection in client authentication" support; Ashley is the main contributor for the "Automatic proxy detection" and the upcoming "Save password option in login dialog box" support.
- What do you think about the new user experience?
- What other improvement do you like to see?
So, what are you still waiting for? Download Mustang today and experience these improvements! Most importantly, tell us what you think.
- Stanley
P.S. Thanks Andy and Rita in the Java Deployment team for doing a nice job on the implementation! Special thanks go to Chris Le Dantec, Coleen Baik, and Mike Albers in the xDesign team.
The common theme across all the new features and enhancements in various deployment technologies in Tiger is Ease-of-Deployment. Specifically, the primary motivation behind adding many new deployment features and enhancements is to simplify your life as a deployer.
For Java Web Start and Java Plug-in, developers told us that they wanted a more consistent deployment model between these products. Moreover, we received many feature requirements for enterprise deployment support, better networking support, security enhancements, desktop integration improvements, etc. Obviously, we must have a robust and extensible architecture if we are to continue evolving these deployment technologies, but we already reached saturated points in various subsystems that made feature extension difficult. Thus, we decided to rearchitecture Java Web Start and Java Plug-in in J2SE 5.0 to ensure there is a good foundation to evolve our deployment platforms going forwards. After the rearchitecture, ~70% of Java Web Start and Java Plug-in are supported by a common set of subsystems, including configuration, security, networking, download, etc. It does not only make many of the existing features behave more consistently across deployment platforms, but it also allows us to add features and enhancements in a rapid rate that has not been seen before.
Why does it matter to you? As a developer and a deployer, these changes will enable you to develop and deploy Java applets and JNLP applications more easily and avoid many deployment surprises. Let's take a more detailed look at some of the new deployment features and enhancements introduced in J2SE 5.0.
To make configuration customization easier, Java Web Start and Java Plug-in now share a configuration infrastructure. Users only need to customize their settings once, and the affected configuration would be applied to both products. We also provide a Java Control Panel to help users easily customize their configuration.
Moreover, to enable flexible configuration customization, Java Web Start and Java Plug-in expose a variety of configuration options, e.g. proxy settings, certificate stores, security policies, etc. These options allow users and administrators to have more controls over their deployments.
Futhermore, to make configuration deployment more manageable, Java Web Start and Java Plug-in support enterprise configuration and lock-down. The feature is very useful in enterprise deployment, because it enables administrators to provide default values of selected set of configuration settings for users in the enterprise, and it also allows administrators to lock-down selected set of configuration settings altogether to prevent users from customization.
To allow web servers to determine the identification of the client, HTTPS client authentication is now fully supported by Java Web Start and Java Plug-in. If your applications connect to a server that requires client authentication, your users will be prompted to select a client certificate available on the system.
Many companies used the browser keystore as a way to distribute certificates and keys to be used by their internal applications. To make distribution of certificates and keys easier for Java applications, Java Web Start and Java Plug-in now support browser keystore in Internet Explorer and Mozilla. Certificates and keys in the browser keystore are used for signature signing verification, HTTPS server authentication, and HTTPS client authentication. If the certificates and keys are on the smartcards, they are also recognized by Java Web Start and Java Plug-in if they are exposed in the browser keystore.
To avoid deployment surprise due to expired certificate, Java Web Start and Java Plug-in now support signature timestamp. Consider the following situation: an application is signed without signature timestamp and is rolled out for mass deployment, and the certificate is expired afterwards. In this case, this application would be considered to be signed by an expired certificate, and your users may be present with security dialog that indicates the certificate has expired when your users try to run the application. On the other hand, with signature timestamp support, developers may generate a timestamp along with the signature when the applet or application is signed. Java Web Start and Java Plug-in will consider that the application is properly signed if it has a valid signature timestamp even though the certificate has expired at time-of-deployment.
Pack200 is a hyper-compression technology defined in JSR 200, and it achieves approximately 1-to-9 compression ratio for JAR. To minimize application download size, Java Web Start and Java Plug-in support HTTP compression and Pack200 compression for JAR download. If a server returns gzip-compressed or Pack200-compressed JAR during JAR download, Java Web Start and Java Plug-in will uncompress these files automatically.
Java Web Start now supports SOCKSv4 and SOCKSv5, in additional to support for HTTP, HTTPS and FTP. Moreover, Java Web Start also fully support automatic proxy script on Windows.
To allow JNLP applications to take advantages of session management, Java Web Start now provides build-in cookie support in HTTP and HTTPS connections.
We have added many enhancements in the JNLP 1.5 specification implemented by Java Web Start in J2SE 5.0. First, the version-id syntax has been extended, so developers have more flexibility in specifying version-range. Moreover, the "java-vm-args" has been added to support various "safe" VM options for your JNLP deployment.
To improve desktop integration support, developers now have more controls over how their JNLP applications are integrated through shortcuts creation, file extensions and MIME association, and related content integration in the start menu. On Windows, Java Web Start also automatically integrates downloaded JNLP applications with the "Add/Remove Programs" in the Windows Control Panel, so users may uninstall any JNLP application directly from the Windows Control Panel.
To allow JNLP application to run as a singleton, a new javax.jnlp.SingleInstanceService was introduced into the JNLP APIs. In addition, there are various minor API enhancements as well.
To allow JNLP application to be deployed from a different source, Java Web Start now contains an import facility. This can be used for CD installs, where code is initially loaded from one location and then updated from another. It can also be used to pre-install applications and libraries in either the user or system cache without running the applications.
The major change was the elimination of Java Web Start Application Manager. Having a separate application to manage downloaded JNLP applications is important, but we believe that the most natural way to launch JNLP applications is through either the desktop shortcut or JNLP link in the browsers. The majority of the functionality of the Application Manager is now contained in the Java Control Panel.
We have also beefed up the debugging support in Java Console in Java Web Start similar to the one already available in Java Plug-in, so developers may debug their applications more easily.
We have added a new animation to provide better feedback to end users when the applet is loaded. If you run applets using Java Plug-in in Internet Explorer, Mozilla, or Firefox, you will see a new animation appear in the applet whitebox area during applet loading.
We have also made many improvements in JavaScript/Java communication by tightening up the security model, and by speeding up the performance up to 4 times in some cases. These improvements should improve the responsiveness of your applets in the browser.
The main improvements are better proxy support on Windows 2003 as well as authenticated proxy support for all Windows platforms. Also, we have further reduced the minimal download size of the JRE 5.0 online installer compared to 1.4.2, despite there are many new 5.0 feature.
As you could see, the list of improvements in deployment in J2SE 5.0 is very long, and there are still many features and enhancements I did not cover in this blog. Nonetheless, I hope I have given you a high level overview of the new deployment features and enhancements available in J2SE 5.0, so you may leverage these new features and enhancements in your next deployment.
o Which are the new deployment features in J2SE 5.0 you like?
o Which are the new deployment features in J2SE 5.0 you plan to use in your next deployment?
o How will these new deployment features improve your life as a deployer?
Let me know.
- Stanley
Note: Some of you may also be interested to know about what's coming in Mustang, and I will talk about "Java Deployment in J2SE 6.0" in an upcoming blog. Stay tuned.
Stanley: What do you see as the major obstacle of your deployment?
Developer: JRE download size is definitely a problem, especially on Windows.
Stanley: Why?
Developer: The JRE is 25MB, and it is beyond what most consumers are willing to download.
Stanley: Actually, the full download size of JRE 5.0 is ~15MB, and you could get a minimal download of ~7.2MB if you use the online installer.
Developer: Really? I didn't know that. I thought the JRE was '25MB' because that's what I've always heard.
Obviously, there are some misunderstanding among Java developers about the JRE download size. It is time to do a reality check!
Apparently, the offline installation is ~15MB, and the online installation is only 221KB?! Note: most JRE installers for Linux/Solaris are also ~15MB.
What exactly are the online and offline installation?
The online installer was introduced in Mantis (J2SE 1.4.2) in response to the critical install issues at that time:
1. The JRE was big for most end users to download, and average users only needed the core features.
2. If you developed in Java long enough, you probably still remember there were separate English and I18N bundles. If you installed the wrong bundle on the machine, the only way to fix it was to uninstall the wrong bundle and reinstall the right bundle. Obviously, there were lots of frustration over this issue, and it forced most developers to always bundle the (larger) I18N JRE with their applications.
3. There were a few goodies developers wanted but they were only bundled with the JDK, e.g. sound bank, color profiles, additional fonts, etc. Their unavailability generally discouraged their usage.
4. Lack of incremental update between JRE releases.
To address these issues, William Harnois, Kumar Srinivasan, and I decided to implement the install-on-demand feature in the Windows JRE installer. Our approach was to separate the Windows JRE into three features:
1. Core: Western JRE, with Java Plug-in and Java Web Start. This feature is always installed.
2. Other: Other languages and locales. It is installed by default on platforms that support non-Western locales
3. Extras: Sound bank, Color profile pycc.pf, Lucida fonts beyond Lucida Sans Regular.
Architecture of the Windows JRE online installer:
Architecture of the Windows JRE offline installer:
With install-on-demand, the online installer is simply a bootstrap that understands how to pull down the rest of the features from the web, while the offline installer basically packages everything into a single bundle.
Using the online installer, end users only need to download the critical features of the JRE to run their Java applications. One thing to notice is that the full set of J2SE APIs are available if you download only the core feature, so there is no platform fragmentation issue. The next obvious question is: what is the minimal download using the online installer?
If you look at the online installer architecture carefully, the only components that users must download are: bootstrap, MSI database, and the core feature, and the total size of these components is what we called the minimal download size. In JRE 5.0, the minimal download size is ~7.2MB. If you install the Other and Extras features, it is ~10MB.
Wait a minute! If the total download size for all features is only 10MB, how come the size of the offline installer is ~15MB? This is because some Windows systems do not have the Windows installer engine installed, but the installer engine is required for our JRE installation, so the offline installer must bundle the engine together for distribution, and it adds additional overhead! The online installer does not have this issue because it could download the Windows installer engine from the web if necessary.
The good news is: the online installer could give you ~7.2MB minimal download, and it is much smaller than many developers thought!
There is more! If you install a JRE update release (i.e. W.X.Y_Z) using an online installer and you already have an installed JRE in the same family (i.e. W.X.Y or other W.X.Y_Z) on the machine, the download could be further reduced through incremental update with base image patching!
What is incremental update?
In traditional patching, a patch is often generated between the last up-to-date version and the new version of the software image.
In this approach, you could run into all sorts of versioning slew issues. For example, a patch can only be applied if the installed image matches the version that the patch targets. Synching up the mismatch version of the installed image could be very painful, and users may be forced to install dozens of patches in order to bring their local images up to date. Patch installation also yields a different user experience than installing your regular application, and patch management could be very complicated. Fortunately, most of these issues could be avoided in base image patching.
In base image patching, a particular version of the software is defined as the baseline, and its software image is defined as the base image. Patches for subsequent releases of the software are always generated with respect to the base image:
In this approach, a patch can be applied only if the base image is already installed. What is really interesting is that the patch size generated through base image patching is very comparable to the one from traditional patching when we applied this scheme to the JRE.
Since Mantis, we have supported incremental update by leveraging base image patching under the cover of the JRE installer. When a JRE installer is built, the installer actually contains a base image and a patch. For example,
- 1.4.2 -> 1.4.2 base image + no-op patch
- 1.4.2_01 -> 1.4.2 base image + 1.4.2_01 patch
- 1.4.2_02 -> 1.4.2 base image + 1.4.2_02 patch
- 1.4.2_0X -> 1.4.2 base image + 1.4.2_0X patch
- 5.0 -> 5.0 base image + no-op patch
- 5.0u1 -> 5.0 base image + 5.0u1 patch
- 5.0u2 -> 5.0 base image + 5.0u2 patch
- 5.0u3 -> 5.0 base image + 5.0u3 patch
- 5.0uX -> 5.0 base image + 5.0uX patch
When a JRE is installed, the installer will always attempt to store the base image in a repository on the machine first in case the base image is absent. If the base image is present, the installer will retrieve the base image and apply its patch to generate the appropiate version of the JRE image for your installation.
Why does it matter to you? If you always use the offline installer, it probably won't make any difference. However, if you use the online installer, the incremental update support could significantly reduce your minimal JRE download size even further. For example, if you already have a 5.0 family JRE installed on the machine, installing a new 5.0 family JRE will only download ~2MB (patch + installer overhead). As a result, the download is much smaller, and the installation is much faster!
Because of the benefits in download reduction, we actually used the online installer for java.com and Java Update in many download scenarios.
As a side note: Java Update is a software to deliver JRE update to end users regularly and automatically. Contrary to common belief, Java Update does NOT do any patching; it is actually the downloaded online installer that does the magic!
Consider bundling the JRE online installer.
If you need to bundle a JRE with your application, you may consider bundling the JRE online installer and take advantages of all the download reduction benefits I outlined above. The bootstrap is less than 250KB! Incremental update is only ~2MB! Even if you don't have a base image on the system, the minimal download is only ~7.2MB. There is a catch though Your users must be online during the JRE installation, and the success rate of the installation is subjected to the network and server condition.
Hopefully, you should now have a better understanding on the JRE download size issue. Of course, we will continue to think hard on how to reduce the download size even further. Tell us what you think!
- Stanley
Note 1: Some of you may wonder how I came up with download size number. The number was determined by simply adding up all the necessary files the online installer must download for the core feature. However, the online installer is hosted on the server side, so it is difficult for general developers to calculate the number themselves.
Note 2: The Windows JDK installer also follows similar architecture.
]]>A signed Java application is simply just, signed. It does not mean that the application should be granted with all permission automatically, because the application could be signed by anyone, including hacker. Therefore, before Java Web Start runs a signed application, it is critical to ask users if they want to trust the signer before granting all permissions to the application for execution.
Security warning UI should reveal proper level of information to help users making the right decision.
However, there is a fine line between revealing too little information that misleads the users to make the wrong decision, and revealing too much information that scares the users away. Based on the feedbacks, the current security warning UI in Java Web Start is simply too scary looking, and many developers have hard time to convince users to run their applications during deployment.
Here comes the good news!
We have decided to revamp the user experience in Java Web Start in Mustang. Even better, the promoted Mustang build 34 contains the initial putback of the new security warning dialog box, and you can download it today! To give you a taste, here are some screenshots:
Signed application with valid certificate:
Signed application with expired certificate:
Self-signed application:
Note: The messages in the security warning UI are changed based on the nature of the signer certificate. You should try out Mustang to see how the new security warning dialog box will look like for your application.
We understand that any change like this has a profound impact to the success of your Java deployment, and your feedback will certainly help us to shape the final product to satisfy your needs. We will continue to make a series of UE/UI changes in Java Web Start before Mustang code freeze, including graphics and layouts, so the input we want at this point is around the messages in the dialog box, not the cosmetic UI issues:
- Do the messages still make the dialog box look scary to your users?
- Do the messages reveal proper level of security related information?
- What improvement do you like to see in the new dialog box?
Tell us what you think.
]]>