Sean Sheedy's Blog

A "Fragmentation Program Office"?

The MEDD conference was exactly what I thought it would be: learning about cool stuff (SunSPOTS!), meeting old friends and making new ones. And it turned out that one topic stood out above the rest - confronting fragmentation in the mobile space.

Looking at the lightning talk feedback we find the most highly rated lightning talk was possibly Terrence Barr's "Do We Need a Mobile Developer Alliance"? Were the ratings also votes for making such an alliance a reality?

]]> The direction of the last sessions of both days suggests that the answer is "yes". The technical session feedback shows that among the highest rated sessions were the panel on "Developing and Deploying Content in the Real World", and the "Fish Bowl" on the last day, both of which dove into the topic of fragmentation and never left it.

Undoubtedly, the current java.net poll was inspired by this theme. As of this writing, by far, voters believe that the two greatest sources of fragmentation relief are the manufacturers, and Sun. Why aren't operators, who could team together to coordinate signing and certification requirements, considered a group that can solve this major point of fragmentation?

One answer lies in the Fish Bowl conversation. One developer spoke about how fragmentation was really a problem, but what was really killing him was all the signing and certification hurdles they had to face. In other words, we have different perceptions as to what constitutes fragmentation and what doesn't.

The point is that it is really hard to discuss fragmentation when we don't even have a common language to describe it or can agree on its scope.

This came up again in an open discussion on fragmentation at Sun's offices on Friday. (More to come on that once the meeting notes are written up.) One of the participants mentioned that their company actually had a comprehensive glossary of fragmentation-related terms, but nobody else in the meeting was aware of it.

Even the discussion of fragmentation is fragmented, with major corporations in this space aware of it, but discussing it, raising it as an issue, or actively pursuing programs to address it in relative isolation. Some of that work has been effective (such as JSR-248, in pulling together many JSRs under one umbrella), but other work has been duplicative or has not happened at all. It is kind of like building a wheel with a some good spokes, some broken spokes, some missing spokes, and no hub.

Part of the reason that industry collaboration does not exist in many areas is that when people do want to collaborate with their competitors to solve a joint problem, their managers only hear the word "competitor" and big caution flags come out.

There are many aspects of fragmentation that need to be addressed, and it will require work by ALL of the parties listed in the java.net poll. There are several crucial things that must happen for this to occur:

A common definition and understanding of the problem must occur (hub)
The problem needs to be divided into pieces that can be easily described (spokes)
Environments that bring competitors together to address each piece need to be created (spokes)
Coordination must occur to avoid duplication and enable communication between groups tackling different areas of the problem (hub), yet
There must be enough freedom for ad-hoc groups that have the support of the community to run with whatever part of the problem they want to (spokes)

I have considered the JCP to be a good place to center a "Fragmentation Program Office" to lead? manage? keep track of? launch? different fragmentation reduction efforts. But judging from the response to the java.net survey and the apparent call for a Developer Alliance at the MEDD conference, a hub for defragmentation activity could spring from anywhere the community will accept it to be.

SunSPOTs and ham radio

This week I decided to drive from San Diego to Santa Clara to this week's MEDD conference rather than fly home and back again. This gave me the opportunity to try out a new ham radio product called the TinyTrak4 which is about to be released by its creator, Byon Garrabrant.

The TinyTrak4 provides me with an APRS setup in my car to track my position as I moved along. (I also beaconed that I could be reached by voice on 144.39 MHz, which is the ham radio national calling frequency for two meters, and kept a second radio on for that purpose.)

Links:

Present location
Ham stations near me
APRS users around the country

Now, there is a really cool product called the TinyTrak4 which has been through a year of beta testing (that's my TT4 on Byon's web page) and is about to go live. You hook up a GPS receiver to the TT4 and it creates packets and uses smart beaconing to send it into the ham radio APRS network via an attached ham radio.

The TT4 can do more than this, and one of the cool things about the TT4 is that it can take sensor data and attach it to these packets.

There are devices called "digipeaters" which hams set up to hear radios like mine, and retransmit them, the idea being that other radios in the area can more easily hear them. Transmissions can go over multiple hops or none at all depending on the path that the user sets in the tracker.

Furthermore, there are "i-gates" which are receivers hooked up to the Internet, which take packets (that are not flagged to not be gated) and post them to the Internet, where you can look at them on maps such as at the above links.

There are 30,000 or so APRS users out there, and a device that can include sensor data with a scheduled transmission is useful. This brings us to the other neat feature of the TT4, which is that it can be used as a data modem or "TNC" as it is known in ham radio speak.

APRS is not only used for tracking yourself (in fact, use of this mode with no other information is discouraged by some because it really does not contribute much other than a spot on a map, and does lead to more congestion of the channel.) Some other uses include letting people know how you can be contacted (as I did, by including the frequency I'd listen to), or send messages back and forth (many APRS systems are capable of messaging), or transmit weather station information, or many other uses.

What I wanted to put out there is that the SunSPOT is a device that is easily programmable using Java and has a number of interesting sensors that are of interest to hams, the original Makers. Temp sensor, accelerometer, inputs/outputs, etc. What could be done if you connect a highly programmable sensor like this to a network like APRS? What could you get from a network of sensors working simultaneously or that can receive and act on messages sent to it remotely?

Open Discussion of Developer Issues

Over the years, there has been much written and spoken about the challenges facing mobile developers. This will be no different in some of the sessions and in the hallways at the Mobile & Embedded Developer Days conference and at BarCampME in Santa Clara next week.

To facilitate converting talk into action, a room has been been reserved from 9 AM to 1 PM on Friday January 25th near the MEDD site for an extended face-to-face discussion.

The timing coincides with the conclusion of the MEDD conference to take advantage of the momentum caused by the conference, and so that members of the community who are flying in to attend the MEDD conference can participate in this discussion.

The discussion is not limited to any particular technology or aspect of the mobile application ecosystem.

The agenda is fluid but is looking something like this:

Introductions - getting to know each other and why we are interested in developer issues
Vision/Goals - what would we like to accomplish in the interests of developers? In 2008? In the next quarter?
Recent Activity - What was attempted last year that worked or did not work?
From Talk to Action - Blogging and writing articles has helped us understand but not necessarily led to change. What new *actions* can we take during the next quarter towards reaching these goals?

We might only reach the second agenda item, but leaving with a better knowledge of who is genuinely interested in addressing mobile developer issues, having a common vision, and sharing email addresses and phone numbers, will in itself be a great first step.

If you are interested in attending, please email Terrence, or myself at developermeeting@gmail.com.

Pack your sleeping bags!

Pack your sleeping bags! The MiniBarCamp, "BarCampME", which bridges the two days of the Mobile & Embedded Developer Days (MEDD) conference, is on!

My friend Dan Green is due a big Thank You from me and everyone who will participate, because late last week he found a secret FooCamp/BarCamp veteran who came through with the support we needed to make it happen.

The MiniBarCamp starts at 8:30 PM Wednesday January 23rd, after the MEDD dinner, and runs until 8:30 AM the next morning. Information can be found here:

BarCampME

In case you were wondering, here is the Wikipedia entry for BarCamp.

Thank you Terrence Barr and Roger Brinkley for putting this on the MEDD agenda!!!

Android's "do not fragment" clause...

The OHA members have signed an anti-fragmentation clause. "They've basically agreed not to fragment nor do things that would result in different versions of the platform", says Rich Miner in the PC World interview. And the JCP has that too - called MSA. But as Java ME developers know very well, there are many facets to fragmentation extending beyond the platform and well into the surrounding ecosystem.

A strict "do not fragment" clause is impossible to implement without putting every aspect of the platform and ecosystem through a committee, or relinquishing creativity and possibly IP to a central authority. This is because the other face of fragmentation is called innovation and differentiation.

If many vendors are playing similar roles in the ecosystem then I don't care if there is an anti-fragmentation clause in OHA, you are going to see some fragmentation. Some of this is beneficial: you need to allow new features to appear in devices and new services to be offered. "Fragmentation management" is needed so you don't eliminate the innovative process when eliminating redundancies that appear as a result of that process.

What this clause is, however, is a huge, essential step in the right direction. It establishes a positive spirit of cooperation. It reminds competing parties why they need to be at the table. It justifies discussions on processes throughout the ecosystem, such as application certification, signing processes, distribution, rewarding the creation of new features and services, open issue tracking and sharing, collaboration on documentation and developer support, interoperability testing and "bake offs", etc.

One of my objectives in my candidacy for one of the open Java ME EC seats is to drive the EC to expand its scope to address the areas of fragmentation that go beyond the platform. Having everyone sit down and sign a "do not fragment" statement, no matter how symbolic, is perhaps a great way to set the tone for the collaboration required to do what everyone already wants: turn the symbolism in concrete practices.

On the Android announcement - a bullet list

Android has existed for probably about 4 years, created by people who have real handset experience, so a delivery timeframe of 2H08 is not unreasonable. Google acquired Android back in August 2005 and it had already been running for 22 months by then.
This old (March 2007) article tells us perhaps more technically than most of the news about the phone that came out in the past day.
Given that article's date, and its lack of mention of a lot of other players, and the growing leaks of Gphone news in the press prior to the announcement, the other players in the announcement were probably brought on board shortly after this and probably have been working on this coming on 4-6 months...
Sun was noticeably absent but Jonathan Schwartz blogged his full support yesterday and promised full support in NetBeans. This is really great news for Java developers.
If you peruse the press releases from the companies in Open Handset Alliance you will find a little more tech info which shows that it's a Java play as far as developers are concerned and quite possibly that Linux will not be exposed to developers (or at least not to anyone but close partners.) Could be wrong about that last part, and this is what we're all dying to see in the Nov 12 release of the SDK.
This is underscored by the presence of Esmertec and Aplix who have created the Java ME platforms for a lot of mobile phones.
If you look closely at all the companies in the OHA list, and do a little dreaming, you can piece together a vision for a handset with some pretty intense and useful capabilities that will be yet another nail in the coffin of Frankenphones we have to live with today. (I'll provide my definition of Frankenphones later, but the short answer is that it is what you get when product managers are allowed to make major functional and design decisions for handsets.)
The presence of Nuance as part of the voice solution suggests that this may provide a stepping stone towards giving the VOIP industry the vision it needs to get out of its saturated but under-realized state. Personal IVRs may be right around the corner. Home phones where you have one line that rings through your whole house and you can't make a call while someone else is on the phone will be as passé and unacceptable as party lines would be (though you might see stuff like Selective Ring, since you may want all handsets in the house to ring, but need to know who in the house it is for.) OK, maybe this is stretch here, but VOIP is an industry ready for a focusing as much as applications and technologies we are discussing here have focused mobile data after its early fits in starts in the late 90s and early 2000s.
Expect to see telephony integration in the PC taken to the next step as a fallout of this.
That they cobbled together 34 companies for the OHA announcement so secretly is amazing. The NDA must have some pretty big teeth. Even people who should have been involved in this did not know about its full extent until yesterday.
Apple has proven that a sufficiently sized outsider is a gorilla in this space and will ravage the china shop if ignored.
You cannot ignore Google; they are a bigger gorilla than Apple both in market cap (about 40% greater) and arguably, penetration of their respective industries, depending on what industries you consider.
Still, there must be a pretty darn good business case for the parties involved in Google's pitch deck. I would love to see an unedited version of that.
Not to mention that those on the OHA missed the iPhone boat, so there may be a bit of fear of not being on board for this "second chance". Operators are saying that they recognize the days of closed phones are nearing the end (this is mentioned a ways down in the article by one of those interviewed.)
How open is OHA? At the end of the FAQ on the alliance on www.openhandsetalliance.org is an email for questions about joining OHA. Of course, I sent them an email asking what the process is for joining. (I already know the general answer, but why not get an official one?)
Interesting that Qualcomm is on the list, obviously because of chipset implications, but also because this directly competes with their relatively closed BREW business. What does this spell for BREW?
Also interesting, but not surprising, that AT&T is not joining, but they already have their own pet gorilla.
Nokia is also not on the list which is also not surprising as it has its own plans for a soup-to-nuts ecosystem around its handsets as evidenced by its recent location acquisitions. At last month's CTIA conference, NAVTEQ held a full day session which emphasized the many yet-unrealized possibilities surrounding location, including taking location to the pedestrian level - which is where Google will play with Android. In the Wall Street Journal, Nokia said they already have an open system in their high end phones.
PC Magazine reports that OHA members are not dropping other OSs for Android.
Additionally, there are reports that signing on to the alliance does not mean that a company will create a "google phone". This makes sense, as Google said that it did not want to build a phone - it would probably not want to deal with phones any more than it wants to deal with a Google PC - but it does need an environment that meets its needs.
So given this, and with OHA's website not really being more than a link or two deep, is OHA more than a bunch of press releases? Today, probably not. Will they eventually standardize this work, or push the standards work out to places like the JCP, W3C, OMA, IETF, 3GPP and CDG?
I also have to comment that the video on the OHA website plays like the tours of Pixar in the DVD extras of films like Toy Story. Don't all the robots look just totally cool?
Finally, a reference to the JCP. Opening a dialog with OHA should be a priority - if Android has meat (which it does by all indications) then working with them is going to be imperative.

Note that this is all written before seeing the Android SDK. However, there are a lot of smart people behind this, and with people like Jonathan Schwartz supporting it (who probably has insight into what is under the covers - I will leave the commentary on this for others), there will probably be some very interesting stuff for Java developers.

Sean

Voting is Good but Making Your Voice Heard is Better

After spending several years in the Java ME standards circuit for a major operator, I'm now on my own as a consultant, and running for one of the open Executive Committee seats as an Individual. I'd really like to know what burning topics Java ME developers would like to see the winning candidates bring to the EC.

Java ME has been fabulously successful with well over 2 Billion devices shipped. A lot is being done right, but it does not mean that the platform is without issues. Some are longstanding, and current EC members tell me that the desire for correcting them is universal. However, the nature of corporate life makes initiating these discussions difficult. How do you tell your boss that you need travel budget to discuss fragmentation with your biggest competitors? "You're going to discuss what with who?"

An Individual can facilitate discussions on these issues. They have the advantage of not having corporate encumbrances. They are freer to start conversations among competitors to bring those to the table who need to be there for change to happen. They can highlight other Members' public actions that demonstrate their commitment to the Community Process, without being accused of "aiding the enemy." They are freer to question private actions within the EC that undermine those public actions, without burning vendor/customer relationships.

In my position statement, I listed many potential objectives, but with only 15 people in the EC, you have to focus in order to get anything done. So I am interested in hearing what people consider to be the first issue for a new EC member to raise after they are elected to the Executive Committee.

In the JCP, a Community with over 700 members, about 2/3 are Individual Members and of the remaining 1/3, a significant number are smaller companies. However, between the EC and MSA - the two bodies that drive ME - all but one member is a large corporation. This is not to say that these companies do not represent developers. From personal experience I know they do, but developer interests are represented through a corporate lens.

What do individual developers want to accomplish when that lens is removed?

SE/EE Executive Committee Candidates

Being focused on Java ME, I found that after perusing the candidate list for the SE/EE EC on jcpelection2007.org, I still was not sure who to vote for. So I put this together to be able to compare the candidates and understand their motivations for running for an EC seat.

I'm posting this so that maybe a few people or the candidates themselves can help fill this out, or comment on who they are voting for and why.

All candidates should be commended for selecting EC participation as their way of contributing to the Java community and helping to guide Java's future.

I paraphrased what I found in their candidate statements and things they linked to, so there are bound to be errors. Please let me know if you find any and I will correct them.

For each, a couple of questions came to mind and are listed. For example, I was hoping to see more information in the candidates statements about why they were running for a seat - only two of the seven candidates provided this information.

Disclaimer: I am a candidate for a seat on the Java ME EC.

Voting ends 11/12.

]]> CodeGear (Borland)

JBuilder developer tools
First all-Java development environment
Helped define the Java beans specification

compiler, debugger, ejb, servlets, jsp, jsf, jbi
JCP EC for...?

Goals for the EC:

Goals for Java:

Ravi Kumar, Principal Architect, Java tools group
David Intersimone, VP Developer Relations, Chief Evangelist - responsible for CodeGear developer network - 6M SW developers

What are your goals for your EC participation?
What do you plan to do to change Borland's past EC participation stats?

=============

Eclipse Foundation, Inc

Hosts Eclipse projects, 100% in Java
Tooling support for JSRs

220, 222, 235, 291, 317
New JCP member, joined in 2007

Open source licensing of Java technologies
Maintaining a culture of openness and innovation

Goals for Java:

None mentioned on the election site

Who do you plan to send to EC meetings, and what are their credentials?
What specifically do you propose regarding "maintaining a culture of openness and innovation"?

==============

Ericsson AB

Mobile device and infrastructure provider
Contributing to the open source community

Currently participating in 21 JSRs including 281 and 319.
JCP EC for past six years

Offer technology leadership
Evolve the JCP
(from website) "presently ramping up our engagement in JCP".

Promote Java as the technology of choice for the telecom industry

Jens Jensen, spec lead for JSR-319

In what ways do you wish to see the JCP evolve?
Where do you plan to provide technology leadership?
In what ways are you ramping up your engagement in the JCP?

================

Google Inc.

It's Google
Extensive use of the Java platform

Participated actively in 23 JCP EGs
JCP EC for past 3 years

Goals for the EC:

Deeply believes in giving back to the community

Joshua Bloch, Chief Java Architect and author of a number of Java books, previously Distinguised Engineer at Sun

What are your objectives for your JCP EC participation?

=================

Interface21, Inc.

Creator and supporter of Spring framework

JSF 2.0, JDO 2.0, JPA 2.0, Servlet 2.3, Java EE 6.0, JMX

Goals for the EC:

Goals for Java:

Rod Johnson, author or several best-selling Java EE books and top speaker at JavaOne, Java Champion and EG member including Java EE 6

What are your reasons for running for a position on the EC? What are your EC objectives?

=======================

Klaus Meffert (Individual)

Works for Brandt & Partner (software consulting and implementation) on Java and SAP.
Author of a book on JUnit

JSRs:

Goals for the EC:

Goals for Java:

Self

What are your goals for your EC participation?

====================

Perret Pierre-Henry (Individual)

Works for Cross Systems in Switzerland.

JSRs:

Goals for the EC:

Goals for Java:

Self

What are your objectives for your EC participation?

A Call for "Open Enrollment" for MSA (Mobile Service Architecture) Advanced

MSA Advanced is chartered to define the next generation Mobile Java platform. So, it seems logical to get the experience and expertise of many users and citizens and developers into the discussions surrounding this new platform.

However, there are no individual experts in MSA Advanced.

The problem is that once an Expert Group (EG) has been selected, it's common for a JSR to stop accepting participants and remain closed at the discretion of the spec leads. Somehow this EG formed without Individual experts. MSA Advanced's web page on JCP.org shows that it started in summer 2004, and was scheduled to finish a little over a year later. A lot has changed in the mobile space since then, and the individual perspective would contribute valuable insights.

There is a precedent for joining JSRs after "open enrollment" ends. Motorola's Mike Milikich, the spec lead for MIDP3, has an open-door policy for any JCP member wanting to get involved with that JSR. The result is that many eyeballs have improved the richness and quality of the specification. The "too many cooks" problem has not materialized.

In his keynote at JavaOne 2006, Jonathan Schwartz said, "This is not a heavy-weight corporate kind of environment. You go to JCP.org, you register as an individual . . . We want to make sure that all voices are heard in the process . . . This is not about corporations defining a technology platform, this is about users and citizens and developers defining that platform."

I know many of the participants in MSA Advanced. They are intelligent and kind, and they and their companies have repeatedly demonstrated how much they value the role and the input of individual developers. Furthermore, MIDP3 has shown that there is great value in the contributions of individual developers in the EG process. Maybe a little reminder of this is all that is needed for the MSA Advanced EG to announce "open enrollment" in the very near future.

How Java M&E Developer Days came about...

Finally, everything I'd like to see in a mobile & embedded developer-focused Java conference.

It started in mid-June with this email from Roger to folks like CEO and Eric Giguere:

"Terrence and I are thinking of starting a Mobile & Embedded Conference that would occur some time in the late fall/early winter. This would be a conference solely devoted to Mobile & Embedded developers..."

]]> Heck, every one of us in this space had been thinking how great a focused conference like this would be, and it certainly came out in the responses to this email.

Roger is very sensitive to keeping this developer focused and keeping the costs down to make it easier for individual developers to attend. Which means it's being held in Sun's Santa Clara campus auditorium - save boatloads on venue and instead spend it on unlimited pizza and bottomless sodas...

Seriously, given the excitement generated every time Roger has written about this, it's inevitable that there will be key Java ME experts roaming the halls at this conference. This means not only great sessions, but also those great late evening discussions over beer with Java giants, that I look so forward to at standards meetings and at JavaOne. The folks Roger consulted live and work around the world, so I expect a lot of people will be flying in for this one.

So yeah, this announcement is just a call for papers now, but I think it's safe to say that waking up early on November 1 when registration opens is a good idea... how many people can the Sun campus auditorium possibly hold?

Making your point on standards matters

The MIDP3 Expert Group (JSR-271) is meeting this week, and if you have something on your mind that you want the EG to take notice of, now would be a good time to act. Feel strongly about something? Try the personal approach.

The MIDP3 spec lead, Mike Milikich, has done an outstanding job getting the individual developer's perspective by making the EG easy for individual JCP members to join. Additionally, many of the manufacturers and operators in the EG have close ties with their companies' developer and partner programs, and speak from the perspective of developer needs. Mike has actively encouraged the surfacing of developer perspectives since MIDP3 began.

Yet with all this work, developers tend to be underrepresented on MIDP3 and Java ME JSRs in general, simply because it is generally not feasible for individuals to travel around the globe to standards meetings on their own dime. Although operators and manufacturers try to speak from the developer perspective, it is not the same as representing yourself.

One way to get heard in MIDP3 is to email the EG at jsr-271-comments@jcp.org. Received just before this week's face-to-face meeting, your email will be fresh on the minds of the EG members.

But if you really want to make a point, make a personal connection with a like-minded EG member. Take a look at the list of participants on the JSR's home page at jcp.org. Google any individuals you find and shoot them an email, or better yet, dig further, find a phone number, and call them. You'd be surprised how few people actually do this. It really does make a difference and makes traveling to these meetings that much more worthwhile.

Remember that individual EG members cannot represent other individuals (and please don't share any of your IP), but we most certainly can raise points that we feel are important to the community. The personal approach is hardly used yet is one of the most effective methods for getting your voice heard.

MIDP3 EG face-to-face wraps up

Another MIDP3 Expert Group face-to-face meeting wrapped up today. I have to say, although I went as an individual JCP member and paid my own way to Spain, it was worth every moment - the friendships, the networking, the feeling of accomplishment as we worked through spec issues, dusting off high school foreign language skills, being able to have intelligent conversations with geniuses in the mobile space... it was wonderful. There was a LOT of working going on - many people tired if not burned out from all the travel, the 9 1/2 hour days, the office email that one must attend to back at the hotel - all of this making the time after hours that much more anticipated.

Speaking of after-hours activities, unlike other standards bodies, you don't find JSR EGs running off to boondoggles in the French Riviera or Thailand - they generally meet in ordinary cities in Europe or somewhere in the US, alternating continents each meeting. Someone from the EG volunteers to host a meeting in one of their offices, pays for food to be catered, and takes folks out to dinner one night for the "EG Dinner" (which prospective EG hosts should know is NOT mandatory.)

Before and after meetings, some of us explore the city we've come to for our meetings...]]>
Our meetings this week wrapped up with a walking tour and bar hop of Madrid led by Oscar Gutierrez, the representative from Vodafone Spain, and we were fortunate enough to have Kay Glahn, one of the spec leads from MSA, join us. I think we were better off than many tourists who do not have the luxury of Spanish tour guide to show us where the locals eat and drink, away from the tourists and their trappings.

Tomorrow it is time for skiing... yes, skiing, in Madrid, 85-degree-summertime-Madrid. I have most of the day free before my flight home, and a few of us are going to head to Europe's largest indoor ski slope, kept at -2C all year long, located just outside Madrid.

Why am I going on about this? Because I was the only individual developer at the meeting. We really need more developers providing the developer perspective to standards discussions. Everyone else was corporate, as in operators and manufacturers. We developers cannot just sit back and monitor KVM-INTEREST for what's going on, we need to get involved by attending these meetings and asking for what we need. It is definitely not us-versus-them - it turns out that we're all thinking very much alike. It's a matter of explaining our side of the story.

I hope I have given a few folks some additional incentives to monitor JSRs and sign up to go when there is a call for participants at an expert group face-to-face meeting. Once again, it's easy for me to snap pictures and show you the after-hours highlights. A lot of work gets done in the meetings - we're talking 9 to 10 hour sessions - and after all this, people have to get back to their hotels and check the days' email and do the other work they're expected to do. But it sure helps when you can take a little time out of a busy EG schedule and relax.

Clarifying the MIDP3 pauseApp proposal

The intent of this post is to summarize and address the developer comments regarding the MIDP3 proposal to deprecate pauseApp, and hopefully clarify some misconceptions about the proposal. I want to thank Enrique Ortiz for using his position in the industry to help get the message on this topic out to developers via his blog and post to KVM-INTEREST which is reflected in the Mobile & Embedded forums at java.net.

Mike Milikich, the MIDP3 EG lead, has asked for developer input on two specific questions:

What percentage of your applications implement pauseApp, and for the ones that do, what functionality is present in pauseApp?
For any applications that implement pauseApp, what would be the effect of running that application in an environment in which pauseApp was never called?

If you don't make it through this rather long post, these are the two most pressing questions at this point.

Also, I will be going to next week's MIDP3 face-to-face meeting. If there is a particular aspect of this topic or any other you would like me to emphasize, post a reply, or call me at +1-703-771-8527. The developer community is underrepresented in face-to-face meetings because of the substantial time and financial burden that attending these presents, and frankly we need more of the "street smarts" experience of developers.

What is driving the proposal?

The MIDP3 pauseApp proposal revolves around these points:]]>

MIDP3 is introducing new APIs to provide better homes for the functionality that implementers have burdened pauseApp with in non-portable ways.

MIDP3 is also introducing concurrency, and is trying to eliminate the ability of a platform to place a MIDlet into a PAUSED state (that is, cease providing CPU time to threads, or arbitrarily pull resources).

The proposal proponents wish to force implementers to stop the overloading of pauseApp, which is creating fragmentation between devices, by closing this loophole, while providing standardized methods to accomplish the same tasks.

The Expert Group wishes to learn all the different ways that developers have been using pauseApp so that no one gets left out in the cold, either by addressing concerns through these new APIs, or if the impact is severe enough, dropping the proposal.

The EG literally realized that when these were accomplished, there would be nothing left for pauseApp or the related methods to do. Therefore, pauseApp, resumeRequest, and notifyPaused would essentially have nothing but a legacy purpose, and could become no-ops for applications written for MIDP3.

And to the legacy point, the EG has discussed whether or not implementations should be allowed to continue to provide legacy functionality for these APIs, if the MIDlet is labeled "MIDP-2.x" or earlier. It's totally legal by the MIDP1/2 spec and TCK to make these methods no-ops (and this is what Nokia does); if MIDP3 mandates this for ALL implementations, how much of a problem will it be for you to port your application to a MIDP3 device?

Is the EG "removing" pauseApp?

No! The interpretation of the word "deprecate" has turned this into a real debate. The proposal includes the use of the deprecation as a tool to remind people that this has occurred via compiler warnings - not to remove the API. We're thinking more along the lines of this definition, keeping in mind that eventual removal of the API is not going to happen any time soon because of the huge impact this would have on existing applications.

I'm afraid that I inadvertently started the idea that the EG was removing pauseApp with the title of my original post (Message from MIDP3: Goodbye pauseApp!) What I meant was, goodbye to all the issues it has caused, and I'll be happy never to have to include an empty pauseApp method in my MIDlets again. (The new API is removing the abstract modifier to make that possible.)

What is replacing the functionality that pauseApp is overloaded with?

The MIDP1/2 specification provides the pauseApp, notifyPaused and resumeRequest methods for the following purposes:

To allow a MIDlet to be signaled that the implementation has placed it into the PAUSED state
To allow a MIDlet to signal that is wishes to be placed into the PAUSED state
To allow a MIDlet to signal that it wishes to be removed from the PAUSED state

This begs the question, what exactly does it mean to be PAUSED. The answer has been subject to different interpretations since MIDP 1.0. In my original post, I pointed out that the PAUSED state existed so that early implementers would be better able to use MIDP on platforms not originally designed for it. If an implementation being built in 2000 really needed CPU time for a phone call, it could simply call pauseApp, halt execution, take back memory and other resources, and handle the phone call.

In 2007, handsets are designed with Java ME in mind, and many implementers have solutions to the phone call problem. MIDP3 is introducing concurrency so a MIDlet's thread can execute even when it does not have the display and other MIDlets are running.

There is an argument that the PAUSED state must exist to allow MIDP3 to run on the lowest common denominator of devices, but if you look at where MIDP has gone since MIDP1, it has basically priced itself out of that market. MIDP has found its niche and it is not at the bottom of the market. New VMs such as the Squawk VM are filling that gap. I'd be happy to argue about this in a different thread.

So with no PAUSED state (outside of that prior to the first call of startApp), that leaves us with the ways pauseApp has been overloaded by implementers in non-portable ways:

As a signal that the app is about to lose the UI, as during an incoming phone call
And in other ways that implementations are leveraging pauseApp. This is a key area where the EG is trying to find out if the APIs in MIDP3 are covering these cases sufficiently - please provide these cases!

pauseApp provided me with a way of doing (x) - how can I do that in MIDP3?

The signal that the app is about to lose the display for a phone call is one interpretation that permeates the iDEN platform, which was the first US Java ME platform launched in 2001. The low level lcdui class Canvas provides hideNotify as a notification that you've lost the display, but there was no equivalent high level API notification - you had to poll Screen.isShown, which was possible in the PAUSED state on the iDEN platform (iDEN has allowed threads to run in PAUSED state since alpha in 2000) but this is kind of hard to do when threads are stopped in other platforms. Providing a way to be notified of display state changes is a major feature addition in MIDP3 that was not ready in time for the early draft review, but is planned for the upcoming public review. This is one area where MIDP3 is replacing functionality in pauseApp that is not implemented consistently across the ME universe.

As far as detecting an incoming phone call, on iDEN that was a guess, because pauseApp could be called for a number of reasons, including when the user actually paused the application through the device's application manager.

As noted in the discussion, there are many other reasons that a phone may lose control of the display. Some of these can be handled through the new mechanisms in MIDP3 or existing mechanisms in other JSRs (Bluetooth, etc) and hopefully (a point for another entire debate!) these implementers get it right and the TCKs catch where they got it wrong. The new javax.microedition.event package should be of great help here. The EventData class includes things like BATTERY_LEVEL, EXTERNAL_POWER, and VOICE_CALL (events outside of MIDP's purview, such as those related to SMS, are the domain of other JSRs.) Developers should look at other fields in this class by downloading the early draft review from the JSR-271 page at JCP.org, and let the EG know what else they'd like to see.

Won't this break compatibility between MIDP 1/2 and MIDP3?

Actually, the way pauseApp is currently defined and implemented, we have incompatibility today between platforms, with developers creating porting layers run by porting teams to deal with these incompatibilities whenever a new device comes out. Defining new APIs that replace the non-portable behaviors in pauseApp with standardized behavior is the whole point of this.

The pauseApp mechanism is so loosely defined that nothing today is preventing an implementer from changing their pauseApp behavior when they move to a new platform (and I know cases where this has happened.) They will still pass the TCK with flying colors. There is no way to bolster the TCK the way these APIs are currently defined - they were deliberately left open to allow the widest range of platforms to implement MIDP, back at the turn of the century.

There is a question that developers need to weigh in on, and that is whether or not implementers of MIDP3 would be allowed (or forced) to retain compatibility with applications written for MIDP 1/2 platforms. In other words, if an app was written to deal with a specific platform's implementation of pauseApp, it might have a chance of running on a new MIDP3 platform without modification.

While this sounds good on the outside, there are several major issues with this, including:

This requirement could greatly increase the complexity of certain MIDP3 implementations and emulators.
The requirement for apps to work in MIDP3 as they did in MIDP2 is not testable by a TCK.
A particular implementer may have several implementations that are legal but incompatible, so the requirement would be impossible to comply with.

The EG is strongly leaning towards the simpler solution of making this change apply regardless of how a MIDlet is labeled. This means that if your existing application is, for example, counting on pauseApp for a particular notification and will misbehave if pauseApp is not called when that behavior occurs, then your application will need to be rewritten before it will run on that new device.

It would be good to know the extent of this issue, and how bad it really is given that many apps (unfortunately) need to be touched whenever a new version of a device comes out. Some posters commented that a pauseApp-related change was relatively minor compared to some of the other porting issues they encounter when bringing a new device onboard.

What areas has this thread suggested that the EG has to look into more/respond to?

One area is the interruption of MIDlets (loss of display and/or keyboard) to various notifications by the platform:

Presentation of a permissions request
Low battery
Out of coverage
Incoming call - request to accept it
Incoming notifications
Bluetooth and other device changes

MIDP3 does provide the javax.microedition.event API for notification of events, but there is no coupling between the new Display notifications to allow a MIDlet to determine why they were interrupted.

I feel that trying to find a cause for every interruption leads an exercise in boiling the ocean. Still, it is useful to provide some coupling between the DisplayListener API and the event mechanism if for no other reason than knowing that it is the same event coming from two different perspectives, possibly in different order on different platforms.

How to couple events from other JSRs that also lead to a DisplayListener event is another question.

In conclusion... isn't this a minor issue in the bigger fragmentation picture?

Simon Maspero's response to Hartti Suomela's Forum Nokia Blog on this topic summed it up well:

...the point is not really to know if pauseApp is necessary or not, but to make sure that every device will behave the same.

Fragmentation spans many areas of the Java ME ecosystem and needs to be addressed in a coordinated way. This will help in one small way, but there are many other points of fragmentation, including broken behavior, operator certification hurdles, barriers to certain APIs, user interface differences between MIDlet and native, phones interrupting apps for who knows what, etc. And, many people will tell you that not all fragmentation is bad. Some of it comes from the necessity of providing new features before a standard can be wrapped around them. Some of it comes from the need to provide flexibility to allow implementations to flourish. pauseApp is an example of the latter whose time has passed and is no longer needed.

MSA addresses certain points of fragmentation but only a subset of those encountered. Individual companies like Sun, every company who is on MSA, and other standards organizations like 3G Americas are also very concerned and making progress.

But the largest fragmentation issues cannot be solved by a single company or JSR. The issue of fragmentation must be taken up at the industry level and be a legitimate coordinated effort involving all in the Java ME ecosystem. It especially needs to involve developers who don't have time to participate in EGs because they are too busy getting code out the door. It needs to solve the problem of getting input from these key people, who can't afford to fly around the world to meetings on their own dime.

It cannot be solved in closed JSRs where none of the members are developers working at the street level where these issues are encountered. It is the biggest threat to Java ME because it stops budding developers in their tracks and eats away at the investment we've made into Java ME.

Finally, I want to say that actively bringing dialog from an active JSR expert group, which is closed, out to the community in order to get feedback, is something that MIDP has done before, and is an experiment that should happen more often. Thanks to Mike Milikich, the 271 spec lead, for opening this up and letting us bypass the normal JCP rules that allow JSRs to operate in a closed manner.

Message from MIDP3: Goodbye pauseApp!

[Clarification and developer bias: The EG is providing finer grained notification functionality such as DisplayListener (see the EDR javadoc) that will move some of the platform-dependent functionality overloaded into pauseApp (and NOT defined in the MIDP specification), into platform-independent APIs whose behavior IS defined by the specification. In other words, if the capability provided by pauseApp is better provided elsewhere, isn't the natural result that can we reduce the confusion and fragmentation around pauseApp by making it a no-op (as implementations like Nokia's already do today?)

So the question to developers could be, what platform-specific stuff are developers doing with pauseApp that MIDP3 may need to accommodate if pauseApp is never called?

And if that capability is provided elsewhere, pauseApp's remaining function is to allow the environment to notify the MIDlet that the MIDlet is about to lose CPU time. Do we, as developers really implementations be able to "pause" us, to stop our threads, given that today's platforms are powerful enough to allow them to get at least some CPU time? What reasons might we want to keep the Paused state around? ]

To help reduce fragmentation, the MIDP3 expert group wishes to deprecate MIDlet.pauseApp, and wants your input - especially if you are negatively impacted. The idea is not to remove the API, but simply require that implementations never call it, and eliminate the need for developers to provide an empty pauseApp method.

There are several questions that the expert group would like the community to weigh in on. They are:

Are there any VM/platform providers who will be severely impacted by this change? That is, is pauseApp woven into their implementations so deeply, or required by the nature of their platforms, that not being able to signal a MIDlet with pauseApp would create fundamental issues?
Are there any software developers whose applications would require fundamental and expensive changes in order to run in an environment where pauseApp is never called?
Are there any other impacts the EG should consider?
Should this apply regardless of whether or not the MIDlet is designated MIDP3? That is, should platforms be permitted to provide legacy pauseApp functionality to MIDlets indicating MicroEdition-Profile less than MIDP-3.0?

I suppose we're not expecting a huge outcry of opinion on this, but if you have something to say about it or are impacted, definitely say so. You can leave a comment here and/or email the MIDP3 EG at jsr-271-comments@jcp.org. This post is rather long, so if something jumps out at you, comment on it; don't feel you need to read or understand the whole thing before doing so.

A copy of the proposed API javadoc can be found at
http://seansheedy.com/standards/midp3/pauseApp/MIDlet.html.

]]> (You can scroll to the bottom of this post for the deprecation proposal from Roger Riggs as posted in the MIDP3 artifacts tracker.)

Why does pauseApp exist in the first place?

One of the reasons this API exists is to provide a tool to help applications deal with platforms that cannot concurrently run MIDlets and handle key native events such as phone calls. By calling pauseApp, an environment informs the MIDlet about entry into the PAUSED state. What the platform requires of a MIDlet is not defined in MIDP and is platform dependent. In the PAUSED state, a platform may deny a MIDlet all CPU time, or require that it give up resources so that the platform can do other things - or none of these. pauseApp was designed to give early implementers one less excuse for not including MIDP in those early mobile devices!

Why is pauseApp no longer considered technically necessary?

As Moore's Law has made devices more sophisticated, the need for this accommodation has diminished. Platforms are able to perform multiple tasks without completely shutting down MIDlets. The possibility of errant MIDlets forced designers to place the responsibility of releasing resources on the implementation, not the application. APIs now provide listeners to help MIDlets learn about and manage the acquisition and loss of resources. MIDlets are expected to register listeners for notification of events that impact them. Essentially, pauseApp is no longer needed in MIDP 3.0. However, this dialog arises out of the MIDP3 EG's concern about backwards compatibility issues.

How do various platforms handle pauseApp today?

pauseApp has been a source of confusion and misinterpretation for both implementers and developers. The implementations of pauseApp are quite varied. On some devices, pauseApp signals that the MIDlet has a limited amount of time to release resources before losing all CPU time. In others, pauseApp is called when the system is about to lose the display and keypad to some other process like an incoming call, but all threads created by the MIDlet continue to execute. Finally, some implementations never call pauseApp throughout the life of the MIDlet!

What else is impacted?

The deprecation includes resumeRequest, which is defined as merely a hint from the application that it be resumed from a paused state. resumeRequest also has varied implementations and generally cannot be relied upon across platforms - sometimes it does nothing, and sometimes it is obtrusive. With pauseApp eliminated, resumeRequest is no longer necessary. notifyPaused also has little meaning.

Roger’s proposal also requires that an IllegalStateException be thrown when MIDlets labeled MIDP 3 call these methods. Is it necessary that these methods throw an exception, possibly introducing instability in applications that have been “upgraded” to use some other MIDP3 APIs? Is it sufficient for notifyPaused and resumeRequest to simply return without doing anything? As these were only hints anyway, simply ignoring them is an option.

pauseApp's abstract qualifier will be removed so that developers no longer need to include an empty pauseApp method in their MIDlet class.

Why do some of us want to get rid of pauseApp?

This opportunity to eliminate pauseApp is an opportunity to decrease platform fragmentation. EG members seem to prefer to be rid of pauseApp and all the baggage it carries, regardless of what MIDP version a MIDlet is written for. Deprecating pauseApp for ALL MIDlets running on MIDP3 platforms would accelerate the elimination of this point of fragmentation. But it could break some existing apps. Is the value of eliminating this point of fragmentation greater the cost of rewriting some (how many?) applications that may be broken by this change (in other words, to paraphrase the great Spock, are the needs of the many greater than the needs of the few?)

What is the impact to VM/platform implementers?

Actually, because of the wide range of behaviors for present implementations, this is the key area where the EG wants to get feedback.

One aspect under debate is whether or not deprecation should apply to all MIDlets, or only those labeled as MIDP-3.0 MIDlets. For example, at least one implementation contains an LOC (Licensee Open Class - aka OEM class or proprietary API) that depends on the present pauseApp functionality. As explained to the EG, this implementation requires MIDlets to release resources provided by the LOC when pauseApp is called. This is consistent with the definition of pauseApp. Deprecating pauseApp only for MIDlets labeled MIDP3 would allow these implementations to retain this behavior for MIDP 2 content.

However, there is nothing preventing an implementer from ignoring this legacy behavior anyway. While the present proposal requires that MIDP 1 & 2 applications “continue to function as before", it’s impractical, if not impossible, to build a TCK test that can verify this. In fact, an implementation that formerly called pauseApp for MIDP2 implementations, when moving to MIDP3, could cease calling pauseApp altogether, and would still pass the TCK and be fully MIDPx compliant. Calls to notifyPaused and resumeRequest could simply do nothing, if these implementations never call pauseApp.

What is the impact to developers?

The EG seems to feel that the impact to developers will generally be positive. Many developers already have to deal with different implementations of pauseApp, and this change is simplifying.

Developers who are writing apps for many devices are probably leveraging porting layers. Whenever a new handset appears on their radar, a porting team adapts the porting layer to accommodate it. The porting teams should find the elimination of calls to pauseApp simplifying. (Question to porting teams: is this simplifying?)

Apps that may be most impacted are those written for devices that were hand chosen for a particular set of capabilities. A subset of these will depend on pauseApp to be called for some device-specific reason or another, that the EG is not aware of. The EG would like these developers to speak up and let us know how they are impacted.

For example, a recurring theme that I saw on the iDEN platform, was that some developers wanted to see their application return promptly to the display after a phone call completed. On iDEN platforms, these applications were using pauseApp as a signal that they were losing the display, and employed strategies such as repeatedly calling resumeRequest in order to get their application surfaced again when the call ended. This was a very device-dependent solution, enabled by this particular interpretation/implementation of pauseApp and resumeRequest.

Another example is that some developers use notifyPaused as a means of telling the implementation that they no longer need the display. However, this functionality is not defined nor guaranteed by the MIDP1/2 specifications, and in fact does not work in all devices. The lcdui package in MIDP3 will better define how applications can indicate that they no longer need a display.

What is the question the EG has for developers?

The question to developers is, how are you impacted, and would you prefer reducing fragmentation by

deprecating pauseApp only for MIDP3-labeled apps
deprecating pauseApp regardless of version, or
not deprecating pauseApp?

and why?

What are your opinions? What other impacts have we forgotten?

------

Roger Riggs' pauseApp deprecation proposal:

The function of MIDlet.pauseApp has been the source of confusion to developers and implementors and seems to have
limited value. Further its use by the AMS has been a cause of fragmentation across implementations.

On several occasions it has been suggested to deprecate use of pauseApp.

In order to more fully evaluate the changes and the impact the attachment contains javadoc with the updates needed to
MIDlet class and package documentation and diffs between the proposed javadoc and the previous javadoc. Please read and
comment.

For convenience the proposal is:

For backward compatibility with MIDP 2.0, the function of pauseApp cannot be removed completely. MIDP 2.0 applications
must continue to function as before.
Changes (for MIDP 3.0 MIDlet suites only):
- The pauseApp method MUST not be called.
- The pauseApp method will be changed to remove the “abstract” qualifier so MIDlet suites do not need to provide an
  empty pauseApp method.
- The notifyPaused and resumeRequest methods are modified throw IllegalStateException (only for MIDP 3.0 MIDlets)
- However, the package documentation must retain a complete description of the PAUSED state and required behavior for
  MIDP 2.0 MIDlet suites.
- Also included in the diffs; deprecate of the MIDlet.checkPermission method in favor of AccessController.
  checkPermission (for MIDP 3.0 MIDlet suites only).