Janice J. Heiss's Blog

J2ME Archives

Dancing While Gaming – Multitasking VMs on Real Devices

The JavaOne session, “Dancing While Gaming – Multitasking VMs on Real Devices”, focused on helping developers gain a wider and deeper understanding of the Multitasking VM implementation on real devices based on the interaction between Sun and Samsung who are engaged in a joint collaboration through the work of Jae Hyou Lee of Samsung Electronics and Yaniv Shani of Sun.

MVM deployment is based on the Sun JWC1.1.3 (Java Wireless Client 1.1.3) which comes with multi-tasking capabilities. The MVM deploys most of the MSA (Mobile Service Architecture) JSRs. It has been tested with hundreds of games and applications from leading content providers.

So what is MVM? The Multi-Tasking Virtual Machine is a technology that allows the end-user to execute multiple Java applications simultaneously. MVM enables preferred and rich applications like MP3 playing, Instant Messaging and mail clients. Users of MVM want to run multiple Java applications simultaneously and want to be able to switch applications very rapidly. The typical MVM player might be a teenager who loves to listen to MP3 players, chat with friends over the Net, and play video games simultaneously. Multi-tasking capabilities enable all these activities to be conducted on the same device at the same time.

The Isolation Model

To execute multiple applications, the isolation model was introduced in the VM. Each application runs in its own logical Virtual Machine environment, called an isolate. Within the JVM, each isolate is represented as a task. A task consists of one or more Java threads of execution. All tasks execute in the same single OS task context. All tasks share constant data structure and runtime support functions. Each task encapsulates non-sharable program state MVM.

“In the CLCD space, we would like to address limited operating systems that lack native process modeling support,” said Shani. “This is the main reason why in the CLCD space, the VM implementation conducts both the Java task and Java thread scheduling internally within the VM.”

Implementation Details

Each task has a separate, private copy of the static variables of all the loaded Java classes. Static variables are only global to all threads within a particular task. Each task maintains a private copy of ‘count’. The Class representation is shared. Synchronization has to be task private. Threads in one task cannot block threads in another task. Blocking threads across tasks isn’t allowed.

Resource Management

The low-level part of the VM is responsible for all CPU time and memory resources in order to ensure that all tasks get their “fair” share. Other resources, such as network bandwidth and communication ports are managed by the profile layer.

Resource Management – Scheduling

Tasks and threads are scheduled internally by the JVM. Fair scheduling algorithm is used for task scheduling in order to prevent one task from taking disproportionate CPU time and causing other tasks to stop. The ‘isolate’ class offers a set of APIs to modify the priority levels for each task. In most cases MIDlets that execute in the foreground get to have more CPU cycles compared to others that run in the background.

Resource Management – Memory

The VM requires a large consecutive memory block from the underlying OS. Allocations for all tasks are made from this same global heap region. The VM has a bookkeeping mechanism that accounts for each task’s total heap memory consumption. The VM injects OutOfMemoryError as needed to inform tasks that heap space has become scarce.

The Profile Level

The presentation turned to what must be done at the profile level to transform their implementation into one capable of multi-tasking. They have been using Sun’s latest JWC1.1.3, which comes with multi-tasking capabilities. It allows only a single MIDlet to execute in the foreground at a certain time.

A MIDlet application is said to be in the foreground when:

-- Its displayable controls the display.
-- It handles events from the user input mechanism since they are automatically routed to it.

An application is in the background when:

-- Its displayable does not control the display.
-- It does not handle the user input mechanism.
-- Zero or more MIDlets can execute in the Background
at a time.

The application manager is critical for task switching. They use a resident Java MIDlet for the application manager; it is launched in the background when the device ends its boot phase. When users switch it to the foreground, they see a list of all running applications and can switch between them quickly. The app manager also displays background MIDlet alerts, when one wants to drive the user intention.

They next displayed some demos showing how to switch MIDlets to the background and switching between MIDlet states.

Switching a MIDlet to the Background

A short ‘hot’ key press is used for switching between running apps.
Upon a short ‘hot' key press the foreground MIDlet will be
placed in the background and the main device menu will be
displayed on the screen.

Upon a long ‘Hot’ key press the foreground MIDlet will be
placed in the background and the ‘Application manager’ will
be displayed on the screen. The user can see the list of running apps and can switch to another app.

Upon calling setCurrent(null) the MIDlet will be placed in the
background and the ‘idle’ screen of the device will be
displayed. This enables MIDlets to place themselves in the background.

Interrupting the User

A demo of a user experience focused on ways to interrupt the user when a background MIDlet wants to drive the user’s attention.

-- Background MIDlet calls Display.setCurrent(alert)
-- Background MIDlet tries to access a protected API and a security dialog is prompted.
-- Noteworthy event occurred in the background, e.g., Incoming IM while playing a game.

Managing Resources

The session turned to resource management. In a single-tasking environment, the MIDlet has access to all of the available resources. In a multitasking environment, MIDlets have to compete for available resources

Resource management mechanisms include:

Reservation -- A reservation mechanism sets aside a certain amount of a resource for a MIDlet and keeps it available for that MIDlet and that MIDlet alone

Quota -- A quota mechanism permits a MIDlet to allocate up to a
certain amount of a resource. If the MIDlet attempts to allocate more resources than its quota, the attempt fails, even if resources are actually available on the system.

Revocation -- The system can revoke a resource from a MIDlet and give it to another MIDlet. Resource revocation examples include CPU cycles, display access, and audio playback. Resource revocation doesn’t always have a dramatic effect on
the MIDlet’s execution

The session went on to cover LCD display, and sound issues and offered guidelines for resource policy selection. The key points were:

-- Resource policy definition relies on the underlying platform
capabilities and requirements.

-- A fixed partition policy should be used for fundamental resources that are required by the majority of the applications.

-- An open policy should be used for resources that are used by
specific MIDlets and have limited availability (e.g. Bluetooth, Location).

-- The Display and Sound requires special handling.

Multitasking Safety

Regarding safety, the native JVM task can run native code for any task, hence the code must be made aware of the task on whose behalf it is being called and possibly allocating resources. When the task context is established, the code is considered multitasking safe. Sun’s JWC software is multitasking safe.

The session closed with an exploration of such topics as static and global variables, issues of native code execution, MIDlet guidelines, and basic resource awareness principles.

The take-home message from my perspective is that MVM technology fits nicely with what is happening, for better or worse, not only in youth culture in the US and much of the world, but among older generations. We are living speeded up lives and engage in multi-tasking, out of both necessity and pleasure and we want to be able to do it comfortably and quickly from the same device.

“Dancing while gaming” is here to stay and Sun and Samsung are on to something. More power to them.

Where the Software Met the Road

It's day two of JavaOne '06, and I’m over at the Slot Car Racing Programming Challenge area, where the track is heating up. Developers are standing in line, some for the 20th or 30th time! to see if their latest tweaks will pan out as they hoped. A lot has been happening here. The ABC Morning News guys were here with their cameras. Developers are taken with the chance of being onstage with James Gosling and spending some time with him. Gosling himself has been trying out the slot cars off and on for a few days. Some contestants admit that the slot car race brings them back to their childhood. Developer aren’t exactly closet game junkies.

I approach one, who asked to be referred to as “Turtle,” and ask him if this is harder than he expected.

“It’s very difficult,” says Turtle. (Yes, the name gives away something about his times.) “I have no experience with real-time. There is a lot of tuning, a lot of unknown variables – every car runs a little different. So even if you tweak it right for one car, it may not work for the next. This is about my 15th attempt and I’ve missed two sessions already.” (I’m beginning to understand why he doesn’t want his real name used.) “I’m getting better each time, and finally made it into turn three.”

When I ask him how this is affecting him he says he wants to build a race track in his home. “This is a lot of fun. Once your car runs, the times are posted for your machine and you can figure out how fast your car is going at each sensor point. You could copy and paste this if you wanted, but it would be preferable if they gave you all of your races together so you could analyze all of the numbers.”

Turtle informs me that the man standing next to him is tied for the lead. His name is Peter Whitfield.

“I’m probably the worst Java programmer here – I’m more of a manager than a programmer,” admits Whitfield. “It will be embarrassing if anyone looks at my code. I have no experience with real-time Java, though 20 years ago, I worked with real-time systems. This is a lot of fun and a good opportunity to see how the real-time Java stuff works. It’s easy.” Different strokes for different folks.

Peter -- who is quite generous with his time as he keeps eyeing the monitor above the track listing the names of who's in the lead along with who is currently racing -- completed the race in 25.49 seconds somewhere around his 20th trial and estimates that maybe 10 or 12 people out of 100 have made it to the finish line without crashing. Clearly the hardest thing is just making it around the track. I ask him the secret to his success.

“Keep it simple -- do the absolute minimum necessary to get the result,” Whitfield explains. “There are guys doing some very sophisticated algorithms but I have not actually modified the sample algorithm at all, I’ve just been changing the parameters. Nothing in this application has been hard. I have an engineering background rather than a pure software development background and this is about manipulating voltages to control the speed, and that’s second nature to me. So my background helps. I have only now started changing the way the real-time stuff works. Until now, I’ve only changed the parameters of the way the speed changes as it goes around the track, so a lot if it involves trying to figure out how, if I change the voltage in one place, it affects the speed in another. That’s what people are missing -- they don’t understand how if they go too fast in one part of the track, it will make the car unstable later. The Java side of it is very easy to pick up because we are given a sample program. We have not had to set up the framework for it. Real-time programming is about understanding physical systems; it's not as abstract as writing UI software. So you need to understand the physics of what you are trying to do and the timing.”

I ask if he spends a lot of time visually contemplating the track. “Absolutely – I stare at the track trying to understand where things happen. It’s a spatial thing and physical thing. I can run the same program 10 times and get different results each time. There are 5 different cars that all perform differently. We don’t know which car they will run, so we can’t tune it for a specific car. The challenge is to write software that is in tune with the reality of what you are trying to make happen.”

Peter went on to say: “I learned that I don’t program in Java often enough, so I hope people aren’t looking over my shoulder when I look up Google to figure out how I do multi-dimensional arrays. You need to be aware of where you are putting your thread to sleep and where it will wake up and if you are doing it in more than one place you need to think of the different permutations and where it can all happen.”

And what else is Peter doing at JavaOne?.. “I’ve spent way too much time here. I don’t have a boss so I can sit here and just have fun.”

I talk to a few other contestants waiting their turn. Jonathan O’Keefe, who does database programming, is trying out real-time for the first time. And there is Ulrich from Denmark who does web applications and GUI stuff and business logic, and Michael from Germany who does desktop programming and Carl, an American, from a company called Triego Network Security, who manages security information. None have experience with real-time. The line ebbs and flows at various times, but there's always someone racing, and there are a lot folks sitting in front of the machines set up for them on which they work on their real-time programs.

Only Carl admits to big ambitions. “At first,” says Carl, “I was over-complicating it by treating it like a J2ME product, but if you are very conservative about the objects you allocate, you can treat it as a regular Java program. You have to keep it simple and not allocate anything in the loop or you will run out of memory. I’m going to try to knock someone off the leader board -- I’ve just started with the base program so there is a lot of room for improvement.”

Carl sees uses for real-time Java at his company, Triego. “We do some close to real-time work and try hard to keep our GC under control and use a concurrent generational collector, so the real-time Java technology is very interesting. We could find a lot of use for this technology in the work we do. It’s interesting and the applicability of the predictability of execution time is great.”

As I walk away, I see that Robert Chu is tied with Peter Whitfield’s 25.49 seconds time while Richard Yee is third, at 26.18, and Rivera (I don't catch his first name) is fourth at 28.60. I wonder if any of these folks will be in the top three when James Gosling gives his keynote on Friday morning. Time and times will tell.

Stay tuned. The winners will be announced and compete one final time at the Gosling keynote on Friday. Should be a lotta fun!

Finally, here's a story about the Slot Car Racing Programming Challenge yours truly wrote over on java.sun.com:

The 2006 JavaOne Conference Slot Car Racing Programming Challenge
http://java.sun.com/javaone/sf/slot_car.jsp

And, in the form of shameless self-promotion, here's an interview that I did not far back with Sun Distinguished Engineer, Greg Bollella, on real-time Java:

Programming in Real-Time Specification for Java (RTSJ): A Conversation with Distinguished Engineer Greg Bollella
http://java.sun.com/developer/technicalArticles/Interviews/Bollella_qa2.html

Here we go! JavaOne is almost here!

Among the topics currently scheduled for my editorial beat:

-- The Slot Car Programming Challenge wherein developers get to test their skills with Real-Time Java. Standard 1/24 scale cars run around a track with 80 sensors spread along its length with a power supply controlled by an A/D converter driven from a workstation. Contestants have to write software that controls the car while going as fast as possible. The sensors are simple photocell gates like those used to detect paper moving through a printer. The Real-Time Java program senses the track position of a slot car and sets the voltage to the track -- and thus the speed of the car. Attendees with the 3 shortest lap times will have a final run-off during James Gosling’s keynote on Friday.

-- The activities of Tommy, the autonomous, unmanned Java technology-powered robotic dune buggy, who will be making myriad appearances. Tommy's software is built on a Java technology-based platform called the Mobile Autonomous X-bot (MAX) developed by Perrone Robotics, Inc. (PRI). PRI-MAX runs on the Java Platform, Standard Edition (Java SE, formerly known as J2SE), and uses the Java Communications API, while Tommy's microprocessors rely on a hardware-based Java Virtual Machine (JVM) running Java Platform, Micro Edition (Java ME, formerly known as J2ME).

-- The always-lively discussion at the Fireside Chat wherein JavaOne conference Alumni get to engage with Java luminaries.

-- Sun’s Project Tango which addresses interoperability between applications built on Microsoft's Web Services Communications Foundation (WCF, a.k.a. Indigo) and those built with Sun's Java Web Services technologies.

-- SOA and developments in open source.

-- Twelve Reasons to Use NetBeans Software.

Plus numerous other sessions, along with exhibits in the Pavilion and the flexibility to go where the action is. I already feel sleep deprived!

RTSJ and the JavaOne Slot Car Challenge

Conscientious Software

Project Looking Glass: An Expanding Universe on Your Desktop

I'm at TS-7992 where Project Looking Glass (LG3D), a Java technology-based open source project that brings a richer user experience to the desktop through 3D windowing and visualization capabilities, is being presented to an audience of, I guesstimate, 800 people. LG3D sprang from the very creative heart and mind of Sun's Hideya Kawahara. Recognizing that desktops had not changed substantially in 20 years, he set out to make them more aesthetically appealing and powerful. Operating on the assumption that the next user interfaces would be 3D, he initiated a side project that would consume at least two hours a day of his spare time, plus most of his weekends and holidays for more than a year before taking hold at Sun. To put it mildly, it has taken hold. It's the most popular "app" on java.net (http://lg3d.dev.java.net) with 26,600 source code downloads, plus 600 members (and counting) since the 2004 JavaOne Conference where it was open sourced.

So what's the latest? Hideya and Paul Byrne, LG3D project owners, demo'ed a range of 3D images, a music player, scenes in which you could alter the backgrounds with a click, "Alice" an award winning 3D media player (http://alice.dev.java.net) that is the first to utilize the 3D capacity of Looking Glass, and more.

CosmoSchedulerD, a three-dimensional application running on LG3D software, created by developers at the Kyushu Institute of Technology in Japan, won a Duke's Choice Award (http://java.sun.com/javaone/sf/dukes_choice_awards.jsp). As a schedule book, it recreates outer space, with your personal solar system built in by arranging the planets according to their dates. The front of the orbit represents the current time, while the size of the planet symbolizes an appointment's importance, which makes it hard to forget an event even a few light years from now. CosmoSchedulerD contains features that ordinary schedule notebooks don't have, such as automatic scheduling, networking, and a workspace manager. Imaginative desktops seem to inspire even more imaginative apps to be built on them.

(I can't escape the feeling that talking about innovations on a gorgeous 3D desktop is like a donkey carrying a load of books. Have to shake it off. By all means go check out "Philco" running LG3D, the mock-up on the cell phone, and LG3D on a 3D LCD display, and all the rest on the pavilion floor!)

Hideya and Paul gave a brief summary of how to create a "deep" 3D environment. It's built on Java 3D with specialized classes that include a component model, animation system and SceneManager interaction. The LG3D 0.7 release has just arrived. There is now WebStart support (http://lg3d-webstart.dev.java.net) for running the "developers" mode of Looking Glass. It operates in application mode so LG3D can run on top of a user's existing desktop. Java 3D 1.4 now enables performance improvements like shader support. It has Open Solaris support.

In the pipeline is tool integration, a visualization library, and "SwingNode" support. There will be greater inclusion of identity and collaboration features and a more task-oriented UI.

To run it: http://lg3d-webstart.deve.java.net

To get it: http//lg3d-core.dev.java.net

To learn more about LG3D: http://java.sun.com/developer/technicalArticles/javaopensource/plg .html

http://java.sun.com/developer/technicalArticles/J2SE/Desktop/look ingglass/

This work is licensed under a Creative Commons License.