Janice J. Heiss's Blog

Community: Java Games Archives

Dancing While Gaming – Multitasking VMs on Real Devices

The JavaOne session, “Dancing While Gaming – Multitasking VMs on Real Devices”, focused on helping developers gain a wider and deeper understanding of the Multitasking VM implementation on real devices based on the interaction between Sun and Samsung who are engaged in a joint collaboration through the work of Jae Hyou Lee of Samsung Electronics and Yaniv Shani of Sun.

MVM deployment is based on the Sun JWC1.1.3 (Java Wireless Client 1.1.3) which comes with multi-tasking capabilities. The MVM deploys most of the MSA (Mobile Service Architecture) JSRs. It has been tested with hundreds of games and applications from leading content providers.

So what is MVM? The Multi-Tasking Virtual Machine is a technology that allows the end-user to execute multiple Java applications simultaneously. MVM enables preferred and rich applications like MP3 playing, Instant Messaging and mail clients. Users of MVM want to run multiple Java applications simultaneously and want to be able to switch applications very rapidly. The typical MVM player might be a teenager who loves to listen to MP3 players, chat with friends over the Net, and play video games simultaneously. Multi-tasking capabilities enable all these activities to be conducted on the same device at the same time.

The Isolation Model

To execute multiple applications, the isolation model was introduced in the VM. Each application runs in its own logical Virtual Machine environment, called an isolate. Within the JVM, each isolate is represented as a task. A task consists of one or more Java threads of execution. All tasks execute in the same single OS task context. All tasks share constant data structure and runtime support functions. Each task encapsulates non-sharable program state MVM.

“In the CLCD space, we would like to address limited operating systems that lack native process modeling support,” said Shani. “This is the main reason why in the CLCD space, the VM implementation conducts both the Java task and Java thread scheduling internally within the VM.”

Implementation Details

Each task has a separate, private copy of the static variables of all the loaded Java classes. Static variables are only global to all threads within a particular task. Each task maintains a private copy of ‘count’. The Class representation is shared. Synchronization has to be task private. Threads in one task cannot block threads in another task. Blocking threads across tasks isn’t allowed.

Resource Management

The low-level part of the VM is responsible for all CPU time and memory resources in order to ensure that all tasks get their “fair” share. Other resources, such as network bandwidth and communication ports are managed by the profile layer.

Resource Management – Scheduling

Tasks and threads are scheduled internally by the JVM. Fair scheduling algorithm is used for task scheduling in order to prevent one task from taking disproportionate CPU time and causing other tasks to stop. The ‘isolate’ class offers a set of APIs to modify the priority levels for each task. In most cases MIDlets that execute in the foreground get to have more CPU cycles compared to others that run in the background.

Resource Management – Memory

The VM requires a large consecutive memory block from the underlying OS. Allocations for all tasks are made from this same global heap region. The VM has a bookkeeping mechanism that accounts for each task’s total heap memory consumption. The VM injects OutOfMemoryError as needed to inform tasks that heap space has become scarce.

The Profile Level

The presentation turned to what must be done at the profile level to transform their implementation into one capable of multi-tasking. They have been using Sun’s latest JWC1.1.3, which comes with multi-tasking capabilities. It allows only a single MIDlet to execute in the foreground at a certain time.

A MIDlet application is said to be in the foreground when:

-- Its displayable controls the display.
-- It handles events from the user input mechanism since they are automatically routed to it.

An application is in the background when:

-- Its displayable does not control the display.
-- It does not handle the user input mechanism.
-- Zero or more MIDlets can execute in the Background
at a time.

The application manager is critical for task switching. They use a resident Java MIDlet for the application manager; it is launched in the background when the device ends its boot phase. When users switch it to the foreground, they see a list of all running applications and can switch between them quickly. The app manager also displays background MIDlet alerts, when one wants to drive the user intention.

They next displayed some demos showing how to switch MIDlets to the background and switching between MIDlet states.

Switching a MIDlet to the Background

A short ‘hot’ key press is used for switching between running apps.
Upon a short ‘hot' key press the foreground MIDlet will be
placed in the background and the main device menu will be
displayed on the screen.

Upon a long ‘Hot’ key press the foreground MIDlet will be
placed in the background and the ‘Application manager’ will
be displayed on the screen. The user can see the list of running apps and can switch to another app.

Upon calling setCurrent(null) the MIDlet will be placed in the
background and the ‘idle’ screen of the device will be
displayed. This enables MIDlets to place themselves in the background.

Interrupting the User

A demo of a user experience focused on ways to interrupt the user when a background MIDlet wants to drive the user’s attention.

-- Background MIDlet calls Display.setCurrent(alert)
-- Background MIDlet tries to access a protected API and a security dialog is prompted.
-- Noteworthy event occurred in the background, e.g., Incoming IM while playing a game.

Managing Resources

The session turned to resource management. In a single-tasking environment, the MIDlet has access to all of the available resources. In a multitasking environment, MIDlets have to compete for available resources

Resource management mechanisms include:

Reservation -- A reservation mechanism sets aside a certain amount of a resource for a MIDlet and keeps it available for that MIDlet and that MIDlet alone

Quota -- A quota mechanism permits a MIDlet to allocate up to a
certain amount of a resource. If the MIDlet attempts to allocate more resources than its quota, the attempt fails, even if resources are actually available on the system.

Revocation -- The system can revoke a resource from a MIDlet and give it to another MIDlet. Resource revocation examples include CPU cycles, display access, and audio playback. Resource revocation doesn’t always have a dramatic effect on
the MIDlet’s execution

The session went on to cover LCD display, and sound issues and offered guidelines for resource policy selection. The key points were:

-- Resource policy definition relies on the underlying platform
capabilities and requirements.

-- A fixed partition policy should be used for fundamental resources that are required by the majority of the applications.

-- An open policy should be used for resources that are used by
specific MIDlets and have limited availability (e.g. Bluetooth, Location).

-- The Display and Sound requires special handling.

Multitasking Safety

Regarding safety, the native JVM task can run native code for any task, hence the code must be made aware of the task on whose behalf it is being called and possibly allocating resources. When the task context is established, the code is considered multitasking safe. Sun’s JWC software is multitasking safe.

The session closed with an exploration of such topics as static and global variables, issues of native code execution, MIDlet guidelines, and basic resource awareness principles.

The take-home message from my perspective is that MVM technology fits nicely with what is happening, for better or worse, not only in youth culture in the US and much of the world, but among older generations. We are living speeded up lives and engage in multi-tasking, out of both necessity and pleasure and we want to be able to do it comfortably and quickly from the same device.

“Dancing while gaming” is here to stay and Sun and Samsung are on to something. More power to them.

Where the Software Met the Road

It's day two of JavaOne '06, and I’m over at the Slot Car Racing Programming Challenge area, where the track is heating up. Developers are standing in line, some for the 20th or 30th time! to see if their latest tweaks will pan out as they hoped. A lot has been happening here. The ABC Morning News guys were here with their cameras. Developers are taken with the chance of being onstage with James Gosling and spending some time with him. Gosling himself has been trying out the slot cars off and on for a few days. Some contestants admit that the slot car race brings them back to their childhood. Developer aren’t exactly closet game junkies.

I approach one, who asked to be referred to as “Turtle,” and ask him if this is harder than he expected.

“It’s very difficult,” says Turtle. (Yes, the name gives away something about his times.) “I have no experience with real-time. There is a lot of tuning, a lot of unknown variables – every car runs a little different. So even if you tweak it right for one car, it may not work for the next. This is about my 15th attempt and I’ve missed two sessions already.” (I’m beginning to understand why he doesn’t want his real name used.) “I’m getting better each time, and finally made it into turn three.”

When I ask him how this is affecting him he says he wants to build a race track in his home. “This is a lot of fun. Once your car runs, the times are posted for your machine and you can figure out how fast your car is going at each sensor point. You could copy and paste this if you wanted, but it would be preferable if they gave you all of your races together so you could analyze all of the numbers.”

Turtle informs me that the man standing next to him is tied for the lead. His name is Peter Whitfield.

“I’m probably the worst Java programmer here – I’m more of a manager than a programmer,” admits Whitfield. “It will be embarrassing if anyone looks at my code. I have no experience with real-time Java, though 20 years ago, I worked with real-time systems. This is a lot of fun and a good opportunity to see how the real-time Java stuff works. It’s easy.” Different strokes for different folks.

Peter -- who is quite generous with his time as he keeps eyeing the monitor above the track listing the names of who's in the lead along with who is currently racing -- completed the race in 25.49 seconds somewhere around his 20th trial and estimates that maybe 10 or 12 people out of 100 have made it to the finish line without crashing. Clearly the hardest thing is just making it around the track. I ask him the secret to his success.

“Keep it simple -- do the absolute minimum necessary to get the result,” Whitfield explains. “There are guys doing some very sophisticated algorithms but I have not actually modified the sample algorithm at all, I’ve just been changing the parameters. Nothing in this application has been hard. I have an engineering background rather than a pure software development background and this is about manipulating voltages to control the speed, and that’s second nature to me. So my background helps. I have only now started changing the way the real-time stuff works. Until now, I’ve only changed the parameters of the way the speed changes as it goes around the track, so a lot if it involves trying to figure out how, if I change the voltage in one place, it affects the speed in another. That’s what people are missing -- they don’t understand how if they go too fast in one part of the track, it will make the car unstable later. The Java side of it is very easy to pick up because we are given a sample program. We have not had to set up the framework for it. Real-time programming is about understanding physical systems; it's not as abstract as writing UI software. So you need to understand the physics of what you are trying to do and the timing.”

I ask if he spends a lot of time visually contemplating the track. “Absolutely – I stare at the track trying to understand where things happen. It’s a spatial thing and physical thing. I can run the same program 10 times and get different results each time. There are 5 different cars that all perform differently. We don’t know which car they will run, so we can’t tune it for a specific car. The challenge is to write software that is in tune with the reality of what you are trying to make happen.”

Peter went on to say: “I learned that I don’t program in Java often enough, so I hope people aren’t looking over my shoulder when I look up Google to figure out how I do multi-dimensional arrays. You need to be aware of where you are putting your thread to sleep and where it will wake up and if you are doing it in more than one place you need to think of the different permutations and where it can all happen.”

And what else is Peter doing at JavaOne?.. “I’ve spent way too much time here. I don’t have a boss so I can sit here and just have fun.”

I talk to a few other contestants waiting their turn. Jonathan O’Keefe, who does database programming, is trying out real-time for the first time. And there is Ulrich from Denmark who does web applications and GUI stuff and business logic, and Michael from Germany who does desktop programming and Carl, an American, from a company called Triego Network Security, who manages security information. None have experience with real-time. The line ebbs and flows at various times, but there's always someone racing, and there are a lot folks sitting in front of the machines set up for them on which they work on their real-time programs.

Only Carl admits to big ambitions. “At first,” says Carl, “I was over-complicating it by treating it like a J2ME product, but if you are very conservative about the objects you allocate, you can treat it as a regular Java program. You have to keep it simple and not allocate anything in the loop or you will run out of memory. I’m going to try to knock someone off the leader board -- I’ve just started with the base program so there is a lot of room for improvement.”

Carl sees uses for real-time Java at his company, Triego. “We do some close to real-time work and try hard to keep our GC under control and use a concurrent generational collector, so the real-time Java technology is very interesting. We could find a lot of use for this technology in the work we do. It’s interesting and the applicability of the predictability of execution time is great.”

As I walk away, I see that Robert Chu is tied with Peter Whitfield’s 25.49 seconds time while Richard Yee is third, at 26.18, and Rivera (I don't catch his first name) is fourth at 28.60. I wonder if any of these folks will be in the top three when James Gosling gives his keynote on Friday morning. Time and times will tell.

Stay tuned. The winners will be announced and compete one final time at the Gosling keynote on Friday. Should be a lotta fun!

Finally, here's a story about the Slot Car Racing Programming Challenge yours truly wrote over on java.sun.com:

The 2006 JavaOne Conference Slot Car Racing Programming Challenge
http://java.sun.com/javaone/sf/slot_car.jsp

And, in the form of shameless self-promotion, here's an interview that I did not far back with Sun Distinguished Engineer, Greg Bollella, on real-time Java:

Programming in Real-Time Specification for Java (RTSJ): A Conversation with Distinguished Engineer Greg Bollella
http://java.sun.com/developer/technicalArticles/Interviews/Bollella_qa2.html

Here we go! JavaOne is almost here!

Among the topics currently scheduled for my editorial beat:

-- The Slot Car Programming Challenge wherein developers get to test their skills with Real-Time Java. Standard 1/24 scale cars run around a track with 80 sensors spread along its length with a power supply controlled by an A/D converter driven from a workstation. Contestants have to write software that controls the car while going as fast as possible. The sensors are simple photocell gates like those used to detect paper moving through a printer. The Real-Time Java program senses the track position of a slot car and sets the voltage to the track -- and thus the speed of the car. Attendees with the 3 shortest lap times will have a final run-off during James Gosling’s keynote on Friday.

-- The activities of Tommy, the autonomous, unmanned Java technology-powered robotic dune buggy, who will be making myriad appearances. Tommy's software is built on a Java technology-based platform called the Mobile Autonomous X-bot (MAX) developed by Perrone Robotics, Inc. (PRI). PRI-MAX runs on the Java Platform, Standard Edition (Java SE, formerly known as J2SE), and uses the Java Communications API, while Tommy's microprocessors rely on a hardware-based Java Virtual Machine (JVM) running Java Platform, Micro Edition (Java ME, formerly known as J2ME).

-- The always-lively discussion at the Fireside Chat wherein JavaOne conference Alumni get to engage with Java luminaries.

-- Sun’s Project Tango which addresses interoperability between applications built on Microsoft's Web Services Communications Foundation (WCF, a.k.a. Indigo) and those built with Sun's Java Web Services technologies.

-- SOA and developments in open source.

-- Twelve Reasons to Use NetBeans Software.

Plus numerous other sessions, along with exhibits in the Pavilion and the flexibility to go where the action is. I already feel sleep deprived!

Conscientious Software

Putting the Server in Your Pocket

At the June 29, Wednesday morning Platinum session, held from 8:30 to 9:15, Nokia Chief Technology Officer and Senior Vice President Pertti Korhonen provided a vision of the future that promises to take Sun's motto "The network is the computer" to another level, by putting the server in your pocket.

Steve Meloan does a really good job of covering the technical moves Nokia and the industry are making to enable this here:

so that allows me to wax philosophical :) in this blog.

It's tempting to make this story symbolic of something or other - I don't really know what. Maybe something like the story of the pilot of the plane who doesn't know where he is going but is proud of the fact that he is breaking all speed records. It's all happening so fast, and there is always the law of unintended, unforeseeable consequences.

I found myself wanting Korhonen, and everyone else, to get specific about how this technology can help us. The vision can't just be technological, but one that looks more deeply into the nuances of the implications for human life.

Project Looking Glass: An Expanding Universe on Your Desktop

I'm at TS-7992 where Project Looking Glass (LG3D), a Java technology-based open source project that brings a richer user experience to the desktop through 3D windowing and visualization capabilities, is being presented to an audience of, I guesstimate, 800 people. LG3D sprang from the very creative heart and mind of Sun's Hideya Kawahara. Recognizing that desktops had not changed substantially in 20 years, he set out to make them more aesthetically appealing and powerful. Operating on the assumption that the next user interfaces would be 3D, he initiated a side project that would consume at least two hours a day of his spare time, plus most of his weekends and holidays for more than a year before taking hold at Sun. To put it mildly, it has taken hold. It's the most popular "app" on java.net (http://lg3d.dev.java.net) with 26,600 source code downloads, plus 600 members (and counting) since the 2004 JavaOne Conference where it was open sourced.

So what's the latest? Hideya and Paul Byrne, LG3D project owners, demo'ed a range of 3D images, a music player, scenes in which you could alter the backgrounds with a click, "Alice" an award winning 3D media player (http://alice.dev.java.net) that is the first to utilize the 3D capacity of Looking Glass, and more.

CosmoSchedulerD, a three-dimensional application running on LG3D software, created by developers at the Kyushu Institute of Technology in Japan, won a Duke's Choice Award (http://java.sun.com/javaone/sf/dukes_choice_awards.jsp). As a schedule book, it recreates outer space, with your personal solar system built in by arranging the planets according to their dates. The front of the orbit represents the current time, while the size of the planet symbolizes an appointment's importance, which makes it hard to forget an event even a few light years from now. CosmoSchedulerD contains features that ordinary schedule notebooks don't have, such as automatic scheduling, networking, and a workspace manager. Imaginative desktops seem to inspire even more imaginative apps to be built on them.

(I can't escape the feeling that talking about innovations on a gorgeous 3D desktop is like a donkey carrying a load of books. Have to shake it off. By all means go check out "Philco" running LG3D, the mock-up on the cell phone, and LG3D on a 3D LCD display, and all the rest on the pavilion floor!)

Hideya and Paul gave a brief summary of how to create a "deep" 3D environment. It's built on Java 3D with specialized classes that include a component model, animation system and SceneManager interaction. The LG3D 0.7 release has just arrived. There is now WebStart support (http://lg3d-webstart.dev.java.net) for running the "developers" mode of Looking Glass. It operates in application mode so LG3D can run on top of a user's existing desktop. Java 3D 1.4 now enables performance improvements like shader support. It has Open Solaris support.

In the pipeline is tool integration, a visualization library, and "SwingNode" support. There will be greater inclusion of identity and collaboration features and a more task-oriented UI.

To run it: http://lg3d-webstart.deve.java.net

To get it: http//lg3d-core.dev.java.net

To learn more about LG3D: http://java.sun.com/developer/technicalArticles/javaopensource/plg .html

http://java.sun.com/developer/technicalArticles/J2SE/Desktop/look ingglass/

A Cool MP3 Player at the SwingLabs Exhibit

I’m at Exhibit #1111, Swing/Labs JDesktop Network Components (JDNC). It’s a Sun Microsystems project that allows experimentation with extensions to existing Java Foundation Classes/Swing API components, new JFC/Swing components and various desktop related technologies like Java 2D, AWT. I’m talking now with Sun’s Roman Guy, who explains, “We experiment with a number of projects containing JDNC and Swing components, all of which sit on top of Swing and add extra features, like transparency or animation. There are data-aware components that make it very easy to find data from a database or an XML file, directly to the UI. You have almost nothing to do -- everything is done for you in the code. You just choose the source of code and – that’s it.”

The source code is available on java.net, (http://www.java.net/) as part of the LGPL license. The idea is to explore new ideas in Swing components that may one day be part of the JDK, like various filtering, highlighting and searching features. “It’s a playground for the Swing team,” explains Guy.

Next to Roman, is Sun’s Richard Bair, operating a SwingLabs desktop network component demo, who explains, “We’ve created a music player that reads the music libraries of a very popular music player and will play any MP3 file that you have in your library. The interesting thing we’ve done is show off some of the cool features in our Swing X project, especially the data binding. First, you see the finished project, a nice music player with ‘play,’ ‘previous,’ and ‘next’ buttons and a search field. We have a lot of music and custom cell renderers and so on. On Thursday, June 30, this will go public as: joplin.dev.java.net – Joplin, as in Scott Joplin, is the name of our music player. We have a list of music and a collapsible panel -- you click a button and the music screen comes down as an animated effect. As you select different songs, you notice that the title, album, and artist genre info get updated.”

The screen displays a cool album cover.

“It looks fancy for a demo, but a lot of it is stock components from the Swing X project we put to use here,” says Richard. “We have a JX image panel component, which is nothing more than a component that knows how to extract the image art from an MP3. It’s pretty simple stuff. This table is a normal JX table and a little browser that lets you walk in 3D through all of the album covers in your music library. It’s all implemented in Java 2D, very fast. Once you get your song, you can listen to it. Another screen was written in Java 2D, we call Zoomy; this panel can be retrieved from another open source project on java.net that is called ping.dev.java.net. A search field enables you to filter by artist, album, genre and song. This will be part of James Gosling’s keynote on Thursday.”

Great work! Let’s hope it finds its way into the JDK. And don’t forget Gosling’s Thursday morning keynote.

A Pragmatic Application of SOA (Service-Oriented Architecture)

Over at TS-1640 Pragmatic SOA: A Case Study, with developers Ashok Mollin, Ashesh Badani of Sun Microsystems, Inc. and T.N. Subramaniam, Director of Technology at RouteOne LLC Inc. Ed Ort (http://weblogs.java.net/blog/edort/archive/2005/06/pragmatic_soa.html) and Tim Bray (http://www.tbray.org/ongoing/) of Sun are blogging the session in greater detail. I will be doing a story on this for java.sun.com in a month or two so stay tuned. For now, a sneak preview.

RouteOne provides a web-based Credit Aggregation Management System (CAMS) created to accelerate the automotive finance process for dealers and their finance sources. The session presented “a real-world implementation of an SOA project at RouteOne LLC and helps explain the architecture and various best practices, design patterns, standards, and technologies involved in building an end-to-end business solution.” The presenters define Service Oriented Architecture (SOA) as “an integrated software infrastructure and design approach, leveraging Web computing standards, for delivering business functions as shared and reusable services.”

A lot of people regard SOA as a real challenge, in part because it typically involves both substantial organizational change, in addition to new ways of organizing IT. It seems like a situation in which developers have to be, not only technically sharp, but good listeners who can understand the culture of the company they are working with. Creating a shared service requires technical design prowess — architects have to know who, when, and where in the business process, services will be consumed. And then there is the whole history of ERP enterprise resource planning, in which some CIOs and IT managers experienced a lot of growing pain involving expensive projects that required changing processes across the enterprise as part of automation. It has not always worked out. Some people fear that SOA will bring a lot of pain to companies because it may be bigger than an ERP implementation. So businesses are sometimes understandably cautious. There can be a lot to rethink: development methodology, business impact, infrastructure, budget, organizational design and more.

Another challenge involves creating high-level business components that can be re-used and re-configured. One problem is that the requirements that yielded the original component interface were different enough from the new ones so that they required the re-write of substantial functionality.

The basic approach of Sun’s Ashok Mollin and Ashesh Badani, along with RouteOne’s T.N. Subramaniam, seems sensible and cautious: Projects need to generate ROI in 12-18 months so start small and be opportunistic. Minimize disruption to existing infrastructure; reduce risk with fewer web services initially while climbing the skill curve. Wrap legacy/existing applications using adapters + WS; Java and .NET interoperability. Evolve into a flexible, standardized architecture; does not mean “rip and replace”. Foster cultural change to encourage reuse. Take a top down approach – let the business drive!

They addressed several critical problems:

Single Sign On

Transparent login from lender portal

Get Dealer Information

Accessing volatile dealer information at runtime

Import Credit Application

Start the process on another system – DSP

Orchestration

Maintain state and coordinate document exchange in long running transactions with Lender

A summary of their “Pragmatic SOA wisdom:

Start simple, don’t let the “alphabet SOAup” overwhelm you.

Let business identify the service.

Think XML documents not objects.

Use SOAP as an envelope, but not for binding.

Use WDSL 2.0 for description not code generation.

Think asynchronous conversations.

Use WSBPEL to orchestrate the process.

Use JBI for integration.

Keep learning, this is not finished.

Question as I left: Did the large Java developer audience there come away enthusiastic about SOA? There certainly were a lot of smart, probing questions in the brief, casual q &a after this session, if that's any sign.

This work is licensed under a Creative Commons License.