The Source for Java Technology Collaboration
User: Password:



Malcolm Davis's Blog

July 2006 Archives


The Java DB, or better yet, the JRE DB, is just a RFP item.

Posted by malcolmdavis on July 20, 2006 at 11:49 AM | Permalink | Comments (10)

Sales and marketing:
Companies routinely add functionality to a product that will never be used by a client. The reason for the un-used features is to meet an entry on a Request For Proposal (RFP). The sales department can then present the functionality to potential clients.

Many times a RFP item is something that multiple clients have requested, but serves little or no purpose. [The absurdity of the RFP are well understood by people that have worked for the Federal government or Fortune 100 companies. After 4 years on the NASA's Space Station Freedom project, I can tell horror stories about the RFP process.]

Choice and discontentment:
Embedded Java databases have existed for years. As Java developers, we have the added luxury of picking the database of our choosing. [Yet, some have argued that all the choices have bred discontentment toward Java.] Microsoft's claim to fame is their one homogenous, well-integrated, do it all environment. Even though Microsoft VisualStudio falls way short of free tools like Eclipse and NetBeans, and the .NET architecture is lacking features like an Object/Relational Mapper (ORM), the developers don't have all those decisions to make, they just make what they have work.

Microsoft:
Microsoft has introduced the embedded database with the release of SQL Server 2005. As it matures, the database will become a standard component of the .NET platform. Microsoft already provides SQL Server Express for free, which can be embedded into application clients.

So, what does Sun do? Introduce an embedded database. Sun has to have that RFP item checked off the list.

As Microsoft has taught us, it is not necessarily about technology feasibility, competences, or merit, but marketing. Microsoft has satisfied many RFP items in name, but not in spirit.

Reality check:
Welcome to the world of reality. When Jon Stewart has a book author on his show, Jon Stewart did not necessarily think the book was a good idea, rather the publisher bought airtime.

I personally think Sun could spend the energy more judicially by cleaning up the Java API of the deprecated code. However, I am not a marketing guy.



Scale Up vs. Scale Out

Posted by malcolmdavis on July 19, 2006 at 02:40 PM | Permalink | Comments (3)

For years I've had the discussion with developers, database administrators, and system admins on big-box vs. the multi-box solutions. The question is "How is scalability achieved"?

Application scalability can be defined as the ability to increase the application throughput in proportion to the hardware that is being used to host the application. In other words, if an application is able to handle 100 users on a single CPU hardware, then the application should on be able to handle 200 users when the number of process are doubled.

Vertical scalability is adding more memory and CPUs to a single box, or scaling up. Vertical scalability or scaling up is well suited for database tier. Databases have the following needs:

  • Large shared memory space,
  • Many dependent threads,
  • Tightly-coupled internal interconnect

Horizontal scalability is adding more boxes of similar memory and CPU, or scaling out. Scale out is ideal for web-tier, and has some of the following characteristics:

  • Small non-shared memory space
  • Many independent threads
  • Loosely-coupled external interconnect [an important point]
  • Possibly many OS's

The following link is a study conducted by BEA on vertical vs. horizontal platform scalability: http://e-docs.bea.com/wls/docs81/capplan/capappa.html

The IT shop
I cannot count the number of discussions I have had with system administrators about hardware configuration for application. Many IT shops treat the application problem much like the database problem. Instead of an array of rack mounted blade servers that are inexpensive and easy to add or replace, administrators purchase big boxes.

The administrator's argument about the big boxes:

  • Bigger bang for the buck
  • Easier to admin on box than multiple boxes
  • Easier to make a single purchase instead of multiple small purchases
  • N squared hardware failure, the more boxes the more likely to fail.

The reality:

  • As seen from the BEA study, less bang for the buck for application scalability,
  • More difficult to admin because of the conflicting needs of all the independent process. Huge impacts occur due to inadvertent changes to root or patches to sub-systems or OS.
  • Costly mis-sizing of hardware that cost the organization. It cost more to add hardware to a big-box, then to slide in another blade server.
  • Software, rather then hardware, is generally the root cause of problems in production

From an application perspective, scaling out provides other advantages.

  • Administration of conflicting needs Many times independent process require different versions of the same software, or worse, different versions of a shared library (*.so). Conflicting needs occur when independent processes are required to run on the same box, in the same user and JVM process space. Due to the conflicting needs of all the independent process, multi-box solutions is easier to admin. [Many people argue this simple solved with multiple Virtual Machines (VM), however VMs presents another set of risk and cost.]
  • Root cause analysis Problems will occur when software that is defective or troublesome is delivered. If all the software runs on a single box, in a single process space, determining the cause can be difficult to discover. Scaling out separate process to different boxes provides an easier method of determining the problem.
  • Defective software isolation Defective/Troublesome/Buggy software impacts everybody. However, the impact can be minimized of the defective software is isolated to a single box rather the impacting the entire application.
  • Failover The common approach in the J2EE world is to provide numerous boxes of the same application and configuration. If a single box fails, the traffic is routed to another box. The user session is maintained by a constant serialization of user session data between boxes. Constant serialization of data among boxes has a myriad of issues. For instances, transient versus non-transient data, session logging propagation, limitations on certain design patterns, a unique awareness of global data. Defective software is the largest problem with the multi-box failover model. If a flaw in specific user flow is the cause of a production failure, moving the user to another box will only cause the second box to fail. Since most failures are due to defective software, spreading the software in multi-box failover model does not fix the problem.
  • Load balancing Simple round-robin request between boxes does not solve the problem of load balancing. Determining loads of specific processes, and properly design for the specific processes, is the best way of handling application load. Scaling out provides greater opportunities for tuning the operating system and processes.
  • Right sizing It easier to right size the multi-box architecture. Sliding another blade server into a rack cost less then adding hardware to a big-box
  • Security In every large IT organization I consulted, the developers are not allowed access to the production servers. The security aspect hampers the ability to evaluate problems in production. Utilizing logging to determine behavior leads to its own set issues including increased load the server, and extensive amount of coding.
  • Process Expansion In pipeline architecture, a process can be arbitrarily 'scaled' by substituting any number of identical sub-processes. This takes advantage of the Rules for Queues (many processes may feed from a Queue, a Queue may be fed by many processes)


  • Emails 'pose threat to IQ'.

    Posted by malcolmdavis on July 02, 2006 at 10:33 AM | Permalink | Comments (0)

    Messaging barrage
    My first job out of college was with Boeing Aerospace. Engineers occupying rows of desk in a large open room, and the constant distraction of telephones and conversations, I wondered how anybody got anything done.

    Jump ahead 20 years, "Did you get my email?" a colleague asked. "No" I replied, "I haven't checked my email recently".

    The colleague was frustrated. He had expected a near instance response, not something hours later. The lack of response on my part was because I was not running my mail client or IM while I was working.

    Now there is evidence that this constant stream of interrupts not only screws up a programmers "Flow State", but could also lower a persons IQ. Depending on the degree of interruptions, the brain drain can be equivalent to smoking marijuana.

    Constant crunch
    One of the appreciations I have of extreme programming is the balance between work and life. However, many managers still like to push, thinking they can get more out of a person if that person is kept in crisis mode. However studies conducted show, that over an extended period of time increased hours do not increase productivity.

    Sleep
    If you require an alarm clock to wake up, you are not getting enough sleep. Its amazing the number of zombies walking around the office on a Friday afternoon. Just as you shouldn't buy a car built on a Monday (blue-Mondays due to the hangovers from the weekend), I have often wondered about the number of coding defects relative to the day of the week, or hour of the day.

    The after action report of the first gulf war indicated a serious problem with military officers. Officers' had made bad decisions that ended up in fratricide (killing friendly soldiers). The root cause of the mental mistakes was sleep fatigue. Only women with newborns had greater sleep fatigue then a combat officer. Even intern-doctors get more sleep. The report helped create new policies that required officers to receive a specific amount of sleep for every 24-hour period.

    Final thoughts
    The empirical evidence is overwhelming. We are all hurt, (especially the employer), by constant interruptions, working long hours, and walking around in a daze due to sleep fatigue. Yet, in 2006, every environment I have consulted on this year has looked so close that first job out of college.

    The abuse of the IM technology has made the situation worse, not better for the manager. In "Planning: Managers need to sell their trucks", I argue that managers should remove the electronic leash called a BlackBerry from employees. Maybe it is time to think about the same for email and IM.





    Powered by
    Movable Type 3.01D
     Feed java.net RSS Feeds