David Van Couvering 's Blog

March 2007 Archives

SDN podcast/video about databases at Sun

This just in: a new podcast/videocast from Sun Developer Network where Simon Phipps is interviewing all my old buds (Josh Berkus, Francois Orsini, Rick Hillegas) from Sun's Database Technology Group talk about Sun and open source databases. Check it out!

NetBeans wins JOLT Award for Best Development Environment

I just heard the news that NetBeans wins the JOLT Award for Best Development Environment (see Section 7). Given that I work on NetBeans, this is super cool. I'd like to know why it won, but that doesn't seem to be available.

My goal (vision? wild and crazy dream?): get NetBeans to win the award for Best Database Tool. Right now that honor goes to Visual Studio 2005 Team Edition for Database Professionals. Those MS folks are very good at building this kind of tooling. Time to take a page from their book, see what they're doing, and get some of this into a product that is portable (runs on more than Windows), free and open source.

The runners-up also look very interesting, and I plan to take a look at them as well. Why re-invent the wheel if somebody's already done a good job of it.

Google Web Toolkit for NetBeans

When I first heard about Google Web Toolkit, I was kind of bummed because it looked so nice but it was closed source. But then they announced they were open sourcing it under the Apache license, opening the door to all sorts of possibilities.

One great consequence is the announcement I just read on the NetBeans dev alias from Thomas Szlota and Tomas Zezula . They have started a project on java.net to support the Google Web Toolkit in NetBeans.

As a Java guy who has struggle with JavaScript, and as somebody who very very much appreciates the help an IDE like NetBeans gives to a developer, I was very excited to hear this. Of course I'd like to participate, but only so much time in the day. But I look forward to see what they're going to do here and am excited to see these two technologies coming together For The Greater Good.

Why use Atom Publishing Protocol for REST?

I've been working with a group of folks here on prototyping an application architecture that presents relational entities as REST endpoints (yes, I'm sure some of you have already done this, but now it's our turn :)).

We decided to build our REST services using the Atom Publishing Protocol, or APP. This is a common recommendation, and what's good for Google was good enough for us :).

But as we did the work, it seemed like a lot of overhead to put all this APP framework around a very specific application. Why not just use HTTP and JSON and have done with it?

Well, we didn't have a clear answer, so we thought we'd ask the experts here at Sun, and sent an email off to Tim Bray and Dave Johnson. Rather than (mis)quote what they had to say (or the conclusions of the other folks on the team), I'll summarize what my conclusions were at the end of this discussion. I'll let Tim and Dave (or anybody else) comment if they feel so inclined. But I do want to thank Tim and Dave for their very helpful and valuable input.

First of all, I think we should stick with Atom, even with the overhead. Why? Here are some of my reasons:

• APP lets you post a new entry without having to know its URI. The protocol lets the server tell you where it put it as part of the response. This allows the server to allocate things as it sees fit, and decouples the client and server. This feature also seems key to me for solutions that need to scale -- you don't want to hardcode locations into the client.
  
  
• Because APP and Atom are growing and popular standards, lots of products are or will be working with it. As a simple example, when I was testing my Atom server with my browser, at first I was just getting raw XML displayed (through a plugin). But when I finally got the response right and sent a valid Atom feed, all of a sudden Google Reader popped in and asked me if I wanted to subscribe to the feed. Sweet!
  
  I tried to subscribe, but it didn't work, because Google didn't know how to connect to localhost. But Firefox Live Bookmarks worked great!. My content is in JSON so it's not too helpful, but if I had an alternative HTML representation, people could subscribe to my service without any effort on my part.
  
  
• Because APP and Atom are growing and popular standards, people are providing and will continue to provide tools to make it easy to build APP servers and clients. ROME and Apache Abdera are just two examples of this.
  
  
• Finally, supporting APP makes me buzzword-compliant. All things being equal, there's something to be said for that :)

So, tell me what you think. Am I missing anything? When do you think it doesn't make sense to use Atom/APP for your REST service?

If you're free, DabbleDB is free

I really like DabbleDB, but one of the things I complained about was that you had to pay to use it, rather than pay for services or support. I understand their motivation, but I really think it limits your ability to get big adoption. If you have a ton of people adopting your platform, then you can leverage that in oh-so many ways. But without volume, as our own Mr. Schwartz is always hammering into us, you got nothin'.

Well, I just got an email saying that if you are willing to put your database created under DabbleDB under the Creative Commons license, then you can use DabbleDB for free. Now that is cool.

So take a look, it is a very powerful tool. It's free, as long as you are free.

Next step: make data stored in DabbleDB available through a REST interface, and allow people to build new apps on top of it. If they do that, then we're really talkin'

Cloudscape is no more

Over 10 years ago, a bunch of very smart database engineers, many of them friends and colleagues of mine from Sybase, disappeared. We heard they had started a new secret company in Oakland. They were doing something "cool and weird", but for over a year it was all hush-hush. Then it came out: they were building a relational database completely in Java. WTF? What were they thinking? How would it perform? Who would want it?

Years went by, Cloudscape was bought up by Informix, who was then bought by IBM, and IBM ended up with this little Java database in their portfolio and weren't sure what to do with it. I personally was convinced that that cool Java database had seen its last days and its team would be assimilated by the Borg of DB2

But now the Cloudscape database, under the moniker Apache Derby, is in widespread use and is being shipped in Sun's JDK (using Sun's distribution of Apache Derby, Java DB). It has shown competitive performance with those old fuddy-duddy databases written in C, and with one build runs on numerous platforms from Windows to Solaris to an IBM mainframe to a J2ME CDC device. It can be installed and run from a browser invisibly and embedded in a Java application so the user never knows it is there.

I have said this before, but I'll say it again: I want to thank IBM and the Cloudscape team for having the inspiration, the guts and the commitment to get Cloudscape into open source so this could happen.

But now IBM has decided they no longer want to provide official, paid-for support for Derby. They were shipping their supported version under the original name of the product Cloudscape.

IBM has made it very clear that this was just a decision about providing support, not about ending their involvement in Derby. I believe them: I can't imagine them disinvesting as there are numerous projects within IBM that ship with and depend upon Derby.

But with this decision, the Cloudscape name, and brand, is now only a (TM) in IBM's bundle of trademarks and nothing more. So, a moment of sadness at the disappearance of this name, and a moment of honoring the history of Apache Derby

I guess there is one good thing: now there are only two names for the same thing :)

Freebase not letting me in

Oh, poo, I can't get into the FreeBase beta program. Sigh...

We received your email registration. We're excited and overwhelmed by the enthusiasm.

You may be wondering if "Information Wants to be Free" why is our Alpha test currently limited? For the time being, we're open to a small set of users to get critical feedback. We'll use that feedback to make it all a little less 'alpha'. At that point, we'll open up registration to a wider group.

For more on this and other common questions, see our FAQ: http://www.freebase.com/signin/faq

Also, check out our blog as further details are revealed publicly: http://roblog.freebase.com

Freebase: free-wheelin' semantic web

We all love metatagging sites like deli.cio.us and Flickr because they let us organize our data however we want.

But if I want to write a program that lets me in some controlled way query, filter, and sort ala SQL, metatagging is pretty difficult. That's why the Semantic Web is so attractive.

The problem with the Semantic Web is that you have to build an ontology, and most of us can't be bothered, to be honest. And how do you share these ontologies, and evolve them, in a way that is accessible for as many users as possible? We want the flexibility and usability of metatagging with the programmatic power of a structured schema.

Enter freebase. Right now it's in alpha, and you have to apply to play around with it. Maybe they'll let me in. But for now you can read Tim O'Reilly's blog to get a mouth-watering taste of what they're doing.

As a database guy who understands the power of SQL, but also values the free-form and evolving nature of metatagging and a read-write web, this looks like a very good fit, and I definitely want to see more. If you combine this with an easy query-building interface, kind of like Yahoo! Pipes*, then this gets serious fast.

Reality check: we have to see how all this is going to scale. It is going to have follow the basic principles of REST, for starters. There is the additional scaling challenge of handling joins, grouping and so on, which traditionally require all data to be in a single place before you can do the operation. When you are operating across multiple, perhaps hundreds, of data sources, this can be a real issue.

The general rule for improving client/server query performance is to keep the code as close to the data as possible. I can see people quickly realizing they need to create some form of "stored procedure" that can execute filtering and other logic against the data at the source before all the data is shipped over the web. SalesForce realized this and recently extended their web service query language to support procedural logic.

But I am hopeful we can solve these scaling challenges -- there are a lot of smart people out there, some backed with lots of money who want to see this happen.

So, stay tuned. It looks like database application development is about to meet the web, and the combination will enable a completely new way to build applications.

*well, something like it: Yahoo! Pipes does this for RSS feeds, not structured content

How To Find Out Who You Are

Sometimes I would really like to know the identity of the folks commenting on my blog entries. But java.net just gives me an email address of "foo@bar.com" and a java.net userid, which for the life of me I can't figure out how to dereference to an actual person.

I'd like to see your blogs, see what you're working on, what company you work for, what you're talking about. I can't do any of that.

If anyone has any tips, I would greatly appreciate it.

Magic: My Blog Becomes a Forum

A magic transformation has happened, where my blog about how to get a job in software has turned into a forum of people weighing in about what you should expect in a Java programmer, focusing particularly around the static initializer construct.

I have seen so many blogs going around and around on this topic, with the folks who hate detailed interview questions and those who think it's very important. So I'm not going to go there, and it's not really my point here.

What strikes me more than that is that it's happening on my li'l ol blog. It was meant to be a quick and interesting observation, but it has served as the catalyst for a fairly heated discussion. I'm just sitting on the sidelines and watching.

What it reveals to me is that as a whole we are a passionate group of people, who love software and care about it deeply. We have invested ourselves personally into our work. Nothing wrong with that! Can you imagine two dishwashers arguing passionately about the right way to organize dishes on a rack?

Wait, strike that -- I have seen dishwashers do this. At one job just out of high school, I remember getting long lectures from an old-timer about how to lay out the dishes, and then getting an alternate lecture from another old-timer.

Oh well, maybe it's just human nature...

Swing Application Framework and Beans Binding in NetBeans

Check out Roman's demo on how to make use of the Swing Application Framework and Java Beans Binding in NetBeans

This is the kind of stuff I remember doing in Access, making it so easy to build simple database applications. It looks like Java, with NetBeans, is getting the same kind of functionality.

Great work, Swing and Matisse teams!

How To Get a Job in Software

Every now and then a friend contacts me or hooks me up with someone saying they are interested in getting into high tech, particularly software. I have come to understand that because of open source, how you do this has completely changed since I got into the industry.

Here's a copy of an email exchange I had recently with someone asking how to get started...

"Joe" says:

Hello David

Recently, I discovered the Information Technologies industry and found it most appealing.

I have always been interested in computer work and found my way learning software while utilizing my organizational skills in different industries.

I am looking for a more stable environment to raise my family in. My ambition is to start a new career in High Tech, yet I don’t know how to penetrate into the industry.

What are the opening positions that offer a person the opportunity to learn and get a foot in the door? Can one get ahead without formal education in the field?

Here is my response. I really believe this is the best way to get into the industry these days. Open source has really opened things up so anyone with any enthusiasm can get in the door...

Hi, "Joe". I can tell you're very excited about working in high tech - you must have fun with it. I would say that's the most important asset you can have :)

Ten years ago, I would have told you to get some training and try to get an entry-level position in a tech company.

Things have changed since then. Now there is this fastly growing phenomenon of open source software. This means that you do not need to take a class, go job hunting, interview, etc., etc., to start getting some real-world experience and meeting and learning from other folks in technology.

So instead what I recommend is to poke around and find out what open source projects there are that interest you. There are projects for almost everything, from little utilities for Windows, to the massive project for the Apache Web Server, which has little niches of its own inside it.

Then all you do is start volunteering your time. Start using and learning the product, and ask questions on the list. Learn how to download the source and build it. Read up on the technologies they are using, and play around with them. Then, when you feel ready, if you see a bug, see if you can fix it. You send in the changes and then someone who has the rights will put the fix into the code. You learn the people in the community, and they learn to know you. They should be friendly and helpful -- if they're critical and flaming all the time, move on to another project.

Then before you know it (a) you've learned some technology and (b) you've established some cred and some contacts. I would say if you do this for about a year, you should be able to find a good entry-level job in the same field, maybe working for a company that is paying people to work on the project you already have experience with.

Classes and trainings and books are all well-and-good, but there's nothing like real-world experience, and these days it's so easy to do because of open source.

David

This work is licensed under a Creative Commons License.