Skip to main content

davidvc's Blog

Internet content by reference, not by value

Posted by davidvc on May 9, 2008 at 12:07 PM EDT

Note: if we had DRY working for blogs, then I could have just embedded the content from my other blog and not have had to enter things in twice. Posting a link is not the same as having the content immediately available, which is why I decided to just copy the blog. Oh well...

You may have noticed that I am very interested in how data is managed on the Internet as a platform, at a web scale. In that light, I have been having some very illuminating and interesting conversations with an old friend and colleague, Nitin Borwankar. His thoughts on data property rights and DRY data are concepts that if implemented could result in a major shift in how we manage data on the web.

Data property rights is about laying out a "bill of rights" for data that goes far beyond "the right to move". It also includes the right to access, modify, remove and own your data. So often it happens that once you upload your content to a site, you no longer have full rights to that content, as if somehow in the act of uploading it it is no longer yours. It's like living in a serfdom where you do all the work to plow, seed, tend and harvest the land, but the fruit of your labor is not yours, just because you are using the land that someone else owns.

DRY data is about following the principle of Don't Repeat Yourself for web content. Web applications need to start applying this principle, so that rather than you having to load copies of your content across multiple sites (and often losing some degree of ownership of it in the process), you place it in one location (your "home" on the Web, as it were), and then you refer application providers to that one place. They can focus on providing added value (for instance, referring it to your friends, enabling collaboration, or helping you organize it or present it in useful ways) rather than on the overhead of building and deploying a scalable storage architecture.

Nitin calls this architecture YINAS (YINAS Is Not A Silo).

The value of DRY for the user is obvious - I only have to put my stuff in one place, and I get to really own my stuff, rather than the vendor owning it. DRY is also very valuable for the vendor, as they can save overhead and complexity by delegating the work of scalable storage and indexing to a "data service provider" rather than having to do it themselves. It's even good for the environment, because you need fewer disk farms sucking up power and space.

It's funny, it makes so much sense, but nobody is really doing this.

I pulled Tim Bray aside at Java One to talk to him about these ideas after reading his blog about changing his address, and he suggested that concepts are good, but a simple proof of concept is better. Hm... let me think about that ... :)

Blog Links >>

You can help with NetBeans database tooling

Posted by davidvc on October 18, 2007 at 5:46 PM EDT

survey.jpg

I know this isn't my main blog page any more (I'm hanging out at http://davidvancouvering.blogspot.com), but I wanted to reach out to you all and get your input.

We are looking at what we want to do next with database tooling in NetBeans, and there are some features I'm just not sure how important they are. You can help guide the ship in the right way.

I have put together a very quick survey, no more than ten minutes. If you're so inspired, please check it out.

Blog Links >>

Moving to new digs

Posted by davidvc on June 27, 2007 at 4:19 AM EDT

I've decided to move my blog to a new location. You can now find me at http://davidvancouvering.blogspot.com.

Why am I making the move? Well, this was a hard decision. I really like the java.net community, and I like how the editor takes care to call out cool and interesting stuff.

But the problem is, sometimes I want to talk about more than Java, and more than my work. And java.net really wants and needs to stay focused on Java technology and things that impact Java (like Ruby). If my blog waxes poetic about politics or the cutest thing my kid did the other day, it tends to diffuse the focus of this site. People come here for serious Java stuff.

I also am a little constrained by the fairly rigid walls of java.net as a blogging platform. Personally, I'm tired of typing in raw HTML. I want to have a few widgets, not many, on my page. I want Google Reader to show the whole post, not just the blurb. Things like that.

If you don't want to see cute pictures from time to time, or hear about my opinions on education, and you can subscribe just to the "technology" label. There's also a quicklink label, for useful delicious links that I have auto-posted to my blog.

So, see you at my new digs!

P.S. Yes those are probably the dullest photos on my "recent Flickr Photos" widget. But I promise they'll change from time to time :) (Most of you will probably be reading my through an aggregator anyway and won't even see my fun widgets. Too bad...)

Blog Links >>

Replication? In Java DB? Wow.

Posted by davidvc on June 22, 2007 at 4:46 PM EDT

Open source continues to amaze me. For so many years, I have worked on projects where customers would clamor for features, and we just didn't have enough time in the day or people to get them all built. We would say "yeah, that sounds cool" but with all the other things on our plate, when would we ever get to it?

Well, in open source, it's a whole different ball game. The latest example is the contribution of Egil Sorensen's master's thesis. Egil is a student in the Computer Science department at NTNU in Trondheim, Norway, and he has contributed code that provides replication and hot standby for Apache Derby (discovered through Max Mortazavi's blog).

This paper and attached source code describes the work done to add hot standby replication functionality to the Apache Derby Database Management System.

By implementing a hot standby scheme in Apache Derby several features are added. The contents of the database is replicated at run time to another site providing online runtime backup. As the hot standby takes over on faults availability is added in that a client can connect to the hot standby after a crash. Thus the crash is masked from the clients. In addition to this, online upgrades of software and hardware can be done by taking down one database at the time. Then when the upgrade is completed the upgraded server is synchronized and back online with no downtime.

A fully functional prototype of the Apache Derby hot standby scheme has been created in this project using logical logs, fail-fast takeovers and logical catchups after an internal up-to-crash recovery and reconnection.

In case you missed that, this feature provides basic high availability to Derby. This is not small potatoes, and it was done as an independent contribution to Derby. Wow.

Blog Links >>

Derby 10.3 Beta Available

Posted by davidvc on June 22, 2007 at 2:02 PM EDT

Derby 10.3 beta is available for testing. If you are using Derby/Java DB, you should try your local tests with this beta and make sure everything's working for you.

This page describes what's in this release. Of note are security improvements, performance improvements, and the ability to drop or rename a column using ALTER TABLE. And, as usual, a slew of bugfixes.

One thing to be aware of is that the Network Server starts with a security manager installed by default now. This is a good thing as it provides you, by default, strong protection from potentially malicious Java code running in your server.

But what this also means is that if you have Java procedures installed in your database that do funny things like file I/O or open network sockets, you may get exceptions saying you don't have permissions to do this. You can fix this by modifying the security policy file to grant permissions, or by disabling the security manager by running the Network Server with the -noSecurityManager option.

Blog Links >>

Spam Vegetable Strudel

Posted by davidvc on June 21, 2007 at 8:10 PM EDT

I noticed today that I had 754 messages in my Spam folder in Gmail.

I decided to open the folder, and GMail had decided to place the following sponsored link at the top of the folder:

"Spam Vegetable Strudel - Bake 20 minutes or until golden, serve with soy sauce"

I just did a refresh, and now it says "Spam Imperial Tortilla Sandwiches - To serve, cut each roll in half"

Yum, yum. Another reason to avoid my spam folder

Blog Links >>

Legal Blogging

Posted by davidvc on June 20, 2007 at 7:47 PM EDT

I am a techie, and generally am not interested in the legal world. That changed a bit when I got into open source, where just to be able to contribute code you have to have some pretty solid understanding of open source licenses.

I wasn't sure what to think when I heard Sun's general counsel, Mike Dillon, had started a blog. What the heck, I thought, I'll subscribe

He doesn't blog very often, but when he does, it can be a real eye opener for me. His latest one on litigation is just great. It has just the right level of detail for a layman like me, it educates, and it makes a point: ... it's important to remember that litigation is just a tool. And, as with all tools, it is effective only when used dispassionately, in the right way and for the right reasons.

Thanks, Mike, and keep it up!

Blog Links >>

Java DB Authorization using GRANT and REVOKE

Posted by davidvc on June 17, 2007 at 10:23 PM EDT

The default behavior of Java DB is that you have two level of access control: full access and read-only. Again, this is I believe due to the legacy of Cloudscape being originally an embedded-only database. In this environment, the embedding application is responsible for controlling access to the database, and always only logs in as one user, so there is no real need for finer-grained access control in the database

But that changes when you have a client/server environment. So as of 10.2, Java DB has support for the SQL standard GRANT/REVOKE. However, again for backward compatibility, you have to explicitly enable support for it, otherwise you get the default behavior that anybody who is able to log in has full access.

You enable GRANT/REVOKE access control by adding an entry in your derby.properties file: 

derby.database.sqlAuthorization=true

Once you have done this, only the owner of a resource (such as a table, view, index, etc.) has any rights to view, modify, or delete the resource. The owner is the one who created the resource.

The owner can then grant and revoke specific rights for that resource to other users. The rights supported by Java DB are: DELETE, EXECUTE, INSERT, SELECT, REFERENCES, TRIGGER, UPDATE. For more information on SQL authorization using GRANT and REVOKE, you can check out the Derby documentation

So, for example, lets say you create a table named "races". You can then let the Marx Brothers see the races by executing the following command: 

GRANT SELECT ON TABLE races TO groucho, harpo, chico

Blog Links >>

No More Java DB Password in The Clear

Posted by davidvc on June 15, 2007 at 7:21 PM EDT

As I mentioned in my previous blog, Java DB's legacy is in the embedded world, where there is no such thing as sending the password over the wire.

But when you introduce the network client, this becomes an issue. And sending users and passwords in the clear is just not acceptable. So this blog tells you how to stop that from happening.

Java DB allows you to configure how credentials are sent to the server using four different modes, documented here. and here.

  • CLEAR_TEXT_PASSWORD_SECURITY (0x03) - User and password are sent in the clear. This is the default
  • USER_ONLY_SECURITY (0x04) - Only the user is sent. This doesn't work if authentication is enabled on the server
  • STRONG_PASSWORD_SUBSTITUTE_SECURITY (0x08) - "When using this mechanism, a strong password substitute is generated and used to authenticate the user with the network server. The original password is never sent in any form across the network." (from the docs)
  • ENCRYPTED_USER_AND_PASSWORD_SECURITY (0x09) - The user and password are encrypted using a pluggable encryption mechanism

Now, why on earth am I including the hex value for the security mechanism here? Because (and I kid you not), when you specify the security mechanism you want to use on your URL, you have to pass the numeric value for the mechanism, not the name.

Needless to say, when I discovered this, I immediately logged a bug. It didn't help that nowhere do the docs actually tell you you need to do this. I had to figure this out through a series of experiments and failures.

And get this, you have to convert the hex value to an integer, if you pass in 0x08 you get a NumberFormatException. Sweet.

Anyway, I'm here to help. If you want security, you probably want to choose STRONG_PASSWORD_SUBSTITUTE or ENCRYPTED_USER_AND_PASSWORD. I personally don't know the difference in terms of strength/breakability, but I do know that if you choose ENCRYPTED_USER_AND_PASSWORD, then you also need to install IBM JCE and configure the JRE to use an appropriate security provider. You have to do this for the JRE used by both the client and the server. If you are deploying your client to a wide swath of machines, this particular approach is not feasible, as far as I can tell. I am pinging the Derby user group about this, more info will be posted here if I get it.

So, personally, I'm opting for STRONG_PASSWORD_SUBSTITUTE (or 8 if you're thinking in integers). Here's how you do it:

First, if you want to enforce that only this security mechanism be used, you can configure the network server by putting this line in derby.properties: 

derby.drda.securityMechanism=STRONG_PASSWORD_SUBSTITUTE_SECURITY

Next, when you connect, include the securityMechanism property in your URL, e.g 

jdbc:derby://localhost:1527/mydb;create=true;securityMechanism=8

Now your password is no longer flying over the wire in the clear.

UPDATE: After some discussion on the derby-user list, I have some more clarity about securing your network pipe.

First of all, the reason you can't use Sun's Java runtime to do password encryption with Java DB is because Java DB network communication uses DRDA, a standard network protocol (used primarily by IBM). DRDA's encryption protocol uses a 256-bit key, which is considered too short (and thus too weak) by Sun's encryption engine. IBM JCE does support a 256-bit key, so that's why you would need to install it. I have asked if there other encryption engines that support a 256-bit key.

The STRONG_PASSWORD_SUBSTITUTE mechanism I describe above was implemented as an alternative because standard encryption has these issues with support and configuration overhead.

There was also general agreement on the list that what you really want is encryption using industry-standard SSL (or its new incarnation, TLS). Java DB 10.3, which is getting ready to be released, will include support for SSL/TLS. Using this mechanism is recommended unless you are concerned about the performance impact of encrypting all communications and you are fine with just protecting the password.

Blog Links >>

Authentication and Authorization with the Java DB Network Server

Posted by davidvc on June 15, 2007 at 6:44 PM EDT

The original Cloudscape was built as an embedded-only database -- that is, it could only run inside the VM of another application. It did not have a client/server mode. That was added later.

In some areas, Java DB continues to carry have this embedded "legacy" with it. One particular area you should be aware of is authentication and authorization.

By default when you start up the Java DB Network Server, it accepts connections on localhost only [1], but it allows any user/password combination, and it allows any user to do anything they darn well please. And on the client, by default, the user and password are sent in the clear.

If you don't like this (and you shouldn't if you are running Java DB with sensitive data and/or or listening on ports other than localhost), there are some things you can do (NOTE: I'm assuming you're using Java DB 10.2 or greater):

  • Enable authentication on the server
  • Configure the client to not send the password in the clear
  • Control access and privileges using GRANT/REVOKE

In this blog I'll talk about how to enable authentication on the server. In subsequent blogs I'll talk about configuring the client to not send the password in the clear, and how to use GRANT/REVOKE.

First of all, you need to tell Java DB that you want to actually authenticate users. You do this by setting the system property (on the command line or in derby.properties [1]): 

derby.connection.requireAuthentication=true

Java DB authentication is pluggable - you can use LDAP, builtin authentication, or authentication driven by a user program. For more information, see the Derby manual section on authentication

The simplest solution is to use builtin authentication for all your databases, and I'll describe how to do that here.

You basically enter in a series of users and passwords in to derby.properties (this means, of course, that you need to set up proper permissions on the derby.properties file) 

# Enable builtin authentication
derby.authentication.provider=BUILTIN

# Define users and passwords.  User names and passwors
# are *case sensitive* and can be quoted if desired
derby.user.groucho=cigar
derby.user.harpo=toottoot
derby.user.chico=sanityclause

Now when you try to connect with any old user or password, you will be pleasantly foiled.

More on authentication and authorization in upcoming blogs. I thought it better to do them in simple pieces so you're not overwhelmed by too much information in a single blog.

[1] If you want to listen on a different host or port, use derby.drda.host and derby.drda.portNumber in your derby.properties file

[2] If you're wondering where you should put derby.properties: by default it is in the directory where you start the server or the VM that embeds the server. You can change this by setting the system directory, e.g. 

java -Dderby.system.home=/export/home/groucho/movies derbyrun.jar server start
Blog Links >>