Skip to main content

Web Crawling

Posted by daniel on October 11, 2004 at 10:29 AM EDT

A low bandwidth weekend

I spent the weekend in a hotel where the connection to the web was telephone dial-up. This is a good exercise for many reasons. First, this is how many people experience the web and it's good for me to take a look at our site through their eyes. Second, it reminded me of how much I take for granted when I use broadband to connect. See a project that looks interesting, go ahead and download it. When the download time is less than a minute or so, that's an easy decision to make. When it might be hours, that's another matter. I remember the early days of ftp'ing the TeX files off of the archive sites hoping that the connection would hold long enough form me to get the big files. I could (and did) go back and get the smaller ones in separate sessions.

More than a nostalgia session, it is important to test your applications using the various technologies that your targeted end-users will use. Some use Java aphorism of "write once, test everywhere" to claim that the WORA promise was never fulfilled. Testing everywhere can help you identify user problems that you may not notice on your development machine.

Brett Simmons writes about trying to replicate a user problem in The opposite of Murphy's Law. He writes "In software development, what's worse than Murphy's Law is its opposite: that whatever can go wrong, will not go wrong. Today I'm working on an intermittent crash a few people have reported when marking all as read in a group. I have been completely unable to reproduce it, even using the subscription list of one of the people who reported the crash. In other words, the thing that should go wrong - the crash - just positively refuses to go wrong."

In today's Weblogs Tom Ball sees parallels between circus trapeze artists and software developers in The Problem with Unit Testing he writes, "As a development engineer, all I want from unit testing is a safety net, not a huge, QA-approved test monster (for awhile, JavaSoft only had the JCK to do automated testing, which required a very large disk partition and took hours to run). So over the years, I've built several safety nets, keeping them hidden and avoiding names with words like "test", "quality", "verification", etc. The problem with a single tool as a safety net is that like all software it grows over time and becomes its own maintenance headache (to my horror, management learned of one of my test tools and shipped it!)."

Calvin Austin thanks the Tiger umbrella expert group and also solicits you to submit any bugs you may come across. Stuart Simms considers why kids are hooked on games and not Calculus in Gaming in another world .

More Mustang suggestions in today's Forums. BJB writes that IP Sockets "is really a missing technology. http://java.sun.com/j2se/1.5.0/docs/guide/net/socketOpt.html [..] It is not a security nightmare. Permission for using RAW socket will be not granted by default by the VM policy. You have to update the policy file to get it. Once updated, you will still rely on the OS policy to get this working. For instance on XP, you have to be Administrator to get access to the RAW socket API, AFAIK."

Armhold writes that " The single thread model for Swing really places a burden on developers, and even when we manage to figure out how to make it work we end up with some very messy code. You can't block the event dispatch thread, so you fork a new thread to deal with a slow task... but then your slow task needs to update the UI occasionally, so you have to deal with SwingUtilities.invokeLater(), or in some cases invokeAndWait."

Luggypm writes "Here's what I'd like to see in Mustang: 1. MUCH finer grained security permissions in WebStart. [..] 2. Bring JDOM into the core API.3. A copy method on the file object."

In Also in Java Today , in his article on Alerts in MIDP 2.0, Eric Giguere describes the recent additions to the alert mechanism. He explains "During a lengthy operation, it's good to report the operation's progress. Reassurance that the application hasn't stopped reduces the user's uncertainty and impatience. You can add a gauge component to any alert, which will act as an activity or progress indicator [..] Alerts now also support user-defined commands. In MIDP 1.0, a single implicit command was used to dismiss an alert. In MIDP 2.0, you can explicitly add commands to the alert, as you do to other screens."

You don't have to jump to 5.0 to start working with annotations. In fact, it might not be a good idea, since "application server vendors don't always fully support dot-zero releases right away; IDE, profiler, and other tool support can lag; many new bugs have to be fixed; and businesses are leery of building on anything so new." In Bridging the Gap: J2SE 5.0 Annotations, Kyle Downey shows how to use metadata/metaprogramming both in J2SE 5.0 and in several frameworks that are compatible with earlier versions of Java, such as XDoclet, JBoss annotations, and his own P.Anno.

In Projects and Communities , The Mac Java Community has a tip about converting old, OSX-incapable Macs to Linux boxes, allowing them to run Java 2 by way of IBM's Java developer kits for Linux.

Daniel Brookshier reports on the new JELC projects including the girls' Java Cafe, "a female-only community for high school students learning Java."

In today's java.net News Headlines :

Registered users can submit news items for the java.net News Page using our news submission form. All submissions go through an editorial review before being posted to the site. You can also subscribe to thejava.net News RSS feed.

Current and upcoming Java Events :

Registered users can submit event listings for the java.net Events Page using our events submission form. All submissions go through an editorial review before being posted to the site.

Archives and Subscriptions: This blog is delivered weekdays as the Java Today RSS feed. Also, once this page is no longer featured as the front page of java.net it will be archived along with other past issues in the java.net Archive.

A low bandwidth weekend
Blog Links >>