Recently, I caught up with Lee Bieber, a Sun engineering manager in charge of, among other projects, Drizzle, Sun's open-source project for a lightweight, optimized SQL database for cloud and Web applications. He mentioned a demo application being developed by his team to create a highly scalable search engine that retrieves, indexes, and searches for data. The application will be a key demo at the Drizzle and memcached POD at CommunityOne West next Monday, June 1, and at JavaOne later in the week.Besides Drizzle, the application takes advantage of three other open-source technologies:

• Gearman, a job-distribution system that dispatches jobs to clients best suited for the tasks involved
• Sphinx, an SQL full-text search engine
• memcached, a high-performance, distributed memory object-caching system that
  speeds up Web applications by alleviating database load

Among the project engineers is Eric Day, whose recent blog postings impressively chronicle the progress of the development process. For details, see today's writeup Narada - A Scalable Open Source Search Engine, complete with an illustration of the task flow.

According to Eric, a primary purpose of the demo is to showcase those four potential-filled projects, which have benefited from many contributions from their community members since inception. In particular, Patrick Galbraith, whose book on developing Web applications will be off the press soon, has been a major player.

Why Drizzle, Gearman, memcached, and Sphinx? Eric explains, "Open source is naturally a major reason. That means the code is free for enhancement and customization by the public. Second, scalability rules. To head off bottlenecks, Gearman will break the app's tasks into small pieces and start as many 'workers' as appropriate. Plus, these projects have been attracting more and more adopters in the recent months with concurrent hikes in site traffic. Many folks have joined the mailing lists, too, sharing their experience, expertise, and ideas."

Attending CommunityOne or JavaOne next week? Be sure to drop by the Drizzle and memcached POD to see Drizzle, Gearman, memcached, and Sphinx in concerted action.