Skip to main content

Google Developer Day Report

Posted by arungupta on May 31, 2007 at 7:17 PM PDT

Follow up from
here.
It's a very long entry, so feel free to skip around, look at the pictures or
read notes from sections of your choice.

I reached San Jose McEnery Convention Center around 9:15am and the crowd
started building closer to 9:30am.

The keynote hall opened at 9:30am. A video with different
mahsups (primarily using Google Maps) was shown until the keynote started
(around 10 minutes late). Here are some of the websites that were shown in
the video: map.movin-360.com (seems
like a virtual tour of Berlin city),
prague360.com (name
says it),
dudewheresmyusedcar.com
(looking for a used car near a zip code),
wepoco.com/maps/observations.php
(making weather and climate information useful),
arena.tv/gastronomie/barfinder.jsp
(finding a bar in Germany),
cleancruising.com.au
(Australian cruising site, another Map mashup),
turnhere.com/sfbayarea (local
Bay area guide), fishingmap.jp,
bimrace.com (tracks your progress in race,
shows elevation for a race and animates).

Anyway, what follows is notes/impressions from the different

sessions I attended
today.


Opening Keynote

The Opening Keynote was given by
Jeff Huber, Vice
President Engineering.



Jeff explained the rationale behind conducting the event at 10 cities
simultaneously that instead of bringing the developers to conference, take the
conference to developers. There is approximately 1500 in Bay area, 5000 over
all, 160 sessions over all.

housingmaps.com digged into the
APIs hidden in JavaScript which motivated Google to create the Maps API. There
are over 50,000 sites using these APIs all over the web.



Evolution of development over last several years: Start from open-source
building blocks, Linux + Apache + MySQL + [Ruby+PHP+Java+Python]. Set of Web
services as building blocks, for example Amazon S3 for storage.



Google's Developer Products are centered around three areas

  • Integrate (Ability to take Google services to create your
    applications)
  • Reach (How do I get distribution of my products, traffic to my
    site)
  • Build (Building next gen web based applications)

Jeff talked about how Google provide infrastructure to build some of the
products like [jogplanner, apartmentfinder, moviemaps. coolphotoeditor,
syncingservice, certifiedusedcars, ok-i'm-kidding].google.com :)



Integrate (Google APIs + Mashup Editor)

GData APIs - Single RESTful Web service model, uses  APP and RSS for
reading/writing to Google's application. For example
picnik (online photo editing),
spanningsync (syncing mechanism to ical
and isync on Mac), Honda
Certified used cars
(Google Base GData APIs, setting custom search).



Announced: Google Mashup
Editor
- Experimental product, online application to create mashups. When I
went online to sign up, then I got the following message:

Google Mashup Editor
The Google Mashup Editor is currently in beta and access is limited to 
small number of developers during this testing period. If you would like to
be notified when spots are available to participate in the beta please sign
up here.

The demo created a mashup of CA state parks, feed from external
source, fed to Google Maps, and then link with Google Base for camping gear in 3
lines of code + HTML wrappers. Finally publish on googlemashups.com or
igoogle.com.



Reach (Gadget +

Mapplets
)


Google's network is the largest network, 500 million users per month. Gadgets
API. Very open model, 100,000 gadgets have been created. Todo List Gadget had
6.7 million page views in last week, Expedia Fare Calendar Gadget (updates every
time you are online with the latest roundtrip fares) had 4.7 million page views
in last week.

10:35am: Seems like they turned on the Air Conditioner and it's
too loud!!



Announcement: Google Mapplets
- Combined Gadgets and Maps API.



Orbitz mapplet, Weatherbug. The cool thing
is you can show data from different sources such as Hotel and Weather on the
same map. Combined Orbitz and WeatherBug with Google Maps - this combination is
more useful than any of the individual sites. Called as "enabling a mashups of
mashups".



Build (GWT
+

Gears
)




Announcement:

Google Gears
- offline access for web apps, cross-browser, cross-platform,
BSD-license. A way by which developers can add off line capability to your
applications in a standard manner. Browser extension to allow web apps to work
offline - Firefox, IE, Safari + Mac, Linux, Windows. Launching as open source
project.



This is an evolutionary approach where end users continue using their
application as they are using today w/o worrying about network connection,
Developers continue building the application using their existing skills and
code base, simply adapt to Gears API.



Also launching Google Reader Online -
Working with Adobe, Opera and Firefox for industry collaboration. Kevin Lynch
from Adobe showed a Flex/Apollo application with very similar online and offline
experience.



Sergey Brin (co-founder
of Google) made a cameo appearance. He talked about how the "person" involved in
maintaining the Internet needs to be out-of-the-loop for it to truly
self-sustaining and how

Mapplets
are leading towards that. And of course, payback to community which
created such a fantastic ecosystem.

 


Google Maps API Introduction

The complete presentation is available at
doug.ricket.com/gdd2007. Doug
Ricket, the speaker, used the preso from this link.

API is all about writing little bit of JavaScript and do cool things. Here
are some examples of how Google Maps API have been used:

  • onNYturf.com - Subway map
    of NY city on Google maps (classic API set - some data put on map, useful to
    users)
  • trulia.com -
    Real estate search
  • Panoramio - Upload your photos
    and tag them with a specific geographic location
  • wikimapia.org - People are
    adding content all over the world to communicate (entirely user generated
    content)

The trend is towards more collaborative.

Load maps API package (indicated by "file=api"). There used to
be a limit to the number of times this API key can be used in a day. But this
limit is now removed. However if 5 million users are expected for the key, then
contact Google to ensure sufficient server capacity.

"v=2" indicate that there is a big change in the API and
required changes in the application code. In reality, you'll get the latest 2.x
release, but behind the scenes versions are getting incremented and you get the
latest version.

2.s is a more stable release for enterprise customers, solid and no bugs. You
can also specify an exact version.

The fact that you can drag the map is the "big feature". Usability studies
indicate that the cursor need to be changed to hand instead of mouse pointer (at
least 2.51) - 2.50 you've pointer, 2.51 has a hand.

Because exact version numbers can be specified, almost infinite backward
compatibility is available.

Key is registered for your specific website. For the public API, completely
free but your site also need to be freely available. The basic API set has to be
open to public. If you want to run it inside corporate firewall, then you need
to use Enterprise and you get support from the Google team.

Follow along the presentation
for all the detailed examples.

In Google Maps terms, InfoWindow is what is called as Balloon, Bubble or
Pop-up. Marker is the red balloon, Title option to Marker is the tooltip.

Overlays is anything that can go over the map, for example Marker, Polylines,
Polygons, Image and others. Can also create a custom overlay, basically any HTML
can be converted into an overlay. There are few overlays that are available as
default and then forums where people have shared their overlays.

Q. How far can you zoom in a particular level ?

A. Access to whatever is available to Google maps, in the area of 22 zoom
levels. Certain level that is available all over the world. No API to get that
level. The documentation says the default level available at all levels, but
it's likely 18.

Q. When Maps are not available, can people provide their own tiles

A. Yes.

Icons: Any icon (local or remote) can be used on the map. After 200 markers
(on Firefox) and 100 on IE shows performance degradation.

Asynchronous HTTP: Static data in your JavaScript file. Or separate the data
from the code. GDownloadUrl downloads XML file, parse the data (GXml.parse) and
create a marker (addOverlay(new GMarker(..))). If it's huge amount of markup,
then there could be a performance hit.

Geocoder: Translating in address into lat/long and mapping it.

Full documentation:
www.google.com/apis/maps/documentation

Q. Will Google add the ability to print satellite views of the maps

A. Not right now but working on it.

Q. Do you have Routing APIs

A. Come to the advanced session.

Q. Is there a way to highlight marker

A. Yes through JavaScript, use GMarker.setImage and highlight the marker that
user mouseover.

Q. Can the Geocoder return the quality of geocode

A. It just returns null, check the documentation.

Q. Can you display a title for each marker without mouse over

A. Markers use an image, open source publicly create Overlay called TextMarker
would serve that purpose, or create your own in 20-30 lines.

Q. Can Flash be embedded in Info windows

A. Put any HTML in the Info Window.

Q. Displaying Raster data on the map like a bitmap

A. Come to Advanced session. Take an image, put it down on the map anywhere and
then add overlays. Or an API where you can specify tiles for different zoom
levels.

"Go Forth and Create Mashups" was the key message.

The multi-cuisine (Asian, Mexican, Italian ...) lunch gave a feeling of what
Googlers might be eating
every day.


A Look Behind the Scenes at Google

Jeff Dean, Google Fellow
gave this talk explaining "Building a Computer System for the Worlds'
Information". The talk was all about behind the scenes look at Google systems
infrastructure.

Every Google service sees continuing growth in computational needs = More
queries + More data + More results.

The goal is to create very large scale, high performance computing
infrastructure => Hardware + Software systems to make it easy to build products.
Focus on price/performance and ease of use.

The Hardware Design Philosophy is:

  • Buy as many commodity class PCs (Prefer low-end server/PC-class designs)
  • Larger number of unreliable computers than a big unreliable computer
    because reliability can be handled in software
  • Rather parallelize and get latency down
  • "Ultra-reliable" hardware makes programmers lazy
    • Fault-tolerance to be built into software

The following set of pictures show how Google's infrastructure evolved over
the years:

 

The three main components in Google's infrastructure are:

  • Why GFS ?
    • Unique file system requirements
      • Huge read/write bandwidth
      • Reliability over thousands of nodes
      • Most operating on large data blocks
      • Need efficient distributed operations
    • Have control over apps, libraries and operating systems

MapReduce - a
programming model and library to simplify large-scale computations on large
clusters.

Map phase extract relevant information from each record of the input, Reduce
phase collects data together and produces final output.

MapReduce is

  • Fast (locality optimization, optimized sorter, lots of tuning work)
  • Robust (handles machine failure, bad records)
  • Easy to Use (little boilerplate, supports many formats)
  • Scalable (can easily add more machines to handler more data or reduce
    the run-time)
  • Widely applicable (can solve a broad range of problems)

Lots of products within Google use it.

BigTable - A
large-scale storage system for semi-structured data

  • Higher level API that a raw file system
    • Somewhat like a database, but not as full-featured
  • Useful for structured/semi-structured data
    • URLs
      • Contents, crawl metadata, links, anchors, pagerank ...
    • Per-user data
      • User preference settings, recent queries/search results ...
    • Geographic Data
      • Physical entities, roads, satellite imagery ...
  • Scales to large amount of data

Allows to do low-level storage optimization help performance significantly

  • Distributed multi-level map
  • Fault-tolerant, persistent
    • Uses GFS underlying
  • Scalable
    • Thousands of servers
    • Terabytes of in-memory data
    • Petabytes of disk-based data
    • Millions of reads/writes per second, efficient scans
  • Self-managing
    • Servers can be added/removed dynamically, adjust to load imbalance

Lot of internal pre-processing such as satellite imagery. Largest BigTable
manages ~3000TB of data spread over several thousand machines (larger cells
planned)

Used by Google Print, My Search History, Orkut, Crawling/Indexing pipeline

Q. What flavors of Linux ?

A. Some using 2.4 and variants of 2.6

Q. Do you do destructive testing ?

A. Definitely yes. A battery of tests, take a data center offline, much more
load than expected and see if it degrades gracefully and much more.

Q. Do you make an attempt to measure that interesting data grows than raw
data ?

A. Raw data requires computation to make interesting data. Not looked at
different class of data.

Q. Will we see an open source version of GFS ?

A. There have been some discussions about it which would make difficult to open
source it. There is an open source structure (Hudop?) that created a mimic of
MapReduce.

Q. Do you modify the Linux kernel ?

A. Do modify as problems are discovered, lot of them are contributed back to the
community.

Q. Few 1000 thousand servers, what policy is about recycling and green
computing ?

A. Generally donate after outlived it's reasonable life and try to repair the
machine instead of throwing it away.


Mashups the Easy Way

First mashup - housingmaps.com. Paul did it before the Maps API.

Whole bunch of things before you can make/release the mashup.

Three parts to create a Mashup:

  • Feeds
  • User Interface
    • Common set of Ajax UI elements (cross-browser compatible)
    • Simple declaration of Ajax components (similar to using HTML tags)
    • Flexible way of using UI controls programmatically
  • Infrastructure
    • Lots of machines and serving capacity
    • Hosting or serve infrastructure with no setup up
    • Simple data store modeled after a feed
    • Ability to sandbox application for testing
    • One click deployment of application

Experimental project built by a small team in NY

  • An Ajax application that lets you built other Ajax applications
  • JavaScript library
  • Hosting container at googlemashups.com.

Feeds -> UI -> Test -> Publish -> Distribute

Mashups and Simple Applications made easy.


Google Desktop Gadgets - Access, Share and Personalize Information

The speaker started with a survey and the results were:

  • 80% use gadgets on desktop
  • 50% use Google gadgets
  • 5% develop gadgets

2007 - The Year of the Gadget ?

Specialized mini-application that runs in a web page or on the desktop. There
are two types of widgets:

  • Universal gadget
    • Minimal web app
    • No downloads necessary
    • Needs online connection
    • HTML + JavaScript
  • Desktop Gadget
    • Runs on the desktop
    • Advanced functionality
    • Works online/offline
    • XML + JavaScript

The talk focused on desktop gadgets.

Why develop desktop gadget ? Instantly reach millions of users, Easily
distribute content, Increase traffic to your sites.

Gadget is a zip archive with .gg extension and the following structure:

An easier way to develop a gadget from scratch -
Google Desktop Gadget
Designer
- allows you to easily experiment with WYSIWYG interface. It then
showed a simple demo to create the gadget using Designer.

Advanced APIs - Add Flash, Graphics, Drag/Drop, Transparency, Download
streams or Internet but following are the interesting ones:

  • Communication API - send and receive data using the Google Talk
    framework. For example, multi-user games, social networking gadgets.
  • Query API - find related information by searching for a specific term in
    the user index. For example, music or photo collection organizers. Searches
    the local desktop and prepare a list.
  • Event API - Provide contextual information based on the users actions.
    For example, For web pages containing an address, show a map of the place.

Useful links:

Next steps:

  • Submit your gadget to Google to reach more users.
  • The gadget will appear in the online gadget gallery and in the 'Add
    gadgets' panel on the users machine.
  • Gadgets are automatically targeted and recommended to users who are most
    likely to use them.
  • Populare gadgets will give you the chance to get into
    Google Desktop Hall of Fame

Key Take-Aways

  • 2007 - Year of the Gadget
  • Desktop Gadget benefits:
    • Easy to develop and install (Only JavaScript + XML required and
      WYSIWYG Designer)
    • Easy to use, low resource requirements
    • Powerful APIs offer opportunities for feature-rich designs
  • Through the Google Desktop Gadget Gallery developers can:
    • Reach millions of users around the world
    • Target your gadget to users most likely to see it
    • Distribute your content and increase traffic to your website
    • Become famous as a top gadget developer


Custom Search Engine - Search in Your Hands

Ramanathan Guha
started Custom Search Engine at Google.
Google Custom Search
extends Open Publishing to search.

Why Custom Search ?

  • Anyone, anywhere can create a substantially better search experience for
    their chosen topic
  • No more complexity that creating a webpage
  • The usual incentives apply: fame, fortune, pride ...

What Custom Search provides ?

  • Harness the power of Google search
  • Put search results on your website
  • Customize the look & feel
  • Invite friends
  • Make money

Allows producing a custom search on demand.

That's the end of sessions. All in all, it was a great crash course for
different Google offerings.

Technorati:
googledeveloperday
gdd
gdd07
gdd07us

Related Topics >>