Skip to main content

JavaOne 2013 Impressions #4: How to Utilize Hadoop to Process 30 Characters in just 34 Seconds! And How to Do Much, Much More...

Posted by editor on September 26, 2013 at 12:37 AM PDT

What happens if you configure a Hadoop-centric scalable big data infrastructure running on the Amazon EC2 cloud, that's programmed to perform some very basic processing on delimited ASCII data records (about 30 characters in each record), and you feed it a single data record? How quickly do you think that massively powerful system would process that teensy, trivial input data set? Milliseconds? Microseconds? Nah, surely that type of system would blast through a single record in nanoseconds, right? Or even picoseconds or femtoseconds or attoseconds? In zeptoseconds? Yoctoseconds? (Anyone know what comes next?)

The answer, as it turns out, is 34 seconds. I found this out in the Wednesday JavaOne session Pragmatic Big Data Architectures in the Cloud: A Developer