JFrog's New Bintray Platform, and the Technology Behind It
Last week, JFrog's online software binaries repository Bintray was made available to the public. What's Bintray? JFrog's announcement calls it "the first social platform for developers to publish, download, store, promote, and share software libraries (binaries) across a unified community." One way of looking at Bintray is that it's a "GitHub for binaries."
Bintray already has some major users. For example, Netflix is distributing its denominator executable JAR package through Bintray. Denominator "is a building block for portably controlling DNS mappings across cloud providers. denominator exposes features present on a subset of DNS providers, such as GEO or Latency mapping."
As you can see in the image, Bintray has a social aspect as well. In fact, its design illustrates how some of the communication techniques that have been made popular by social networking sites like Facebook can be put to good use by cloud-based platforms that serve the needs of developers. For example, the Netflix Open Source Projects page includes a feed that shows the organization's latest activity (repositories created, packages created, etc.).
And, as a registered user, you can join, watch, or message other users. So, for example, say you're interested in knowing when Netflix OSS adds a new binary to Bintray. All you have to do is watch them, and you'll be notified whenever that happens.
After I created my account, I saw a message "Try out the JCenter repository if you're here for Java packages!" There, I found that more than 44,000 Java packages are already on Bintray!
Under the Bintray hood
After looking at some of the HTML source for Bintray pages, I became curious about the site's underlying technology. So, I asked JFrog co-Founder and CTO Yoav Landman about this. I think Java developers will find his answer quite interesting:
Bintray is composed of a clustered web layer that serves all UI and REST interactions, and of an independent, multi-datacenter download service.
The web layer is written using Grails and Jersey. It interacts with an ElasticSearch cluster for searches and with a MongoDB backend cluster for persistency. Grails' GORM is heavily used with some own customizations.
A thin download service utilizes Nginx, Grizzly and Jersey, talking to CouchDB clusters and to a distributed Object Store that is synced up to the CDN.
Redis with the lightweight Jedis java client library is used for job management, such as building repository indexes and stats aggregation.
Bintray is hosted on a multi-cloud environment, composed of both dedicated, physical servers and cloud instances.
That's a pretty Java-laden set of infrastructure! Not that this should surprise us, given JFrog's prominence in the Java community. Maybe the JFrog team will consider giving a presentation on how they put all this together at JavaOne this September. That would be a very interesting session, and the JavaOne Call for Papers is still open!