September 09, 2015
Everything you wanted to know about writing async, high-concurrency HTTP applications in Java, but were afraid to ask With tens of millions of clients continuously downloading binaries from our repositories, we decided to offer an OSS client that natively supports these downloads. In this talk, we will share the main challenges in developing a highly-concurrent, resumable, async download library on top of Apache HTTP client. We will cover other libraries we tested and why we decided to reinvent the wheel. We will see important pitfalls we came across when working with HTTP and how using the right combination of techniques can improve performance by an order of magnitude. We will also see why your initial assumptions may completely change when faced with other players on the network. Consider yourself forewarned: lots of HTTP internals, NIO and concurrency ahead!