16th*, 17th & 18th August 2017 info@golanguk.com @golangukconf

Video archive

Complex Concurrency

Video details

Speaker: Evan Huus
Length: 0:53:45

Description

There is a lot of power available through Go’s concurrency primitives, but they are still just that: primitives. Typical examples of channel and goroutine usage show pipelines and worker pools, but real applications are rarely that simple. Production systems can easily end up with hundreds of goroutines running dozens of interdependent functions; reasoning about the resulting interactions is not easy.

Shopify’s Sarama library is an open-source fat client for LinkedIn’s Kafka project. It is heavily concurrent, making use of Go’s concurrency primitives to build several complex patterns, including fanout-and-shuffle among others. This talk will cover some of the patterns used in Sarama, including the challenges faced, problems overcome, and trade-offs made. Sarama has been running in production for nearly a year and a half across Shopify’s infrastructure, processing hundreds of millions of messages per day.

Stay up-to-date with Golang UK

Enter your email address to join the Golang UK Conference mailing list and be the first to hear our latest news and announcements.