The Scale and Transport team is responsible for the feature store for Quantcast’s real time bidding infrastructure, handling ~2 million requests per second over a 10+ TB datastore with p99 latencies <5 ms. Their big challenges include;
supporting a growing array of business critical use-cases for real-time data at scale; handling a high-availability, high-throughput, low-latency real time bidding environment; ensuring consistency and usefulness of offline feature data for use in model training; working with modeling scientists to facilitate effective model training; scoring, and featurization, with an eye towards rapid experimentation and maintaining and extending client software to facilitate ergonomic and efficient use of the datastore.
The team also maintains Quantcast's custom distributed computing infrastructure, including a high performance map-reduce framework, and Quantcast’s open source hdfs-compatible filesystem, QFS.
As a Sr Engineering Manager on the Scale & Transport team, you would grow and develop a skilled group of distributed computing engineers to solve these significant problems of scale. The ideal candidate will understand how to align the roadmap of a platform team with a customer focused company, and bring demonstrated expertise in managing and developing skilled engineers in a technically demanding area. Reliability, efficiency, and cost control are all key to team success!
Responsibilities:
Scale the feature store w/the bidding system while effectively controlling costs and meeting demanding latency and throughput requirements
Work closely with principal engineers to develop, circulate, and execute on a technical roadmap
Develop positive relationships with your customers
Mentor and develop the members of your team from both a technical and soft skill perspective
Build a well-balanced team by partnering with talent acquisition
Preferred Qualifications:
5+ years of engineering experience, and demonstrated experience developing technically strong teams
Experience developing and operating real time low latency, high volume distributed systems AND/OR high throughput distributed computing clusters such as map-reduce or spark
Experience with all phases of software development (design, development, test, operations, end-of-life)
Experience with one of the major cloud platforms
Quantcast is an advertising technology company and the creator of an innovative intelligent audience platform that empowers brands, agencies and publishers to know and grow their audiences online. Our solutions are leveling the playing field for our customers when it comes to effectively reaching audiences online and helping them power a thriving free and open internet for everyone. Founded in 2006 and headquartered in San Francisco, Quantcast has employees in 20 offices across 10 countries.
We are committed to building an inclusive and diverse environment where everyone can be their authentic self. We work to ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.
