I love that they call adding adverts inline into content enrichment :)
It was like changing airplanes mid-flight. They slowly added the new infrastructure around their third-party service, and when it was ready, they would rip it out.:
- They started by injecting the ad selector into the request path, having it purely as a proxy. The system did the same thing it was doing before, but with the ad selector in place. This allowed them to scale up requests through the ad selector without without actually doing anything.
- Then, instead of just being a proxy, they implemented and rolled out native ad selection in the ad selector service. Now, the ad selector would process the requests internally, but still act as a proxy and pass the request to the third party. The system would still use the third party response.
- Then, they added Event Logger to implement logging for native responses, and set up Kafka.
- They continued building out the rest of services, starting with stub services, and added logic along the way.
- Eventually, once everything was in place, they cut off the third party ad server.
I also like that they recommend using a framework/libraries - as long as the packages used are lightweight, importable as distinct units (no interdependencies) and don't get in the way, I think this is a really good idea. The main argument for this as I see it is consistency - if you have a large codebase or several services, the last thing you want is people choosing which library to use without constraints. It's better to have a common foundation for all the code, which typically is or becomes a framework (whether out of house or in-house). Low dependencies is great, but for things like logging services, it's essential that everyone uses the same thing. It's easier to navigate the code base, easier to get new hires up to speed, and easier to write new code, as decisions have been made for you. I suspect companies/people who don't want to use a framework end up writing their own anyway in piecemeal fashion - which is fine, but it's still a framework.