Wednesday, October 5, 2011

Separation of Software Development & Production

When I was managing software engineering/development and data center operations teams at excite, IGN, Fox Interactive, and vudu, a large percentage of my staff questioned my methods. I required the dev team to manage its app in production--including the app servers. I required the ops team to manage the httpd/OS/network stack. I had a separate DBA team to manage the data stores. The dev team wanted to code--not deploy/operate. The ops team didn't want devs to access their servers and wanted the trust to run it all. I wanted specialized teams who I could hold accountable.

I dreamt of the day I'd have enough resources to split each team into dev/ops subgroups. I tried various approximations: ops role for a week; devoting fixed half-days to dev; outsourcing after-hours ops (DBA; netops).

google's Site Operations has the resources to address this dilemma. Here's an interesting video by one of goog's Site Operations Engineers.