Contribution Activity Metrics – Early attempts and fails

As we examined with the intro post, the need for contribution activity metrics for different contribution areas in Mozilla has been high. It was only logical that many attempts were made to address this issue, mainly on the area-level (and not in Mozilla-wide level). Almost all of them had zero interaction between each other, and there was a general lack of vision for an holistic approach to the problem.

After one of our initial gatherings as the (then meta-) Community Building Team, a couple of people brainstormed together a possible solution to our problem. Together with Josh Matthews, Giorgos Logiotatidis, Ricky Rosario and Liz Henry a new approach was born. Enter project Blackhole!

Project Blackhole was a collaborative effort to develop and maintain an infrastructure of gathering and serving raw contribution data within Mozilla. We created a data architecture and flow together with a data Schema and specification to describe contribution activities for the first time in Mozilla. The project went far enough (thanks to Josh) to create a working prototype for back-end and front-end.

What went right:

Having a single project to drive multiple metrics efforts forward got people engaged. Everyone saw the value of de-duplicating efforts and tapping into that as a resource. Also during the process of designing and testing it we were able to self-identify as a group of people that share interest and commitment towards a common goal. Most of those people went on to become active members of the Systems and Data Working Group. Finally, we ended up with a common language and descriptions around contribution activities, a really valuable asset to have for the future of cross-project tracking.

What went wrong:

Building *anything* from scratch can be hard. Really hard. First, everyone (rightfully) questions the need to build something instead of re-using what is out there. Once you get everyone on board, development and deployment resources are hard to find especially on a short notice. On top of that Blackhole’s architecture *seemed* logical enough in theory, but was never tested in scale so everyone involved was not 100% sure that our architecture would survive stress tests and the scale of Mozilla’s contribution ecosystem.

PRO TIP: Changing the project name does not help. We went from “Blackhole” to “Wormhole” (and back to “Blackhole”?), to better reflect the proposed data flow (data would not disappear forever!) and people got confused. Really confused. Which is obviously something that is not helpful during conversations. Pick a name, and stick to it!

Lack of a team dedicated to it and inability to get the project listed as a personal goal of people (or teams), halted any progress leading us to a fearsome dead-end.

What we learned:

As with most failures, this one was also really valuable. We learned that:

  • we need to be a top line goal for people and teams
  • we need to examine really well what is out there (internally or externally to Mozilla) and investigate the possibility of re-using it.
  • we need a clear and common language to make communications as effective as possible
  • we need to be inclusive in all our procedures as a working group, with volunteers as well as all paid staff.
  • and in true Mozilla fashion: we need to start small, test and iterate with a focus on modularity.

A way forward?

Having those lessons learned from the process, we sat down last December as a group and re-aligned. We addressed all 5 issues and now we are ready to move forward. And the name of it? Baloo. Stay tuned for more info on our next detailed post :)