KPI Dashboard on

Mozilla Reps as a program is full of activities. Reps around the world, do extraordinary  things everyday, promoting Mozilla’s mission and getting new contributors on board.

Moving forward, trying to identify how those activities align with top-tier initiatives, Mozilla Reps program wanted a way to visualize some Key Progress Indicators around the program.

We (the Participation Infrastructure team) sat down with the programmatic owners of Reps (Nuke & Rosana) and identified what numbers and metrics we would like to expose in a much more digestible way, so we can assess the progress of the program on many levels.

We identified 3 different KPIs:

  • Number of Reps (and growth rates)
  • Number of Events (and growth rates)
  • Number of Reports (and growth rates)

… and also 3 different filters you can apply on those numbers:

  • Country
  • Functional Area (of Mozilla)
  • Initiative (associated with Rep, Event or Report)

You can find the spec for this initial iteration here.

We decided to have the filters as drop-downs, applied on the whole page (in combination or one-by-one). Then for each KPI group we would have a time graph for the past 6 weeks (fixed for now) with a HUD of basic numbers and growth rates.

Screenshot from 2015-04-24 13:39:29Technology-wise, we tied the coding of this new dashboard to the delivery of a proper API for the Reps Portal (more info on that soon). The new API enabled us to easily create custom endpoints to calculate the numbers needed for our Reps KPI graphs (based on the existing Conversion Points). Nemo and Tasos did a fantastic work to deliver the new API and the custom endpoints, while making sure that this is not heavy on our DB.

Nikos then worked on the front-end using D3.js as the visualization library to create the graphs dynamically (each time you access the page or you filter using Country, Area or Initiative).

Screenshot from 2015-04-24 13:37:05The overall result is smooth and easily helps you assess progress of various Areas and Initiatives on specific Countries, for Reps, Events and Reports.

You can check out the dashboard here.

Next step would be to introduce a time-slider for customizing the time range you want to be displayed.

FOSDEM 2015 Bug sprint tool

“Reclama” Experiment report


FOSDEM 2015 is a premier open source developers’ event in Europe. Mozilla is heavily participating in the event for over 10 years now, with booths and dev-rooms. This year the Community Development team decided to run an experimental approach on recruiting new contributors by promoting good-first bugs.


Given the highly technical nature of the audience in FOSDEM 2015, the approach decided has a straight-forward promotion of bugs and prizes for them, for people to track and get involved.

Following sign-off meeting with the stakeholders (William Quiviger, Francisco Piccolini and Brian King) the specifications agreed for the first iteration of the experiment were as follows:

  1. A straight-forward interface displaying hand-selected good first bugs, ready to be worked on
  2. Next to bugs, prizes should be displayed.
  3. Viewing from mobile devices should be also accounted for.
  4. Public access to all content. Ability to edit and add events on selected accounts (hardcoded for v1)
  5. The interface has to be properly Mozilla branded (high visibility and promotion in an event)
  6. It needs to be future-proof for event to come (easily add new events and/or bugs after deployment)
  7. Solution URL should be short and easily memorable and digestible.
  8. It needs to be delivered before the start of FOSDEM 2015 😉
  9. A report should be compiled with usage statistics and impact analysis for the experiment


Given the extremely short timeframe (less than 3 days), ready, quick or off-the-shelf solutions were evaluated like:

  • An online spreadsheet (gdocs)
    • Not meeting requirements #3, #5, #6, #7
  • A bugzilla query
    • Not meeting requirements #2, #7, #6
  • A scrappy HTML plain coded solution
    • Not meeting requirements #4, #6

Thus, given the expertise of the group it was decided to create a django application to meet all requirements in time.

Following 2 days of non-stop development (fantastic work from Nikos and Nemo!), testing and iteration we met all requirements by developing the app which is codenamed “reclama” (Italian -shortof-  for claim).

Code can be found here:

Screen Shot 2015-02-27 at 14.57.26Deployment

In order to meet requirement #7 (short and memorable URL) and given the timeframe, we decided to acquire a URL quickly ( ) and deploy the application.

For usage statistics, awstats was deployed on top of the app to track incoming traffic.

Usage statistics

During the weekend of FOSDEM 2015, 500 people with almost 5000 hits visited the website. That’s almost 10% of the event participants.mozbugsprint-stats

Saturday 31-Jan-2015 traffic analysis

mozbugsprint-saturdayBooths and promotion of the experiment started at 9:00 as expected. With mid-day (noon) pick which is consistent with increased traffic in booths area of the event.

Traffic continues to flow in steadily even after the end of the event, which indicates that people keep the URL and interact with our experiment substantial time after the face to face interaction with our booth. Browsing continues through the night, and help might be needed (on call people/mentors ?) during that too.

Sunday 1-Feb-2015 traffic analysis

mozbugsprint-sundaySecond day in FOSDEM 2015 included a Mozilla dedicated dev-room. The assumption that promotion through the dev-room would increase the traffic in our experiment proved to be false, as the traffic continued on the same levels for day 2.
As expected there was a sharp cut-off after 16:00 (when the final keynote starts) and also people seem to not come back after the event. Thus the importance of hyper-specific event-focused (and branded) challenge seems to be high, as people relate to that understanding the one-off nature of it.


32 coding bugs from different product areas were presented to people. 9 of them were edited (assigned, commented, worked on) during or immediately (one day) after FOSDEM 2015. Out of those 9 , 4 ended up on first patch submission (new contributors) and 3 received no response (blocked contributor) from mozilla staff (or core contributors).


  • Re-do experiment in a different cultural environment, but still with related audience, so we can cross-compare.
  • Continue the experiment by implementing new functionality as suggested by the stakeholders (notes, descriptions etc).
  • Experiment random sorting of bugs, as current order seemed to affect what has been worked on.
  • Submission of bugs to be featured on an event should be coordinated by event owner and related to event theme and topics.
  • A/B test how prize allocation affects what bugs are worked on.
  • Expand promotional opportunities throughout the event. (booth ideas?)
  • On call developers on promoted bugs would eliminate the non-answered section of our bugs

PS. Special thanks to Elio for crafting an excellent visual identity and logo once again!

Mozilla Arabic Meetup 2014 and thoughts on Regional Support

Last weekend I had the pleasure to be among the Mozilla Arabic meetup for their annual community meeting, this time in Istanbul, Turkey.

The meetup schedule was packed for two full days, and we barely had time to cover all planned items. We made it though, thanks to the fantastic organizing team (Melek, Sofien, Majda, Rami (who joined remotely), Migdadi and Nefzaoui)

Note to self #1 This is once again a reminder that such 30-people meetups that happen annually (or in less frequency) need to run beyond 2 days. The addition of half a day on Friday would tremendously help, enabling everyone to sync up, bringing people up to speed and informing the schedule of the next two days.

The first day was dedicated on meta-community organization issues. Arabic community is a group of regional communities that are coming together under same goals (especially around l10n). The challenge on having such a meta-community is that the regional ones already have structure, leadership, pace and goals in place, and those might not necessarily be compatible between each other. We initially spent some time to determine the shared functions, roles and goals that should be dealt on a meta-community level rather then individual community one (things like: l10n oversight, Arabic community visibility, cross-community events and activities etc). The structure proposed (which I totally support) is forming a coordination committee with a rolling chair. Each community gets to be the chairing (“hosting”) one, driving and coordinating the meta-community for a period of 6-months. Then another community takes over.

The notable pros of this approach is the shared load over time, the visibility this brings to individual communities, the helpful exposure to different coordination styles and the sense of involvement and leadership all communities will get to experience. The ball is already rolling with this approach and a meeting next week will determine the first chairing community and finalize the way forward.

IMG_20141102_235026-SMILESecond day was more project specific. We had 3 core themes (L10n, FirefoxOS and Webmaker) and we split up in groups to have sessions on those. Partially training, partially brainstorming on next activities on the region, it was a productive experience for both participants and session owners. Haven’t showcased WebIDE to people? Introduce them to the magic of developing apps with Firefox Desktop and watch them drool.

During the meetup we also had a long session on participation and community building (which was kinda different from the approach taken on previous meetings). This time we introduced the idea of “Innovation from the edges” to people and brainstormed under two arcs: “Innovative ideas that you would like to work on” and “Ways that the rest of the Mozilla project could help you“.

Stating with the realization that Mozilla Project (supported by Mozilla Corporation and Mozilla Foundation) could not plan, execute, innovate and support all possible activities and projects that advance our Mozilla Mission, we let people loose to come up with regional (and global) activities and projects that would bring innovation to Mozilla and help us advance our mission. The response was enthusiastic and informing. People quickly came up with ideas that they would like to work on ranging from engineering projects to partnerships with other projects on the ground. More interestingly, patterns emerged under the arc of “how the rest of Mozilla can support you“. Hands-on training (technical or not), mandate to represent Mozilla, access to tools and systems (in an open way) and resources around IT, were some recurring themes that we identified. All these will be taken back to the Mozilla Community Building team and the appropriate Working Groups to inform our strategy for the near future and enable us to support regional and functional communities better.

Note to self #2 Budget and Swag (our default go-tos for regional support) were not even mentioned on the “how we can support you” session. We may need to rethink many of our assumptions moving forward.

I am confident that the Arabic Community has a solid way forward planned after this meetup, and I can’t wait to see the results of it. As for the learnings that we got out of this weekend, we need to evaluate them and plan the way forward for participation strategy informed by such inputs.

Event wiki page:
Analysis of the community:
Action plan:

Systems and Data principles

For a year now the Systems and Data Working Group of Mozilla has been meeting, brainstorming about community building systems, designing and implementing them and pioneering new ways to measure contribution activity across Mozilla.

In the process of evaluating existing systems (like and creating new ones (like Baloo) it was obvious that we needed a common set of principles to apply on all systems that are in service of community building within Mozilla. That would enable Mozillians to easily access tools and contribute in a way that maximizes impact. We as the Systems and Data Working Group recommend these principles be adapted for all tools used by Mozilla.

The principles written in buckets are:

  • Unified Identity
    • Tools should have single source of truth for people data
      • Integration with HRIS
      • already has staff and volunteer information, so it is a good candidate at the single source of tr
    • Tools should de-duplicate people information by integrating with a single source of truth
    • e.g. Reps: Not integrated with, lots of duplicate information on two profiles
  • Unified Authentication and Authorization
    • Tools should use a single identity platform that provides permissions-based access to tools (like
    • e.g Reps: add people to the Reps group on to give them permission to use as a Rep
  • Accessible Metrics
    • Tools should track each contribution a Mozillian makes and provide it in an accessible way to create a holistic view of contributions
  • Localization
    • Tools should be localized so they are accessible to our global community
  • Education
    • Tools should teach the user how to use the tool, answer common usage questions, and have general documentation
  • Recognition
    • Tools should recognize the contributions that they enable
  • Participation
    • Tools should enable anyone to improve the tool by filing bugs, suggesting ideas and bringing those ideas to life
  • Content de-duplication
    • Tools should de-duplicate the content that is created in those tools, making it accessible to other systems
  • Fun
    • Tools should be personal and written in the Mozilla voice

This has been a collaborative effort involving various stakeholders of tools within Mozilla that have been reviewing those and providing feedback during our meetings. We are seeking more feedback moving forward especially with regards to how those impact the Roadmap of various tools and translate to actual features. Feel free to comment here or join our discussions in the community-building mailing list.


Connecting Baloo with

It was only a matter of time to connect the dots. As we saw in a previous post, we have been working with Adam Lofting on publishing a public dashboard for contribution activity metrics. The data we had were based on one-off exports from Github for demo purposes. The intention was to feed the dashboard with data from Baloo, our single source of truth about contribution activity in Mozilla.

Thanks to the hard work of Sheeri Carbal, Anurag Phadke and community builders on various contribution areas across Mozilla, this connection is now live.areweamillion_balooNavigating to you can see the total counts of Active Contributors in Mozilla with drill-downs to specific teams and systems.

The data flow can be briefly described as this: Databases for integrated systems (Github, Bugzilla, SuMo for now) are scrapped for activity info based on our Schema, resulting in a formatted database full of raw contribution data. Then we apply aggregations per system and per area as defined by Community Builders in our Conversion Points tables to create active contributor counts while de-duplicating them across projects. Aggregations are then exported and captured by a nodejs app feeding info to our public dashboard.

More systems are in the pipeline to be integrated (Reps, MDN, Location Services and others) really soon. You can track the progress (and request integrations) through the Baloo wiki page.

Damned Lies and Contribution Metrics

The power of numbers is unquestionable. I never fully understood though, what it is. Possibly the urge of everyone to explain the world rationally. Or the need for reference to make any decision an “informed one”. Whatever it is, it drives people. Mozilla wouldn’t be an exception.

The ask was simple enough:

How many active contributors do we have in Mozilla?

No one knew last year, that a year in today we would only have scratched the surface of this question. But in the process of doing so we laid a solid foundation to move us forward.

Yesterday Adam Lofting announced the unified Mozilla Foundation and Mozilla Corporation contributors dashboard which you can check out visiting This has been a collaborative effort between both teams and an incredible journey so far exploring and articulating notions of contribution metrics across the Mozilla Project.Screen Shot 2014-06-12 at 1.06.33 PMProject Baloo (intro post here) is underway to supply all the data that will fuel the unified dashboard, starting with Bugzilla, and Github data, thanks to Sheeri and Anurag from BI team. Next in line are Reps, SuMo and MDN. All data will be gathered in a central database, in a common schema, updated almost instantly by the systems that activities are happening. You can track the progress here.

Adam’s post has all the technical details about the current implementation (so I will not go into details here) but I would like to expand a bit around the importance of deduplication and cross-examination of metrics between different teams of Mozilla.

Being a Community Builder inside Mozilla you want metrics for your contribution area. You can see people come and go, but you have no idea whether those people are moving to other teams or leaving Mozilla completely. With cross examination of contribution metrics we will be able to see trends and movements of people across different projects and teams for the first time.

Using deduplication of identities (based on emails) we will get a much more accurate count of people, that will improve even more once we integrate with and Workday so we can deduplicate people using multiple emails. Anecdotally (and based on the initial real data we have) we know for sure that the actual count of active contributors will be considerably lower that the sum of active contributors on all teams.

Deduplication(1)Expect more updates to come as we roll new integrations in and new data-sets become available.

Mozilla Location Services – A story of intentionality and growth

Since the start of this year I joined the Community Building team with a task (among others) to abstract community building best practices and apply them to teams that haven’t had any dedicated community building resource, forming a strategy around community with them.

Over 7 months ago the Services team of Mozilla announced a new project. The Mozilla Location Service (MLS for short). Given the priority of the project, community excitement and my passion about geo-related projects I was assigned as a community builder in a supportive function.

Since it started, over 4 thousand people have been contributing to the project. What is interesting about this contribution activity is that although engagement of potential contributors was relatively low, and the call to action was not widely advertised, the influx of people was steady and global. Though we can speculate in general about the source those contributions, we can also safely say that the vision behind the project and the low barrier to entry contributed a lot towards this influx of people.

As the months went by, location services team wanted to understand better the contribution that was happening, assess it, and act based on a community building strategy. The immediate need was the definition of a contribution path. Given the structure of the program that was fairly straightforward. A contributor downloads MozStumbler, installs it and then starts walking around. The next step for a contributor would be to opt-in for a nickname associated to his/her contributions so that he/she participate on the leaderboard and for us to have more meaningful contribution activity data. Articulating a pathway also helps on identifying bottlenecks and the overall health of the community, and we are now in the process of defining the active contribution thresholds.

At that point onwards, the question that was raised within our community building meetings for MLS was around the “intentionality” of the community building. It is one thing to have a program open for contributions and a totally different one to facilitate and encourage contributions, assessing the community health in parallel. The shift towards intentionality for community building, requires a significant resource commitment that any team within an organization would naturally be reluctant to make. As a supportive community builder I proposed a community building pilot approach to evaluate the community engagement and contribution possibilities.

Quoting Erin Lancaster, one of the key drivers of this effort:

A community builder is essential in order to connect the technical team directly to the very people who care enough about the project in order to devote their free time to helping us out. [A community builder is] also key to ensuring that the community is empowered with the details so they can hit the ground running and contribute while being able to distill information back to the dev team.

Our fantastic community in India was selected as the host for the first pilot. For our first event we would try to get people together for a stumbling-party in Bangalore and assess the contribution rates, spikes and ripples that the event would create, against our investment towards the event. Deb, Vineel and Galaxy, our awesome local leaders organized the event and by tweaking existing event-in-a-box templates from older Mozilla projects and using Mozilla Reps for supporting the event set the date for 26th of April.

14167122357_37c2ff0e53_bThe event was really successful. 30 people showed up and started stumbling and the local team made some slight twists on the event structure to facilitate better community engagement. (extended stumbling period, assigned areas for stumbling etc). What was really important for our pilot was to evaluate the contribution activity that we got from this small scale, low on resource event, and the result was stunning. We saw a 10x spike in our contribution rates in India for the following 2 weeks, and once the spike was over we were already 3x from the rates before the event (contribution activity ripples).hanno_data_2There were some concrete learnings from our first pilot, especially regarding the format, structure and communications needed before and after the event. In order to fortify our learnings and fine-tune the event format (for larger scale implementation) we decided to run a second pilot in three Indian cities (8th of June) in parallel with the same core team. Our first pilot clearly showcased the value of community contributions in MLS and based on the combined results of those two events we will be forming a community building growth strategy for MLS team transitioning towards a fully intentional approach.

All this would not be possible without the help of the fantastic people in MLS team (Vishy Krishnamoorthy, Erin Lancaster, Asa Dotzler, Richard Barnes, Hanno Schlichting, Ravikumar Dandu, Doug Turner) that have been really supportive since the early discussions around MLS community. A huge thanks, to all of you and onwards we go!

Contribution Activity Metrics – Early attempts and fails

As we examined with the intro post, the need for contribution activity metrics for different contribution areas in Mozilla has been high. It was only logical that many attempts were made to address this issue, mainly on the area-level (and not in Mozilla-wide level). Almost all of them had zero interaction between each other, and there was a general lack of vision for an holistic approach to the problem.

After one of our initial gatherings as the (then meta-) Community Building Team, a couple of people brainstormed together a possible solution to our problem. Together with Josh Matthews, Giorgos Logiotatidis, Ricky Rosario and Liz Henry a new approach was born. Enter project Blackhole!

Project Blackhole was a collaborative effort to develop and maintain an infrastructure of gathering and serving raw contribution data within Mozilla. We created a data architecture and flow together with a data Schema and specification to describe contribution activities for the first time in Mozilla. The project went far enough (thanks to Josh) to create a working prototype for back-end and front-end.

What went right:

Having a single project to drive multiple metrics efforts forward got people engaged. Everyone saw the value of de-duplicating efforts and tapping into that as a resource. Also during the process of designing and testing it we were able to self-identify as a group of people that share interest and commitment towards a common goal. Most of those people went on to become active members of the Systems and Data Working Group. Finally, we ended up with a common language and descriptions around contribution activities, a really valuable asset to have for the future of cross-project tracking.

What went wrong:

Building *anything* from scratch can be hard. Really hard. First, everyone (rightfully) questions the need to build something instead of re-using what is out there. Once you get everyone on board, development and deployment resources are hard to find especially on a short notice. On top of that Blackhole’s architecture *seemed* logical enough in theory, but was never tested in scale so everyone involved was not 100% sure that our architecture would survive stress tests and the scale of Mozilla’s contribution ecosystem.

PRO TIP: Changing the project name does not help. We went from “Blackhole” to “Wormhole” (and back to “Blackhole”?), to better reflect the proposed data flow (data would not disappear forever!) and people got confused. Really confused. Which is obviously something that is not helpful during conversations. Pick a name, and stick to it!

Lack of a team dedicated to it and inability to get the project listed as a personal goal of people (or teams), halted any progress leading us to a fearsome dead-end.

What we learned:

As with most failures, this one was also really valuable. We learned that:

  • we need to be a top line goal for people and teams
  • we need to examine really well what is out there (internally or externally to Mozilla) and investigate the possibility of re-using it.
  • we need a clear and common language to make communications as effective as possible
  • we need to be inclusive in all our procedures as a working group, with volunteers as well as all paid staff.
  • and in true Mozilla fashion: we need to start small, test and iterate with a focus on modularity.

A way forward?

Having those lessons learned from the process, we sat down last December as a group and re-aligned. We addressed all 5 issues and now we are ready to move forward. And the name of it? Baloo. Stay tuned for more info on our next detailed post :)


Contribution Activity Metrics – An Intro

Yesterday was my 3rd anniversary of joining Mozilla as a paid contributor. We’ve come a long way over those 3 years, and especially on my domain (community building) we experimented, deployed and pioneered new ideas on how to build a global community.

For me, Mozilla Reps was the experimentation focus. Together with William Quiviger we created a program with a specific set of tools to help Mozillians to organize and/or attend events, recruit and mentor new contributors, document and share activities, and support their local communities better. Other parts of the organization also ramped up their contributor engagement efforts, namely Webmaker, SuMo, Addons, QA and Coding, explored new ways to interact and incentivize contributions for their areas.

Inherently with experimentation and growth, some needs are surfaced. The need to have visibility, have a deeper understanding of the inner workings of your community, have hard figures to look at while assessing the progress of a program in order to take informed decisions for the future of it. To meet those needs, each team tried to create their own metrics around their contribution systems (websites, tools, communities), having a narrow scope, only covering their immediate needs.

Starting late 2012, a meta team of community builders was formed across Mozilla (Community Building Team aka CBT), thanks to the efforts of David Boswell trying to get us all on the same page. It was easy to see early-on that we needed to standardize our language around contribution metrics, align and define a unified way forward. The advantages of a unified approach were obvious. For the first time, we would be able to track contributions across all Mozilla projects and cross compare activity metrics for each project. Establishing a common language and definition of a contribution would also be the key to unlock all communications between community builders in Mozilla. We would be speaking the same language and measure activity the same way. For sure interpretations, thresholds and actions after the metrics might differ for each project, but hey! we would have de-duplicated a lot of efforts after all!

Since late December 2013, (almost 2 months now) it is my honor to be part of a newly formed Community Building Team under David. In my new role, I am tasked to lead the Systems and Data working group, delivering these unified tools and language around contribution metrics in Mozilla. You can learn more about the Systems and Data working group reading this intro post. For over a year now, this (initially informal) Working Group has been conceptualizing and rethinking the way we measure contribution across Mozilla, and with this newly formed team the time has come to deliver those. We are well underway on building the first iteration of those tools and getting the first results published.

This post is just the first of many to come in a series of posts around Contribution Activity Metrics and our progress so far. If you love metrics and you are interested in community building, you can join us by being part of the Working Group.

People are doing awesome stuff in Mozilla. Let’s measure them :)

Mozilla Contribution Madlibs

Michelle Marovich is preparing to run the Designing for Participation workshop for the People team. She’s looking for some real world examples that can help make the concepts concrete for everyone, so she set up a Contribution Madlib template for people to fill out.

Here is mine, on a contribution idea I have for sometime now.

I want to get more people to contribute to Mozilla Location Service, I need thousands of people to help me on it therefore I will reach out to all established geo-communities and post on all relevant forums and mailing lists in order to publicize the work.

Then I will organize regular open meetings and local meetups with communities on the ground in order to engage with the people who are interested. I break the work down into tasks by creating local teams that can self-organize the work that needs to be done locally.

I communicate those tasks by a form of an online game, so people become more engaged with the contribution opportunities. So that we can work effectively together, I always make sure that we have an open channel on IRC with people on call to answer questions. I continue to raise awareness of the work by evangelizing what the team is doing on the already established channels of the Geolocation industry.

I communicate decisions and progress by delegating this to the people on the Location Services team that are handling progress reports. When we achieve a milestone, reach a goal, or someone does something amazing I recognize them by cool prizes, badges and local recognition titles.