Will >> Will's blog

purpose: Will Kahn-Greene's blog of Miro, PyBlosxom, Python, GNU/Linux, random content, PyBlosxom, Miro, and other projects mixed in there ad hoc, half-baked, and with a twist of lemon
Page 1 of 2  >> (less recent)

Fri, 12 Sep 2014

Input status: September 12th, 2014


High-level summary:

  • Updated to ElasticUtils v0.10 which will allow us to upgrade our cluster to Elasticsearch 1.1. I'm working on a fix that'll let us to go to Elasticsearch 1.2, but that hasn't been released, yet.
  • Integrated the spicedham library prototype and set it up to classify abusive Input feedback. It's not working great, but that's entirely to be expected. I'm hoping to spend more time on spicedham and classification in Input in 2014q4. Ian did a great job with laying the foundation! Thank you, Ian!
  • Implemented a data retention policy and automated data purging.
  • Made some changes to the Input feedback GET and POST APIs to clarify things in the docs, fix some edge cases and make it work better for Firefox for Android and Loop.
  • Fixed the date picker in Chrome. Thank you, Ruben!

Landed and deployed:

  • c4e8e34 [bug 1055520] Update to ElasticUtils v0.10
  • e023fa4 [bug 1055520] Fix two reshape issues post EU 0.10 update
  • f9ba829 [bug 1055785] Codify data retention policy
  • 91396a8 Generalize About page text so it works for all products
  • 6fc03bf [bug 1053863] Update django to 1.5.9
  • 85709b2 [bug 1055788] Implement data purging
  • c0677a1 [bug 965796] Add a products update page
  • 121588d [bug 1057353] Update django-statsd and pystatsd
  • fe1c740 Add PII-related notes to the API fields
  • f77ecfa [bug 799562] Clarify API field documentation
  • c5eec03 [bug 1055789] Restrict front page dashboard and api to 6 months
  • 0892546 [bug 1059826] Add max_length to url field in API
  • f192f84 [bug 1057617] Fix url data validation
  • aad961d [bug 1030901] Document Input GET API
  • 2f212c5 [bug 1015788] Add flake8 linting
  • d673947 Update coding conventions
  • 27a1b6b Add "maximum" arg to GET API
  • 4f671e4 [bug 1062436] Add flags app and Flag model
  • 9d03d4b Fix flake8_lint issues
  • 0411d91 [bug 1062453] Add flagged view
  • 56f7e24 [bug 1062439] Celery task for classification
  • 7aa2930 [bug 1062455] Add spicedham to vendor (Ian Kronquist)
  • 0d90df3 We don't need spicedham under vendor/packages (Ian Kronquist)
  • a2a491d fix bug 1012965 - Date picker looks broken in chrome (Ruben Vereecken)
  • 0c42213 [bug 1063825] Integrate spicedham into fjord
  • 78a2d63 [bug 1062444] Initial training data
  • 5ca816e [bug 1020307] Prepare for adding gradient to generic form

Current head: 5ca816e

Rough plan for the next two weeks

  1. Working on Dashboards-for-everyone bits. Documenting the GET API. Making it a bit more functional. Writing up some more examples. (https://wiki.mozilla.org/Firefox/Input/Dashboards_for_Everyone)
  2. Gradients (https://wiki.mozilla.org/Firefox/Input/Gradient_Sentiment)

What I need help with

  1. (django) Update to django-rest-framework 2.3.14 (bug #934979) -- I think this is straight-forward. We'll know if it isn't if the tests fail.
  2. (django, cookies, debugging) API response shouldn't create anoncsrf cookie (bug #910691) -- I have no idea what's going on here because I haven't looked into it much.

For details, see our GetInolved page:


If you're interested in helping, let me know! We hang out on #input on irc.mozilla.org and there's the input-dev mailing list.

Additional thoughts

I've been codifying project plan details on the wiki:


I have no idea who's going to use that information or whether it helps. If you see things that are missing, let me know. It'll help me hone the project management templates I'm using and know which information is important to keep up to date and which information I can let slide until rainy days.

That's it!

Tue, 19 Aug 2014

Input status: August 19th, 2014


High-level summary:

It's been a slower two weeks than normal, but we still accomplished some interesting things:

  • L Guruprasad finished cleaning up the Getting Started guide--that work helps all future contributors. He did a really great job with it. Thank you!
  • Landed a minor rewrite to rate-limiting/throttling.
  • Redid the Elasticsearch indexing admin page.
  • Fixed some Heartbeat-related things.

Landed and deployed:

  • cf2e0e2 [bug 948954] Redo index admin
  • f917d41 Update Getting Started guide to remove submodule init (L. Guruprasad)
  • 5eb6d6d Merge pull request #329 from lgp171188/peepify_submodule_not_required_docs
  • c168a5b Update peep from v1.2 to v1.3
  • adf7361 [bug 1045623] Overhaul rate limiting and update limits
  • 7647053 Fix response view
  • f867a2d Fix rulename
  • 8f0c36e [bug 1051214] Clean up DRF rate limiting code
  • 0f0b738 [bug 987209] Add django-waffle (v0.10)
  • b52362a Make peep script executable
  • 461c503 Improvie Heartbeat API docs
  • 8f0ccd3 [bug 1052460] Add heartbeat view
  • d1604f0 [bug 1052460] Add missing template

Landed, but not deployed:

  • ed2923f [bug 1015788] Cosmetic: flake8 fixes (analytics)
  • afdfc6a [bug 1015788] Cosmetic: flake8 fixes (base)
  • 05e0a33 [bug 1015788] Cosmetic: flake8 fixes (feedback)
  • 2d9bc26 [bug 1015788] Cosmetic: flake8 fixes (heartbeat)
  • dc6e990 Add anonymize script

Current head: dc6e990

Rough plan for the next two weeks

  1. Working on Dashboards-for-everyone bits. Documenting the GET API. Making it a bit more functional. Writing up some more examples. (https://wiki.mozilla.org/Firefox/Input/Dashboards_for_Everyone)
  2. Update Input to ElasticUtils v0.10 (bug 1055520)
  3. Land all the data retention policy work (bug 946456)
  4. Gradients (https://wiki.mozilla.org/Firefox/Input/Gradient_Sentiment)
  5. Product administration views (bug 965796)

Most of that is in some state of half-done, so we're going to spend the next couple of weeks focusing on finishing things.

What I need help with

  1. (django) Update to django-rest-framework 2.3.14 (bug 934979) -- I think this is straight-forward. We'll know if it isn't if the tests fail.
  2. (django, cookies, debugging) API response shouldn't create anoncsrf cookie (bug 910691) -- I have no idea what's going on here because I haven't looked into it much.
  3. (html) Fixing the date picker in Chrome (bug 1012965) -- The issue is identified. Someone just needs to do the fixing.

For details, see our GetInolved page:


If you're interested in helping, let me know! We hang out on #input on irc.mozilla.org and there's the input-dev mailing list.

Additional thoughts

We're in the process of doing a Personally Identifiable Information audit on Input, the systems it's running on and the processes that touch and move data around. This covers things like "what data are we storing?", "where is the data stored?", "who/what has access to that data?", "does that data get copied/moved anywhere?", "who/what has access to where the data gets copied/moved to?", etc.

I think we're doing pretty well. However, during the course of the audit, we identified a few things we should be doing better. Some of them already have bugs, one of them is being worked on already and the otehrs need to be written up.

Some time this week, I'll turn that into a project and write up missing bugs.

That's about it!

Mon, 04 Aug 2014

Input status: August 4th, 2014


This is the status report for development on Input. I publish a status report to the input-dev mailing list every couple of weeks or so covering what was accomplished and by whom and also what I'm focusing on over the next couple of weeks. I sometimes ruminate on some of my concerns. I think one time I told a joke.

Work summary

  • Continued work on the development environment with awesome help from L. Guruprasad
  • Implement the bits required to support Loop
  • Add smoketest coverage for Firefox OS feedback form
  • Fix some technical debt issues


Landed and deployed:

  • 6097571 [bug 1040919] Change version to only have one dot
  • dcf8f91 [bug 1040919] Fix .0.0 versions to .0
  • 4012933 Remove CEF-related things
  • 4887c49 [bug 1030905] Add product/version tests
  • 1bdd3a8 [bug 1042222] update Vagrantfile to use 14.04 LTS (L. Guruprasad)
  • c1bdfbb Add L. Guruprasad to CONTRIBUTORS
  • 3e607a5 [bug 1042560] Remove unused packages in dev VM (L. Guruprasad)
  • c0759d1 Update django-mozilla-product-details
  • bcf8ec8 Update product details
  • 58579ab [bug 987801] peep-ify requirements
  • 248b0ab [bug 987801] Exit if requirements are mismatched
  • 5db57bc [bug 987801] Redo how we figure out whether to use vendor/
  • df8400d Fix sampledata generation to use bulk create
  • fa0caac [bug 1041622] Capture querystring slop
  • 98745b5 [bug 1021155] Add basic FirefoxOS smoketest
  • 6cf9e39 [bug 1041664] Capture slop in Input API
  • ff4b711 Show context in response view
  • 976ebbf [bug 1045942] Add category to response table
  • 34622ad Add docs for category and extra context for Input API
  • d1a8dd5 Add an additional note about keys
  • fec1319 [bug 998726] Quell Django warnings
  • 288afe4 Rename fjord/manage.py to fjord/manage_utils.py
  • d40e8a9 Update dennis to v0.4.3

Landed, but not deployed:

  • 28cd90f [bug 1047681] Fix mlt call to be more resilient

Current head: 28cd90f

Over the next two weeks

  1. Keep an eye out for any Loop or Heartbeat related work--that's top priority.
  2. Work on gradient support and product picker support

What I need help with

  1. (Google chrome, JavaScript, CSS, HTML) Investigate what's wrong with the date picker in Chrome and fix it [bug 1012965]
  2. Test out the Getting Started instructions. We've had a few people go through these already, but it's definitely worth having more eyes. http://fjord.readthedocs.org/en/latest/getting_started.html

If you're interested in helping, let me know! We hang out on #input on irc.mozilla.org and there's the input-dev mailing list.

That's it!

Sun, 20 Jul 2014

Input status: July 20th, 2014


This is the status report for development on Input. I publish a status report to the input-dev mailing list every couple of weeks or so covering what was accomplished and by whom and also what I'm focusing on over the next couple of weeks. I sometimes ruminate on some of my concerns. I think one time I told a joke.

Last status report was at the end of June. This status report covers the last few things we landed in 2014q2 as well as everything we've done so far in 2014q3.


Landed and deployed:

  • 6ecd0ce [bug 1027108] Change default doc theme to mozilla sphinx (Anna Philips)
  • 070f992 [bug 1030526] Add cors; add api feedback get view
  • f6f5bc9 [bug 1030526] Explicitly declare publicly-visible fields
  • c243b5d [bug 1027280] Add GengoHumanTranslater.translate; cleanup
  • 3c9cdd1 [bug 1027280] Add human tests; overhaul Gengo tests
  • ff39543 [bug 1027280] Add support for the Gengo sandbox
  • 258c0b5 [bug 1027280] Add test for get_balance
  • 44dd8e5 [bug 1027280] Implement Gengo Human push_translations
  • 35ae6ec [bug 1027280] Clean up API code
  • a7bf90a [bug 1027280] Finish pull_translations and tests
  • c9db147 [bug 1027286] Gengo translation system status
  • f975f3f [bug 1027291] Implement spot Gengo human translation
  • f864b6b [bug 1027295] Add translation_sync cron job
  • c58fd44 [bug 1032226] en-GB should copyover, too
  • 7480f87 [bug 1032226] Tweak the code to be more defensive
  • 7ac1114 [bug 1032571] CSRF exempt the API
  • ac856eb [bug 1032571] Fix tests to catch csrf issues in the api
  • 74e8e09 [bug 1032967] Handle unsupported language pairs
  • 74a409e [bug 1026503] First pass at vagrantification
  • a7a440f Continued working on docs; ditched hacking howto
  • 44e702b [bug 1018727] Backfill translations
  • 69f9b5b Fix date_end issue
  • e59d4f6 [bug 1033852] Better handle unsupported src languages
  • cc3c4d7 Add list of unsupported languages to admin
  • 32e7434 [bug 1014874] Fix translate ux
  • 672abba [bug 1038774] Hide responses from hidden products
  • e23eca5 Fix a goof in the last commit
  • 6f78e2e [bug 947767] Nix authentication for API stuff
  • a9f2179 Fix response view re: non-existent products
  • e4c7c6c [Bug 1030905] fjord feedback api tests for dates (Ian Kronquist)
  • 0d8e024 [bug 935731] Add FactoryBoy
  • 646156f Minor fixes to the existing API docs
  • f69b58b [bug 1033419] Heartbeat backend prototype
  • f557433 [bug 1033419] Add docs for heartbeat posting

Landed, but not deployed:

  • 7c7009b [bug 935731] Switch all tests to use FactoryBoy
  • 2351fb5 Generate locales so ubuntu will quite whining (Ian Kronquist)

Current head: 7ea9fc3


At a high level, this is:

  1. Landed automated Gengo human translation and a bunch of minor fixes to make it work more smoothly.
  2. Reworked how we build development environments to use vagrant. This radically simplifies the instructions and should make it a lot easier for contributors to build a development environment. This in turn should lead to more people working on Input.
  3. Fixed a bug where products marked as "hidden" were still showing up in the dashboard.
  4. Implemented a GET API for Input responses. (https://wiki.mozilla.org/Firefox/Input/Dashboards_for_Everyone)
  5. Implemented the backend for the Heartbeat prototype. (https://wiki.mozilla.org/Firefox/Input/Heartbeat)
  6. Also, I'm fleshing out the Input section in the wiki complete with project plans. (https://wiki.mozilla.org/Firefox/Input)

Over the next two weeks

  1. Continue fleshing out project plans for in-progress projects on the wiki.
  2. Gradient sentiment and product picker work.

What I need help with

  1. We have a new system for setting up development environments. I've tested it on Linux. Ian has, too (pretty sure he's using Linux). We could use some help testing it on Windows and Mac OSX.

Do the instructions work on Windows? Do the instructions work on Mac OSX? Are there important things the instructions don't cover? Is there anything confusing?


  1. I'm changing the way I'm managing Fjord development. All project plans will be codified in the wiki. A rough roadmap of which projects are on the drawing board, in-progress, completed, etc is also on the wiki. I threw together a structure for all of this that I think is good, but it could use some review.

Do these project plans provide useful information? Are there important questions that need answering that the plans do not answer?


If you're interested in helping, let me know! We hang out on #input on irc.mozilla.org and there's the input-dev mailing list.

I think that covers it!

Tue, 01 Jul 2014

Input: 2014q2 post-mortem

I'm going to start doing quarterly post-mortems for Input development. The goal is to be more communicative about what happened, why, what's in the works and what I need more help with.

NB: "Fjord" is the name of the codebase that runs Input.

Bug and git stats


Bugs created:        63
Bugs fixed:          54


Total commits: 151

      Will Kahn-Greene : 142  (+14758, -4599, files 438)
        ossreleasefeed : 3  (+197, -42, files 9)
          Anna Philips : 2  (+734, -6, files 24)
          Joshua Smith : 2  (+65, -31, files 5)
     Swarnava Sengupta : 1  (+2, -2, files 1)
         Ricky Rosario : 1  (+0, -0, files 0)

Total lines added: 15756
Total lines deleted: 4680
Total files changed: 477

We added a lot of lines of code this quarter:

  • April 1st, 2014: 15195 total, 6953 Python
  • July 1st, 2014: 20456 total, 9247 Python

That's a pretty big jump in LOC. I think a bunch of that is the translation-related changes.

Contributor stats

5 non-core people contributed to Fjord development.

I spent some time over the weekend finishing up Vagrant provisioning script and rewriting the docs. I'm planning to spend some more time in 2014q3 reducing the complexity and barriers for setting up a Fjord development environment to the point where someone can contribute.

Additionally, I'm planning to create more bugs that are contributor-friendly. I started doing that in the last week. I think a good goal for Input is to have around 20 contributor-y bugs hanging around at any given time.


Site health dashboard: I wrote a mediocre site health dashboard that's good enough to give me a feel for how the site is performing before and after a deployment. This still needs some work, but I'll schedule that for a rainy day.

Client side smoke tests: I wrote smoke tests for the client side. I based it on the defunct input-tests code that QA was maintaining up until we rewrote Input. There are still a bunch of tests that I want to write to have a better coverage of things, but having something is way better than nothing. I'm hoping the smoke tests will reduce the amount of manual testing I'm doing, too.

Vagrant: I took some inspiration from Erik Rose and DXR and wrote a Vagrant provisioning shell script. This includes a docs overhaul as well. This work is almost done, but needs some more testing and will probably land in the next week or two. This will make peoples' lives easier.

Automated translation system (human and machine): I wrote an automated translation system. It's generalized so that it isn't model/field specific. It's also generalized so that we can add plugins for other translation systems. It's currently got plugins for Dennis, Gengo machine translation and Gengo human translation. I turned the automated human translation on yesterday and it seems to be working well. That was a HUGE project. I'm glad it's done.

One thing it includes is a lot of auditing and metrics gathering. This will make it possible for me to go back in time and look at how the translation system worked on various Input feedback responses and hone the system going forward to reduce the number of human translations we're doing and also reduce the number of problems we have doing them.

Better query syntax: We were upgraded to Elasticsearch 0.90.10. I switched the query syntax for the dashboard search field to use Elasticsearch simple_query_string. That allows users to express search queries they weren't previously able to express.

utm_source and utm_campaign handling: I finished the support for handling utm_source and utm_campaign querystring parameters. This allows us to differentiate between organic feedback and non-organic feedback.

More like this: I added a "more like this" section to the response view. This makes it possible for UA analyzers to look at a response and see other responses that are similar.

Dashboards for you, dashboards for everyone!

I'm putting this in its own section because it's intriguing. I'll write another blog post about it later in July as things gel.

On Thursday, a couple of days after d3 training that Matt organizied, I threw together a better GET API for Input feedback responses. It's not documented, it probably has some bugs and it's probably going to change a bit, but the gist of it is that it lets you more easily build a dashboard that meets your needs against live Input data.

Here's a proof-of-concept:


That's looking at live Input data using the new GET API. The code is in a GitHub gist. It auto-updates every 2 minutes.

The problem is that I've got a ton of Input work to do and I just can't write dashboard code on Input fast enough. Further, of the people I've talked to that use the front page dashboard, they all have really different questions they're asking of the data. I'm hoping this alleviates that bottleneck by letting you and everyone else write dashboards that meet your needs.

I encourage you to take my proof-of-concept, fork the gist, tweak it, use bl.ocks.org or something to "host" the gist. Build the dashboard that answers your questions. Share it with other people. Plus, let me know about it. If you have issues with the API, submit a bug and tell me.

If this scratches the itch I think needs scratching, it should result in a bunch of interesting dashboards. If that happens, I'll write some code in Input to create a curated list of them so people can find them more easily.


This was a really crazy quarter and parts of it really sucked, but we got a lot accomplished and we laid some groundwork for some really interesting things for 2014q3.

Mon, 23 Jun 2014

Input status: June 23rd, 2014

I publish a status report to the input-dev mailing list every couple of weeks or so covering what was accomplished and by whom and also what I'm focusing on over the next couple of week. I sometimes ruminate on some of my concerns. I think one time I told a joke.

Since the last report:

Landed and deployed:

Landed, but not deployed:

HEAD: 1d9e67a

Mostly I spent the last couple of weeks working on automated Gengo human translation support. This involved some infrastructure rewriting plus some additional infrastructure so that when we push all this out, we can see what's going on as it is happening.

Additionally, I went through and updated the mentor metadata for mentored bugs, added a bunch of new mentored bugs and worked with two potential contributors on them.

Over the next week (last week in 2014q2):

  1. finish up automated Gengo human translation work

First thing in 2014q3, I'll spend some time "opening up" the development side of the project. This will make it easier/possible to follow and participate in development. I'm still figuring out some of the details and it's likely I'll continue to change how things work over the course of the quarter, but plan to follow advice from the Community Building team and Erik Rose who seems to be doing really super with DXR.

(Updated) Using the bug_mentor field with the Bugzilla REST API to get mentored bugs

In my previous post, I mentioned how Bugzilla grew a mentor field and had some code on how to query Bugzilla using the old and new APIs for the list of mentored bugs. This updates that information.

Gerv and Byron pointed out there's a isnotempty operator that's better to use than the way I cobbled together to query for bugs that have data in the bug_mentor field.

Thus, the code should look like this:

import requests

# Using the old BzAPI: https://wiki.mozilla.org/Bugzilla:REST_API
r = requests.get(
    'https://api-dev.bugzilla.mozilla.org/latest' +
data = r.json()
print len(data['bugs'])  # Prints 9

# Using the new BMO API. https://wiki.mozilla.org/BMO/REST
r = requests.get(
    'https://bugzilla.mozilla.org/rest' +
data = r.json()
print len(data['bugs'])  # Prints 9

Thu, 19 Jun 2014

Using the bug_mentor field with the Bugzilla REST API to get mentored bugs

Updated June 23rd, 2014:
There's a better way to query the data. See the update blog post

Bugzilla grew a mentor field recently. This is really fantastic as it solves some interesting problems and makes it easier to track various aspects of mentoring which have been previously difficult to track. Yay to everyone involved in making that happen!

Migrating from the old way (sticking [mentor=xxx] in the whiteboard field) to the new way caused a problem that I spent a while working on today. I heard reports of other people having the same problem, hence this blog post.

There are a bunch of Bugzilla-symbiotic systems which would show a list of mentored bugs by checking to see if the string mentor= was in the whiteboard field. That no longer works. Instead we have to check to see if the bug_mentor field is empty. However, this is difficult to express with the old Bugzilla REST API (BzAPI).

The bug_mentor field is unique in that it holds email addresses which have the @ in them. So we can (ab)use this property by seeing if the bug_mentor field contains the @ character.

I did this with the GetInvolved/input.mozilla.org page. Here is the diff in case that's helpful.

Here's some Python that shows this with the old BzAPI and the new BMO API which pulls mentored bugs for Input:

import requests

# Using the old BzAPI: https://wiki.mozilla.org/Bugzilla:REST_API
r = requests.get(
    'https://api-dev.bugzilla.mozilla.org/latest' +
data = r.json()
print len(data['bugs'])  # Prints 9

# Using the new BMO API. https://wiki.mozilla.org/BMO/REST
r = requests.get(
    'https://bugzilla.mozilla.org/rest' +
data = r.json()
print len(data['bugs'])  # Prints 9

Tue, 13 May 2014

Input: changed query syntax across the site

Better search syntax is here!

Yesterday I landed the changes for bug 986589 which affects all the search boxes and search feeds on Input. Now they use the Elasticsearch simple-query-search query instead of the hand-rolled query parser I wrote.

This was only made possible in the last month after we were updated from Elasticsearch 0.20.6 (or whatever it was) to 0.90.10.

Tell me more about this ... syntax.

I'm pretty psyched! It's pretty much the minimum required syntax for useful searching. It's kind of lame it took a year to get to this point, but so it goes.

To quote the Elasticsearch 0.90 documentation:

+  signifies AND operation
|  signifies OR operation
-  negates a single token
"  wraps a number of tokens to signify a phrase for searching
*  at the end of a term signifies a prefix query
( and )  signify precedence

Negation and prefix were the two operators my hand-rolled query parser didn't have.

What does this mean for you?

It means that you need to use the new syntax for searches on the dashboard and other parts of the site.

Further, this affects feeds, so if you're using the Atom feed, you'll probably need to update the search query there, too.

Also, we added a ? next to search boxes which links to a wiki page that documents the syntax with examples. It's a wiki page, so if the documentation is subpar or it's missing examples, feel free to let me know or fix it yourself.

Fri, 20 Dec 2013

SUMO: 2013 retrospective

It was a big year for SUMO. In 2012, we got a lot accomplished: new search, new information architecture,

One thing I didn't do was make my year-end script product better output.

Anyhow---on with stats!

Twas the year: 2013


Bugs created: 889

               a.topal : 156
              rrosario : 100
                willkg : 87
           scoobidiver : 85
                  ibai : 58
                mverdi : 54
               mcooper : 42
                feer56 : 35
          krystaiceman : 26
                rdalal : 19
            david.weir : 15
                shuhao : 13
      swarnavasengupta : 9
       andrei.hutusoru : 8
           me+bugzilla : 7
            tobbi.bugs : 6
                 mluna : 6
          joshua-smith : 6
               tdowner : 6
               leszekz : 6
        yoshi.yokotani : 5
        stephen.donner : 5
                 slurp : 5
                  mana : 5
             madperson : 4
              kbrosnan : 4
             tonnes.mb : 4
               rardila : 4
              pmcclard : 4
               dbialer : 4
             michaljev : 4
                   abc : 4
                  l10n : 4
               pcvrcek : 3
                 rdaub : 3
              fabricio : 2
             rmcguigan : 2
          sudheesh1995 : 2
          alex_mayorga : 2
          simone.lando : 2
            nishant_cs : 2
                  bram : 2
               smolejv : 2
        bob.silverberg : 2
              rtanglao : 2
        kdurant35rules : 2
            amit103065 : 2
         subedimahadev : 2
                lhenry : 2
          thomas.lendo : 2
             shawnsumo : 2
              mhammond : 1
        kdurant35rules : 1
                  djst : 1
               curtisk : 1
        chiorean.ioana : 1
                bermea : 1
               friedel : 1
        bputstudentweb : 1
     margaret.leibovic : 1
             rbillings : 1
       nikitan.dolmart : 1
       georgevidalakis : 1
            nsm.nikhil : 1
              satishb3 : 1
            bwbrowning : 1
              bugzilla : 1
                  coce : 1
              EddyCarr : 1
              gryllida : 1
        mohammed.samad : 1
                  6a68 : 1
         krupa.mozbugs : 1
           John99-bugs : 1
             wjohnston : 1
              barderne : 1
               jan0286 : 1
               fwenzel : 1
               rnewman : 1
           this4midhun : 1
              bjohnson : 1
  bugzilla-fromthedeep : 1
             iamjithin : 1
               bmo2010 : 1
    chrismore.bugzilla : 1
                 evold : 1
              jbertsch : 1
                yousef : 1
          pmjcreations : 1
               rhelmer : 1
              danishka : 1
                  mail : 1
             gphemsley : 1
              Rebeccah : 1
           ckreinbring : 1
               stephen : 1
         berker.peksag : 1
                jezdez : 1
                 nchen : 1
         iamjayakumars : 1
            netfuzzerr : 1
              benjamin : 1

Bugs resolved: 1116

              rrosario : 386 resolved, 273 fixed
                rdalal : 152 resolved, 150 fixed
               a.topal : 121 resolved, 41 fixed
               mcooper : 118 resolved, 105 fixed
                willkg : 72 resolved, 61 fixed
           scoobidiver : 31 resolved, 0 fixed
             michaljev : 20 resolved, 17 fixed
      swarnavasengupta : 18 resolved, 0 fixed
                shuhao : 16 resolved, 13 fixed
           me+bugzilla : 12 resolved, 5 fixed
         berker.peksag : 12 resolved, 12 fixed
                mverdi : 11 resolved, 5 fixed
                  erik : 9 resolved, 9 fixed
                lhenry : 8 resolved, 1 fixed
          krystaiceman : 7 resolved, 1 fixed
            tobbi.bugs : 7 resolved, 6 fixed
          joshua-smith : 6 resolved, 4 fixed
                 jfong : 5 resolved, 5 fixed
               tdowner : 5 resolved, 2 fixed
                feer56 : 5 resolved, 0 fixed
            david.weir : 5 resolved, 4 fixed
            bwbrowning : 5 resolved, 5 fixed
                  bram : 5 resolved, 4 fixed
                  ibai : 4 resolved, 1 fixed
   alastra.mariagrazia : 4 resolved, 2 fixed
                 laura : 4 resolved, 4 fixed
              williamr : 3 resolved, 1 fixed
         buchanae+bugs : 3 resolved, 3 fixed
           bharath_ves : 3 resolved, 3 fixed
              paul+moz : 3 resolved, 3 fixed
          chris.lonnen : 2 resolved, 2 fixed
               smolejv : 2 resolved, 1 fixed
             zcampbell : 2 resolved, 0 fixed
         iamjayakumars : 2 resolved, 0 fixed
               curtisk : 2 resolved, 0 fixed
               leszekz : 2 resolved, 0 fixed
                   abc : 2 resolved, 1 fixed
               pcvrcek : 2 resolved, 0 fixed
          taygunagiali : 2 resolved, 2 fixed
                  mail : 1 resolved, 0 fixed
               bmo2010 : 1 resolved, 1 fixed
               wymette : 1 resolved, 0 fixed
                beaotx : 1 resolved, 1 fixed
                nelson : 1 resolved, 1 fixed
             madperson : 1 resolved, 1 fixed
                  coce : 1 resolved, 0 fixed
              pmcclard : 1 resolved, 1 fixed
             tgavankar : 1 resolved, 1 fixed
  bugzilla-fromthedeep : 1 resolved, 0 fixed
              rtanglao : 1 resolved, 0 fixed
        stephen.donner : 1 resolved, 1 fixed
       guillermo.movia : 1 resolved, 1 fixed
              lorchard : 1 resolved, 1 fixed
              nukeador : 1 resolved, 0 fixed
             rtucker11 : 1 resolved, 0 fixed
            nishant_cs : 1 resolved, 0 fixed
                  stas : 1 resolved, 0 fixed
             mattbasta : 1 resolved, 1 fixed
              satishb3 : 1 resolved, 0 fixed
              ragsagar : 1 resolved, 1 fixed
             rmcguigan : 1 resolved, 1 fixed
                 nchen : 1 resolved, 0 fixed
                kudrom : 1 resolved, 1 fixed
       andrei.hutusoru : 1 resolved, 0 fixed
                  reed : 1 resolved, 0 fixed
           tiziana.sel : 1 resolved, 0 fixed
       chance.zibolski : 1 resolved, 1 fixed
             alice0775 : 1 resolved, 0 fixed
                  ravi : 1 resolved, 0 fixed
            nsm.nikhil : 1 resolved, 0 fixed
              gryllida : 1 resolved, 1 fixed
    deletesoftware+moz : 1 resolved, 0 fixed
          pmjcreations : 1 resolved, 0 fixed
                boerni : 1 resolved, 1 fixed
               rforbes : 1 resolved, 0 fixed
               dbialer : 1 resolved, 1 fixed
                jgross : 1 resolved, 1 fixed

            INCOMPLETE : 43
               WONTFIX : 51
             DUPLICATE : 64
               INVALID : 73
            WORKSFORME : 121
                 FIXED : 764

Research bugs: 15

761582: [research] Add feature: Articles that link to this article
788104: [research] [ux] Support multiple products in the support forum
815089: [research] Investigate telling apart Firefox for Desktop and Firefox for Mobile tweets
816970: [research] SurveyGizmo API to be used in automated exit survey
823060: [research] Use datetime instead of ints in ES mappings
823891: [research] Adding KB revisions feature
825621: [research] Store the templates, article links and images in each article
825624: [research] Investigate how to update to Twitter API v1.1
841412: [research] Bad localization strings shouldn't break the site.
845290: [research] URL bar should fade away on SUMO
854554: [research] Youtube embeds don't work with templates
889884: [research] Open Badges!
889890: [research][discuss] figure out how to improve our l10n situation with search
906992: [research] Add support for multiple ES indexes by doc type
937889: [research] Login users via a URL in email

Tracker bugs: 20

433161: [Tracker] Support for forums in other languages
625891: [tracker] HTML email
721462: [tracker] Taxonomy IA improvements
758598: [Tracker] Search UX suggestions
783262: [tracker] Add rate limiting to protect us from spammers
790785: [Tracker] L10n tools editing part
790786: [Tracker] L10n tools organization part
800962: [Tracker] Add activity history page for KB
815625: [Tracker] Segment dashboards and other contributor pages by product
817540: [tracker] AJAXify the refine+focus panel
825606: [tracker] Switch everything from Webtrends to Google Analytics
827640: [tracker] Localize Questions
838584: [Tracker] Getting ready for Firefox OS launch
845286: [tracker] Use as little bandwidth as possible on mobile version of SUMO
845773: [Tracker] move to an OS charting solution
848520: [tracker] Make all traffic HTTPS
851730: [tracker] Close threads pro-actively
871559: [tracker] update codebase to django 1.4 layout
897057: [tracker] Open Badges -- stage 1
920530: [tracker] support Webmaker on SUMO


Total commits: 1138

         Ricky Rosario : 492  (+16258, -16435, files 2972)
      Will Kahn-Greene : 178  (+8311, -3748, files 438)
           Rehan Dalal : 168  (+13016, -5554, files 680)
           Mike Cooper : 145  (+46955, -22136, files 582)
           Kadir Topal : 39  (+352, -110, files 61)
      Michał Frontczak : 19  (+229, -182, files 78)
         Berker Peksag : 15  (+570, -717, files 73)
             Shuhao Wu : 15  (+1523, -127, files 51)
       Jen Fong-Adwent : 9  (+138, -18, files 17)
                 Tobbi : 8  (+338, -204, files 13)
              browning : 5  (+140, -16, files 12)
             davd Weir : 4  (+15, -1, files 4)
          Joshua Smith : 4  (+94, -87, files 13)
         Tobias Markus : 3  (+8, -8, files 4)
                 Anush : 3  (+4, -1, files 3)
       Gaurav Dadhania : 3  (+3, -3, files 3)
  Bharath Thiruveedula : 3  (+15, -14, files 3)
                  ibai : 3  (+30, -30, files 4)
                kudrom : 2  (+9, -9, files 5)
             Nghi Tran : 2  (+2, -1, files 2)
          Tanner Filip : 2  (+4, -4, files 2)
                Börni : 2  (+30, -15, files 4)
             madperson : 2  (+5, -4, files 2)
         Taygun AGIALI : 2  (+7, -6, files 3)
           TylerDowner : 2  (+3, -3, files 2)
           James Socol : 2  (+37, -27, files 3)
               david-w : 1  (+1, -1, files 1)
              ragsagar : 1  (+16, -1, files 2)
       Guillermo Movia : 1  (+1, -0, files 1)
              rosanaar : 1  (+9, -0, files 1)
              Gryllida : 1  (+26, -6, files 3)
       Beatriz Nombela : 1  (+9, -9, files 6)

Total lines added:   88158
Total lines deleted: 49477
Total files changed: 5048
  1. Ricky does a lot of work! Holy cow!

  2. In 2011, we had 19 people who contributed code changes.

    In 2012, we had 23 people.

    In 2013, we had 32 people.

  3. Like 2011 and 2012, we resolved more bugs than we created in 2013. That's three years in a row! I've never seen that happen on a project I work on.

  4. There are a lot of people braving Bugzilla to write up bugs. Skimming the list, I see developers, non-developers, Support contributors, localizers, support team and a lot of people I don't recognize.

Here's some number comparisons:

name 2011 2012 2013
Bugs created: 1357 938 889
Bugs resolved: 1637 1025 1116
Total commits: 1137 916 1138
Code contributors: 19 23 32

I spent a good chunk of 2013 working on Input, but here's what I remember from SUMO development in 2013:

  1. We rearranged the codebase for better Django 1.4 layout. That was a project. Oy.
  2. We added support for non-English languages to the support forums!
  3. We switched email to be HTML formatted. We also reworked email to be localized.
  4. We switched to Google Analytics.
  5. We implemented Open Badges---though there's still a few important pieces to finish there.
  6. We switched to YouTube for videos.
  7. We added support for Webmaker and Firefox OS. Thunderbird support will be added to SUMO in 2014.
  8. Mike took a lantern, a crust of bread and a big sword and spelunked into the darkest dungeons filled with stinky, squelchy muck and rewrote the showfor code.
  9. We reworked our search code to handle multiple indexes, though we haven't taken advantage of that, yet.
  10. We switched deployment to use Dennis to lint all translated strings before pushing them to production. This has almost assuredly saved us from production fires. I hated those kinds of fires. Hooray for Dennis!
  11. We wrote and switched to Ernest for sprint planning and coordination.
  12. We overhauled everything to add support for Persona authentication, but had to push off deployment indefinitely because of problems with Persona which are being ironed out by the Persona team.
  13. We added an escalation system for questions that haven't received a response in x hours for some positive value of x that is still in flux.
  14. We ditched Highcharts.
  15. We wrote a command-line deployer which tells us exactly what's going out and tells New Relic, too. This gives us a much better idea of what we're deploying and how it affected the site afterwards. This command-line deployer is named chief-james in honor of James who has moved on to greener and well measured pastures.
  16. We added a bunch of new metrics, dashboards, history pages, activity pages, icons, bicons, landing pages, take-off pages, topics, subtopics, toe picks and all kinds of stuff.

That's the gist of the year: it was a lot of work, but we accomplished a ton.

w00t for 2013!