Will >> Will's blog
Fri, 20 Dec 2013
It was a big year for SUMO. In 2012, we got a lot accomplished: new search, new information architecture,
One thing I didn't do was make my year-end script product better output.
Anyhow---on with stats!
Twas the year: 2013 =================== Bugzilla ======== Bugs created: 889 a.topal : 156 rrosario : 100 willkg : 87 scoobidiver : 85 ibai : 58 mverdi : 54 mcooper : 42 feer56 : 35 krystaiceman : 26 rdalal : 19 david.weir : 15 shuhao : 13 swarnavasengupta : 9 andrei.hutusoru : 8 me+bugzilla : 7 tobbi.bugs : 6 mluna : 6 joshua-smith : 6 tdowner : 6 leszekz : 6 yoshi.yokotani : 5 stephen.donner : 5 slurp : 5 mana : 5 madperson : 4 kbrosnan : 4 tonnes.mb : 4 rardila : 4 pmcclard : 4 dbialer : 4 michaljev : 4 abc : 4 l10n : 4 pcvrcek : 3 rdaub : 3 fabricio : 2 rmcguigan : 2 sudheesh1995 : 2 alex_mayorga : 2 simone.lando : 2 nishant_cs : 2 bram : 2 smolejv : 2 bob.silverberg : 2 rtanglao : 2 kdurant35rules : 2 amit103065 : 2 subedimahadev : 2 lhenry : 2 thomas.lendo : 2 shawnsumo : 2 mhammond : 1 kdurant35rules : 1 djst : 1 curtisk : 1 chiorean.ioana : 1 bermea : 1 friedel : 1 bputstudentweb : 1 margaret.leibovic : 1 rbillings : 1 nikitan.dolmart : 1 georgevidalakis : 1 nsm.nikhil : 1 satishb3 : 1 bwbrowning : 1 bugzilla : 1 coce : 1 EddyCarr : 1 gryllida : 1 mohammed.samad : 1 6a68 : 1 krupa.mozbugs : 1 John99-bugs : 1 wjohnston : 1 barderne : 1 jan0286 : 1 fwenzel : 1 rnewman : 1 this4midhun : 1 bjohnson : 1 bugzilla-fromthedeep : 1 iamjithin : 1 bmo2010 : 1 chrismore.bugzilla : 1 evold : 1 jbertsch : 1 yousef : 1 pmjcreations : 1 rhelmer : 1 danishka : 1 mail : 1 gphemsley : 1 Rebeccah : 1 ckreinbring : 1 stephen : 1 berker.peksag : 1 jezdez : 1 nchen : 1 iamjayakumars : 1 netfuzzerr : 1 benjamin : 1 Bugs resolved: 1116 rrosario : 386 resolved, 273 fixed rdalal : 152 resolved, 150 fixed a.topal : 121 resolved, 41 fixed mcooper : 118 resolved, 105 fixed willkg : 72 resolved, 61 fixed scoobidiver : 31 resolved, 0 fixed michaljev : 20 resolved, 17 fixed swarnavasengupta : 18 resolved, 0 fixed shuhao : 16 resolved, 13 fixed me+bugzilla : 12 resolved, 5 fixed berker.peksag : 12 resolved, 12 fixed mverdi : 11 resolved, 5 fixed erik : 9 resolved, 9 fixed lhenry : 8 resolved, 1 fixed krystaiceman : 7 resolved, 1 fixed tobbi.bugs : 7 resolved, 6 fixed joshua-smith : 6 resolved, 4 fixed jfong : 5 resolved, 5 fixed tdowner : 5 resolved, 2 fixed feer56 : 5 resolved, 0 fixed david.weir : 5 resolved, 4 fixed bwbrowning : 5 resolved, 5 fixed bram : 5 resolved, 4 fixed ibai : 4 resolved, 1 fixed alastra.mariagrazia : 4 resolved, 2 fixed laura : 4 resolved, 4 fixed williamr : 3 resolved, 1 fixed buchanae+bugs : 3 resolved, 3 fixed bharath_ves : 3 resolved, 3 fixed paul+moz : 3 resolved, 3 fixed chris.lonnen : 2 resolved, 2 fixed smolejv : 2 resolved, 1 fixed zcampbell : 2 resolved, 0 fixed iamjayakumars : 2 resolved, 0 fixed curtisk : 2 resolved, 0 fixed leszekz : 2 resolved, 0 fixed abc : 2 resolved, 1 fixed pcvrcek : 2 resolved, 0 fixed taygunagiali : 2 resolved, 2 fixed mail : 1 resolved, 0 fixed bmo2010 : 1 resolved, 1 fixed wymette : 1 resolved, 0 fixed beaotx : 1 resolved, 1 fixed nelson : 1 resolved, 1 fixed madperson : 1 resolved, 1 fixed coce : 1 resolved, 0 fixed pmcclard : 1 resolved, 1 fixed tgavankar : 1 resolved, 1 fixed bugzilla-fromthedeep : 1 resolved, 0 fixed rtanglao : 1 resolved, 0 fixed stephen.donner : 1 resolved, 1 fixed guillermo.movia : 1 resolved, 1 fixed lorchard : 1 resolved, 1 fixed nukeador : 1 resolved, 0 fixed rtucker11 : 1 resolved, 0 fixed nishant_cs : 1 resolved, 0 fixed stas : 1 resolved, 0 fixed mattbasta : 1 resolved, 1 fixed satishb3 : 1 resolved, 0 fixed ragsagar : 1 resolved, 1 fixed rmcguigan : 1 resolved, 1 fixed nchen : 1 resolved, 0 fixed kudrom : 1 resolved, 1 fixed andrei.hutusoru : 1 resolved, 0 fixed reed : 1 resolved, 0 fixed tiziana.sel : 1 resolved, 0 fixed chance.zibolski : 1 resolved, 1 fixed alice0775 : 1 resolved, 0 fixed ravi : 1 resolved, 0 fixed nsm.nikhil : 1 resolved, 0 fixed gryllida : 1 resolved, 1 fixed deletesoftware+moz : 1 resolved, 0 fixed pmjcreations : 1 resolved, 0 fixed boerni : 1 resolved, 1 fixed rforbes : 1 resolved, 0 fixed dbialer : 1 resolved, 1 fixed jgross : 1 resolved, 1 fixed INCOMPLETE : 43 WONTFIX : 51 DUPLICATE : 64 INVALID : 73 WORKSFORME : 121 FIXED : 764 Research bugs: 15 761582: [research] Add feature: Articles that link to this article 788104: [research] [ux] Support multiple products in the support forum 815089: [research] Investigate telling apart Firefox for Desktop and Firefox for Mobile tweets 816970: [research] SurveyGizmo API to be used in automated exit survey 823060: [research] Use datetime instead of ints in ES mappings 823891: [research] Adding KB revisions feature 825621: [research] Store the templates, article links and images in each article 825624: [research] Investigate how to update to Twitter API v1.1 841412: [research] Bad localization strings shouldn't break the site. 845290: [research] URL bar should fade away on SUMO 854554: [research] Youtube embeds don't work with templates 889884: [research] Open Badges! 889890: [research][discuss] figure out how to improve our l10n situation with search 906992: [research] Add support for multiple ES indexes by doc type 937889: [research] Login users via a URL in email Tracker bugs: 20 433161: [Tracker] Support for forums in other languages 625891: [tracker] HTML email 721462: [tracker] Taxonomy IA improvements 758598: [Tracker] Search UX suggestions 783262: [tracker] Add rate limiting to protect us from spammers 790785: [Tracker] L10n tools editing part 790786: [Tracker] L10n tools organization part 800962: [Tracker] Add activity history page for KB 815625: [Tracker] Segment dashboards and other contributor pages by product 817540: [tracker] AJAXify the refine+focus panel 825606: [tracker] Switch everything from Webtrends to Google Analytics 827640: [tracker] Localize Questions 838584: [Tracker] Getting ready for Firefox OS launch 845286: [tracker] Use as little bandwidth as possible on mobile version of SUMO 845773: [Tracker] move to an OS charting solution 848520: [tracker] Make all traffic HTTPS 851730: [tracker] Close threads pro-actively 871559: [tracker] update codebase to django 1.4 layout 897057: [tracker] Open Badges -- stage 1 920530: [tracker] support Webmaker on SUMO git === Total commits: 1138 Ricky Rosario : 492 (+16258, -16435, files 2972) Will Kahn-Greene : 178 (+8311, -3748, files 438) Rehan Dalal : 168 (+13016, -5554, files 680) Mike Cooper : 145 (+46955, -22136, files 582) Kadir Topal : 39 (+352, -110, files 61) Michał Frontczak : 19 (+229, -182, files 78) Berker Peksag : 15 (+570, -717, files 73) Shuhao Wu : 15 (+1523, -127, files 51) Jen Fong-Adwent : 9 (+138, -18, files 17) Tobbi : 8 (+338, -204, files 13) browning : 5 (+140, -16, files 12) davd Weir : 4 (+15, -1, files 4) Joshua Smith : 4 (+94, -87, files 13) Tobias Markus : 3 (+8, -8, files 4) Anush : 3 (+4, -1, files 3) Gaurav Dadhania : 3 (+3, -3, files 3) Bharath Thiruveedula : 3 (+15, -14, files 3) ibai : 3 (+30, -30, files 4) kudrom : 2 (+9, -9, files 5) Nghi Tran : 2 (+2, -1, files 2) Tanner Filip : 2 (+4, -4, files 2) Börni : 2 (+30, -15, files 4) madperson : 2 (+5, -4, files 2) Taygun AGIALI : 2 (+7, -6, files 3) TylerDowner : 2 (+3, -3, files 2) James Socol : 2 (+37, -27, files 3) david-w : 1 (+1, -1, files 1) ragsagar : 1 (+16, -1, files 2) Guillermo Movia : 1 (+1, -0, files 1) rosanaar : 1 (+9, -0, files 1) Gryllida : 1 (+26, -6, files 3) Beatriz Nombela : 1 (+9, -9, files 6) Total lines added: 88158 Total lines deleted: 49477 Total files changed: 5048
Ricky does a lot of work! Holy cow!
In 2011, we had 19 people who contributed code changes.
In 2012, we had 23 people.
In 2013, we had 32 people.
Like 2011 and 2012, we resolved more bugs than we created in 2013. That's three years in a row! I've never seen that happen on a project I work on.
There are a lot of people braving Bugzilla to write up bugs. Skimming the list, I see developers, non-developers, Support contributors, localizers, support team and a lot of people I don't recognize.
Here's some number comparisons:
I spent a good chunk of 2013 working on Input, but here's what I remember from SUMO development in 2013:
- We rearranged the codebase for better Django 1.4 layout. That was a project. Oy.
- We added support for non-English languages to the support forums!
- We switched email to be HTML formatted. We also reworked email to be localized.
- We switched to Google Analytics.
- We implemented Open Badges---though there's still a few important pieces to finish there.
- We switched to YouTube for videos.
- We added support for Webmaker and Firefox OS. Thunderbird support will be added to SUMO in 2014.
- Mike took a lantern, a crust of bread and a big sword and spelunked into the darkest dungeons filled with stinky, squelchy muck and rewrote the showfor code.
- We reworked our search code to handle multiple indexes, though we haven't taken advantage of that, yet.
- We switched deployment to use Dennis to lint all translated strings before pushing them to production. This has almost assuredly saved us from production fires. I hated those kinds of fires. Hooray for Dennis!
- We wrote and switched to Ernest for sprint planning and coordination.
- We overhauled everything to add support for Persona authentication, but had to push off deployment indefinitely because of problems with Persona which are being ironed out by the Persona team.
- We added an escalation system for questions that haven't received a response in x hours for some positive value of x that is still in flux.
- We ditched Highcharts.
- We wrote a command-line deployer which tells us exactly what's going out and tells New Relic, too. This gives us a much better idea of what we're deploying and how it affected the site afterwards. This command-line deployer is named chief-james in honor of James who has moved on to greener and well measured pastures.
- We added a bunch of new metrics, dashboards, history pages, activity pages, icons, bicons, landing pages, take-off pages, topics, subtopics, toe picks and all kinds of stuff.
That's the gist of the year: it was a lot of work, but we accomplished a ton.
w00t for 2013!
Fri, 08 Feb 2013
A while back, I wrote a post about poxx.py which talked about a script I based on Ned Batchelder's poxx.py script and overhauled to provide a faux "Swedish Chef" translation of Miro strings allowing me to test localizations of the application.
The transform from English to "Swedish Chef" had the following four impotant properties:
- the output is vaguely readable
- the output is longer than the input which helps us find ui issues
- the output is clearly distinguished from English which helps us find strings that aren't getting translated
- the output is mildly amusing which is sometimes important in dark times
Back in August, I made some changes and pulled it into fjord. This helped us suss out localization issues on a new site. However, I wasn't really happy with it. Amongst other things, I always wondered if "Swedish Chef" was kind of culturally insensitive.
A couple of weeks ago, I overhauled poxx.py again. This time, PIRATE! It continues to have the four properties I think are important for a test locale.
We're using it now for SUMO development. It's the grog to your Jolly Roger:
We're using this script on both SUMO and Fjord now. You can use it for your site, too! The code is at https://github.com/mozilla/kitsune/tree/master/scripts/.
If you see any problems with it, toss me a message in a bottle.
This localization is only available in development environments. Unlike Miro where we shipped the Swedish Chef translation (or used to---I'm not sure if they do anymore), you cannot see this on the -dev, -stage or -prod SUMO sites.
Mon, 31 Dec 2012
This was my first full year at Mozilla and it was intense. I essentially worked on four projects: SUMO, Input, ElasticUtils and Gaia. This blog post talks about the first two which are worked on by the James' Rifles SUMINPUT Megalosaur team.
We accomplished a lot on SUMO this year. I spent a couple of hours last week throwing together a rough "year in review" script that looked at Bugzilla and git and crunched some numbers:
Twas the year: 2012 =================== Bugzilla ======== Bugs created: 938 rrosario : 201 a.topal : 188 willkg : 108 scoobidiver : 51 igarcia : 41 mverdi : 36 swarnavasengupta : 30 james : 29 bram : 19 tobbi.bugs : 17 Bugs resolved: 1025 rrosario : 335 : WORKSFORME 18 : INVALID 16 : DUPLICATE 23 : WONTFIX 7 : FIXED 263 : INCOMPLETE 8 a.topal : 182 : WORKSFORME 36 : INVALID 41 : DUPLICATE 11 : WONTFIX 70 : FIXED 21 : INCOMPLETE 3 willkg : 131 : DUPLICATE 6 : FIXED 110 : WORKSFORME 2 : WONTFIX 11 : INVALID 2 rdalal : 84 : FIXED 84 james : 51 : WORKSFORME 6 : INVALID 5 : DUPLICATE 3 : WONTFIX 15 : FIXED 14 : INCOMPLETE 8 mcooper : 37 : FIXED 36 : INVALID 1 tobbi.bugs : 29 : FIXED 29 tgavankar : 28 : WONTFIX 1 : WORKSFORME 1 : FIXED 26 scoobidiver : 28 : FIXED 4 : DUPLICATE 4 : WORKSFORME 11 : WONTFIX 3 : INVALID 6 bmo2010 : 13 : FIXED 1 : DUPLICATE 3 : WORKSFORME 3 : INVALID 6 INCOMPLETE : 21 DUPLICATE : 61 WORKSFORME : 82 INVALID : 91 WONTFIX : 117 FIXED : 653 git === Total commits: 916 Ricky Rosario : 430 Will Kahn-Greene : 192 Rehan Dalal : 98 Mike Cooper : 44 Erik Rose : 34 Tobbi : 29 Tanay Gavankar : 23 Kadir Topal : 11 Tim Watts : 10 Berker Peksag : 9 James Socol : 7 Victor Neo : 6 Cesar Carruitero : 5 David Lilly : 4 Ibai : 3 Isac Lagerblad : 2 icaaq : 1 TylerDowner : 1 browning : 1 ricky rosario : 1 Anatoli Papirovski : 1 Clauber Stipkovic : 1 Jason Thomas : 1 atopal : 1 Florin Strugariu : 1
There are some interesting bits in there:
Ricky does a lot of work! Holy cow!
There were 23 people who contributed code to Kitsune (the SUMO codebase) this year. Of those, about half are volunteer contributors.
Compare with 2011, we had 19 people who contributed to the code base and less than half were volunteer contributors.
We resolved more bugs than we created in 2012. We did that in 2011 as well, so that's two years in a row. I've never seen that happen before on a project I work on.
The codebase is pretty different now than it was at the beginning of the year. I helped with the following semi-massive overhauls:
- The push for more metrics and dashboards to view the numbers.
- The switch from Sphinx to ElasticSearch.
- The new Information Architecture which affected browsing and searching across the site.
- The site redesign which covered both the desktop and mobile versions of the site.
- The upgrade to Django 1.4.
- The switch from arecibo to sentry.
- The push to switch from fixtures to model makers for all our tests.
- The switch from weekly deployments on Tuesdays to deploying whenever we want. Continuous deployment is fantastic.
- Started switching the whole site from Webtrends to Google Analytics. I saw Ricky write up a bunch of bugs to finish up that work, so I'll say it's in progress.
- During the redesign, Rehan redid all the CSS and switched us to use LESS.
- I spun off some code I wrote for richard, then ported to Fjord, then improved into a project called django-eadred. That makes it a lot easier to generate sample data for a variety of purposes like new contributors, bootstrapping, and large random data sets.
On top of that, we did a lot of work on the documentation and making it easier to get to a working Kitsune development environment. We switched to a sprint-based work flow using Scrumbugz. We also nixed our daily checkin conference call for an IRC-based checkin system that we wrote called Standup.
It's been a big year.
For Input, it was a bigger year. We decided to abandon the old Input codebase (omfg yay) in favor of rewriting it from the ground up. The rewrite took a couple of months and then has sort of been sitting around waiting for a security review. In the meantime, we (actually, Mike did) fixed a bunch of issues with the old site code because that's what's currently in production.
Rewriting Input wouldn't have taken so long except that we did a lot of work fixing bugs in external libraries and updating Playdoh. That work definitely cut into our schedule, but it benefitted a bunch of other groups/people/sites, so that's good.
That's the gist of the year: it was a lot of work, but we accomplished a ton.
w00t for 2012!
Sun, 15 Apr 2012
I haven't had time to blog much in the last few months. At work, I've been spending all my time with elasticsearch, elasticutils, and SUMO bug fixing. I've been working on the conversion from Sphinx search to elasticsearch for SUMO since I started at Mozilla, but I've only recently felt like I'm really getting the hang of it. There are a bunch of elasticutils-related things I want to blog about, but those will come in fugure entries.
In my spare time, I've been working on richard. This project has nothing to do with Richard of air mozilla fame, but rather is a video indexing web application. It's the software that runs pyvideo.org.
pyvideo.org has the distinction of being the first Django application I've built from the ground up. That distinction is both a virtue (yay for first apps!) and a vice (boo for silly things I did when doing it!).
The one thing I did that I'm really proud of is that when building the software, I knew I needed help if it was to succeed and thus I worked to make it easy and inviting for contributors to get involved:
- I wrote documentation: license file, README, documentation covering how to install it for hacking, how to contribute, where to find me, ...
- I parked the code on GitHub to make it easier for people to access. 
- I made sure there were a series of issues in the issue tracker that showed the next round of things that needed to be done.
- I made sure I had an IRC channel and that people knew where to find me to ask questions.
- I quickly got the documentation built on ReadTheDocs.
- I had a test infrastructure set up. 
- I respond to everyone who sends an email, creates a pull request, writes an issue, says hi on IRC, ...
The end result of that is that there are 4 contributors to richard including myself and one of them is very active.
Well, I've got 5 on my video index web application software that I "launched" a month ago. I'm feeling good about that.
|||Several of my friends point out that GitHub kind of takes the D out of DVCS.|
|||Though didn't have any tests when I "launched".|
|||I may fix this paragraph after Asheesh corrects me.|