The purpose of Input is to collect actionable feedback from our user base across each channel of our software development process. The application collects feedback and offers a set of analysis methods for looking at the resulting data.
This is my 2014 retrospective for Input.
The purpose of Input is to collect actionable feedback from our user base across each channel of our software development process. The application collects feedback and offers a set of analysis methods for looking at the resulting data.
This is my 2014q4 retrospective on Input.
It's been 3 months since the last status report. Crimey! That's not great, but it's even worse because it makes this report very long.
First off, lots of great work done by Adam Okoye, L. Guruprasad, Bhargav Kowshik, and Deshraj Yadav! w00t!
Second, we did a ton of stuff and broke 1,000 commits! w00t!
Third, I've promised to do more frequent status reports. I'll go back to one every two weeks.
Onward!
High-level summary:
Lots of code quality work.
Updated ElasticUtils to 0.10.1 so we can upgrade our Elasticsearch cluster.
Heartbeat v2.
Overhauled the generic feedback form.
remote-troubleshooting data capture.
contribute.json file.
Upgrade to Django 1.6.
Upgrade to Python 2.7!!!!!
Improved and added to pre-commit and commit-msg linters.
Landed and deployed:
ce95161 Clarify source and campaign parameters in API
286869e [bug 788281] Add update_product_details to deploy
bbedc86 [bug 788281] Implement basic events API
1c0ff9f [bug 1071567] Update ElasticUtils to 0.10.1
7fd52cd [bug 1072575] Rework smart_timedelta
ce80c56 [bug 1074276] Remove abuse classification prototype
7540394 [bug 1075563] Fix missing flake8 issue
11e4855 [bug 1025925] Change file names (Adam Okoye)
23af92a [bug 1025925] Change all instances of fjord.analytics.tools to fjord.analytics.utils (Adam Okoye)
ae28c60 [bug 1025925] Change instances of of util relating to fjord/base/util.py (Adam Okoye)
bc77280 Add Adam Okoye to CONTRIBUTORS
545dc52 [bug 1025925] Change test module file names
fc24371 [bug 1041703] Drop prodchan column
9097b8f [bug 1079376] Add error-email -> response admin view
d3cfdfe [bug 1066618] Tweak Gengo account balance warning
a49f1eb [bug 1020303] Add rating column
55fede0 [bug 1061798] Reset page number Resets page number when filter checkbox is checked (Adam Okoye)
1d4fd00 [bug 854479] Fix ui-lightness 404 problems
a9bf3b1 [bug 940361] Change size on facet calls
c2b2c2b [bug 1081413] Move url validation code into fjord_utils.js Rewrote url validation code that was in generic_feedback.js and added it to fjord_utils.js (Adam Okoye)
4181b5e [bug 1081413] Change code for url validation (Adam Okoye)
2cd62ad [bug 1081413] Add test for url validation (Adam Okoye)
f72652a [bug 1081413] Correct operator in test_fjord_utils.js (aokoye)
c9b83df [bug 1081997] Fix unicode in smoketest
cba9a2d [bug 1086643] [bug 1086650] Redo infrastructure for product picker version
e8a9cc7 [bug 1084387] Add on_picker field to Product
2af4fca [bug 1084387] Add on_picker to management forms
1ced64a [bug 1081411] Create format test (Adam Okoye)
00f8a72 Add template for mentored bugs
e95d0f1 Cosmetic: Move footnote
d0cb705 Tweak triaging docs
d5b35a2 [bug 1080816] Add A/B for ditching chart
fa1a47f Add notes about running tests with additional warnings
ddde83c Fix mimetype -> content_type and int division issue
2edb3b3 [bug 1089650] Add a contribute.json file (Bhargav Kowshik)
d341977 [bug 1089650] Add test to verify that the JSON is valid (Bhargav Kowshik)
dcb9380 Add Bhargav Kowshik to CONTRIBUTORS
7442513 Fix throttle test
f27e31c [bug 1072285] Update Django, django-nose and django-cache-machine
dd74a3c [bug 1072285] Update django-adminplus
ececdf7 [bug 1072285] Update requirements.txt file
6669479 [bug 1093341] Tweak Gengo account balance warning
f233aab [bug 1094197] Fix JSONObjectField default default
11193d7 Tweak chart display
9d311ca Make journal.Record not derive from ModelBase
f778c9d Remove all Heartbeat v1 stuff
e5f6f4d Switch test__utils.py to test_utils.py
cab7050 [bug 1092296] Implement heartbeat v2 data model
5480c42 [bug 1097352] Response view is viewable by all
46b5897 [bug 1077423] Overhaul generic feedback form dev
da31b47 [bug 1077423] Update smoke tests for generic feedback form dev
e84094b Fix l10n email template
d6c8ea9 Remove gettext calls for product dashboards
e1a0f74 [bug 1098486] Remove under construction page
032a660 Fix l10n_status.py script history table
19cec37 Fix JSONObjectField
430c462 Improve display_history for l10n_status
d6c18c6 Windows NT 6.4 -> Windows 10
73a4225 [bug 1097847] Update django-grappelli to 2.5.6
4f3b9c7 [bug 1097847] Fix custom views in admin
3218ea3 Fix JSONObjectfield.value_to_string
67c6bf9 Fix RecordManager.log regarding instances
a5e8610 [bug 1092299] Implement Heartbeat v2 API
17226db [bug 1092300] Add Heartbeat v2 debugging views
11681c4 Rework env view to show python and django version
9153802 [bug 1087387] Add feedback_response.browser_platform
f5d8b56 [bug 1087387] [bug 1096541] Clean up feedback view code
c9c7a81 [bug 1087391] Fix POST API user-agent inference code
4e93fc7 [bug 1103024] Gengo kill switch
de9d1c7 Capture the user-agent
4796e4e [bug 1096541] Backfill browser_platform for Firefox dev
f5fe5cf [bug 1103141] Add experiment_version to HB db and api
98c40f6 [bug 1103141] Add experiment_version to views
0996148 [bug 1103045] Create a menial hb survey view
965b3ee [bug 1097204] Rework product picker
6907e6f [bug 1097203] Add link to SUMO
e8f3075 [bug 1093843] Increase length of product fields
2c6d24b [bug 1103167] Raise GET API throttle
d527071 [bug 1093832] Move feedback url documentation
6f4eb86 Abstract out python2.6 in deploy script
f843286 Fix compile-linted-mo.sh to take pythonbin as arg
966da77 Add celery health check
1422263 Add space before subject of celery health email
5e07dbd [heartbeat] Add experiment1 static page placeholders
615ccf1 [heartbeat] Add experiment1 static files
d8822df [heartbeat] Add SUMO links to sad page
3ee924c [heartbeat] Add twitter thing to happy page
d87a815 [heartbeat] Change thank you text
06e73e6 [heartbeat] Remove cruft, fix links
8208a72 [heartbeat] Fix "addons"
2eca74c [heartbeat] Show profile_age always because 0 is valid
4c4598b [bug 1099138] Fix "back to picker" url
b2e9445 Add note about "Commit to VCS" in l10n docs
9c22705 Heartbeat: instrument email signup, feedback, NOT Twitter (Gregg Lind)
340adf9 [heartbeat] Fix DOCTYPE and ispell pass
486bf65 [heartbeat] Change Thank you text
d52c739 [heartbeat] Switch to use Input GA account
f07716b [heartbeat] Fix favicons
eff9d0b [heartbeat] Fixed page titles
969c4a0 [heartbeat] Nix newsletter form for a link
dce6f86 [heartbeat] Reindent code to make it legible
dad6d82 [bug 1099138] Remove [DEV] from title
4204b43 fixed typo in getting_started.rst (Deshraj Yadav)
7042ead [bug 1107161] Fix hb answers view
a024758 [bug 1107809] Fix Gengo language guesser
808fa83 [bug 1107803] Rewrite Response inference code
d9e8ffd [bug 1108604] Tweak paging links in hb data view
00e8628 [bug 1108604] Add sort and display ts better in hb data view
17b908a [bug 1108604] Change paging to 100 in hb data view
39dc943 [bug 1107083] Backfill versions
fee0653 [bug 1105512] Rip out old generic form
b5bb54c Update grappelli in requirements.txt file
f984935 [bug 1104934] Add ResponseTroubleshootingInfo model
c2e7fd3 [bug 950913] Move 'TRUNCATE_LENGTH' and make accessable to other files (Adam Okoye)
b6f30e1 [bug 1074315] Ignore deleted files for linting in pre-commit hook (L. Guruprasad)
4009a59 Get list of .py files to lint using just git diff (L. Guruprasad)
c81da0b [bug 950913] Access TRUNCATE_LENGTH from generic_feedback template (Adam Okoye)
9e3cec6 [bug 1111026] Fix hb error page paging
b89daa6 Dennis update to master tip
61e3e18 Add django-sslserver
93d317b [bug 1104935] Add remote.js
ad3a5cb [bug 1104935] Add browser data section to generic form
cc54daf [bug 1104935] Add browserdata metrics
31c2f74 Add jshint to pre-commit hook (L. Guruprasad)
68eae85 Pretty-print JSON blobs in hb errorlog view
8588b42 [bug 1111265] Restrict remote-troubleshooting to Firefox
b0af9f5 Fix sorby error in hb data view
8f622cf [bug 1087394] Add browser, browser_version, and browser_platform to response view (Adam Okoye)
c4b6f85 [bug 1087394] Change Browser Platform label (Adam Okoye)
eb1d5c2 Disable expansion of $PATH in the provisioning script (L. Guruprasad)
59eebda Cosmetic test changes
aac733b [bug 1112210] Tweak remote-troubleshooting capture
6f24ce7 [bug 1112210] Hide browser-ask by default
278095d [bug 1112210] Note if we have browser data in response view
869a37c [bug 1087395] Add fields to CSV output (Adam Okoye)
Landed, but not deployed:
4ee7fd6 Update the name of the pre-commit hook script in docs (L. Guruprasad)
d4c5a09 [bug 1112084] create requirements/dev.txt (L. Guruprasad)
4f03c48 [bug 1112084] Update provisioning script to install dev requirements (L. Guruprasad)
03c5710 Remove instructions for manual installation of flake8 (L. Guruprasad)
a36a231 [bug 1108755] Add a git commit message linter (L. Guruprasad)
Current head: f0ec99d
PTO. It's been a really intense quarter (as you can see) and I need some rest. Maybe a nap. Plus we have a deploy freeze through to January, so we can't push anything out anyhow. I hope everyone else gets some rest, too.
That's it!
Dennis is a Python command line utility (and library) for working with localization. It includes:
a linter for finding problems in strings in .po files like invalid
Python variable syntax which leads to exceptions
a template linter for finding problems in strings in .pot files that
make translator's lives difficult
a statuser for seeing the high-level translation/error status of
your .po files
a translator for strings in your .po files to make development
easier
Since v0.5, I've done the following:
Rewrote the command line handling using click and added an exception handler.
Merged the lint and linttemplate commands. Why should you care
which file you're linting when the linter can figure it out for you?
Added the whimsical double vowel transform.
Added line numbers in the lint output. This will make it possible to
find those pesky problematic strings in your .po/.pot files.
Add a line reporter to the linter.
Getting pretty close to what I want for a 1.0, so I'm pretty excited about this version.
I've updated Denise with the latest Dennis and moved it to a better
url. Lint your .po/.pot files via web service using
http://denise.paas.allizom.org/.
For more specifics on this release, see here: http://dennis.readthedocs.org/en/latest/changelog.html#version-0-6-december-16th-2014
Documentation and quickstart here: http://dennis.readthedocs.org/en/v0.6/
Source code and issue tracker here: https://github.com/willkg/dennis
Source code and issue tracker for Denise (Dennis-as-a-service): https://github.com/willkg/denise
6 out of 8 employees said Dennis helps them complete 1.5 more deliverables per quarter.
Since the beginning of 2014, I've been laying the groundwork to rewrite the feedback form that we use on Input.
Today, after a lot of work, we pushed out the new form! Just in time for Firefox 34 release.
This blog post covers the circumstances of the rewrite.
In 2011, James, Mike and I rewrote Input from the ground up. In order to reduce the amount of time it took to do that rewrite, we copied a lot of the existing forms and styles including the feedback forms. At that time, there were two: one for desktop and one for mobile. In order to avoid a translation round, we kept all the original strings of the two forms. The word "Firefox" was hardcoded in the strings, but that was fine since at the time Input only collected feedback for Firefox.
In 2013, in order to reduce complexity on the site because there's only one developer (me), I merged the desktop and mobile forms into one form. In order to avoid a translation round, I continued to keep the original strings. The wording became awkward and the flow through the form wasn't very smooth. Further, the form wasn't responsive at all, so it worked ok on desktop machines, but mediocre on other viewport sizes.
2014 rolled around and it was clear Input was going to need to branch out into capturing feedback for multiple products---some of which were not Firefox. The form made this difficult.
Related, the smoketest framework I wrote in 2014 struggled with testing the form accurately. I spent some time tweaking it, but a simpler form would make smoketesting a lot easier and less flakey.
Thus over the course of 3 years, we had accumulated the following problems:
The flow through the form felt awkward, instructions weren't clear and information about what data would be public and what data would be private wasn't clear.
Strings had "Firefox" hardcoded and wouldn't support multiple products.
The form wasn't responsive and looked/behaved poorly in a variety of situations.
The form never worked in right-to-left languages and possibly had other accessibility issues.
The architecture didn't let us experiment with the form---tweaking the wording, switching to a more granular gradient of sentiment, capturing other data, etc.
Further, we were seeing many instances of people putting contact information in the description field and there was a significant amount of dropoff.
I had accrued the following theories:
Since the email address is on the third card, users would put their email address in the description field because they didn't know they could leave their contact information later.
Having two cards would reduce the amount of drop-off and unfinished forms than three cards.
Having simpler instruction text would reduce the amount of drop-off.
Anyhow, it was due for an overhaul.
I've been working on the overhaul for most of 2014, but did the bulk of the work in October and November. It has the following changes:
The new form is shorter and clearer text-wise and design-wise.
It consists of two cards: one for capturing sentiment and one for capturing details about that sentiment.
It clearly delineates data that will be public from data that will be kept private.
It works with LTR and RTL languages (If that's not true, please open a bug.)
It fixes some accessibility issues. (If you find any, please open a bug.)
It uses responsive design, mobile first. Thus it was designed for mobile devices and then scaled to desktop-sized viewports.
It's smaller in kb size and requires fewer HTTP requests.
It's got a better architecture for future development.
It doesn't have "Firefox" hardcoded anymore.
It's simpler so the smoketests work reliably now.
The old Input feedback form.
The new Input feedback form.
The old and new forms were instrumented in various ways, so we'll be able to analyze differences between the two. Particularly, we'll be able to see if the new form performs worse.
Further, I'll be checking the data to see if my theories hold true especially the one regarding why people put contact data in the description.
There are a few changes in the queue that we want to make over the course of the next 6 months. Now that the new form has landed, we can start working on those.
Even if there are problems with the new form, we're in a much better position to fix them than we were before. Progress has been made!
Have you ever submitted feedback? Have you ever told Mozilla what you like and don't like about Firefox?
Take a moment and fill out the feedback form and tell us how you feel about Firefox.
Update: 1/18/2021: Input was decommissioned, so this doesn't work anymore.
I've been doing web development since 1997 or so. I did a lot of frontend work back then, but I haven't done anything serious frontend-wise in the last 5 years. Thus this was a big project for me.
I had a lot of help: Ricky, Mike and Rehan from the SUMO Engineering team were invaluable reviewing code, helping me fix issues and giving me a huge corpus of examples to learn from; Matt, Gregg, Tyler, Ilana, Robert and Cheng from the User Advocacy team who spent a lot of time smoothing out the rough edges of the new form so it captures the data we need; Schalk who wrote the product picker which I later tweaked; Matej who spent time proof-reading the strings to make sure they were consistent and felt good; the QA team which wrote the code that I copied and absorbed into the current Input smoketests; and the people who translated the user interface strings (and found a bunch of issues) making it possible for people to see this form in their language.
I've been working on Input for a while now. One of the things I've actively disliked was the chart on the front page. This blog post talks about why I loathe it and then what's happening this week.
First, here's the front page dashboard as it is today:
Input front page dashboard (October 2014)
When I started, Input gathered feedback solely on the Firefox desktop web browser. It was a one-product feedback gathering site. Because it was gathering feedback for a single product, the front page dashboard was entirely about that single product. All the feedback talked about that product. The happy/sad chart was about that product. Today, Input gathers feedback for a variety of products.
When I started, it was nice to have a general happy/sad chart on the front page because no one really looked at it and the people who did look at it understood why the chart slants so negatively. So the people who did look at it understood the heavy negative bias and could view the chart as such. Today, Input is viewed by a variety of people who have no idea how feedback on Input works or why it's so negatively biased.
When I started, Input didn't expose the data in helpful ways allowing people to build their own charts and dashboards to answer their specific questions. Thus there was a need for a dashboard to expose information from the data Input was gathering. I contend that the front page dashboard did this exceedingly poorly--what does the happy/sad lines actually mean? If it dips, what does that mean? If they spike, what does that mean? There's not enough information in the chart to make any helpful conclusions. Today, Input has an API allowing anyone to fetch data from Input in JSON format and generate their own dashboards of which there are several out there.
When I started, Input received some spam/abuse feedback, but the noise was far outweighed by the signal. Today, we get a ton of spam/abuse feedback. We still have no good way of categorizing spam/abuse as such and removing it from the system. That's something I want to work on more, but haven't had time to address. In the meantime, the front page dashboard chart has a lot of spammy noise in it. Thus the happy/sad lines aren't accurate.
Thus I argue we've far outlived the usefulness of the chart on the front page and it's time for it to go away.
So, what happens now? Bug 1080816 covers removing the front page dashboard chart. It covers some other changes to the front page, but I think I'm going to push those off until later since they're all pretty "up in the air".
If you depend on the front page dashboard chart, toss me an email. Depending on how many people depend on the front page chart and what the precise needs are, maybe we'll look into writing a better one.
This is the 2014q3 Input post-mortem. It was a better quarter than 2014q2--that one kind of sucked. Instead 2014q2 started out well and then got kind of busy and then I was pretty overwhelmed by the end.
Things to know:
Input is Mozilla's product feedback site.
Fjord is the code that runs Input.
I unilaterally decided to extend 2014q3 to October 6th.
I am Will Kahn-Greene and I'm the primary developer on Input.
Bugzilla
========
Bugs created: 58
Bugs fixed: 54
git
===
Total commits: 161
Will Kahn-Greene : 150 (+195704, -188071, files 672)
Ian Kronquist : 7 (+201, -53, files 11)
L. Guruprasad : 3 (+8, -18, files 3)
Ruben Vereecken : 1 (+34, -14, files 6)
Total lines added: 195947
Total lines deleted: 188156
Total files changed: 692
We added a bunch of code this quarter:
October 7th. 2014: 23466 total, 11614 Python
Compare to previous quarters:
2014q1: April 1st, 2014: 15195 total, 6953 Python
2014q2: July 1st, 2014: 20456 total, 9247 Python
Nothing wildly interesting there other than noting that the codebase for Input continues to grow.
Ian Kronquist was the Input intern for Summer 2014. He contributed several fixes to Input. Yay!
We spent a bunch of time making our docs and Vagrant provisioning script less buggy so as to reduce the problems new contributors have when working on Input. I talked with several people about things they're interested in working on. Plus several people did some really great work on Input.
Generally, I think Input is at a point where it's not too hard to get up and running, we've got several lists of bugs that are good ones to start with and the documentation is good-ish. I think the thing that's hampering us right now is that I'm not spending enough time and energy answering questions, managing the work and keeping things going.
Anyhow, welcome L. Guruprasad, Adam Okoye and Ruben Vereecken! Additionally, many special thanks to L. Guruprasad who fixed a lot of issues with the Vagrant provisioning scripts. That work is long and tedious, but it helps everyone.
Dashboards for everyone: We wrote an API and some compelling examples of dashboards you can build using the API. It's being used in a few places now. We'll grow it going forward as needs arise. I'm pretty psyched about this since it makes it possible for people with needs to help themselves and not have to wait for me to get around to their work. Dashboards for everyone project plan.
Vagrant: We took the work I did last quarter and improved upon it, rewrote the docs and have a decent Vagrant setup now. Reduce contributor pain project plan.
Abuse detection: Ian spent his internship working on an abuse classifier so that we can more proactively detect and prevent abusive feedback from littering Input. We gathered some interesting data and the next step is probably to change the approach we used and apply some more complex ML things to the problem. The key here is that we want to detect abuse with confidence and not accidentally catch swaths of non-abuse. Input feedback has some peculiar properties that make this difficult. Reduce the abuse project plan.
Loop support: Loop is now using Input for user sentiment feedback.
Heartbeat support: User Advocacy is working on a project to give us a better baseline for user sentiment. This project was titled Heartbeat, but I'm not sure whether that'll change or not. Regardless, we added support for the initial prototype. Heartbeat project plan.
Data retention policy: We've been talking about a data retention policy for some time. We decided on one, finalized it and codified it in code.
Shed the last vestiges of Playdoh and funfactory: We shed the last bits of Playdoh and funfactory. Input uses the same protections and security decisions those two projects enforced, but without being tied to some of the infrastructure decisions. This made it easier to switch to peep-based requirements management.
Switched to FactoryBoy and overhauled tests: Tests run pretty fast in Fjord now. We switched to FactoryBoy, so writing model-based tests is a lot easier than the stuff we had before.
Better than 2014q2 and we fixed some more technical debt further making it easier to develop for and maintain Input. Still, there's lots of work to do.
Update April 21st, 2015
LGuruprasad found a bug in the script that caused commits-by-author information to be wrong. Fixed the script and updated the stats!
In 2014q3, I created the Dashboards for Everyone project. This blog post covers what the project is, what's out there now, some examples of usage and the future.
Input collects sentiment data on Mozilla products. Currently, this data can be seen on the front page dashboard. This dashboard sucks for a variety of reasons amongst which is that it's a slog of data that isn't particularly informative.
Further, it's pretty clear that the greater Mozillaverse has different informational needs. Some people are interested in issues with products in specific locales. Some people are interested in today's hot topic. Some people are interested in comparing the first week after release of one version of a product with another. Etc.
Given that, we have two paths:
Create dashboards that live on Input catering to all these needs.
Create an API that allows people to create their own dashboards and host them wherever.
These two paths aren't mutually exclusive. However, I want to put effort into the second one first for the following reasons:
It enables people to individually and collectively solve their own informational problems.
It enables people to solve informational problems as problems crop up rather than wait for Input developers to have time to write up a dashboard.
As people are solving their informational problems, we'll learn a lot more about what informational problems exist which will guide how we build dashboards that live on Input.
The Dashboards for Everyone project aims to create the infrastructure for enabling people to write their own dashboards using Input data.
Over the course of the quarter we put enough of the bits in place that building your own dashboard is a viable thing now.
First, we wrote and honed an API for getting feedback data.
Then we wrote a bunch of proof-of-concept dashboards that use this data.
I threw together these two dashboards which are defined in Github gists and "hosted" on bl.ocks.org:
Happy feedback: http://bl.ocks.org/willkg/c4d5a272f86ae4510750
Happy vs. Sad broken down by locale for Firefox OS: http://bl.ocks.org/willkg/e6013b21ec16d5e8192a
Ian Kronquist (Input intern for summer 2014) wrote this one:
Happitude comparison: http://bl.ocks.org/iankronquist/90f68ed445f804e63cdb
Those are examples of fetching and manipulating the data into a chart. They're not very informative. We used them to help flesh out the API and their current purpose is as examples for using the API to do things.
However, we've built some real things that use the data and are "in production" now:
I use the Input API for the Response Breakdown graphs on the Input Health Dashboard. That helps me figure out whether we've just pushed out bugs preventing people from leaving feedback. That uses a d3-based charting library I've been toying with to help me flesh out possible data transform needs. It also uses a library that lets me defined the charts in HTML attributes and they get "auto-charted" without me having to write JavaScript.
Cheng used the Input API to build the Hello dashboard for tracking Hello feedback.
I then reused most of his ideas and some of his code to produce a Firefox for Android trends dashboard. That's still got some issues namely that the tokenizing isn't very good and thus there's enough noise in the words lists that it doesn't bring real trending issues into starker relief.
That's where we're at. Now I need to tell people about it so that you know these possibilities exist.
Do you have informational needs for the data on Input?
Can the API help you solve your specific needs?
Are there important things missing that you need to have implemented in order to solve your needs?
If you use the Input API to build your own dashboards and you run into problems, write up a bug.
Bugs, conversations and seeing other peoples' dashboards will inform us as to how we need to grow the API going forward. That will set the priorities for the next version.
Also, I have a few ideas I've been mulling over:
Build an index of charts: If there are a sufficient number of interesting charts out there, then we should build an index of them on Input.
Build tokenizing into the Input API: If you look at the JavaScript code for my Firefox for Android Trends chart, a good portion of it deals with tokenizing. If tokenizing is a common thing people are doing to build charts, then we should pull that tokenizing in-house.
Build an auto-charter or chart widget library: Right now you have to do all the charting by hand. It'd be really nice to be able to throw dashboards together using chart "widgets" and possibly define the dashboard entirely in HTML. I've been toying with this with the Input Health Dashboard. I'm curious to find out if there's a need for throwing dashboards together quickly to follow certain issues (e.g. e10s, Hello, ...).
The Metrics team produces some really great looking dashboards that use MetricsGraphics.js. Maybe we could build a shim on top of that letting people more easily use that library with Input data?
I really want to thank Matt Grimes for coordinating d3 training, Cheng Wang for giving me some compelling code to reuse and Mike Cooper for helping me debug code whose mystery was only exceeded by its brokenness.
I'm using the fruits of this labor now, so at a bare minimum, it solved some of my problems.
Can it help you solve yours?
I've been cutting my own hair since like 1991 or so with two exceptions: a professional haircut before my wedding and one before my wife's sister's wedding.
Back in 1991, my parents bought me a set of Wahl clippers. Over the years, I broke two of the combs and a few of the extensions. Plus it has a crack down the side of the plastic body. At one point, I was cutting hair for a bunch of people on my dorm floor in college. It's seen a lot of use in 23 years.
However, a month ago, it started shorting the circuit. There's a loose wire or frayed something or something something. Between that and the crack down the side of the plastic body, I figured it's time to retire them and get a new set. The new set arrived today.
23 years is a long time. I have very few things that I've had for a long time. I bought my bicycle in 1992 or so. I have a clock radio I got in the mid-80s. I have a solar powered calculator from 1990 or so (TI-36). Everything else seems to fail within 5 years: blenders, toaster ovens, rice cookers, drills, computers, etc.
I'll miss those clippers. I hope the new ones last as long.
High-level summary:
Updated to ElasticUtils v0.10 which will allow us to upgrade our cluster to Elasticsearch 1.1. I'm working on a fix that'll let us to go to Elasticsearch 1.2, but that hasn't been released, yet.
Integrated the spicedham library prototype and set it up to classify abusive Input feedback. It's not working great, but that's entirely to be expected. I'm hoping to spend more time on spicedham and classification in Input in 2014q4. Ian did a great job with laying the foundation! Thank you, Ian!
Implemented a data retention policy and automated data purging.
Made some changes to the Input feedback GET and POST APIs to clarify things in the docs, fix some edge cases and make it work better for Firefox for Android and Loop.
Fixed the date picker in Chrome. Thank you, Ruben!
Landed and deployed:
c4e8e34 [[bug 1055520]] Update to ElasticUtils v0.10
e023fa4 [[bug 1055520]] Fix two reshape issues post EU 0.10 update
f9ba829 [[bug 1055785]] Codify data retention policy
91396a8 Generalize About page text so it works for all products
6fc03bf [[bug 1053863]] Update django to 1.5.9
85709b2 [[bug 1055788]] Implement data purging
c0677a1 [[bug 965796]] Add a products update page
121588d [[bug 1057353]] Update django-statsd and pystatsd
fe1c740 Add PII-related notes to the API fields
f77ecfa [[bug 799562]] Clarify API field documentation
c5eec03 [[bug 1055789]] Restrict front page dashboard and api to 6 months
0892546 [[bug 1059826]] Add max_length to url field in API
f192f84 [[bug 1057617]] Fix url data validation
aad961d [[bug 1030901]] Document Input GET API
2f212c5 [[bug 1015788]] Add flake8 linting
d673947 Update coding conventions
27a1b6b Add "maximum" arg to GET API
4f671e4 [[bug 1062436]] Add flags app and Flag model
9d03d4b Fix flake8_lint issues
0411d91 [[bug 1062453]] Add flagged view
56f7e24 [[bug 1062439]] Celery task for classification
7aa2930 [[bug 1062455]] Add spicedham to vendor (Ian Kronquist)
0d90df3 We don't need spicedham under vendor/packages (Ian Kronquist)
a2a491d fix [bug 1012965] - Date picker looks broken in chrome (Ruben Vereecken)
0c42213 [[bug 1063825]] Integrate spicedham into fjord
78a2d63 [[bug 1062444]] Initial training data
5ca816e [[bug 1020307]] Prepare for adding gradient to generic form
Current head: 5ca816e
Working on Dashboards-for-everyone bits. Documenting the GET API. Making it a bit more functional. Writing up some more examples. (https://wiki.mozilla.org/Firefox/Input/Dashboards_for_Everyone)
Gradients (https://wiki.mozilla.org/Firefox/Input/Gradient_Sentiment)
(django) Update to django-rest-framework 2.3.14 (bug #934979) -- I think this is straight-forward. We'll know if it isn't if the tests fail.
(django, cookies, debugging) API response shouldn't create anoncsrf cookie (bug #910691) -- I have no idea what's going on here because I haven't looked into it much.
For details, see our GetInvolved page:
https://wiki.mozilla.org/Webdev/GetInvolved/input.mozilla.org
If you're interested in helping, let me know! We hang out on
#input on irc.mozilla.org and there's the input-dev mailing list.
I've been codifying project plan details on the wiki:
https://wiki.mozilla.org/Firefox/Input
I have no idea who's going to use that information or whether it helps. If you see things that are missing, let me know. It'll help me hone the project management templates I'm using and know which information is important to keep up to date and which information I can let slide until rainy days.
That's it!