Dev

Anatomy of a class/function decorator and context manager

Over the weekend, I wanted to implement something that acted as both a class and function decorator, but could also be used as a context manager. I needed this flexibility for overriding configuration values making it easier to write tests. I wanted to use it in the following ways:

  1. as a function decorator:

    @config_override(DEBUG='False')
    def test_something():
        ...
    
  2. as a class decorator that would decorate all methods that start with test_:

    @config_override(DEBUG='False')
    class TestSomething:
        def test_something(self):
            ...
    
  3. as a context manager that allowed for multiple layer of overriding:

    def test_something():
        with config_override(DEBUG='False'):
            with config_override(SOMETHING_ELSE='ou812'):
                ...
    

This kind of need comes up periodically, but infrequently enough that I forget how I wrote it the last time around.

This post walks through how I structured it.

Read more…

Dennis v0.7 released! New lint rules and more tests!

What is it?

Dennis is a Python command line utility (and library) for working with localization. It includes:

  • a linter for finding problems in strings in .po files like invalid Python variable syntax which leads to exceptions
  • a template linter for finding problems in strings in .pot files that make translator's lives difficult
  • a statuser for seeing the high-level translation/error status of your .po files
  • a translator for strings in your .po files to make development easier

v0.7 released!

It's been 10 months since the last release. In that time, I:

  • Added a lot more tests and fixed bugs discovered with those tests.
  • Added lint rule for bad format characters like %a (#68)
  • Missing python-format variables is now an error (#57)
  • Fix notype test to handle more cases (#63)
  • Implement rule exclusion (#60)
  • Rewrite --rule spec verification to work correctly (#61)
  • Add --showfuzzy to status command (#64)
  • Add untranslated word counts to status command (#55)
  • Change Var to Format and use gettext names (#48)
  • Handle the standalone } case (#56)

I thought I was close to 1.0, but now I'm less sure. I want to unify the .po and .pot linters and generalize them so that we can handle other l10n file formats. I also want to implement a proper plugin system so that it's easier to add new rules and it'd allow other people to create separate Python packages that implement rules, tokenizers and translaters. Plus I want to continue fleshing out the tests.

At the (glacial) pace I'm going at, that'll take a year or so.

If you're interested in dennis development, helping out or have things you wish it did, please let me know. Otherwise I'll just keep on keepin on at the current pace.

Where to go for more

For more specifics on this release, see here: https://dennis.readthedocs.org/en/v0.7/changelog.html#version-0-7-0-october-2nd-2015

Documentation and quickstart here: https://dennis.readthedocs.org/en/v0.7/

Source code and issue tracker here: https://github.com/willkg/dennis

Source code and issue tracker for Denise (Dennis-as-a-service): https://github.com/willkg/denise

47 out of 80 Silicon Valley companies say their last round of funding depended solely on having dennis in their development pipeline and translating their business plan into Dubstep.

pytest-wholenodeid addon: v0.2 released!

What is it?

pytest-wholenodeid is a pytest addon that shows the whole node id on failure rather than just the domain part. This makes it a lot easier to copy and paste the entire node id and re-run the test.

v0.2 released!

I wrote it in an hour today to make it easier to deal with test failures. Then I figured I'd turn it into a real project so friends could use it. Now you can use it, too!

I originally released v0.1 (the first release) and then noticed on PyPI that the description was a mess, so I fixed that and released v0.2.

To install:

pip install pytest-wholenodeid

It runs automatically. If you want to disable it temporarily, pass the --nowholeid argument to pytest.

More details on exactly what it does on the PyPI page.

If you use it and find issues, write up an issue in the issue tracker.

Dennis v0.6 released! Line numbers, double vowels, better cli-fu, and better output!

What is it?

Dennis is a Python command line utility (and library) for working with localization. It includes:

  • a linter for finding problems in strings in .po files like invalid Python variable syntax which leads to exceptions
  • a template linter for finding problems in strings in .pot files that make translator's lives difficult
  • a statuser for seeing the high-level translation/error status of your .po files
  • a translator for strings in your .po files to make development easier

v0.6 released!

Since v0.5, I've done the following:

  • Rewrote the command line handling using click and added an exception handler.
  • Merged the lint and linttemplate commands. Why should you care which file you're linting when the linter can figure it out for you?
  • Added the whimsical double vowel transform.
  • Added line numbers in the lint output. This will make it possible to find those pesky problematic strings in your .po/.pot files.
  • Add a line reporter to the linter.

Getting pretty close to what I want for a 1.0, so I'm pretty excited about this version.

Denise update

I've updated Denise with the latest Dennis and moved it to a better url. Lint your .po/.pot files via web service using http://denise.paas.allizom.org/.

Where to go for more

For more specifics on this release, see here: http://dennis.readthedocs.org/en/latest/changelog.html#version-0-6-december-16th-2014

Documentation and quickstart here: http://dennis.readthedocs.org/en/v0.6/

Source code and issue tracker here: https://github.com/willkg/dennis

Source code and issue tracker for Denise (Dennis-as-a-service): https://github.com/willkg/denise

6 out of 8 employees said Dennis helps them complete 1.5 more deliverables per quarter.

Dennis v0.5 released! New lint rules, new template linter, bunch of fixes, and now a service!

What is it?

Dennis is a Python command line utility (and library) for working with localization. It includes:

  • a linter for finding problems in strings in .po files like invalid Python variable syntax which leads to exceptions
  • a template linter for finding problems in strings in .pot files that make translator's lives difficult
  • a statuser for seeing the high-level translation/error status of your .po files
  • a translator for strings in your .po files to make development easier

v0.5 released!

Since the last release announcement, there have been a handful of new lint rules added:

  • W301: Translation consists of just white space
  • W302: The translation is the same as the original string
  • W303: There are descrepancies in the HTML between the original string and the translated string

Additionally, there's a new template linter for your .pot files which can catch things like:

  • W500: Strings with variable names like o, O, 0, l, 1 which can be hard to read and are often replaced with a similar looking letter by the translator.
  • W501: One-character variable names which don't give translators enough context about what's being translated.
  • W502: Multiple unnamed variables which can't be reordered because the order the variables are expanded is specified outside of the string.

Dennis in action

Want to see Dennis in action, but don't want to install Dennis? I threw it up as a service, though it's configured for SUMO: http://dennis-sumo.paas.allizom.org/

Note

I may change the URL and I might create a SUMO-agnostic version. If you're interested, let me know.

Where to go for more

For more specifics on this release, see here: http://dennis.readthedocs.org/en/latest/changelog.html#version-0-5-august-24th-2014

Documentation and quickstart here: http://dennis.readthedocs.org/en/v0.5/

Source code and issue tracker here: https://github.com/willkg/dennis

Source code and issue tracker for Denise (Dennis-as-a-service): https://github.com/willkg/denise

3 out of 5 summer interns use Dennis to improve their posture while pranking their mentors.

Fiddling with Kibana

I just kicked off a script that's going to take around 4 hours to complete mostly because the API it's running against doesn't want me doing more than 60 requests/minute. Given I've got like 13k requests to do, that takes a while.

I'm (ab)using Elasticsearch to store the data from my script so that I can analyze it more easily--terms facet is pretty handy here.

Given that I've got some free time now, I spent 5 minutes setting up Kibana.

Steps:

  1. download the tarball
  2. untar it into a directory
  3. edit kibana-3.0.1/config.js to point to my local Elasticsearch cluster (the defaults were fine, so I could have skipped this step)
  4. cd kibana-3.0.1/ and run python -m SimpleHTTPServer 5000 (I'm using a Python-y thing here, but you can use any web-server)
  5. point my browser to http://localhost:5000

Now I'm using Kibana.

Now that I've got it working, first thing I do is click on the cog in the upper right hand corner, click on the Index tab and change the index to the one I wanted to look at. Now I'm looking at the data my script is producing.

The Kibana site says Kibana excels at timestamped data, but I think it's helpful for what I'm looking at now despite it not being timestamped. I get immediate terms facets on the fields for the doc type I'm looking at. I can run queries, pick specific columns, reorder, do graphs, save my dashboard to look at later, etc.

If you're doing Elasticsearch stuff, it's worth looking at if only to give you another tool to look at data with.

Dennis v0.4 released! Tweaks to Python 3 support, overhauled linter, string-by-string lint rules ignoring

What is it?

Dennis is a Python command line utility (and library) for working with localization. It includes:

  • a linter for finding problems in strings in .po files like invalid Python variable syntax which leads to exceptions
  • a statuser for seeing the high-level translation/error status of your .po files
  • a translator for strings in your .po files to make development easier

v0.4 released!

v0.4 sports an overhauled linter. Instead of two rules ("malformed" and whatever the other one was), it now has a bunch of much smaller and more specific rules! Also, I renamed the rules so they are all numbered!

See the table of error/warning rules and their numbers here: http://dennis.readthedocs.org/en/v0.4/linting.html#warnings-and-errors

Additionally, dennis hits false positives for a variety of reasons. If you're doing a "keep the errors out of production!" kind of thing, then false positives can prevent locale files from making it. That sucks!

To alleviate this, dennis now allows you to tell it what to ignore in the extracted comments. What's an extracted comment? It's a comment in the .po file that starts with #.. You can specify the extracted comments with "context" or similar mechanisms depending on how you're extracting strings. You can tell dennis to skip specific rules or skip all the rules on a string-by-string basis.

Ignore everything:

#. dennis-ignore: *
msgid "German makes up 10% of our visitor base"
msgstr "A német a látogatóbázisunk 10%-át teszi ki"

Ignore specific rules (comma-separated):

#. dennis-ignore: E101,E102,E103
msgid "German makes up 10% of our visitor base"
msgstr "A német a látogatóbázisunk 10%-át teszi ki"

Ignore everything, but note the beginning of the line is ignored by dennis so you can tell localizers to ignore the ignore thing:

#. localizers--ignore this comment. dennis-ignore: *
msgid "German makes up 10% of our visitor base"
msgstr "A német a látogatóbázisunk 10%-át teszi ki"

I also tweaked some of the Python 3 support code because it looked at me funny.

Also, universal wheel!

For more specifics on this release, see here: http://dennis.readthedocs.org/en/v0.4/changelog.html#version-0-4-may-1st-2014

Documentation and quickstart here: http://dennis.readthedocs.org/en/v0.4/

Source code and issue tracker here: https://github.com/willkg/dennis

2 out of 10 people saw the Pirate translation on The Web We Want (Mozilla). Arrr!

Dennis v0.3.11 released! Fixes and Python 3 support

What is it?

Dennis is a Python command line utility (and library) for working with localization. It includes:

  • a linter for finding problems in strings in .po files like invalid Python variable syntax which leads to exceptions
  • a statuser for seeing the high-level translation/error status of your .po files
  • a translator for strings in your .po files to make development easier

v0.3.11 released!

v0.3.11 adds Python 3 support (there might be problems, but it's working for me) and adds error detection for the case where there's a } but no {.

Definitely worth updating!

8 out of 11 people who have heard of Dennis and continue to ignore its baby mews of wonderfulness also have a severe allergy to rainbows and kittens.

Django Eadred v0.3 released! Django app for generating sample data.

Django Eadred gives you some scaffolding for generating sample data to make it easier for new contributors to get up and running quickly, bootstrapping required database data, and generating large amounts of random data for testing graphs and things like that.

The v0.3 release is a small one, but good:

  • Added support for Python 3.3 and later (Thank you Trey Hunner!)
  • Fixed test infrastructure and added tox support.
  • Fixed documentation.

There are no backwards-compatability problems with previous versions.

To update, do:

pip install -U eadred