Will's blog

If you're writing code like this:

try:
   foo = somevar.getBlah()["xyz"].split(".")[-1].decode("ascii", "replace")
except:
   pass

Please stop! You're killing the rain forest!

In Miro, we've got long strings that are displayed in the user interface. I think the code that defines these strings is messy and hard to parse. For example:

def some_func():
    description = _("""\
This is a really long description that has multiple sentences and a few \
things that need to %(getfilledin)s and it goes on and on and on and on \
and I'm not really sure what's the best way to format it so that it's happy \
in editors and easier to parse.""") % {"getfilledin": blahblah}

PEP-8 doesn't address this, which is fine. I was curious to see what other projects do.

I'm working on improving the PyBlosxom testing situation and in the process of doing that ran into a problem with nose (version 0.10.0) and coverage (version 2.77). Both installed with easy_install.

When running:

nosetests --verbose --with-coverage --cover-package=Pyblosxom --include unit
nosetests --verbose --with-coverage --cover-package=Pyblosxom --include functional

I bumped into the problem described here (nose.python-hosting.com) and here (code.google.com). The solution is to either:

1. remove coverage.py from /usr/bin, or
2. change the filename from /usr/bin/coverage.py to /usr/bin/coverage

I work for Participatory Culture Foundation as a developer on the Miro internet video player.

Today we released Miro 1.0. For most of the projects I've worked on, the 1.0 mark is exciting to reach, but also somewhat of a downer because it really requires you to hone the list of all possible things you could do down to a small finite list. Inevitably, there are things people will want to do that 1.0 doesn't do. Still, it's a huge milestone for the project. We're already working on post-1.0 development and making changes to decisions that didn't turn out as well as we had hoped.

A good portion of the work we do is in Python with lots of interaction between Python and libraries written in C and C++. For example, we use a lot of Pyrex to speed up critical sections.

Miro is a great product to continue to push open standards for video distribution and consumption on the Internet. I regularly watch GoogleEDU videos through Miro and I'm hoping other Python-related channels will be available as well. At some point, I'd like to create my own Python-videos channel and aggregate good Python-related video content. There's a lot of screencasts and tutorials out there....

I'm really late to the PyPi party. Both of the projects I've worked on over the years are in PyPi, but neither are up to date and neither support easy_install.

Figured I'd update them now. Three things I bumped into while working on Lyntin:

1. the PyPi tutorial is pretty good and walks you through what you need to know,
2. the list of classifiers is here, and
3. the Cheesecake folks have scores for all the PyPi modules here

That last one is kind of interesting. You can see the scores and the Cheesecake log output.

A few days ago, I saw on Grig's blog that Titus created a mailing list covering Testing in Python. I joined the list and it's been very educational. So far the posts have been in the "Hi--my name is _____________ (name) and I use ______________ (framework) and ___________ (framework) and ____________ (explanation for which framework gets used for which specific purposes)." Then this spawns a conversation with thoughts and insights and lots of applause. There have also been questions regarding how to use some of the frameworks and which framework would suit some need the best.

While I'm following the mailing list, I'm taking notes so that I can finish up this round of adding testing to PyBlosxom so that I can do a release this week. We're using nose and I'm using a similar structure that Cheesecake uses since I have some familiarity with their project.

It's a neat list so far. I'm glad Titus put it together--it's definitely solving my immediate needs.

I just discovered the Python Sidebar and I'm not sure how I got along without it.

I got a bunch of really helpful responses from my previous entry and I'm pretty sure that my problems were two-fold: my code was missing some stuff (the code I posted and what I actually had were different--go figure) and my testing program had a bug (or two or three).

Anyhow, my knowledge of file locking on Linux (and possibly other Unixs) is pretty abundant now (or so I think). This helped to fix a problem I had at work as well (not the file locking part, but the thing I needed file locking for helped me at work).

Python has the fcntl module which has a flock and lockf. I'm not entirely sure what the difference is, but there are man pages on both functions as the Python versions call their C counter-parts.

There's a good article on file locking that might be located at /usr/src/linux/Documentation/mandatory.txt (though it wasn't on my system--a Google search helped me out) that's pretty interesting. Also, there's explanations of file locking between processes and the issues involved in Advanced Programming in the Unix Environment and Linux Application Development.

My specific issue was with Exim and PINE. Both have documentation as to how they lock mbox files before playing with them--things a Google search reveals without much effort.

Anyhow, I'm pretty sure my code works now. I'll do some more sophisticated testing this weekend to make sure everything is kosher.

Thanks to Chris, Jason, John and Lance for their help.

Linux Application Development ASIN: 0201308215 Buy at Amazon

Advanced Programming in the Unix Environment ASIN: 0201563177 Buy at Amazon

I wrote my own web-mail client (bluemail) because I wanted a web-mail client that would work along-side PINE and figured I'd just roll my own in Python. I work on it from time to time--mostly when someone requests new features. Right now I'm trying to implement deleting of items in a given folder.

The problem I'm having is a file locking issue. When the user clicks on the INBOX link, it opens up their inbox file (an mbox file in /var/spool/mail/) as the user. That works super. I parse out the mbox and display a bunch of email one-liners. The problem comes in that I can't seem to open the file as r+, lock it, parse it into email, change some of the email (whether by deleting the email from the mbox file or changing the attributes of the email [i.e. marking the email as having been answered]), then write the changed file back to disk without the possibility of having another process (our MTA, for example) slipping in and changing the file beneath me.

Right now, I open the file like this:

   import fcntl

   f = open("/some/file", "r+")
   fcntl.flock(f, fcntl.LOCK_EX)

That should work fine (as far as my research has told me). I did notice some places that say that LOCK_EX only works when you open the file in "a" or "w" mode, but I've got it opened as "r+" which means I'm going to update it later, so I'm puzzled.

Anyhow, using the code above doesn't seem to lock the file. I can run the above code thereby "locking" the file and then in another process open the file, read the file, and write to it. That indicates to me that it's not really locked.

Any ideas how I can resolve this? I need to be able to open a file for reading and updating and lock it so that other processes can't change the file between my read and update cycles. I'm running the code on a Debian GNU/Linux box with Python 2.3.

I keep bumping into circular import problems. While some say it's a symptom of bad design, I'm not wholly sure I agree. Some systems are just really complex. Not everything can be refactored into a pretty little UML diagram.

Anyhow, there's a link on daily python url to Victor Ng's blog entry on his experience with circular import problems with Python. The comments of that blog entry are wonderfully enlightening.