Will's blog

purpose: Will Kahn-Greene's blog of Miro, PyBlosxom, Python, GNU/Linux, random content, PyBlosxom, Miro, and other projects mixed in there ad hoc, half-baked, and with a twist of lemon

[ home | blog home | recent activity ]

Thu, 01 Jul 2004

File locking in Python--help!

I wrote my own web-mail client (bluemail) because I wanted a web-mail client that would work along-side PINE and figured I'd just roll my own in Python. I work on it from time to time--mostly when someone requests new features. Right now I'm trying to implement deleting of items in a given folder.

The problem I'm having is a file locking issue. When the user clicks on the INBOX link, it opens up their inbox file (an mbox file in /var/spool/mail/) as the user. That works super. I parse out the mbox and display a bunch of email one-liners. The problem comes in that I can't seem to open the file as r+, lock it, parse it into email, change some of the email (whether by deleting the email from the mbox file or changing the attributes of the email [i.e. marking the email as having been answered]), then write the changed file back to disk without the possibility of having another process (our MTA, for example) slipping in and changing the file beneath me.

Right now, I open the file like this:

   import fcntl

   f = open("/some/file", "r+")
   fcntl.flock(f, fcntl.LOCK_EX)

That should work fine (as far as my research has told me). I did notice some places that say that LOCK_EX only works when you open the file in "a" or "w" mode, but I've got it opened as "r+" which means I'm going to update it later, so I'm puzzled.

Anyhow, using the code above doesn't seem to lock the file. I can run the above code thereby "locking" the file and then in another process open the file, read the file, and write to it. That indicates to me that it's not really locked.

Any ideas how I can resolve this? I need to be able to open a file for reading and updating and lock it so that other processes can't change the file between my read and update cycles. I'm running the code on a Debian GNU/Linux box with Python 2.3.

Comments:

Posted by Michael on Thu Jul 1 11:25:20 2004
I've never tried, but it looks like the flock(3) man page on FreeBSD implies that locking is only cooperative, not exclusive if the other process doesn't try to obtain the lock. That could be the problem.


Posted by will on Thu Jul 1 11:30:16 2004
Just as a note, I've been working and researching this issue for a few months now.  I've read the man pages on flock and fcntl (the ones that come on Debian GNU/Linux) and I've read the Python documentation on the fcntl and posixfile (deprecated) modules.  I've also read the documentation on PINE (they use two locking mechanisms--one of which is flock) and Exim (the MTA I use which I think also uses flock).  I don't think that's the problem--though there could be something I'm missing in my understanding of my setup.


Posted by John P. Speno on Thu Jul 1 14:43:22 2004
Check your mail delivery agent (procmail perhaps?). It probably uses dotfile locking. I.e. by creating a /var/spool/mail/username.lock file. Primative, but very portable. :-)

Good luck.


Posted by Chris Green on Fri Jul 2 09:34:18 2004
The information you seek is in /usr/src/linux/Documentation/mandatory.txt

You need to pass the mount option mand on the filesystem you want mandatory locks on.

That said, you're probably much better off using advisory locks between processes.  I'd also recommend "Advanced Programming in the Unix Environment" if you want to have a lot more C based example code and very thorough explanations


Posted by Jason McBrayer on Fri Jul 2 10:46:10 2004
Like some other posters have said, flock/fcntl are only advisory -- the other process has to cooperate by checking for the existence of a lock and holding off on writing if it's locked.  Mandatory locking is rare as hen's teeth on Unix-like systems.  The thing to do is to check what kind of locking your mail delivery agent is doing and do the same thing.  I'm not sure exactly how to check this, but the build process for Emacs figures it out (for its 'movemail' helper), so if all else fails, you can build emacs from source and see what it tells you ;)


Post a new comment:

Three things:

  1. New comments get placed in a "draft" status and will NOT show up on the site until I explicitly approve it. Sometimes that happens within 24 hours.
  2. I reserve the right to reject/remove inappropriate comments.
  3. Sometimes I'll reply to a comment directly in email--so make sure your email address is correct.

If you can't for some reason post a comment, send me an email: willg at bluesock dot org.

Your name:


Your e-mail address (this doesn't get displayed to anyone--sometimes I'll reply directly to you):


URL of your website (optional):


Comment:


Yes, I am a human!


pyblosxom::1.5-dev git-master

Copyright 1996 to 2012, Will Guaraldi Kahn-Greene, under the Creative Commons BY-SA 3.0 license

Creative Commons License
Will's Blog by William Kahn-Greene is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.