Credit: Jonathan Feinberg, John Nielsen
You need to lock files in a cross-platform way between NT and Posix, but the Python standard library offers only platform-specific ways to lock files.
When the Python standard library itself doesn’t offer a cross-platform solution, it’s often possible to implement one ourselves:
import os # needs win32all to work on Windows if os.name == 'nt': import win32con, win32file, pywintypes LOCK_EX = win32con.LOCKFILE_EXCLUSIVE_LOCK LOCK_SH = 0 # the default LOCK_NB = win32con.LOCKFILE_FAIL_IMMEDIATELY _ _overlapped = pywintypes.OVERLAPPED( ) def lock(file, flags): hfile = win32file._get_osfhandle(file.fileno( )) win32file.LockFileEx(hfile, flags, 0, 0xffff0000, _ _overlapped) def unlock(file): hfile = win32file._get_osfhandle(file.fileno( )) win32file.UnlockFileEx(hfile, 0, 0xffff0000, _ _overlapped) elif os.name == 'posix': from fcntl import LOCK_EX, LOCK_SH, LOCK_NB def lock(file, flags): fcntl.flock(file.fileno( ), flags) def unlock(file): fcntl.flock(file.fileno( ), fcntl.LOCK_UN) else: raise RuntimeError("PortaLocker only defined for nt and posix platforms")
If you have multiple programs or threads that may want to access a shared file, it’s wise to ensure that accesses are synchronized, so that two processes don’t try to modify the file contents at the same time. Failure to do so could corrupt the entire file in some cases.
This recipe supplies two functions, lock
and
unlock
, that request and release locks on a file,
respectively. Using the
portalocker.py
module is a simple matter of calling the
lock
function and passing in the file and an
argument specifying the kind of lock that is desired:
LOCK_SH
A shared lock (the default value). This denies all processes write access to the file, including the process that first locks the file. All processes can read the locked file.
LOCK_EX
An exclusive lock. This denies all other processes both read and write access to the file.
LOCK_NB
A nonblocking lock. If this value is specified, the function returns
immediately if it is unable to acquire the requested lock. Otherwise,
it waits. LOCK_NB
can be ORed with either
LOCK_SH
or LOCK_EX
.
For example:
import portalocker file = open("somefile", "r+") portalocker.lock(file, portalocker.LOCK_EX)
The implementation of the lock
and
unlock
functions is entirely different on
Unix-like systems (where they can rely on functionality made
available by the standard fcntl
module) and on
Windows systems (where they must use the win32file
module, part of the very popular win32all
package
of Windows-specific extensions to Python, authored by Mark Hammond).
But the important thing is that, despite the differences in
implementation, the functions (and the flags you can pass to the
lock
function) behave in the same way across
platforms. Such cross-platform packaging of differently implemented
but equivalent functionality is what lets you write cross-platform
applications, which is one of Python’s strengths.
When you write a cross-platform program, it’s nice
if the functionality that your program uses is, in turn, encapsulated
in a cross-platform way. For file locking in particular, this is
helpful to Perl users, who are used to an essentially transparent
lock
system call across platforms. More generally,
if os.name==
just does not belong in
application-level code. It should ideally always be in the standard
library or an application-independent module, as it is here.
Documentation on the fcntl
module in the
Library Reference; documentation on the
win32file
module at http://ASPN.ActiveState.com/ASPN/Python/Reference/Products/ActivePython/PythonWin32Extensions/win32file.html;
Jonathan Feinberg’s web site (http://MrFeinberg.com).