2

So I want to write some files that might be locked/blocked for write/delete by other processes and like to test that upfront.

As I understand: os.access(path, os.W_OK) only looks for the permissions and will return true although a file cannot currently be written. So I have this little function:

def write_test(path):
    try:
        fobj = open(path, 'a')
        fobj.close()
        return True
    except IOError:
    return False

It actually works pretty well, when I try it with a file that I manually open with a Program. But as a wannabe-good-developer I want to put it in a test to automatically see if it works as expected.

Thing is: If I just open(path, 'a') the file I can still open() it again no problem! Even from another Python instance. Although Explorer will actually tell me that the file is currently open in Python!

I looked up other posts here & there about locking. Most are suggesting to install a package. You migth understand that I don't wanna do that to test a handful lines of code. So I dug up the packages to see the actual spot where the locking is eventually done...

fcntl? I don't have that. win32con? Don't have it either... Now in filelock there is this:

self.fd = os.open(self.lockfile, os.O_CREAT|os.O_EXCL|os.O_RDWR)

When I do that on a file it moans that the file exists!! Ehhm ... yea! That's the idea! But even when I do it on a non-existing path. I can still open(path, 'a') it! Even from another python instance...

I'm beginning to think that I fail to understand something very basic here. Am I looking for the wrong thing? Can someone point me into the right direction? Thanks!

Improve this question
3
  • What you're experiencing is probably a sharing violation (see Creating and Opening Files), which is implemented in the Windows I/O manager via IoCheckShareAccess. Python opens files with read and write (but not delete) sharing. Other programs may open their files with only read sharing or no sharing at all. Commented Jul 28, 2016 at 18:37
  • 1
    Windows also supports file locking via LockFileEx and UnlockFileEx, which are available in Python via msvcrt.locking. Commented Jul 28, 2016 at 18:38
  • Yay I saw msvcrt.locking‌​ already! But didn't understand the int Error... I was mixing up open() and os.open() before! Thank You eryksun!!! :D Commented Jul 28, 2016 at 19:16

2 Answers 2

Reset to default
1

You are trying to implement the file locking problem using just the system call open(). The Unix-like systems uses by default advisory file locking. This means that cooperating processes may use locks to coordinate access to a file among themselves, but uncooperative processes are also free to ignore locks and access the file in any way they choose. In other words, file locks lock out other file lockers only, not I/O. See Wikipedia.

As stated in system call open() reference the solution for performing atomic file locking using a lockfile is to create a unique file on the same file system (e.g., incorporating hostname and pid), use link(2) to make a link to the lockfile. If link() returns 0, the lock is successful. Otherwise, use stat(2) on the unique file to check if its link count has increased to 2, in which case the lock is also successful.

That is why in filelock they also use the function fcntl.flock() and puts all that stuff in a module as it should be.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Thank you! Seems it's the same on windows with open(). Ehmm ok. I tried the filelock package now but of course it os.removes my file when successful... How would I do the link thing on windows? Am I on the right way here?!?!
0

Alright! Thanks to those guys I actually have something now! So this is my function:

def lock_test(path):
    """
    Checks if a file can, aside from it's permissions, be changed right now (True)
    or is already locked by another process (False).

    :param str path: file to be checked
    :rtype: bool
    """
    import msvcrt
    try:
        fd = os.open(path, os.O_APPEND | os.O_EXCL | os.O_RDWR)
    except OSError:
        return False

    try:
        msvcrt.locking(fd, msvcrt.LK_NBLCK, 1)
        msvcrt.locking(fd, msvcrt.LK_UNLCK, 1)
        os.close(fd)
        return True
    except (OSError, IOError):
        os.close(fd)
        return False

And the unittest could look something like this:

class Test(unittest.TestCase):

    def test_lock_test(self):
        testfile = 'some_test_name4142351345.xyz'
        testcontent = 'some random blaaa'
        with open(testfile, 'w') as fob:
            fob.write(testcontent)

        # test successful locking and unlocking
        self.assertTrue(lock_test(testfile))
        os.remove(testfile)
        self.assertFalse(os.path.exists(testfile))

        # make file again, lock and test False locking
        with open(testfile, 'w') as fob:
            fob.write(testcontent)
        fd = os.open(testfile, os.O_APPEND | os.O_RDWR)

        msvcrt.locking(fd, msvcrt.LK_NBLCK, 1)
        self.assertFalse(lock_test(testfile))

        msvcrt.locking(fd, msvcrt.LK_UNLCK, 1)
        self.assertTrue(lock_test(testfile))
        os.close(fd)

        with open(testfile) as fob:
            content = fob.read()
            self.assertTrue(content == testcontent)
        os.remove(testfile)

Works. Downsides are:

  • It's kind of testing itself with itself
  • so the initial OSError catch is not even tested, only locking again with msvcrt

But I dunno how to make it better now.

Improve this answer

6 Comments

In general the behavior of O_EXCL isn't defined if used without O_CREAT. On Windows it's ignored unless used with O_CREAT, in which case it's a CREATE_NEW open disposition.
Windows file locking applies to a region of a file. You're only checking that the first byte isn't locked. You should attempt to lock the entire file, based on the current file size.
Ah ok thanks! I changed the example accordingly. "lock the entire file, based on the current file size"? Ah alright, I got what you mean. passing the size to msvcrt.locking(fd, msvcrt.LK_NBLCK, nbytes)
msvcrt.locking locks a number of bytes starting at the current file pointer and allows locking beyond the current end of the file. The C runtime is limited to synchronous file access, so it can rely on using the file pointer for the lock offset. OTOH, the underlying Windows API LockFileEx requires explicitly setting the offset in the call's lpOverlapped struct because file access in Windows is asynchronous in general, which means file operations return immediately and can overlap each other, in which case the file pointer isn't updated.
In other words: Am I unable to simulate the behaviour of e.g. Photoshop locking it's Plugins?
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.