short read of networked file

[Allen Pulsifer-3]

This particular issue has probably come up before but I just want to make
sure I understand the answer.

Two computers are simultaneously access a networked file.  One is reading
the file and the other is writing it.

Computer A does a getattr to retrieve the file size.  The FS returns 1000
bytes.  A moment later, Computer B truncates the file to 500 bytes.
Computer A then does a read to retrieve bytes 0-999.  The file however now
only contains 500 bytes.  What should the FUSE-based file system return to
indicate that the returned data only contains 500 bytes, not 1000?  Or
should it return a negative error code?

Thank you,

Allen


-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

[Miklos Szeredi]

On Tue, 8 Jul 2008, Allen Pulsifer wrote:

> This particular issue has probably come up before but I just want to make
> sure I understand the answer.
>
> Two computers are simultaneously access a networked file.  One is reading
> the file and the other is writing it.
>
> Computer A does a getattr to retrieve the file size.  The FS returns 1000
> bytes.  A moment later, Computer B truncates the file to 500 bytes.
> Computer A then does a read to retrieve bytes 0-999.  The file however now
> only contains 500 bytes.  What should the FUSE-based file system return to
> indicate that the returned data only contains 500 bytes, not 1000?

It should return 500.  What will happen after this depends on the
version of the fuse kernel module: in 2.6.25 and earlier this caused
the read to return 1000 and the last 500 bytes were zeroed out.
2.6.26 will return 500 in this case.

>  Or
> should it return a negative error code?

I don't think that would be useful.  I've recently learned that a
similar read vs. truncate race affects all linux filesystems, although
the race window there is so small, nobody seems to bother about it.

Miklos

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

[Allen Pulsifer-3]

> > What should the FUSE-based file system return to indicate
> > the returned data only contains 500 bytes, not 1000?
>
> It should return 500.  What will happen after this depends on
> the version of the fuse kernel module: in 2.6.25 and earlier
> this caused the read to return 1000 and the last 500 bytes
> were zeroed out. 2.6.26 will return 500 in this case.

Thank you.  Is this referring to the version of the Linux kernel, or the
version of FUSE?  If I compiled a FUSE kernel module from the latest FUSE
v2.7.3 release, would it also work like "2.6.26"?


-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

[Nikolaus Rath]

"Allen Pulsifer" <pulsifer3@...> writes:

> This particular issue has probably come up before but I just want to make
> sure I understand the answer.
>
> Two computers are simultaneously access a networked file.  One is reading
> the file and the other is writing it.
>
> Computer A does a getattr to retrieve the file size.  The FS returns 1000
> bytes.  A moment later, Computer B truncates the file to 500 bytes.
> Computer A then does a read to retrieve bytes 0-999.  The file however now
> only contains 500 bytes.  What should the FUSE-based file system return to
> indicate that the returned data only contains 500 bytes, not 1000?  Or
> should it return a negative error code?

This probably doesn't answer your question entirely, but if you use -o
direct_io, you can always safely return the actual number of bytes
read or written. AFAIU, of course.

Best,

   -Nikolaus

--
 »It is not worth an intelligent man's time to be in the majority.
  By definition, there are already enough people to do that.«
                                                         -J.H. Hardy

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

[Allen Pulsifer-3]

> > What should the FUSE-based file system return to indicate the
> > returned data only contains 500 bytes, not 1000?
>
> It should return 500.  What will happen after this depends on
> the version of the fuse kernel module: in 2.6.25 and earlier
> this caused the read to return 1000 and the last 500 bytes
> were zeroed out. 2.6.26 will return 500 in this case.

Does "2.6.26" refer to the version of the Linux kernel, or the version of
FUSE?

If I compiled a FUSE kernel module from the latest FUSE v2.7.3 release,
would it also work like "2.6.26"?

Thank you.


-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

[Miklos Szeredi]

On Tue, 8 Jul 2008, Allen Pulsifer wrote:

> > > What should the FUSE-based file system return to indicate
> > > the returned data only contains 500 bytes, not 1000?
> >
> > It should return 500.  What will happen after this depends on
> > the version of the fuse kernel module: in 2.6.25 and earlier
> > this caused the read to return 1000 and the last 500 bytes
> > were zeroed out. 2.6.26 will return 500 in this case.
>
> Thank you.  Is this referring to the version of the Linux kernel, or the
> version of FUSE?  If I compiled a FUSE kernel module from the latest FUSE
> v2.7.3 release, would it also work like "2.6.26"?

The kernel module from fuse-2.7.3 is more like the one in
linux-2.6.24.  I don't plan to keep the kernel module in fuse-2.7.x in
sync with the latest kernels, and in fuse-2.8.x the kernel module will
be dropped completely.

It's too much work to maintain the two different instances,
unfortunately.

Miklos

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

[Allen Pulsifer-3]

> The kernel module from fuse-2.7.3 is more like the one in
> linux-2.6.24.  I don't plan to keep the kernel module in
> fuse-2.7.x in sync with the latest kernels, and in fuse-2.8.x
> the kernel module will be dropped completely.
>
> It's too much work to maintain the two different instances,
> unfortunately.

Ouch--although I can understand.  Its too much work for me to support
different versions of FUSE, so my standard practice is to tell users to
install the FUSE v2.7.3 kernel module from source :-)

The FUSE source release has a file ChangeLog that lists changes by date and
FUSE version.  Is there an equivalent file that lists changes by Linux
kernel version?

Thank you.


-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

[Miklos Szeredi]

On Wed, 9 Jul 2008, Allen Pulsifer wrote:

> > The kernel module from fuse-2.7.3 is more like the one in
> > linux-2.6.24.  I don't plan to keep the kernel module in
> > fuse-2.7.x in sync with the latest kernels, and in fuse-2.8.x
> > the kernel module will be dropped completely.
> >
> > It's too much work to maintain the two different instances,
> > unfortunately.
>
> Ouch--although I can understand.  Its too much work for me to support
> different versions of FUSE, so my standard practice is to tell users to
> install the FUSE v2.7.3 kernel module from source :-)
>
> The FUSE source release has a file ChangeLog that lists changes by date and
> FUSE version.  Is there an equivalent file that lists changes by Linux
> kernel version?

There is a ChangeLog file for each kernel release, but it contains all
the changes not just the fuse ones.

You can see just the fuse changes here, but you can't see the kernel
versions :)

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=history;f=fs/fuse;hb=HEAD

And if you set up a linux git tree, you can generate a ChangeLog for
just fuse.  E.g. this will show all fuse changes between 2.6.25 and
2.6.25:

git log v2.6.24..v2.6.25 fs/fuse

Miklos

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

[Allen Pulsifer-3]

Hello Miklos,

Back to this question again.

It is my understanding that if the user has the FUSE kernel module built
into Linux kernel v2.6.26 or higher, then a "short read" will return to the
user the number of bytes actually delivered by the file system.  On the
other hand, if the user has an earlier kernel module, then a "short read"
will fill the end of the buffer with zeros and return those zeros to the
user.

It seems to me that the file system would be better off returning EIO in the
latter case, rather than having the user get data that is not in the file.
Do you have any thoughts on this?  Is there some way a file system can
discover whether the FUSE kernel module supports "short reads" and adjust
its behavior accordingly?

Thank you,

Allen


-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

[Miklos Szeredi]

On Mon, 14 Jul 2008, Allen Pulsifer wrote:

> Hello Miklos,
>
> Back to this question again.
>
> It is my understanding that if the user has the FUSE kernel module built
> into Linux kernel v2.6.26 or higher, then a "short read" will return to the
> user the number of bytes actually delivered by the file system.  On the
> other hand, if the user has an earlier kernel module, then a "short read"
> will fill the end of the buffer with zeros and return those zeros to the
> user.

Right.

> It seems to me that the file system would be better off returning EIO in the
> latter case, rather than having the user get data that is not in the file.
> Do you have any thoughts on this?

I don't agreee.  But the point is moot, the behavior of old kernels
cannot be changed retroactively.

>  Is there some way a file system can
> discover whether the FUSE kernel module supports "short reads" and adjust
> its behavior accordingly?

No.  Is this a practical problem for your filesystem?  I.e. does it
cause something nasty to happen in a real life situation?  If so,
what?

If this is just a theoretical corner case, that you are worrying
about, then my advice is just to forget about it.  If it's very
important, then your filesystem will have to require the use of 2.6.26
or later kernel.

Miklos

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel