API question

View: New views
11 Messages — Rating Filter:   Alert me  

API question

by Daniel Benamy-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,
Should readdir fail if the result buffer fills up? The example file system
doesn't, the api documentation on the site doesn't say anything about it, and
the man page for readdir(2) lists it as an error condition.
Thanks,
Dan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

Re: API question

by Miklos Szeredi :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Wed, 16 Jul 2008, Daniel Benamy wrote:
> Should readdir fail if the result buffer fills up? The example file
> system doesn't, the api documentation on the site doesn't say
> anything about it, and the man page for readdir(2) lists it as an
> error condition.

Which example?  fusexmp and fusexmp_fh use different strategies, both
of which are pretty well documented in fuse.h, I think.

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

Re: API question

by Daniel Benamy-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Friday 18 July 2008 07:41:08 Miklos Szeredi wrote:
> On Wed, 16 Jul 2008, Daniel Benamy wrote:
> > Should readdir fail if the result buffer fills up? The example file
> > system doesn't, the api documentation on the site doesn't say
> > anything about it, and the man page for readdir(2) lists it as an
> > error condition.
>
> Which example?  fusexmp and fusexmp_fh use different strategies, both
> of which are pretty well documented in fuse.h, I think.

I'm looking at fusexmp. The documentation for this mode says:

     * 1) The readdir implementation ignores the offset parameter, and
     * passes zero to the filler function's offset.  The filler
     * function will not return '1' (unless an error happens), so the
     * whole directory is read in a single readdir operation.  This
     * works just like the old getdir() method.

(Filler is supposed to only return 1 if the buffer is full.)

In fusexmp, readdir always returns 0, so how can the fuse kernel module know
if the buffer filled up?

Thanks,
Dan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

Re: API question

by Miklos Szeredi :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Fri, 18 Jul 2008, Daniel Benamy wrote:
> In fusexmp, readdir always returns 0, so how can the fuse kernel module know
> if the buffer filled up?

It fills a big buffer in memory, and later copies from this big buffer
to the small buffer requested by the fuse kernel module.  If the
directory is so large it doesn't fit in the memory, then this method
won't work.

Miklos

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

Re: API question

by Daniel Benamy-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Saturday 19 July 2008 03:50:07 Miklos Szeredi wrote:
> On Fri, 18 Jul 2008, Daniel Benamy wrote:
> > In fusexmp, readdir always returns 0, so how can the fuse kernel module
> > know if the buffer filled up?
>
> It fills a big buffer in memory, and later copies from this big buffer
> to the small buffer requested by the fuse kernel module.  If the
> directory is so large it doesn't fit in the memory, then this method
> won't work.

So filler() won't ever return 1 when used in this mode?

Dan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

Re: API question

by Miklos Szeredi :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Tue, 22 Jul 2008, Daniel Benamy wrote:

> On Saturday 19 July 2008 03:50:07 Miklos Szeredi wrote:
> > On Fri, 18 Jul 2008, Daniel Benamy wrote:
> > > In fusexmp, readdir always returns 0, so how can the fuse kernel module
> > > know if the buffer filled up?
> >
> > It fills a big buffer in memory, and later copies from this big buffer
> > to the small buffer requested by the fuse kernel module.  If the
> > directory is so large it doesn't fit in the memory, then this method
> > won't work.
>
> So filler() won't ever return 1 when used in this mode?

It will only return 1 if it failed to allocate memory for the buffer.

Miklos

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

Re: API question

by Daniel Benamy-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Wednesday 23 July 2008 02:51:37 Miklos Szeredi wrote:

> On Tue, 22 Jul 2008, Daniel Benamy wrote:
> > On Saturday 19 July 2008 03:50:07 Miklos Szeredi wrote:
> > > On Fri, 18 Jul 2008, Daniel Benamy wrote:
> > > > In fusexmp, readdir always returns 0, so how can the fuse kernel
> > > > module know if the buffer filled up?
> > >
> > > It fills a big buffer in memory, and later copies from this big buffer
> > > to the small buffer requested by the fuse kernel module.  If the
> > > directory is so large it doesn't fit in the memory, then this method
> > > won't work.
> >
> > So filler() won't ever return 1 when used in this mode?
>
> It will only return 1 if it failed to allocate memory for the buffer.

If it can ever return 1 on error, doesn't readdir need to indicate that? Like
so:

--- fusexmp.c   2006-02-02 09:04:51.000000000 -0800
+++ fusexmp-readdir-error.c     2008-07-25 10:22:36.000000000 -0700
@@ -78,7 +78,7 @@
         st.st_ino = de->d_ino;
         st.st_mode = de->d_type << 12;
         if (filler(buf, de->d_name, &st, 0))
-            break;
+            return -EINVAL;
     }

     closedir(dp);

Am I missing something?

Dan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

Re: API question

by Miklos Szeredi :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Fri, 25 Jul 2008, Daniel Benamy wrote:

> On Wednesday 23 July 2008 02:51:37 Miklos Szeredi wrote:
> > On Tue, 22 Jul 2008, Daniel Benamy wrote:
> > > So filler() won't ever return 1 when used in this mode?
> >
> > It will only return 1 if it failed to allocate memory for the buffer.
>
> If it can ever return 1 on error, doesn't readdir need to indicate
> that? Like so:
>
> --- fusexmp.c   2006-02-02 09:04:51.000000000 -0800
> +++ fusexmp-readdir-error.c     2008-07-25 10:22:36.000000000 -0700
> @@ -78,7 +78,7 @@
>          st.st_ino = de->d_ino;
>          st.st_mode = de->d_type << 12;
>          if (filler(buf, de->d_name, &st, 0))
> -            break;
> +            return -EINVAL;
>      }
>
>      closedir(dp);
>
> Am I missing something?

The filler function will store the error value in the buffer, and the
caller will check that and return the stored error.  So the original
code is correct, and the above patch would actually make it worse:

  - the wrong error value is returned (EINVAL instead of ENOMEM)
  - closedir() is not called, causing memory and a file descriptor to leak

Miklos

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

Re: API question

by Daniel Benamy-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Friday 25 July 2008 14:26:22 Miklos Szeredi wrote:

> On Fri, 25 Jul 2008, Daniel Benamy wrote:
> > On Wednesday 23 July 2008 02:51:37 Miklos Szeredi wrote:
> > > On Tue, 22 Jul 2008, Daniel Benamy wrote:
> > > > So filler() won't ever return 1 when used in this mode?
> > >
> > > It will only return 1 if it failed to allocate memory for the buffer.
> >
> > If it can ever return 1 on error, doesn't readdir need to indicate
> > that? Like so:
> >
> > --- fusexmp.c   2006-02-02 09:04:51.000000000 -0800
> > +++ fusexmp-readdir-error.c     2008-07-25 10:22:36.000000000 -0700
> > @@ -78,7 +78,7 @@
> >          st.st_ino = de->d_ino;
> >          st.st_mode = de->d_type << 12;
> >          if (filler(buf, de->d_name, &st, 0))
> > -            break;
> > +            return -EINVAL;
> >      }
> >
> >      closedir(dp);
> >
> > Am I missing something?
>
> The filler function will store the error value in the buffer, and the
> caller will check that and return the stored error.  So the original
> code is correct, and the above patch would actually make it worse:
>
>   - the wrong error value is returned (EINVAL instead of ENOMEM)
>   - closedir() is not called, causing memory and a file descriptor to leak

Ignore the broken details of my patch for a second. If filler can't allocate
memory, there's no buffer for it to store an error value in. So it returns 1.
With the original code, readdir will still return 0 in that case. So how will
fuse know that there was an error?
Dan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

Re: API question

by Miklos Szeredi :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Mon, 28 Jul 2008, Daniel Benamy wrote:
> Ignore the broken details of my patch for a second. If filler can't
> allocate memory, there's no buffer for it to store an error value
> in. So it returns 1.  With the original code, readdir will still
> return 0 in that case. So how will fuse know that there was an
> error?

It stores the error in the buffer (second argument) passed to
->readdir().  See extend_contents() and readdir_fill() functions in
lib/fuse.c for the implementation.

Miklos

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel

Re: API question

by Daniel Benamy-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Tuesday 29 July 2008 03:15:05 Miklos Szeredi wrote:

> On Mon, 28 Jul 2008, Daniel Benamy wrote:
> > Ignore the broken details of my patch for a second. If filler can't
> > allocate memory, there's no buffer for it to store an error value
> > in. So it returns 1.  With the original code, readdir will still
> > return 0 in that case. So how will fuse know that there was an
> > error?
>
> It stores the error in the buffer (second argument) passed to
> ->readdir().  See extend_contents() and readdir_fill() functions in
> lib/fuse.c for the implementation.

Got it. Thanks for the patient explanation.

Dan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel