Data corruption on bulk read

[Mike Lewis]

I'm wondering if anyone has encountered problem with bulk reads and
data corruption. I'm currently using OpenUSB to talk to two devices.
However, the problem doesn't seem to be in the library, as I see the
correct data immediately after the REAPURBNDELAY ioctl returns. The
problem is made worse by having two devices open. What I see is that
either day is missing or I get 0's back. The weird thing is that the
footer data I expect to see, always comes across, even when all the
other data is 0.

I know this is a very general description, but I'm just poking around
to see if anyone has seen anything like this.

Thanks,
Mike

--
Mike Lewis
milewis1@...

"Be who you are and say what you feel, because those who mind don't
matter and those who matter don't mind."
     --Dr. Seuss

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Tim Roberts]

Michael Lewis wrote:

> I'm wondering if anyone has encountered problem with bulk reads and
> data corruption. I'm currently using OpenUSB to talk to two devices.
> However, the problem doesn't seem to be in the library, as I see the
> correct data immediately after the REAPURBNDELAY ioctl returns. The
> problem is made worse by having two devices open. What I see is that
> either day is missing or I get 0's back. The weird thing is that the
> footer data I expect to see, always comes across, even when all the
> other data is 0.
>
> I know this is a very general description, but I'm just poking around
> to see if anyone has seen anything like this.
>  

If the data is present and later goes missing, doesn't that point rather
clearly at a threading confusion?  One buffer, two threads, someone is
clearing part of the buffer before the real consumer gets to consume it?

--
Tim Roberts, timr@...
Providenza & Boekelheide, Inc.


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Mike Lewis]

On Mon, Oct 6, 2008 at 1:16 PM, Tim Roberts <timr@...> wrote:

> Michael Lewis wrote:
>> I'm wondering if anyone has encountered problem with bulk reads and
>> data corruption. I'm currently using OpenUSB to talk to two devices.
>> However, the problem doesn't seem to be in the library, as I see the
>> correct data immediately after the REAPURBNDELAY ioctl returns. The
>> problem is made worse by having two devices open. What I see is that
>> either day is missing or I get 0's back. The weird thing is that the
>> footer data I expect to see, always comes across, even when all the
>> other data is 0.
>>
>> I know this is a very general description, but I'm just poking around
>> to see if anyone has seen anything like this.
>>
>
> If the data is present and later goes missing, doesn't that point rather
> clearly at a threading confusion?  One buffer, two threads, someone is
> clearing part of the buffer before the real consumer gets to consume it?
>

That was my original thought, however, I turned on usbfs snooping and
discovered that the data usbfs is returning is already corrupt, before
any of my threads get to it. I've also confirmed that what gets sent
across the wire is correct, so it seems to me the problem is
definitely on the kernel driver side. I'm just hoping someone knows
enough that they can help me find out where?

It's interesting to note that I've seen a similiar problem with lost
interrupt reads as well. That is to say that I've seen my device send
interrupt data down the write, but usbfs never receives it (again
according to usbfs snooping).

Thanks,
Mike

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Alan Stern]

On Mon, 6 Oct 2008, Michael Lewis wrote:

> That was my original thought, however, I turned on usbfs snooping and
> discovered that the data usbfs is returning is already corrupt, before
> any of my threads get to it. I've also confirmed that what gets sent
> across the wire is correct, so it seems to me the problem is
> definitely on the kernel driver side. I'm just hoping someone knows
> enough that they can help me find out where?
>
> It's interesting to note that I've seen a similiar problem with lost
> interrupt reads as well. That is to say that I've seen my device send
> interrupt data down the write, but usbfs never receives it (again
> according to usbfs snooping).

If you have confirmed that the data transferred over the bus is
different from the data returned by usbfs, and you're certain the
problem is on the kernel side, then you should post in a
kernel-oriented mailing list rather than a user-library-oriented list.

Can you provide more detailed information?  For example, output from
usbfs_snoop, plus output from usbmon, plus the result of your bus
monitoring?

Alan Stern


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Mike Lewis]

On Mon, Oct 6, 2008 at 3:11 PM, Alan Stern <stern@...> wrote:

> On Mon, 6 Oct 2008, Michael Lewis wrote:
>
>> That was my original thought, however, I turned on usbfs snooping and
>> discovered that the data usbfs is returning is already corrupt, before
>> any of my threads get to it. I've also confirmed that what gets sent
>> across the wire is correct, so it seems to me the problem is
>> definitely on the kernel driver side. I'm just hoping someone knows
>> enough that they can help me find out where?
>>
>> It's interesting to note that I've seen a similiar problem with lost
>> interrupt reads as well. That is to say that I've seen my device send
>> interrupt data down the write, but usbfs never receives it (again
>> according to usbfs snooping).
>
> If you have confirmed that the data transferred over the bus is
> different from the data returned by usbfs, and you're certain the
> problem is on the kernel side, then you should post in a
> kernel-oriented mailing list rather than a user-library-oriented list.

I am going to submit this to the kernel mailing list. I just thought
that since the people who read this list are generally very familiar
with USB issues that it might help to try here too.

> Can you provide more detailed information?  For example, output from
> usbfs_snoop, plus output from usbmon, plus the result of your bus
> monitoring?

What format would you prefer the data in. I don't have data from
usbmon, but I do have it from usbfs_snoop and the catc analyzer in csv
format.

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Alan Stern]

On Wed, 8 Oct 2008, Michael Lewis wrote:

> > If you have confirmed that the data transferred over the bus is
> > different from the data returned by usbfs, and you're certain the
> > problem is on the kernel side, then you should post in a
> > kernel-oriented mailing list rather than a user-library-oriented list.
>
> I am going to submit this to the kernel mailing list. I just thought
> that since the people who read this list are generally very familiar
> with USB issues that it might help to try here too.
>
> > Can you provide more detailed information?  For example, output from
> > usbfs_snoop, plus output from usbmon, plus the result of your bus
> > monitoring?
>
> What format would you prefer the data in. I don't have data from
> usbmon, but I do have it from usbfs_snoop and the catc analyzer in csv
> format.

CATC data is okay, so long as you provide a link for downloading the
viewer application.

usbmon output would be preferable to the usbfs_snoop log.  You can
include both, if you want.

Alan Stern


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Mike Lewis]

On Wed, Oct 8, 2008 at 12:34 PM, Alan Stern <stern@...> wrote:

> On Wed, 8 Oct 2008, Michael Lewis wrote:
>
>> > If you have confirmed that the data transferred over the bus is
>> > different from the data returned by usbfs, and you're certain the
>> > problem is on the kernel side, then you should post in a
>> > kernel-oriented mailing list rather than a user-library-oriented list.
>>
>> I am going to submit this to the kernel mailing list. I just thought
>> that since the people who read this list are generally very familiar
>> with USB issues that it might help to try here too.
>>
>> > Can you provide more detailed information?  For example, output from
>> > usbfs_snoop, plus output from usbmon, plus the result of your bus
>> > monitoring?
>>
>> What format would you prefer the data in. I don't have data from
>> usbmon, but I do have it from usbfs_snoop and the catc analyzer in csv
>> format.
>
> CATC data is okay, so long as you provide a link for downloading the
> viewer application.
>
> usbmon output would be preferable to the usbfs_snoop log.  You can
> include both, if you want.
>
> Alan Stern
>
>

I've actually converted all of it to CSV files and attached them, with
appropriate names. I had to do the conversion because they are
actually image data and I needed to be able to view them that way (I
also have tiff version too if you wanted).

The file: catc_frontwhitepatch.csv contains the CATC data
The file: usbfssnoop_frontwhitepatch.csv contains the data I pulled
from the syslog from usbfs
The file: application_frontwhitepatch.csv contains the data the
application received

The thing that led me to this problem in the first place is the fact
that my application and usbfs data are the same. My first assumption
was that my app was screwing it up, but that appears not to be the
case.

--Mike




-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Mike Lewis]

On Wed, Oct 8, 2008 at 12:48 PM, Michael Lewis <milewis1@...> wrote:

> On Wed, Oct 8, 2008 at 12:34 PM, Alan Stern <stern@...> wrote:
>> On Wed, 8 Oct 2008, Michael Lewis wrote:
>>
>>> > If you have confirmed that the data transferred over the bus is
>>> > different from the data returned by usbfs, and you're certain the
>>> > problem is on the kernel side, then you should post in a
>>> > kernel-oriented mailing list rather than a user-library-oriented list.
>>>
>>> I am going to submit this to the kernel mailing list. I just thought
>>> that since the people who read this list are generally very familiar
>>> with USB issues that it might help to try here too.
>>>
>>> > Can you provide more detailed information?  For example, output from
>>> > usbfs_snoop, plus output from usbmon, plus the result of your bus
>>> > monitoring?
>>>
>>> What format would you prefer the data in. I don't have data from
>>> usbmon, but I do have it from usbfs_snoop and the catc analyzer in csv
>>> format.
>>
>> CATC data is okay, so long as you provide a link for downloading the
>> viewer application.
>>
>> usbmon output would be preferable to the usbfs_snoop log.  You can
>> include both, if you want.
>>
>> Alan Stern
>>
>>
>
> I've actually converted all of it to CSV files and attached them, with
> appropriate names. I had to do the conversion because they are
> actually image data and I needed to be able to view them that way (I
> also have tiff version too if you wanted).
>
> The file: catc_frontwhitepatch.csv contains the CATC data
> The file: usbfssnoop_frontwhitepatch.csv contains the data I pulled
> from the syslog from usbfs
> The file: application_frontwhitepatch.csv contains the data the
> application received
>
> The thing that led me to this problem in the first place is the fact
> that my application and usbfs data are the same. My first assumption
> was that my app was screwing it up, but that appears not to be the
> case.
>
> --Mike
>

As a follow up: Looking at the images it's easier to see (they should
be 100x20 pixel at 8bpp). It looks as if the corrupt data is just
missing a couple hundred bytes, as you can see the image footer in the
corrupt data, and it is correct.

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Alan Stern]

On Wed, 8 Oct 2008, Michael Lewis wrote:

> > CATC data is okay, so long as you provide a link for downloading the
> > viewer application.
> >
> > usbmon output would be preferable to the usbfs_snoop log.  You can
> > include both, if you want.
> >
> > Alan Stern
> >
> >
>
> I've actually converted all of it to CSV files and attached them, with
> appropriate names. I had to do the conversion because they are
> actually image data and I needed to be able to view them that way (I
> also have tiff version too if you wanted).

I can't make heads or tails out of those files.  If you want people to
look at your data, you have to post it in a form they can understand.

For example, usbfs_snoop produces plain-text entries in the system log.  
Why don't you just post those instead of going to the trouble to
convert them to some incomprehensible CSV format?

And why didn't you include any usbmon logs?

> The file: catc_frontwhitepatch.csv contains the CATC data

How am I supposed to view it?

> The file: usbfssnoop_frontwhitepatch.csv contains the data I pulled
> from the syslog from usbfs
> The file: application_frontwhitepatch.csv contains the data the
> application received
>
> The thing that led me to this problem in the first place is the fact
> that my application and usbfs data are the same. My first assumption
> was that my app was screwing it up, but that appears not to be the
> case.

Since the application gets the same data as usbfs_snoop shows, there's
no reason to look at the third file, right?

Alan Stern


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Mike Lewis]

On Wed, Oct 8, 2008 at 2:20 PM, Alan Stern <stern@...> wrote:

> On Wed, 8 Oct 2008, Michael Lewis wrote:
>
>> > CATC data is okay, so long as you provide a link for downloading the
>> > viewer application.
>> >
>> > usbmon output would be preferable to the usbfs_snoop log.  You can
>> > include both, if you want.
>> >
>> > Alan Stern
>> >
>> >
>>
>> I've actually converted all of it to CSV files and attached them, with
>> appropriate names. I had to do the conversion because they are
>> actually image data and I needed to be able to view them that way (I
>> also have tiff version too if you wanted).
>
> I can't make heads or tails out of those files.  If you want people to
> look at your data, you have to post it in a form they can understand.
>
> For example, usbfs_snoop produces plain-text entries in the system log.
> Why don't you just post those instead of going to the trouble to
> convert them to some incomprehensible CSV format?
>
> And why didn't you include any usbmon logs?

I'm sorry you don't understand. The CSV files are just a list of the
bytes of data that came back. 0-255 for each byte I sent/received.
I'll work on pulling out the plain text usb_fs snoop files for you and
get back to you. I didn't realize that looking at an array of bytes in
a CSV file was that difficult. My apologies.

If you're looking for a better way to look at the data. It's a raw
image 100 x 20 pixels, grayscale format.

I don't have any usbmon logs. That's why they were not included, I
thought I mentioned that. I'll work on getting those too.

>
>> The file: catc_frontwhitepatch.csv contains the CATC data
>
> How am I supposed to view it?

It's an array of the bytes I received in each packet strung together.

>
>> The file: usbfssnoop_frontwhitepatch.csv contains the data I pulled
>> from the syslog from usbfs
>> The file: application_frontwhitepatch.csv contains the data the
>> application received
>>
>> The thing that led me to this problem in the first place is the fact
>> that my application and usbfs data are the same. My first assumption
>> was that my app was screwing it up, but that appears not to be the
>> case.
>
> Since the application gets the same data as usbfs_snoop shows, there's
> no reason to look at the third file, right?
>

You're right, the third file is pretty much useless.

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Alan Stern]

On Wed, 8 Oct 2008, Michael Lewis wrote:

> >> I've actually converted all of it to CSV files and attached them, with
> >> appropriate names. I had to do the conversion because they are
> >> actually image data and I needed to be able to view them that way (I
> >> also have tiff version too if you wanted).
> >
> > I can't make heads or tails out of those files.  If you want people to
> > look at your data, you have to post it in a form they can understand.
> >
> > For example, usbfs_snoop produces plain-text entries in the system log.
> > Why don't you just post those instead of going to the trouble to
> > convert them to some incomprehensible CSV format?
> >
> > And why didn't you include any usbmon logs?
>
> I'm sorry you don't understand. The CSV files are just a list of the
> bytes of data that came back. 0-255 for each byte I sent/received.

Earlier you wrote:

 > The file: catc_frontwhitepatch.csv contains the CATC data
 > The file: usbfssnoop_frontwhitepatch.csv contains the data I pulled
 > from the syslog from usbfs

You can understand my confusion.  I thought the files contained a
genuine CATC trace and a bunch of usbfs_snoop log lines.

> I'll work on pulling out the plain text usb_fs snoop files for you and
> get back to you. I didn't realize that looking at an array of bytes in
> a CSV file was that difficult. My apologies.

_Looking_ at an array of bytes isn't difficult.  _Understanding_ them
is, particularly when you don't know what they are supposed to mean.

> If you're looking for a better way to look at the data. It's a raw
> image 100 x 20 pixels, grayscale format.

Since the files contain 101 lines (including a header), I deduce that
each line represents 20 pixels of data.  Grayscale format means that
each line should contain 20 values, ranging between perhaps 0 and
255.  That's not what they actually contain.  Can you explain?

I guess your point is that all three files should contain exactly
the same values, since they are supposed to be the same data
intercepted at various points along the way from the device to your
program.  Right?

As you mentioned, the usbfs data and the application data are indeed
identical.  The CATC data are different, though.  This lead me to
wonder whether you are correctly associating the packets in the CATC
log with the same packets in the usbfs_snoop log.

> I don't have any usbmon logs. That's why they were not included, I
> thought I mentioned that. I'll work on getting those too.

They will probably turn out to be the same as the usbfs_snoop data.  If
they are then you need not bother posting them.

> >> The file: catc_frontwhitepatch.csv contains the CATC data
> >
> > How am I supposed to view it?
>
> It's an array of the bytes I received in each packet strung together.

Hmm...  Let's look at the second line of the application file (the
first line contains column headers):

        0,423,13

Is 423 really a byte value?  Or the fourth line:

        2,720,-20

Are these byte values?

Alan Stern


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Mike Lewis]

On Wed, Oct 8, 2008 at 3:38 PM, Alan Stern <stern@...> wrote:

> On Wed, 8 Oct 2008, Michael Lewis wrote:
>
>> >> I've actually converted all of it to CSV files and attached them, with
>> >> appropriate names. I had to do the conversion because they are
>> >> actually image data and I needed to be able to view them that way (I
>> >> also have tiff version too if you wanted).
>> >
>> > I can't make heads or tails out of those files.  If you want people to
>> > look at your data, you have to post it in a form they can understand.
>> >
>> > For example, usbfs_snoop produces plain-text entries in the system log.
>> > Why don't you just post those instead of going to the trouble to
>> > convert them to some incomprehensible CSV format?
>> >
>> > And why didn't you include any usbmon logs?
>>
>> I'm sorry you don't understand. The CSV files are just a list of the
>> bytes of data that came back. 0-255 for each byte I sent/received.
>
> Earlier you wrote:
>
>  > The file: catc_frontwhitepatch.csv contains the CATC data
>  > The file: usbfssnoop_frontwhitepatch.csv contains the data I pulled
>  > from the syslog from usbfs
>
> You can understand my confusion.  I thought the files contained a
> genuine CATC trace and a bunch of usbfs_snoop log lines.
>
>> I'll work on pulling out the plain text usb_fs snoop files for you and
>> get back to you. I didn't realize that looking at an array of bytes in
>> a CSV file was that difficult. My apologies.
>
> _Looking_ at an array of bytes isn't difficult.  _Understanding_ them
> is, particularly when you don't know what they are supposed to mean.
>
>> If you're looking for a better way to look at the data. It's a raw
>> image 100 x 20 pixels, grayscale format.
>
> Since the files contain 101 lines (including a header), I deduce that
> each line represents 20 pixels of data.  Grayscale format means that
> each line should contain 20 values, ranging between perhaps 0 and
> 255.  That's not what they actually contain.  Can you explain?
>
> I guess your point is that all three files should contain exactly
> the same values, since they are supposed to be the same data
> intercepted at various points along the way from the device to your
> program.  Right?
>
> As you mentioned, the usbfs data and the application data are indeed
> identical.  The CATC data are different, though.  This lead me to
> wonder whether you are correctly associating the packets in the CATC
> log with the same packets in the usbfs_snoop log.
>

Yes, my point is that all three should be the same. However, I did
find that I converted the CSV files incorrectly, so let me try this
again. There were some errors in the CSV files, I have run the data
through the wrong process to get the data. I'm attaching two TIFF
files this time. They contain the data that I collect, plus the
obvious TIFF header. The amount of data I requested is the same (2000
bytes). What I see in the usbfs snoop data is that not all 2000 bytes
get sent over. I'm in fact short by about 200 bytes). However, for
display purposes I've included the zeroed out data that should have
been filled in (basically because I parsed out the usbf_snoop data
into a zeroed buffer of the same size of the image I was expecting).

What I see is that it seem like I'm missing the first few rows of data
in the usbfs_snoop data. One interesting thing to note in that data is
what appears to be dots along the bottom of the image. That's actually
the footer data my device send to let me know the image information.
To me that confirms that the beginning of my data is just
"disappearing".

The thing that really gets me is that if I detect the corruption and
then ask for this data again, I get the right answer. That is to say
the CATC data matches the data I see in my application. So I suspect
that it's a knock on effect of something that happened before I try to
collect the data. Not so coincidentally, there is one odd thing my
application does prior to trying to get this data, please bear with me
as this takes a bit of explanation...

When my device comes up I do a bunch of reads on the interrupt
endpoint. When I get data I immediately issue another read to wait for
more. Once my device is online then I send the DISCARDURB ioctl to
cancel the remaining read on the interrupt endpoint. Prior to
capturing this corrupted data I issue another read on the interrupt
endpoint (this is how my device communicates events to me). The very
first event data that my devices sends to the host gets lost. So I
have the device "ping" the host on the interrupt endpoint knowing that
the first one will be lost and that once I the host gets the second
"ping" then I'm good to go. Right after getting the start event (which
comes after the "pings") I request this data. That's when the
corruption happens.

Why am I telling you all this. Well, that's because, if I ask the
device to send this data again it will come back correct. But then
periodically, it will take a long time for usbfs to get back to me
with future interrupt endpoint data. This leads me to believe the too
problems are related somehow.

>> I don't have any usbmon logs. That's why they were not included, I
>> thought I mentioned that. I'll work on getting those too.
>
> They will probably turn out to be the same as the usbfs_snoop data.  If
> they are then you need not bother posting them.
>
>> >> The file: catc_frontwhitepatch.csv contains the CATC data
>> >
>> > How am I supposed to view it?
>>
>> It's an array of the bytes I received in each packet strung together.
>
> Hmm...  Let's look at the second line of the application file (the
> first line contains column headers):
>
>        0,423,13
>
> Is 423 really a byte value?  Or the fourth line:
>
>        2,720,-20
>
> Are these byte values?

Hopefully these images and my lengthy explaination will clear up any
misunderstanding. Please accept my apologies for getting this wrong
the first time around. And thanks for your help so far.

-- Mike



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Roland Koebler]

hi,

your problem doesn't look like data corruption or a kernel-bug.
instead, it looks like an alignment problem, or like the problem I posted
under "bulk reads < 512 bytes?".

> >> I'll work on pulling out the plain text usb_fs snoop files for you and
> >> get back to you. I didn't realize that looking at an array of bytes in
> >> a CSV file was that difficult. My apologies.
for debugging, it's always better to have a look at the raw data.

> I'm attaching two TIFF
> files this time. They contain the data that I collect, plus the
> obvious TIFF header. The amount of data I requested is the same (2000
> bytes). What I see in the usbfs snoop data is that not all 2000 bytes
> get sent over. I'm in fact short by about 200 bytes).
do you request exactly 2000 bytes? or 2048 bytes?
how many bytes do you get exactly from usbfs?

> What I see is that it seem like I'm missing the first few rows of data
> in the usbfs_snoop data.
no, not the "first few rows", but the first 512 bytes: your two
tiff-images are shifted by exactly 512 bytes (or: pixels). and since
the "normal" bulk-packet-size is 512 bytes, it looks like you're missing
the 1st reply-data-packet.


regards,
Roland


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Alan Stern]

On Thu, 9 Oct 2008, Michael Lewis wrote:

> When my device comes up I do a bunch of reads on the interrupt
> endpoint. When I get data I immediately issue another read to wait for
> more. Once my device is online then I send the DISCARDURB ioctl to
> cancel the remaining read on the interrupt endpoint. Prior to
> capturing this corrupted data I issue another read on the interrupt
> endpoint (this is how my device communicates events to me). The very
> first event data that my devices sends to the host gets lost. So I
> have the device "ping" the host on the interrupt endpoint knowing that
> the first one will be lost and that once I the host gets the second
> "ping" then I'm good to go. Right after getting the start event (which
> comes after the "pings") I request this data. That's when the
> corruption happens.

It sounds like your specialized synchronization scheme is causing you
to lose some data.

> Why am I telling you all this. Well, that's because, if I ask the
> device to send this data again it will come back correct. But then
> periodically, it will take a long time for usbfs to get back to me
> with future interrupt endpoint data. This leads me to believe the too
> problems are related somehow.

This is part of the reason for looking at the usbmon data.  It will
show every USB message received by the computer, including ones that
your program ignores or loses.

Alan Stern


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Mike Lewis]

On Fri, Oct 10, 2008 at 10:10 AM, Alan Stern <stern@...> wrote:

> On Thu, 9 Oct 2008, Michael Lewis wrote:
>
>> When my device comes up I do a bunch of reads on the interrupt
>> endpoint. When I get data I immediately issue another read to wait for
>> more. Once my device is online then I send the DISCARDURB ioctl to
>> cancel the remaining read on the interrupt endpoint. Prior to
>> capturing this corrupted data I issue another read on the interrupt
>> endpoint (this is how my device communicates events to me). The very
>> first event data that my devices sends to the host gets lost. So I
>> have the device "ping" the host on the interrupt endpoint knowing that
>> the first one will be lost and that once I the host gets the second
>> "ping" then I'm good to go. Right after getting the start event (which
>> comes after the "pings") I request this data. That's when the
>> corruption happens.
>
> It sounds like your specialized synchronization scheme is causing you
> to lose some data.

It appears that I am missing that first packet (the data is off by
exactly 512 b