FTP component: Possible bug

[Ryadh Amar]

Hello all,
Well after managing to get the ftp component to work, I stumbled on a possible bug
The routes setup:
=========================
                        String ftpUrl = "ftp://guest@localhost:"
                                        + "21"
                                        + "/incoming?binary=true"
                                        + "&consumer.delay=10000&consumer.append=false&consumer.setNames=true";
                        String fileUrl = "file:///c:/inputdir/?noop=true&autoCreate=true";
                        from(ftpUrl).to(fileUrl);
=========================
the remote directory has the following layout

dir1/file2.txt
dir2/file3.txt
file1.txt
file4.txt

in the local directory, here is how the results looks like
dir1/file1.txt
dir1/file2.txt
dir1/file3.txt
dir1/file4.txt

all the files are now in the dir1 folder!

I am taking a look at the sources to confirm my remarks,
Ryadh.

[Claus Ibsen]

Hi Ryadh

Thanks for sharing this. Please let us know your findings. The FTP component is in for some improvements in Camel 1.5, we have a few tickets on this already - but your finding hasn't been reported before.

You can help by creating a junit test that demonstrates the bug; we prefer to have unit tests when fixing bugs.

The existing unit tests in camel-ftp can be used for inspiration and they use a mock FTP library for easy unit testing.

You are also welcome to create a ticket in our bug database so we wont forget this issue:
http://issues.apache.org/activemq/browse/CAMEL




Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. [phone]
Web: www.silverbullet.dk
-----Original Message-----
From: Ryadh [mailto:magnetic.ganesh@...]
Sent: 19. juli 2008 03:36
To: camel-user@...
Subject: FTP component: Possible bug


Hello all,
Well after managing to get the ftp component to work, I stumbled on a
possible bug
The routes setup:
=========================
                        String ftpUrl = "ftp://guest@localhost:"
                                        + "21"
                                        + "/incoming?binary=true"
                                        + "&consumer.delay=10000&consumer.append=false&consumer.setNames=true";
                        String fileUrl = "file:///c:/inputdir/?noop=true&autoCreate=true";
                        from(ftpUrl).to(fileUrl);
=========================
the remote directory has the following layout

dir1/file2.txt
dir2/file3.txt
file1.txt
file4.txt

in the local directory, here is how the results looks like
dir1/file1.txt
dir1/file2.txt
dir1/file3.txt
dir1/file4.txt

all the files are now in the dir1 folder!

I am taking a look at the sources to confirm my remarks,
Ryadh.
--
View this message in context: http://www.nabble.com/FTP-component%3A-Possible-bug-tp18540318s22882p18540318.html
Sent from the Camel - Users mailing list archive at Nabble.com.

[Ryadh Amar]

Hi Claus,
I coded a failing unit test, looked around the code and found the problem; looks like the pollDirectory() never climbs up to the parent directory when it has finished its work.
I made also a patch (albeit a very modest one, containing just a single line ;) )
I've also created an issue here: https://issues.apache.org/activemq/browse/CAMEL-738
Ryadh
 

Claus Ibsen wrote:

Hi Ryadh

Thanks for sharing this. Please let us know your findings. The FTP component is in for some improvements in Camel 1.5, we have a few tickets on this already - but your finding hasn't been reported before.

You can help by creating a junit test that demonstrates the bug; we prefer to have unit tests when fixing bugs.

The existing unit tests in camel-ftp can be used for inspiration and they use a mock FTP library for easy unit testing.

You are also welcome to create a ticket in our bug database so we wont forget this issue:
http://issues.apache.org/activemq/browse/CAMEL




Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. [phone]
Web: www.silverbullet.dk
-----Original Message-----
From: Ryadh [mailto:[e-mail]]
Sent: 19. juli 2008 03:36
To: [e-mail]
Subject: FTP component: Possible bug


Hello all,
Well after managing to get the ftp component to work, I stumbled on a
possible bug
The routes setup:
=========================
                        String ftpUrl = "ftp://guest@localhost:"
                                        + "21"
                                        + "/incoming?binary=true"
                                        + "&consumer.delay=10000&consumer.append=false&consumer.setNames=true";
                        String fileUrl = "file:///c:/inputdir/?noop=true&autoCreate=true";
                        from(ftpUrl).to(fileUrl);
=========================
the remote directory has the following layout

dir1/file2.txt
dir2/file3.txt
file1.txt
file4.txt

in the local directory, here is how the results looks like
dir1/file1.txt
dir1/file2.txt
dir1/file3.txt
dir1/file4.txt

all the files are now in the dir1 folder!

I am taking a look at the sources to confirm my remarks,
Ryadh.
--
View this message in context: http://www.nabble.com/FTP-component%3A-Possible-bug-tp18540318s22882p18540318.html
Sent from the Camel - Users mailing list archive at Nabble.com.

[Claus Ibsen]

Hi Ryadh

Thanks for finding the cause of the bug and fixing it.
Yes the unit test is too vague, we need to improve it to have a folder structure on the server side, and using the setNames=true option to store it in similar folder structure locally. And then assertions that the local folders exists also.

And then the same bug is in the SFTP part as well.

The regular FTP and SFTP has similar code but different 3rd part client API for the FTP stuff. Sadly there isn't a single good API that supports both of them we could use.


Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. [phone]
Web: www.silverbullet.dk
-----Original Message-----
From: Ryadh [mailto:magnetic.ganesh@...]
Sent: 20. juli 2008 03:56
To: camel-user@...
Subject: RE: FTP component: Possible bug


Hi Claus,
I coded a failing unit test, looked around the code and found the problem;
looks like the pollDirectory() never climbs up to the parent directory when
it has finished its work.
I made also a patch (albeit a very modest one, containing just a single line
;) )
I've also created an issue here:
https://issues.apache.org/activemq/browse/CAMEL-738
Ryadh
 

Claus Ibsen wrote:

>
> Hi Ryadh
>
> Thanks for sharing this. Please let us know your findings. The FTP
> component is in for some improvements in Camel 1.5, we have a few tickets
> on this already - but your finding hasn't been reported before.
>
> You can help by creating a junit test that demonstrates the bug; we prefer
> to have unit tests when fixing bugs.
>
> The existing unit tests in camel-ftp can be used for inspiration and they
> use a mock FTP library for easy unit testing.
>
> You are also welcome to create a ticket in our bug database so we wont
> forget this issue:
> http://issues.apache.org/activemq/browse/CAMEL
>
>
>
>
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. [phone]
> Web: www.silverbullet.dk
> -----Original Message-----
> From: Ryadh [mailto:magnetic.ganesh@...]
> Sent: 19. juli 2008 03:36
> To: camel-user@...
> Subject: FTP component: Possible bug
>
>
> Hello all,
> Well after managing to get the ftp component to work, I stumbled on a
> possible bug
> The routes setup:
> =========================
> String ftpUrl = "ftp://guest@localhost:"
> + "21"
> + "/incoming?binary=true"
> +
> "&consumer.delay=10000&consumer.append=false&consumer.setNames=true";
> String fileUrl = "file:///c:/inputdir/?noop=true&autoCreate=true";
> from(ftpUrl).to(fileUrl);
> =========================
> the remote directory has the following layout
>
> dir1/file2.txt
> dir2/file3.txt
> file1.txt
> file4.txt
>
> in the local directory, here is how the results looks like
> dir1/file1.txt
> dir1/file2.txt
> dir1/file3.txt
> dir1/file4.txt
>
> all the files are now in the dir1 folder!
>
> I am taking a look at the sources to confirm my remarks,
> Ryadh.
> --
> View this message in context:
> http://www.nabble.com/FTP-component%3A-Possible-bug-tp18540318s22882p18540318.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>

--
View this message in context: http://www.nabble.com/FTP-component%3A-Possible-bug-tp18540318s22882p18550626.html
Sent from the Camel - Users mailing list archive at Nabble.com.

[Claus Ibsen]

Hi Ryadh

I would expect the code should be more like this:

        String currentDir = client.printWorkingDirectory();

... then all the polling

        // change back to original current dir
        client.changeWorkingDirectory(currentDir);

Could you test if this code works? The SFTP part doesn't have a change to parent dir. And I do think the above code is more common used.

Now we needed the unit test that covered this ;)


Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. [phone]
Web: www.silverbullet.dk

-----Original Message-----
From: Ryadh [mailto:magnetic.ganesh@...]
Sent: 20. juli 2008 03:56
To: camel-user@...
Subject: RE: FTP component: Possible bug


Hi Claus,
I coded a failing unit test, looked around the code and found the problem;
looks like the pollDirectory() never climbs up to the parent directory when
it has finished its work.
I made also a patch (albeit a very modest one, containing just a single line
;) )
I've also created an issue here:
https://issues.apache.org/activemq/browse/CAMEL-738
Ryadh
 

Claus Ibsen wrote:

>
> Hi Ryadh
>
> Thanks for sharing this. Please let us know your findings. The FTP
> component is in for some improvements in Camel 1.5, we have a few tickets
> on this already - but your finding hasn't been reported before.
>
> You can help by creating a junit test that demonstrates the bug; we prefer
> to have unit tests when fixing bugs.
>
> The existing unit tests in camel-ftp can be used for inspiration and they
> use a mock FTP library for easy unit testing.
>
> You are also welcome to create a ticket in our bug database so we wont
> forget this issue:
> http://issues.apache.org/activemq/browse/CAMEL
>
>
>
>
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. [phone]
> Web: www.silverbullet.dk
> -----Original Message-----
> From: Ryadh [mailto:magnetic.ganesh@...]
> Sent: 19. juli 2008 03:36
> To: camel-user@...
> Subject: FTP component: Possible bug
>
>
> Hello all,
> Well after managing to get the ftp component to work, I stumbled on a
> possible bug
> The routes setup:
> =========================
> String ftpUrl = "ftp://guest@localhost:"
> + "21"
> + "/incoming?binary=true"
> +
> "&consumer.delay=10000&consumer.append=false&consumer.setNames=true";
> String fileUrl = "file:///c:/inputdir/?noop=true&autoCreate=true";
> from(ftpUrl).to(fileUrl);
> =========================
> the remote directory has the following layout
>
> dir1/file2.txt
> dir2/file3.txt
> file1.txt
> file4.txt
>
> in the local directory, here is how the results looks like
> dir1/file1.txt
> dir1/file2.txt
> dir1/file3.txt
> dir1/file4.txt
>
> all the files are now in the dir1 folder!
>
> I am taking a look at the sources to confirm my remarks,
> Ryadh.
> --
> View this message in context:
> http://www.nabble.com/FTP-component%3A-Possible-bug-tp18540318s22882p18540318.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>

--
View this message in context: http://www.nabble.com/FTP-component%3A-Possible-bug-tp18540318s22882p18550626.html
Sent from the Camel - Users mailing list archive at Nabble.com.

[Claus Ibsen]

Hi

Ah the unit test did cover it a little. There is a "a" sub folder. This is hard to spot. You should probably have added a longer folder name so its easier to see there is a difference. And use a different binary file so we know its not the wrong file that has been copied.



Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. [phone]
Web: www.silverbullet.dk

-----Original Message-----
From: Ryadh [mailto:magnetic.ganesh@...]
Sent: 20. juli 2008 03:56
To: camel-user@...
Subject: RE: FTP component: Possible bug


Hi Claus,
I coded a failing unit test, looked around the code and found the problem;
looks like the pollDirectory() never climbs up to the parent directory when
it has finished its work.
I made also a patch (albeit a very modest one, containing just a single line
;) )
I've also created an issue here:
https://issues.apache.org/activemq/browse/CAMEL-738
Ryadh
 

Claus Ibsen wrote:

>
> Hi Ryadh
>
> Thanks for sharing this. Please let us know your findings. The FTP
> component is in for some improvements in Camel 1.5, we have a few tickets
> on this already - but your finding hasn't been reported before.
>
> You can help by creating a junit test that demonstrates the bug; we prefer
> to have unit tests when fixing bugs.
>
> The existing unit tests in camel-ftp can be used for inspiration and they
> use a mock FTP library for easy unit testing.
>
> You are also welcome to create a ticket in our bug database so we wont
> forget this issue:
> http://issues.apache.org/activemq/browse/CAMEL
>
>
>
>
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. [phone]
> Web: www.silverbullet.dk
> -----Original Message-----
> From: Ryadh [mailto:magnetic.ganesh@...]
> Sent: 19. juli 2008 03:36
> To: camel-user@...
> Subject: FTP component: Possible bug
>
>
> Hello all,
> Well after managing to get the ftp component to work, I stumbled on a
> possible bug
> The routes setup:
> =========================
> String ftpUrl = "ftp://guest@localhost:"
> + "21"
> + "/incoming?binary=true"
> +
> "&consumer.delay=10000&consumer.append=false&consumer.setNames=true";
> String fileUrl = "file:///c:/inputdir/?noop=true&autoCreate=true";
> from(ftpUrl).to(fileUrl);
> =========================
> the remote directory has the following layout
>
> dir1/file2.txt
> dir2/file3.txt
> file1.txt
> file4.txt
>
> in the local directory, here is how the results looks like
> dir1/file1.txt
> dir1/file2.txt
> dir1/file3.txt
> dir1/file4.txt
>
> all the files are now in the dir1 folder!
>
> I am taking a look at the sources to confirm my remarks,
> Ryadh.
> --
> View this message in context:
> http://www.nabble.com/FTP-component%3A-Possible-bug-tp18540318s22882p18540318.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>

--
View this message in context: http://www.nabble.com/FTP-component%3A-Possible-bug-tp18540318s22882p18550626.html
Sent from the Camel - Users mailing list archive at Nabble.com.

[Ryadh Amar]

Thanks for the amendments.
changing client.changeToParentDirectory(); to
client.changeWorkingDirectory(currentDir); works too.

If you're not modifying the test to include your remarks, I'll gladly submit another one.

Claus Ibsen wrote:

Hi

Ah the unit test did cover it a little. There is a "a" sub folder. This is hard to spot. You should probably have added a longer folder name so its easier to see there is a difference. And use a different binary file so we know its not the wrong file that has been copied.



Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. [phone]
Web: www.silverbullet.dk

-----Original Message-----
From: Ryadh [mailto:[e-mail]]
Sent: 20. juli 2008 03:56
To: [e-mail]
Subject: RE: FTP component: Possible bug


Hi Claus,
I coded a failing unit test, looked around the code and found the problem;
looks like the pollDirectory() never climbs up to the parent directory when
it has finished its work.
I made also a patch (albeit a very modest one, containing just a single line
;) )
I've also created an issue here:
https://issues.apache.org/activemq/browse/CAMEL-738
Ryadh
 

Claus Ibsen wrote:
>
> Hi Ryadh
>
> Thanks for sharing this. Please let us know your findings. The FTP
> component is in for some improvements in Camel 1.5, we have a few tickets
> on this already - but your finding hasn't been reported before.
>
> You can help by creating a junit test that demonstrates the bug; we prefer
> to have unit tests when fixing bugs.
>
> The existing unit tests in camel-ftp can be used for inspiration and they
> use a mock FTP library for easy unit testing.
>
> You are also welcome to create a ticket in our bug database so we wont
> forget this issue:
> http://issues.apache.org/activemq/browse/CAMEL
>
>
>
>
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. [phone]
> Web: www.silverbullet.dk
> -----Original Message-----
> From: Ryadh [mailto:[e-mail]]
> Sent: 19. juli 2008 03:36
> To: [e-mail]
> Subject: FTP component: Possible bug
>
>
> Hello all,
> Well after managing to get the ftp component to work, I stumbled on a
> possible bug
> The routes setup:
> =========================
> String ftpUrl = "ftp://guest@localhost:"
> + "21"
> + "/incoming?binary=true"
> +
> "&consumer.delay=10000&consumer.append=false&consumer.setNames=true";
> String fileUrl = "file:///c:/inputdir/?noop=true&autoCreate=true";
> from(ftpUrl).to(fileUrl);
> =========================
> the remote directory has the following layout
>
> dir1/file2.txt
> dir2/file3.txt
> file1.txt
> file4.txt
>
> in the local directory, here is how the results looks like
> dir1/file1.txt
> dir1/file2.txt
> dir1/file3.txt
> dir1/file4.txt
>
> all the files are now in the dir1 folder!
>
> I am taking a look at the sources to confirm my remarks,
> Ryadh.
> --
> View this message in context:
> http://www.nabble.com/FTP-component%3A-Possible-bug-tp18540318s22882p18540318.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>

--
View this message in context: http://www.nabble.com/FTP-component%3A-Possible-bug-tp18540318s22882p18550626.html
Sent from the Camel - Users mailing list archive at Nabble.com.