|
»
»
»
SftpProducer bug
|
View:
New views
11 Messages
—
Rating Filter:
Alert me
|
 |
SftpProducer bug
by Bela Vizy
::
Rate this Message:
Hi,
First let me say it's been a huge fun working with Camel! Keep up the good work! I have just built 1.5 from the trunk and it seems there's a bug in the SftpProducer. I know this is not the way to report it, but for now I'm swamped ... I promise I'll submit patches later. The problem is that the mkdir in the buildDirectory throws an exception when the folder exists and it's not caught. With the current code you can only send files to the "." (i.e. without specifying a folder). Here is my humble change: protected static boolean buildDirectory(ChannelSftp sftpClient, String dirName) throws IOException, SftpException { boolean atLeastOneSuccess = false; final StringBuilder sb = new StringBuilder(dirName.length()); final String[] dirs = dirName.split("\\/"); for (String dir : dirs) { sb.append(dir).append('/'); String directory = sb.toString(); if (LOG.isDebugEnabled()) { LOG.debug("Trying to build directory: " + directory); } try { sftpClient.mkdir(directory); atLeastOneSuccess = true; } catch ( SftpException e ) { // Nothing. Keep trying .. } } return atLeastOneSuccess; } Thanks, Bela |
|
|
RE: SftpProducer bug
by Claus Ibsen
::
Rate this Message:
Hi Bela
Glad you enjoy riding the Camel. Thanks for reporting the issue. I have created a ticket in JIRA: https://issues.apache.org/activemq/browse/CAMEL-758 Lets see if there is a existDir or similar method on the client so we will only try to build the folder if it doesn't exists. Patches is much appreciated. Just remember to grant the Apache License when attaching the file. As we haven't a mock SFTP Server for unit testing we would appreciate if you can verify the fix when we get it committed to the trunk. Med venlig hilsen Claus Ibsen ...................................... Silverbullet Skovsgårdsvænget 21 8362 Hørning Tlf. +45 2962 7576 Web: www.silverbullet.dk -----Original Message----- From: Bela [mailto:bvizy@...] Sent: 24. juli 2008 23:07 To: camel-user@... Subject: SftpProducer bug Hi, First let me say it's been a huge fun working with Camel! Keep up the good work! I have just built 1.5 from the trunk and it seems there's a bug in the SftpProducer. I know this is not the way to report it, but for now I'm swamped ... I promise I'll submit patches later. The problem is that the mkdir in the buildDirectory throws an exception when the folder exists and it's not caught. With the current code you can only send files to the "." (i.e. without specifying a folder). Here is my humble change: protected static boolean buildDirectory(ChannelSftp sftpClient, String dirName) throws IOException, SftpException { boolean atLeastOneSuccess = false; final StringBuilder sb = new StringBuilder(dirName.length()); final String[] dirs = dirName.split("\\/"); for (String dir : dirs) { sb.append(dir).append('/'); String directory = sb.toString(); if (LOG.isDebugEnabled()) { LOG.debug("Trying to build directory: " + directory); } try { sftpClient.mkdir(directory); atLeastOneSuccess = true; } catch ( SftpException e ) { // Nothing. Keep trying .. } } return atLeastOneSuccess; } Thanks, Bela -- View this message in context: http://www.nabble.com/SftpProducer-bug-tp18640519s22882p18640519.html Sent from the Camel - Users mailing list archive at Nabble.com. |
|
|
RE: SftpProducer bug
by Claus Ibsen
::
Rate this Message:
Hi Bela
I have committed a fix in camel-ftp. http://svn.apache.org/viewvc?view=rev&revision=679979 Could you try the latest code from trunk? Or jump on 1.5-SNAPSHOT if you are using maven. Med venlig hilsen Claus Ibsen ...................................... Silverbullet Skovsgårdsvænget 21 8362 Hørning Tlf. +45 2962 7576 Web: www.silverbullet.dk -----Original Message----- From: Claus Ibsen [mailto:ci@...] Sent: 25. juli 2008 09:40 To: camel-user@... Subject: RE: SftpProducer bug Hi Bela Glad you enjoy riding the Camel. Thanks for reporting the issue. I have created a ticket in JIRA: https://issues.apache.org/activemq/browse/CAMEL-758 Lets see if there is a existDir or similar method on the client so we will only try to build the folder if it doesn't exists. Patches is much appreciated. Just remember to grant the Apache License when attaching the file. As we haven't a mock SFTP Server for unit testing we would appreciate if you can verify the fix when we get it committed to the trunk. Med venlig hilsen Claus Ibsen ...................................... Silverbullet Skovsgårdsvænget 21 8362 Hørning Tlf. +45 2962 7576 Web: www.silverbullet.dk -----Original Message----- From: Bela [mailto:bvizy@...] Sent: 24. juli 2008 23:07 To: camel-user@... Subject: SftpProducer bug Hi, First let me say it's been a huge fun working with Camel! Keep up the good work! I have just built 1.5 from the trunk and it seems there's a bug in the SftpProducer. I know this is not the way to report it, but for now I'm swamped ... I promise I'll submit patches later. The problem is that the mkdir in the buildDirectory throws an exception when the folder exists and it's not caught. With the current code you can only send files to the "." (i.e. without specifying a folder). Here is my humble change: protected static boolean buildDirectory(ChannelSftp sftpClient, String dirName) throws IOException, SftpException { boolean atLeastOneSuccess = false; final StringBuilder sb = new StringBuilder(dirName.length()); final String[] dirs = dirName.split("\\/"); for (String dir : dirs) { sb.append(dir).append('/'); String directory = sb.toString(); if (LOG.isDebugEnabled()) { LOG.debug("Trying to build directory: " + directory); } try { sftpClient.mkdir(directory); atLeastOneSuccess = true; } catch ( SftpException e ) { // Nothing. Keep trying .. } } return atLeastOneSuccess; } Thanks, Bela -- View this message in context: http://www.nabble.com/SftpProducer-bug-tp18640519s22882p18640519.html Sent from the Camel - Users mailing list archive at Nabble.com. |
|
|
RE: SftpProducer bug
by Claus Ibsen
::
Rate this Message:
Hi Bela
Beware that I have also fixed CAMEL-654 that is related to FTP. You can disable this feature by setting the option consumer.exclusiveRead to false. But the option shouldn't affect the "can not create directory bug" Med venlig hilsen Claus Ibsen ...................................... Silverbullet Skovsgårdsvænget 21 8362 Hørning Tlf. +45 2962 7576 Web: www.silverbullet.dk -----Original Message----- From: Claus Ibsen [mailto:ci@...] Sent: 26. juli 2008 13:13 To: camel-user@... Subject: RE: SftpProducer bug Hi Bela I have committed a fix in camel-ftp. http://svn.apache.org/viewvc?view=rev&revision=679979 Could you try the latest code from trunk? Or jump on 1.5-SNAPSHOT if you are using maven. Med venlig hilsen Claus Ibsen ...................................... Silverbullet Skovsgårdsvænget 21 8362 Hørning Tlf. +45 2962 7576 Web: www.silverbullet.dk -----Original Message----- From: Claus Ibsen [mailto:ci@...] Sent: 25. juli 2008 09:40 To: camel-user@... Subject: RE: SftpProducer bug Hi Bela Glad you enjoy riding the Camel. Thanks for reporting the issue. I have created a ticket in JIRA: https://issues.apache.org/activemq/browse/CAMEL-758 Lets see if there is a existDir or similar method on the client so we will only try to build the folder if it doesn't exists. Patches is much appreciated. Just remember to grant the Apache License when attaching the file. As we haven't a mock SFTP Server for unit testing we would appreciate if you can verify the fix when we get it committed to the trunk. Med venlig hilsen Claus Ibsen ...................................... Silverbullet Skovsgårdsvænget 21 8362 Hørning Tlf. +45 2962 7576 Web: www.silverbullet.dk -----Original Message----- From: Bela [mailto:bvizy@...] Sent: 24. juli 2008 23:07 To: camel-user@... Subject: SftpProducer bug Hi, First let me say it's been a huge fun working with Camel! Keep up the good work! I have just built 1.5 from the trunk and it seems there's a bug in the SftpProducer. I know this is not the way to report it, but for now I'm swamped ... I promise I'll submit patches later. The problem is that the mkdir in the buildDirectory throws an exception when the folder exists and it's not caught. With the current code you can only send files to the "." (i.e. without specifying a folder). Here is my humble change: protected static boolean buildDirectory(ChannelSftp sftpClient, String dirName) throws IOException, SftpException { boolean atLeastOneSuccess = false; final StringBuilder sb = new StringBuilder(dirName.length()); final String[] dirs = dirName.split("\\/"); for (String dir : dirs) { sb.append(dir).append('/'); String directory = sb.toString(); if (LOG.isDebugEnabled()) { LOG.debug("Trying to build directory: " + directory); } try { sftpClient.mkdir(directory); atLeastOneSuccess = true; } catch ( SftpException e ) { // Nothing. Keep trying .. } } return atLeastOneSuccess; } Thanks, Bela -- View this message in context: http://www.nabble.com/SftpProducer-bug-tp18640519s22882p18640519.html Sent from the Camel - Users mailing list archive at Nabble.com. |
|
|
RE: SftpProducer bug
by Bela Vizy
::
Rate this Message:
Claus,
It works only if the last folder is missing. It seems the sftp servers don't create intrermediate folders for you. At least my server doesn't (out of the box Fedora 7). So I think we have to bring back the code to create the folders on by one. Let me know if you want me to take a crack at it. But there's good news: The delete option works nicely on the consumer. How about the move (rename) like the file consumer? -Bela
|
|
|
RE: SftpProducer bug
by Bela Vizy
::
Rate this Message:
I attached a patch to CAMEL-758. Did I do it right? This is my first ,,,
-B
|
|
|
RE: SftpProducer bug
by Claus Ibsen
::
Rate this Message:
Hi Bela
Thanks for the patch. Yes it is correct. The most people forget is that the patch file needs the Apache License granted before we can accept and apply them to the code base. But you remembered. I have applied it to the codebase, and also the regular FTP producer as well. Could you test it again? Med venlig hilsen Claus Ibsen ...................................... Silverbullet Skovsgårdsvænget 21 8362 Hørning Tlf. +45 2962 7576 Web: www.silverbullet.dk -----Original Message----- From: Bela [mailto:bvizy@...] Sent: 28. juli 2008 02:07 To: camel-user@... Subject: RE: SftpProducer bug I attached a patch to CAMEL-758. Did I do it right? This is my first ,,, -B Bela wrote: > > Claus, > > It works only if the last folder is missing. It seems the sftp servers > don't create intrermediate folders for you. At least my server doesn't > (out of the box Fedora 7). So I think we have to bring back the code to > create the folders on by one. Let me know if you want me to take a crack > at it. > > But there's good news: The delete option works nicely on the consumer. How > about the move (rename) like the file consumer? > > -Bela > > > Claus Ibsen wrote: >> >> Hi Bela >> >> I have committed a fix in camel-ftp. >> http://svn.apache.org/viewvc?view=rev&revision=679979 >> >> Could you try the latest code from trunk? >> Or jump on 1.5-SNAPSHOT if you are using maven. >> >> > > -- View this message in context: http://www.nabble.com/SftpProducer-bug-tp18640519s22882p18682198.html Sent from the Camel - Users mailing list archive at Nabble.com. |
|
|
RE: SftpProducer bug
by Claus Ibsen
::
Rate this Message:
Hi Bela
About the move ============== How sophisticated do you want the move operation? The file component *only* supports setting a pre and post fix to the name. With the prefix you can move it to a subfolder. And with the postfix you can append ".done" or what you like to the filename. If you move it to a subfolder then we can potential have the same problem that if the folder doesn't exists then we need to create it as well. But I agree it is a valid point of having the same move options in camel-ftp as we have in the FileConsumer. As you have a SFTP server I assume you can help test it on this server? We only have unit test servers for the regular FTP (= non SFTP). So we most relay on the codebase is 100% identical between FTP and SFTP to be more sure that the SFTP part is also working. I have created CAMEL-764 to track it. Med venlig hilsen Claus Ibsen ...................................... Silverbullet Skovsgårdsvænget 21 8362 Hørning Tlf. +45 2962 7576 Web: www.silverbullet.dk -----Original Message----- From: Bela [mailto:bvizy@...] Sent: 28. juli 2008 02:07 To: camel-user@... Subject: RE: SftpProducer bug I attached a patch to CAMEL-758. Did I do it right? This is my first ,,, -B Bela wrote: > > Claus, > > It works only if the last folder is missing. It seems the sftp servers > don't create intrermediate folders for you. At least my server doesn't > (out of the box Fedora 7). So I think we have to bring back the code to > create the folders on by one. Let me know if you want me to take a crack > at it. > > But there's good news: The delete option works nicely on the consumer. How > about the move (rename) like the file consumer? > > -Bela > > > Claus Ibsen wrote: >> >> Hi Bela >> >> I have committed a fix in camel-ftp. >> http://svn.apache.org/viewvc?view=rev&revision=679979 >> >> Could you try the latest code from trunk? >> Or jump on 1.5-SNAPSHOT if you are using maven. >> >> > > -- View this message in context: http://www.nabble.com/SftpProducer-bug-tp18640519s22882p18682198.html Sent from the Camel - Users mailing list archive at Nabble.com. |
|
|
RE: SftpProducer bug
by Bela Vizy
::
Rate this Message:
Sure, I'll test it, no problem. I'll create the test cases.
In regard of the sophistication, being the same as the file endpoint would be nice. Since we have the "delete", actually these aren't the most important things to have. The delete was important so camel doesn't suck out the files again when you restart. Bela
|
|
|
RE: SftpProducer bug
by Claus Ibsen
::
Rate this Message:
Hi Bela
I have added the move feature to the SFTP part as well. Feel free to test it when you have time and report your findings. I would like to know if the logging is to verbose if you move the file after consuming. If the operations fails for some reason it is logged at WARN level, but since the SFTP API hasn't a Boolean return value if the file was moved/deleted then I can be in doubt if the API throws an exception or not. So if you have time to test: - moving files to sub folders that doesn't exists - moving files where an existing file already exists (camel should try to delete the existing file first) - moving files to same folder Thanks a lot. You can report in JIRA at CAMEL-764 then I am sure to spot it. Med venlig hilsen Claus Ibsen ...................................... Silverbullet Skovsgårdsvænget 21 8362 Hørning Tlf. +45 2962 7576 Web: www.silverbullet.dk -----Original Message----- From: Bela [mailto:bvizy@...] Sent: 28. juli 2008 16:06 To: camel-user@... Subject: RE: SftpProducer bug Sure, I'll test it, no problem. I'll create the test cases. In regard of the sophistication, being the same as the file endpoint would be nice. Since we have the "delete", actually these aren't the most important things to have. The delete was important so camel doesn't suck out the files again when you restart. Bela Claus Ibsen wrote: > > Hi Bela > > About the move > ============== > How sophisticated do you want the move operation? > > -- View this message in context: http://www.nabble.com/SftpProducer-bug-tp18640519s22882p18689214.html Sent from the Camel - Users mailing list archive at Nabble.com. |
|
|
RE: SftpProducer bug
by Bela Vizy
::
Rate this Message:
Updated CAMEL-764
|
| Free Forum Powered by Nabble | Forum Help |