Native windows build - issue with long path names
|
View:
New views
4 Messages
—
Rating Filter:
Alert me
|
 |
Native windows build - issue with long path names
by Ryan How
::
Rate this Message:
Hi,
I did a few searches, but I couldn't find any issues discussed about long path names in the native windows build. I have been having issues with the 255 char limit (or round abouts) in pathnames, but I am unsure as to why because I using the native windows build (which shouldn't have this limit?). The error originates in os.pyc (which, I have no python experience, but isn't an rdiff-backup source file?). I get the same error when using dos paths (ie. c:\blah) and UNC paths (which otherwise seem to work well with the native windows build, yay). I could understand the issue with the dos paths, but not with the UNC paths. I have a feeling this may actually be a python issue? Error is below. Cheers, Ryan Processing changed file Documents and Settings/ryan/Local Settings/Temp/test_dir_980382089/deep1/deep1_1/deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/deep1_1_1_1_1_1_1/deep1_1_1_1_1_1_1_1 Exception '[Error 206] The filename or extension is too long: '\\\\127.0.0.1\\c$\\Documents and Settings\\ryan\\Local Settings\\Temp\\test_dir_2042584358/rdiff-backup-data/increments/Documents and Settings/ryan/Local Settings/Temp/test_dir_980382089/deep1/deep1_1/deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/deep1_1_1_1_1_1_1'' raised of class '<type 'exceptions.WindowsError'>': File "rdiff_backup\Main.pyc", line 302, in error_check_Main File "rdiff_backup\Main.pyc", line 322, in Main File "rdiff_backup\Main.pyc", line 278, in take_action File "rdiff_backup\Main.pyc", line 344, in Backup File "rdiff_backup\backup.pyc", line 38, in Mirror File "rdiff_backup\backup.pyc", line 232, in patch File "rdiff_backup\rorpiter.pyc", line 284, in __call__ File "rdiff_backup\backup.pyc", line 622, in start_process File "rdiff_backup\longname.pyc", line 203, in get_mirror_inc_rps File "rdiff_backup\longname.pyc", line 189, in find_inc_pair File "rdiff_backup\longname.pyc", line 128, in check_new_index File "rdiff_backup\longname.pyc", line 111, in wrap_call File "rdiff_backup\longname.pyc", line 122, in make_parent File "rdiff_backup\rpath.pyc", line 964, in makedirs File "os.pyc", line 171, in makedirs Traceback (most recent call last): File "rdiff-backup", line 23, in <module> File "rdiff_backup\Main.pyc", line 302, in error_check_Main File "rdiff_backup\Main.pyc", line 322, in Main File "rdiff_backup\Main.pyc", line 278, in take_action File "rdiff_backup\Main.pyc", line 344, in Backup File "rdiff_backup\backup.pyc", line 38, in Mirror File "rdiff_backup\backup.pyc", line 232, in patch File "rdiff_backup\rorpiter.pyc", line 284, in __call__ File "rdiff_backup\backup.pyc", line 622, in start_process File "rdiff_backup\longname.pyc", line 203, in get_mirror_inc_rps File "rdiff_backup\longname.pyc", line 189, in find_inc_pair File "rdiff_backup\longname.pyc", line 128, in check_new_index File "rdiff_backup\longname.pyc", line 111, in wrap_call File "rdiff_backup\longname.pyc", line 122, in make_parent File "rdiff_backup\rpath.pyc", line 964, in makedirs File "os.pyc", line 171, in makedirs WindowsError: [Error 206] The filename or extension is too long: '\\\\127.0.0.1\\c$\\Documents and Settings\\ryan\\Local Settings\\Temp\\test_dir_2042584358/rdiff-backup-data/increments/Documents and Settings/ryan/Local Settings/Temp/test_dir_980382089/deep1/deep1_1/deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/deep1_1_1_1_1_1_1' _______________________________________________ rdiff-backup-users mailing list at rdiff-backup-users@... http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki |
|
|
Re: Native windows build - issue with long path names
by Ryan How
::
Rate this Message:
Just to add, I should point out it is a fatal error in that the whole
backup fails at that point. It isn't just those files that don't get backed up. Ryan How wrote: > Hi, > > I did a few searches, but I couldn't find any issues discussed about > long path names in the native windows build. > > I have been having issues with the 255 char limit (or round abouts) in > pathnames, but I am unsure as to why because I using the native > windows build (which shouldn't have this limit?). The error originates > in os.pyc (which, I have no python experience, but isn't an > rdiff-backup source file?). I get the same error when using dos paths > (ie. c:\blah) and UNC paths (which otherwise seem to work well with > the native windows build, yay). I could understand the issue with the > dos paths, but not with the UNC paths. I have a feeling this may > actually be a python issue? > > Error is below. > > Cheers, > > Ryan > > > Processing changed file Documents and Settings/ryan/Local > Settings/Temp/test_dir_980382089/deep1/deep1_1/deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/deep1_1_1_1_1_1_1/deep1_1_1_1_1_1_1_1 > > Exception '[Error 206] The filename or extension is too long: > '\\\\127.0.0.1\\c$\\Documents and Settings\\ryan\\Local > Settings\\Temp\\test_dir_2042584358/rdiff-backup-data/increments/Documents > and Settings/ryan/Local > Settings/Temp/test_dir_980382089/deep1/deep1_1/deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/deep1_1_1_1_1_1_1'' > raised of class '<type 'exceptions.WindowsError'>': > File "rdiff_backup\Main.pyc", line 302, in error_check_Main > File "rdiff_backup\Main.pyc", line 322, in Main > File "rdiff_backup\Main.pyc", line 278, in take_action > File "rdiff_backup\Main.pyc", line 344, in Backup > File "rdiff_backup\backup.pyc", line 38, in Mirror > File "rdiff_backup\backup.pyc", line 232, in patch > File "rdiff_backup\rorpiter.pyc", line 284, in __call__ > File "rdiff_backup\backup.pyc", line 622, in start_process > File "rdiff_backup\longname.pyc", line 203, in get_mirror_inc_rps > File "rdiff_backup\longname.pyc", line 189, in find_inc_pair > File "rdiff_backup\longname.pyc", line 128, in check_new_index > File "rdiff_backup\longname.pyc", line 111, in wrap_call > File "rdiff_backup\longname.pyc", line 122, in make_parent > File "rdiff_backup\rpath.pyc", line 964, in makedirs > File "os.pyc", line 171, in makedirs > > Traceback (most recent call last): > File "rdiff-backup", line 23, in <module> > File "rdiff_backup\Main.pyc", line 302, in error_check_Main > File "rdiff_backup\Main.pyc", line 322, in Main > File "rdiff_backup\Main.pyc", line 278, in take_action > File "rdiff_backup\Main.pyc", line 344, in Backup > File "rdiff_backup\backup.pyc", line 38, in Mirror > File "rdiff_backup\backup.pyc", line 232, in patch > File "rdiff_backup\rorpiter.pyc", line 284, in __call__ > File "rdiff_backup\backup.pyc", line 622, in start_process > File "rdiff_backup\longname.pyc", line 203, in get_mirror_inc_rps > File "rdiff_backup\longname.pyc", line 189, in find_inc_pair > File "rdiff_backup\longname.pyc", line 128, in check_new_index > File "rdiff_backup\longname.pyc", line 111, in wrap_call > File "rdiff_backup\longname.pyc", line 122, in make_parent > File "rdiff_backup\rpath.pyc", line 964, in makedirs > File "os.pyc", line 171, in makedirs > WindowsError: [Error 206] The filename or extension is too long: > '\\\\127.0.0.1\\c$\\Documents and Settings\\ryan\\Local > Settings\\Temp\\test_dir_2042584358/rdiff-backup-data/increments/Documents > and Settings/ryan/Local > Settings/Temp/test_dir_980382089/deep1/deep1_1/deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/deep1_1_1_1_1_1_1' > > > > > > _______________________________________________ > rdiff-backup-users mailing list at rdiff-backup-users@... > http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users > Wiki URL: > http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki > _______________________________________________ rdiff-backup-users mailing list at rdiff-backup-users@... http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki |
|
|
Re: Native windows build - issue with long path names
by Josh Nisly
::
Rate this Message:
I looked into this a bit today. Windows does throw a different exception
on long file names than unix (a WindowsError (derived from OSError, derived from EnvironmentError), with the errno set to 206, which doesn't map to anything valid on unix.) However, when I added a check for this to the places that already check for an ENAMETOOLONG exception, it still failed. JoshN Ryan How wrote: > Just to add, I should point out it is a fatal error in that the whole > backup fails at that point. It isn't just those files that don't get > backed up. > > Ryan How wrote: >> Hi, >> >> I did a few searches, but I couldn't find any issues discussed about >> long path names in the native windows build. >> >> I have been having issues with the 255 char limit (or round abouts) >> in pathnames, but I am unsure as to why because I using the native >> windows build (which shouldn't have this limit?). The error >> originates in os.pyc (which, I have no python experience, but isn't >> an rdiff-backup source file?). I get the same error when using dos >> paths (ie. c:\blah) and UNC paths (which otherwise seem to work well >> with the native windows build, yay). I could understand the issue >> with the dos paths, but not with the UNC paths. I have a feeling this >> may actually be a python issue? >> >> Error is below. >> >> Cheers, >> >> Ryan >> >> >> Processing changed file Documents and Settings/ryan/Local >> Settings/Temp/test_dir_980382089/deep1/deep1_1/deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/deep1_1_1_1_1_1_1/deep1_1_1_1_1_1_1_1 >> >> Exception '[Error 206] The filename or extension is too long: >> '\\\\127.0.0.1\\c$\\Documents and Settings\\ryan\\Local >> Settings\\Temp\\test_dir_2042584358/rdiff-backup-data/increments/Documents >> and Settings/ryan/Local >> Settings/Temp/test_dir_980382089/deep1/deep1_1/deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/deep1_1_1_1_1_1_1'' >> raised of class '<type 'exceptions.WindowsError'>': >> File "rdiff_backup\Main.pyc", line 302, in error_check_Main >> File "rdiff_backup\Main.pyc", line 322, in Main >> File "rdiff_backup\Main.pyc", line 278, in take_action >> File "rdiff_backup\Main.pyc", line 344, in Backup >> File "rdiff_backup\backup.pyc", line 38, in Mirror >> File "rdiff_backup\backup.pyc", line 232, in patch >> File "rdiff_backup\rorpiter.pyc", line 284, in __call__ >> File "rdiff_backup\backup.pyc", line 622, in start_process >> File "rdiff_backup\longname.pyc", line 203, in get_mirror_inc_rps >> File "rdiff_backup\longname.pyc", line 189, in find_inc_pair >> File "rdiff_backup\longname.pyc", line 128, in check_new_index >> File "rdiff_backup\longname.pyc", line 111, in wrap_call >> File "rdiff_backup\longname.pyc", line 122, in make_parent >> File "rdiff_backup\rpath.pyc", line 964, in makedirs >> File "os.pyc", line 171, in makedirs >> >> Traceback (most recent call last): >> File "rdiff-backup", line 23, in <module> >> File "rdiff_backup\Main.pyc", line 302, in error_check_Main >> File "rdiff_backup\Main.pyc", line 322, in Main >> File "rdiff_backup\Main.pyc", line 278, in take_action >> File "rdiff_backup\Main.pyc", line 344, in Backup >> File "rdiff_backup\backup.pyc", line 38, in Mirror >> File "rdiff_backup\backup.pyc", line 232, in patch >> File "rdiff_backup\rorpiter.pyc", line 284, in __call__ >> File "rdiff_backup\backup.pyc", line 622, in start_process >> File "rdiff_backup\longname.pyc", line 203, in get_mirror_inc_rps >> File "rdiff_backup\longname.pyc", line 189, in find_inc_pair >> File "rdiff_backup\longname.pyc", line 128, in check_new_index >> File "rdiff_backup\longname.pyc", line 111, in wrap_call >> File "rdiff_backup\longname.pyc", line 122, in make_parent >> File "rdiff_backup\rpath.pyc", line 964, in makedirs >> File "os.pyc", line 171, in makedirs >> WindowsError: [Error 206] The filename or extension is too long: >> '\\\\127.0.0.1\\c$\\Documents and Settings\\ryan\\Local >> Settings\\Temp\\test_dir_2042584358/rdiff-backup-data/increments/Documents >> and Settings/ryan/Local >> Settings/Temp/test_dir_980382089/deep1/deep1_1/deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/deep1_1_1_1_1_1_1' >> >> >> >> >> >> _______________________________________________ >> rdiff-backup-users mailing list at rdiff-backup-users@... >> http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users >> Wiki URL: >> http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki >> > > > _______________________________________________ > rdiff-backup-users mailing list at rdiff-backup-users@... > http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users > Wiki URL: > http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki > _______________________________________________ rdiff-backup-users mailing list at rdiff-backup-users@... http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki |
|
|
Re: Native windows build - issue with long path names
by Andrew Ferguson-3
::
Rate this Message:
Well, here's the answer:
http://msdn.microsoft.com/en-us/library/aa365247.aspx Some thoughts: - I believe longname.py is designed just for handling individual filenames longer than 255 characters, but I could be wrong. - Unix systems seem to have a PATH_MAX of 1024 or 4096 ... Windows is less than 260, unless you use the Windows API (as opposed to ANSI C, like Python uses). This is why you have run into this bug, but there have been few (no?) reports of problems before. So, the next step would be to work out whether longname.py can handle situations in which the path name grows longer than PATH_MAX on Unix. If so, the logic can be built-on for Windows, if not, rdiff-backup will need work on that front. Then, a patch would need to be written which basically moves all file I/O to the Windows API, prepends the "\\?\" (or "\\?\UNC\"), and (perhaps) translates to Unicode. Yech. I think this is a star example of the stupidity of the Windows kernel & API -- that magic prepend business is ridiculous. :-/ some other references: http://msmvps.com/blogs/access/archive/2008/09/12/maximum-path-length-is-248-characters.aspx http://msdn.microsoft.com/en-us/library/aa363858(VS.85).aspx Andrew On Oct 11, 2008, at 10:31 PM, Josh Nisly wrote: > I looked into this a bit today. Windows does throw a different > exception on long file names than unix (a WindowsError (derived from > OSError, derived from EnvironmentError), with the errno set to 206, > which doesn't map to anything valid on unix.) However, when I added > a check for this to the places that already check for an > ENAMETOOLONG exception, it still failed. > > JoshN > > Ryan How wrote: >> Just to add, I should point out it is a fatal error in that the >> whole backup fails at that point. It isn't just those files that >> don't get backed up. >> >> Ryan How wrote: >>> Hi, >>> >>> I did a few searches, but I couldn't find any issues discussed >>> about long path names in the native windows build. >>> >>> I have been having issues with the 255 char limit (or round >>> abouts) in pathnames, but I am unsure as to why because I using >>> the native windows build (which shouldn't have this limit?). The >>> error originates in os.pyc (which, I have no python experience, >>> but isn't an rdiff-backup source file?). I get the same error when >>> using dos paths (ie. c:\blah) and UNC paths (which otherwise seem >>> to work well with the native windows build, yay). I could >>> understand the issue with the dos paths, but not with the UNC >>> paths. I have a feeling this may actually be a python issue? >>> >>> Error is below. >>> >>> Cheers, >>> >>> Ryan >>> >>> >>> Processing changed file Documents and Settings/ryan/Local Settings/ >>> Temp/test_dir_980382089/deep1/deep1_1/deep1_1_1/deep1_1_1_1/ >>> deep1_1_1_1_1/deep1_1_1_1_1_1/deep1_1_1_1_1_1_1/deep1_1_1_1_1_1_1_1 >>> Exception '[Error 206] The filename or extension is too long: '\\\ >>> \127.0.0.1\\c$\\Documents and Settings\\ryan\\Local Settings\\Temp\ >>> \test_dir_2042584358/rdiff-backup-data/increments/Documents and >>> Settings/ryan/Local Settings/Temp/test_dir_980382089/deep1/deep1_1/ >>> deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/ >>> deep1_1_1_1_1_1_1'' raised of class '<type >>> 'exceptions.WindowsError'>': >>> File "rdiff_backup\Main.pyc", line 302, in error_check_Main >>> File "rdiff_backup\Main.pyc", line 322, in Main >>> File "rdiff_backup\Main.pyc", line 278, in take_action >>> File "rdiff_backup\Main.pyc", line 344, in Backup >>> File "rdiff_backup\backup.pyc", line 38, in Mirror >>> File "rdiff_backup\backup.pyc", line 232, in patch >>> File "rdiff_backup\rorpiter.pyc", line 284, in __call__ >>> File "rdiff_backup\backup.pyc", line 622, in start_process >>> File "rdiff_backup\longname.pyc", line 203, in get_mirror_inc_rps >>> File "rdiff_backup\longname.pyc", line 189, in find_inc_pair >>> File "rdiff_backup\longname.pyc", line 128, in check_new_index >>> File "rdiff_backup\longname.pyc", line 111, in wrap_call >>> File "rdiff_backup\longname.pyc", line 122, in make_parent >>> File "rdiff_backup\rpath.pyc", line 964, in makedirs >>> File "os.pyc", line 171, in makedirs >>> >>> Traceback (most recent call last): >>> File "rdiff-backup", line 23, in <module> >>> File "rdiff_backup\Main.pyc", line 302, in error_check_Main >>> File "rdiff_backup\Main.pyc", line 322, in Main >>> File "rdiff_backup\Main.pyc", line 278, in take_action >>> File "rdiff_backup\Main.pyc", line 344, in Backup >>> File "rdiff_backup\backup.pyc", line 38, in Mirror >>> File "rdiff_backup\backup.pyc", line 232, in patch >>> File "rdiff_backup\rorpiter.pyc", line 284, in __call__ >>> File "rdiff_backup\backup.pyc", line 622, in start_process >>> File "rdiff_backup\longname.pyc", line 203, in get_mirror_inc_rps >>> File "rdiff_backup\longname.pyc", line 189, in find_inc_pair >>> File "rdiff_backup\longname.pyc", line 128, in check_new_index >>> File "rdiff_backup\longname.pyc", line 111, in wrap_call >>> File "rdiff_backup\longname.pyc", line 122, in make_parent >>> File "rdiff_backup\rpath.pyc", line 964, in makedirs >>> File "os.pyc", line 171, in makedirs >>> WindowsError: [Error 206] The filename or extension is too long: '\ >>> \\\127.0.0.1\\c$\\Documents and Settings\\ryan\\Local Settings\ >>> \Temp\\test_dir_2042584358/rdiff-backup-data/increments/Documents >>> and Settings/ryan/Local Settings/Temp/test_dir_980382089/deep1/ >>> deep1_1/deep1_1_1/deep1_1_1_1/deep1_1_1_1_1/deep1_1_1_1_1_1/ >>> deep1_1_1_1_1_1_1' >>> >>> >>> >>> >>> _______________________________________________ >>> rdiff-backup-users mailing list at rdiff-backup-users@... >>> http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users >>> Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki >>> >> >> >> _______________________________________________ >> rdiff-backup-users mailing list at rdiff-backup-users@... >> http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users >> Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki >> > > > > _______________________________________________ > rdiff-backup-users mailing list at rdiff-backup-users@... > http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users > Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki _______________________________________________ rdiff-backup-users mailing list at rdiff-backup-users@... http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki |
| Free Forum Powered by Nabble | Forum Help |
