OSX considered harmfull

[Wichert Akkerman]

We're seeing more problems with releases made on OSX: those contain
._ files with the OSX resource fork information. Normally that is a
cosmetic problem only, but for translations it is worse: Zope will
abort with an error when it encounters those in
locales/<lanage>/LC_MESSAGES.

So please: double-check if your upload does not contain those before
uploading it.

Wichert.

--
Wichert Akkerman <wichert@...>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
Plone-developers@...
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Wichert Akkerman]

Previously Wichert Akkerman wrote:
> We're seeing more problems with releases made on OSX: those contain
> ._ files with the OSX resource fork information. Normally that is a
> cosmetic problem only, but for translations it is worse: Zope will
> abort with an error when it encounters those in
> locales/<lanage>/LC_MESSAGES.
>
> So please: double-check if your upload does not contain those before
> uploading it.

And for those of you who have upgraded to Leopard: apparently
COPY_EXTENDED_ATTRIBUTES_DISABLE no longer works - you have to use
COPYFILE_DISABLE now.

Wichert.

--
Wichert Akkerman <wichert@...>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
Plone-developers@...
https://lists.sourceforge.net/lists/listinfo/plone-developers

[grahamperrin]

._ files with the OSX resource fork information.

If a file on NTFS or ZFS or HFS+ has a resource fork (a rarity), you can strip that fork before copying, but even if you:

* copy an unforked file
* using Finder
* using a protocol, or to a filing system, that lacks support

then you will probably find, to your surprise, a
._ dot underscore file at the destination

-- at least, for as long as the file is being copied.

For a more detailed explanation, see
<http://www.plone4artists.org/products/plone4artistsaudio/issues/32>.

These are not bugs in Finder or Mac OS X. More often, people simply misunderstand the richness of filing systems such as HFS+ ... Finder is simply doing its job, ensuring that data/metadata is not lost.

When I last considered alternatives/complements to Finder, muCommander was most interesting.
Mentioned at <http://groups.google.com/group/MacFusion-devel/browse_frm/thread/8af1d90b40c4d97c/>.

FWIW in a broader WebDAV context I hope that
wdfs <http://noedler.de/projekte/wdfs/> may be preferable to
/System/Library/Filesystems/webdav.fs/
in Mac OS X 10.4. If you're interested in this aspect, see
<http://groups.google.com/group/MacFusion-devel/browse_frm/thread/2089d41bae8fb1a8> and
<http://groups.google.com/group/macfuse-devel/browse_frm/thread/26a891bb3143701c>.

Best regards

Graham
MacFusion project

[Ricardo Newbery]

On Nov 14, 2007, at 8:38 PM, Graham Perrin wrote:

>
>
>
>> ._ files with the OSX resource fork information.
>>
>
> If a file on NTFS or ZFS or HFS+ has a resource fork (a rarity),  
> you can
> strip that fork before copying, but even if you:
>
> * copy an unforked file
> * using Finder
> * using a protocol, or to a filing system, that lacks support
>
> then you will probably find, to your surprise, a
> ._ dot underscore file at the destination
>
> -- at least, for as long as the file is being copied.
> For a more detailed explanation, see
> <http://www.plone4artists.org/products/plone4artistsaudio/issues/32>.
>
> These are not bugs in Finder or Mac OS X. More often, people simply
> misunderstand the richness of filing systems such as HFS+ ...  
> Finder is
> simply doing its job, ensuring that data/metadata is not lost.

I'm not sure but I believe the issue Wichert is referring to is *not*  
the behavior we have to deal with when copying files from an HFS+  
file system.  He's not talking about file transfer but rather the  
packaging up of releases.

The issue is with the use of the 'tar' function on the OS X command  
line.  From what I understand, the tar function is creating these ._*  
files in order to preserve the resource forks.  If this is not what  
you want, one solution is to add the following line to your .profile  
file:

export COPY_EXTENDED_ATTRIBUTES_DISABLED=true

or if you're on Leopard:

export COPYFILE_DISABLE=true

Ric






-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
Plone-developers@...
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Wichert Akkerman]

Previously Graham Perrin wrote:

>
>
>
> > ._ files with the OSX resource fork information.
> >
>
> If a file on NTFS or ZFS or HFS+ has a resource fork (a rarity), you can
> strip that fork before copying, but even if you:
>
> * copy an unforked file
> * using Finder
> * using a protocol, or to a filing system, that lacks support
>
> then you will probably find, to your surprise, a
> ._ dot underscore file at the destination
>
> -- at least, for as long as the file is being copied.
>
> For a more detailed explanation, see
> <http://www.plone4artists.org/products/plone4artistsaudio/issues/32>.
>
> These are not bugs in Finder or Mac OS X. More often, people simply
> misunderstand the richness of filing systems such as HFS+ ... Finder is
> simply doing its job, ensuring that data/metadata is not lost.

I understand the richness of the filesystem very well. What I don't
understand is why OSX tools prefer to create lots of special files
instead of using standard options to hide them. For example the tar
format is quite flexible and allows you to add custom block types. If
Apple had used that this problem would not exist.

Wichert.

--
Wichert Akkerman <wichert@...>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
Plone-developers@...
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Ricardo Newbery]

On Nov 14, 2007, at 11:32 PM, Wichert Akkerman wrote:

> I understand the richness of the filesystem very well. What I don't
> understand is why OSX tools prefer to create lots of special files
> instead of using standard options to hide them. For example the tar
> format is quite flexible and allows you to add custom block types. If
> Apple had used that this problem would not exist.
>
> Wichert.


It's my understanding that the metadata support in tar that was added  
in 10.4 is still experimental, but I believe the intent was to plug  
into the same API that the other OS X tools use to keep track of this  
metadata.  It probably seemed natural to represent the metatadata  
using the AppleDouble format, which after all has been around for  
awhile and is supported by a host of other tools.

Perhaps, as you say, it might have been useful to add a "custom block  
type" to handle this.  But not knowing the technical issues involved,  
I'm hesitant to criticize these sorts of decisions.  I understand  
that the regular tar utility doesn't even *see* the metadata because  
it doesn't know how to get this information from the HFS+ filesystem  
API.  So they need to add this ability to tar.  I suspect that in  
order to get two-way translation of this metadata, you probably need  
a sane fallback in case someone uses a non-apple tar on a non-HFS+  
filesystem, so the AppleDouble format with hidden files may make a  
lot of sense.  Could you even get this sort of sane fallback using  
custom block types?

What I think they fumbled on is not providing some sort of command  
line option to turn off the retention of metadata when using tar.  
Using an environment variable to turn this off seems a little  
clumsy.  I noticed that the Mac version of rsync has an "extended-
attributes" option "-E" that turns on support for metadata.

Ric



-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
Plone-developers@...
https://lists.sourceforge.net/lists/listinfo/plone-developers

[grahamperrin]

Ricardo Newbery wrote:

What I think they fumbled on is not providing some sort of command line option to turn off the retention of metadata when using tar. Using an environment variable to turn this off seems a little
clumsy.  

The variables are definitely useful (thanks wichert) but I might hesitate before adding to my
~/.profile

I noticed that the Mac version of rsync has an "extended-attributes" option "-E" that turns on support for metadata.

Similarly in Mac OS X 10.5, for the cp command
<http://developer.apple.com/documentation/Darwin/Reference/ManPages/man1/cp.1.html>

-X    Do not copy Extended Attributes (EAs) or resource forks.

So users of 10.5 who are less keen on
COPYFILE_DISABLE
might instead
cp -XR
then tar the result.

-X is not an option for cp in 10.4, unfortunately.

[grahamperrin]

For archiving with special attention to resource forks and extended attributes, there's also the lesser-known
ditto
but I don't know whether CPIO or PKZip archiving are acceptable alternatives to tar.

From my Mac OS X 10.4.10 man for ditto:

     --norsrc      do not preserve resource forks and HFS meta-data.  If both
                   --norsrc and --rsrc are passed, whichever is passed last
                   will take precedence.  Both options override DITTONORSRC.

man for the more recent version of ditto is at
<http://developer.apple.com/DOCUMENTATION/Darwin/Reference/ManPages/man1/ditto.1.html> and here's another option

--noextattr

[grahamperrin]

wichert wrote:

Zope will abort with an error when it encounters [ ._ files] in
locales/<lanage>/LC_MESSAGES.

If
._
is problematic but
.DS_Store
is not problematic, then muCommander
<http://www.mucommander.com/> may be useful.

muCommander is apparently blind to resource forks,
<http://www.mucommander.com/forums/viewtopic.php?t=509>.

A brief test this morning showed me that .tar created by muCommander:

* includes .DS_Store files
* excludes ._ files.  

I'll quieten down now, that's more than enough!

[Wichert Akkerman]

Previously Graham Perrin wrote:
> If
> ._
> is problematic but
> .DS_Store
> is not problematic, then muCommander
> <http://www.mucommander.com/> may be useful.

.DS_Store is just as bad.

Wichert.

--
Wichert Akkerman <wichert@...>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
Plone-developers@...
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Steve McMahon]

 > So users of 10.5 who are less keen on
 > COPYFILE_DISABLE
 > might instead
 > cp -XR
 > then tar the result.

I'm pretty sure that Leopard's tar is generating and adding the ._*
files itself. So, this is probably not a solution.

Graham Perrin wrote:

>
> Ricardo Newbery wrote:
>> What I think they fumbled on is not providing some sort of command line
>> option to turn off the retention of metadata when using tar. Using an
>> environment variable to turn this off seems a little
>> clumsy.  
>>
>
> The variables are definitely useful (thanks wichert) but I might hesitate
> before adding to my
> ~/.profile
>
>
>
>> I noticed that the Mac version of rsync has an "extended-attributes"
>> option "-E" that turns on support for metadata.
>>
>
> Similarly in Mac OS X 10.5, for the cp command
> <http://developer.apple.com/documentation/Darwin/Reference/ManPages/man1/cp.1.html>
>
> -X    Do not copy Extended Attributes (EAs) or resource forks.
>
> So users of 10.5 who are less keen on
> COPYFILE_DISABLE
> might instead
> cp -XR
> then tar the result.
>
> -X is not an option for cp in 10.4, unfortunately.
>

--
______________________________________________________

Steve McMahon
Reid-McMahon, LLC
steve@...
steve@...

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
Plone-developers@...
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Marco De Vitis]

Il 15/11/2007 15:02, Graham Perrin ha scritto:

> If
> ._
> is problematic but
> .DS_Store
> is not problematic, then muCommander
> <http://www.mucommander.com/> may be useful.

Sorry for being so late, but... if this thread was meant for developers
going to pack and upload their products, and developers are ok with
using a GUI app like muCommander for packing, then why not CleanArchiver?
http://www.sopht.jp/cleanarchiver/
It can exclude all OSX special files from the archives it creates.

--
Ciao,
   Marco.


-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Plone-developers mailing list
Plone-developers@...
https://lists.sourceforge.net/lists/listinfo/plone-developers