Requesting comments: portable file names

[Duncan Lithgow-2]

Hi, I've made a start on a guide to using portable file names, that
is, file names for files you can move between different file systems,
operating systems and physical media without the names being corrupted
or lost.

I'd love some comments and help:
http://gramps-project.org/wiki/index.php?title=Portable_Filenames

Duncan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[Greg Troxel]

Starting to read, I am confused as to what the point is: how to choose
file names for any likely operating system, or also to support ISO9660
w/o Joliet or Rock Ridge?

You use the work "case sensitive", and say that this is just about
windows.  By default, Mac OS X comes with "case preserving" HFS+, where
files keep their original case, but upper and lower compare as equal.
This is different from "case insensitive" like Windows, I think.

You imply POSIX-compliant means case sensitive, but I'm not sure that's
true.  It would be good to have a reference.

POSIX defines a portable character set, with no lower case letters:

http://www.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap03.html#tag_03_276

If that set works everywhere else, it might be good just to recommend
it.

To me, this wiki entry has an awful lot of content, and simply saying
"to be portable use only this set, and see these references for why"
would be more likely to accomplish the goal.

space is never safe; it's awkward and therefore error-prone in
command-line environments.

~ is a shell metacharacter in unix; it shouldn't be used in filenames.


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[steve_geo]

Hi Duncan,
> Hi, I've made a start on a guide to using portable file names, that
> is, file names for files you can move between different file systems,
> operating systems and physical media without the names being corrupted
> or lost.
>  
Another take on portability can be found here, I think it covers most
the same material you already covered.
http://www.controlledvocabulary.com/imagedatabases/filename_limits.html

I will also re-assert that 'spaces' in filenames (and directories) are a
bad thing, just don't do it. If you do, then get out of the habit of
using them.


In another thread you spoke of applying the 'safe characters' to create
meaningful names.

>The next step is to use the available characters to make meaningful file names, for example:

>[birt][plac]London,_England[date]1974-03-12_[name]Smith,_John_[note]birth_certificate.jpg

Have a look at
http://www.controlledvocabulary.com/imagedatabases/filenaming.html   for
their take on managing image names. (keeps with 8.3 format)

Slightly off topic and not directly related to this topic, there is
aways the point of view meaningful labeling should also be inserted into
the image itself, using IPTC/XMP standards
http://www.rideau-info.com/genealogy/digital/label.html

Steve


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[Gerald Britton-2]

I would advocate sticking to alphanumerics with a few extras like
period, hyphen, underscore, dollar sign. I wouldn't support 8.3
however. Also, remember that the filenames probably have to be
case-insensitve to work on windows.


On 7/20/08, Stephen George <steve_geo@...> wrote:

> Hi Duncan,
>> Hi, I've made a start on a guide to using portable file names, that
>> is, file names for files you can move between different file systems,
>> operating systems and physical media without the names being corrupted
>> or lost.
>>
> Another take on portability can be found here, I think it covers most
> the same material you already covered.
> http://www.controlledvocabulary.com/imagedatabases/filename_limits.html
>
> I will also re-assert that 'spaces' in filenames (and directories) are a
> bad thing, just don't do it. If you do, then get out of the habit of
> using them.
>
>
> In another thread you spoke of applying the 'safe characters' to create
> meaningful names.
>
>>The next step is to use the available characters to make meaningful file
>> names, for example:
>
>>[birt][plac]London,_England[date]1974-03-12_[name]Smith,_John_[note]birth_certificate.jpg
>
> Have a look at
> http://www.controlledvocabulary.com/imagedatabases/filenaming.html   for
> their take on managing image names. (keeps with 8.3 format)
>
> Slightly off topic and not directly related to this topic, there is
> aways the point of view meaningful labeling should also be inserted into
> the image itself, using IPTC/XMP standards
> http://www.rideau-info.com/genealogy/digital/label.html
>
> Steve
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Gramps-users mailing list
> Gramps-users@...
> https://lists.sourceforge.net/lists/listinfo/gramps-users
>

--
Sent from Gmail for mobile | mobile.google.com

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[Nigel Horne]

> I would advocate sticking to alphanumerics with a few extras like
> period, hyphen, underscore, dollar sign. I wouldn't support 8.3
> however. Also, remember that the filenames probably have to be
> case-insensitve to work on windows.

Dollar signs would not be a good idea because of its use as a reserved
character in the *nix shell.

-Nigel


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[Brad Rogers]

On Sun, 20 Jul 2008 21:51:07 -0400
"Gerald Britton" <gerald.britton@...> wrote:

Hello Gerald,

> I would advocate sticking to alphanumerics with a few extras like
> period, hyphen, underscore, dollar sign. I wouldn't support 8.3

Hyphen & $ have uses in various command line environments.  Both are
best avoided in file names.

--
 Regards  _
         / )           "The blindingly obvious is
        / _)rad        never immediately apparent"

The public gets what the public wants
Going Underground - The Jam


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[Duncan Lithgow-2]

2008/7/21 Brad Rogers <brad@...>:
> Hyphen & $ have uses in various command line environments.  Both are
> best avoided in file names.
Could one of you find a reference for me on that? Especially about the hyphen...

My point with this page is that I need to understand the information
in order to sort out my own data and make a suggestion that's useful
for others. I know the page is big and messy, I'm working on it. One
reason it's so big is that I haven't foudn a similar resource, now
I'll look at those links and see what they can provide.

As for portability, I want a character set which works all the likely
places my files could end up, CD/DVD, servers, USB keys, Windows
machines...

Thanks for the good response

Duncan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[Gerald Britton-2]

You can go here:

http://www.gnu.org/software/bash/manual/

On Mon, Jul 21, 2008 at 12:55 PM, Duncan Lithgow <dlithgow@...> wrote:

> 2008/7/21 Brad Rogers <brad@...>:
>> Hyphen & $ have uses in various command line environments.  Both are
>> best avoided in file names.
> Could one of you find a reference for me on that? Especially about the hyphen...
>
> My point with this page is that I need to understand the information
> in order to sort out my own data and make a suggestion that's useful
> for others. I know the page is big and messy, I'm working on it. One
> reason it's so big is that I haven't foudn a similar resource, now
> I'll look at those links and see what they can provide.
>
> As for portability, I want a character set which works all the likely
> places my files could end up, CD/DVD, servers, USB keys, Windows
> machines...
>
> Thanks for the good response
>
> Duncan
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Gramps-users mailing list
> Gramps-users@...
> https://lists.sourceforge.net/lists/listinfo/gramps-users
>

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[Brad Rogers]

On Mon, 21 Jul 2008 18:55:44 +0200
"Duncan Lithgow" <dlithgow@...> wrote:

Hello Duncan,

> Could one of you find a reference for me on that? Especially about
> the hyphen...

Gerald's done that, I believe.

> As for portability, I want a character set which works all the likely
> places my files could end up, CD/DVD, servers, USB keys, Windows

In which case, I'd also suggest avoiding accented characters, too.  In
fact, avoid characters that aren't in the basic ASCII set.

--
 Regards  _
         / )           "The blindingly obvious is
        / _)rad        never immediately apparent"

You don't entertain ideas you simply bore them
I Don't Like You - Stiff Little Fingers


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[Duncan Lithgow-2]

Ok, the page is now heavily revised and much shorter, also much more
compact and direct. Thanks for the help.

What do you think?

I know the file name length limit of 31 is rather conservative, but
more is hardly needed as far as I can see.

Next step is a semantic controlled vocabulary. I'm thinking of
something like this:

_PLAC--   = place marker
_INDV--    = individual marker
_EVNT--  = event marker
_DATE--   = date marker
_SOUR--   = source marker
_SURN--   = family name marker
_FIRS--    = first name marker
_NOTE--   = note marker
_          = space indicator
__        = comma followed by space indicator

First these abbreviations need to be matched to those of GEDCOM for consistency

This would give file names like

_EVNT--marriage_SURN--jones_FIRS--mary_angus__SURN--williams_FIRS--matthew_DATE--1923-12-02_NOTE--william_angus_to_right_of_mary.jpg

This could be parsed (by GRAMPS?) as the description:

* Event: Marriage
* Surname: Jones
* Firstname: Mary Angus
* Surname: Williams
* Firstname: Matthew
* Date: 2nd Jan, 1923
* Note: William angus to the right of mary

or the text:

Mary Angus Jones and Matthew Williams, marriage 2nd Jan 1923. (William
angus to the right of mary)

Comments?

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[Martin Ewing]

Duncan,

You asked for thoughts.  Let me share a few.

It seems your intention is to use the filesystem's namespace in what
looks (to me) to be a strained way.  That is, the name is a long
string that tries to summarize the context if not the content of a
file.   There are redundant elements from one file name to the next.
(Maybe I don't understand your intent, but it seems you value "human
readable" filenames more than I might.)

I can think of at least two other approaches.  The simplest, though
less friendly, would be to name the files in a simple way: g00001,
g00002, etc.  Then provide an index file with as much explanatory
information as needed.  You could have a simple gramps filebrowser
application that would let users see everything together in full
glory.  (Well, that's what gramps *is*, right? ;)

A different approach is to use folders to categorize files.  So you
would have key1/.../keyN/filename.  But maybe you prefer a "flat" set
of files?  Of course you would wrap everything in a tar or zip file
for easy transport.  In my system, I tend to group files in folders by
person.

Even if transport is not an issue, it's hard to have a single naming
or folder scheme that covers all cases.  What about a jpeg that has
two cousins in it? ;)  You might want logical links so that one file
can show up in two places.

The file system is an elementary database.  Mapping a complex db into
it is tricky.  And, the lowest common denominator (like isofs) will
have the hardest names thing to work with.

I don't know which is best, but I'm not sure about long filenames . As
you point out, some OSes don't much like them either!

Cheers,
Martin

On Mon, Jul 21, 2008 at 4:06 PM, Duncan Lithgow <dlithgow@...> wrote:

> Ok, the page is now heavily revised and much shorter, also much more
> compact and direct. Thanks for the help.
>
> What do you think?
>
> I know the file name length limit of 31 is rather conservative, but
> more is hardly needed as far as I can see.
>
> Next step is a semantic controlled vocabulary. I'm thinking of
> something like this:
>
> _PLAC--   = place marker
> _INDV--    = individual marker
> _EVNT--  = event marker
> _DATE--   = date marker
> _SOUR--   = source marker
> _SURN--   = family name marker
> _FIRS--    = first name marker
> _NOTE--   = note marker
> _          = space indicator
> __        = comma followed by space indicator
>
> First these abbreviations need to be matched to those of GEDCOM for consistency
>
> This would give file names like
>
> _EVNT--marriage_SURN--jones_FIRS--mary_angus__SURN--williams_FIRS--matthew_DATE--1923-12-02_NOTE--william_angus_to_right_of_mary.jpg
>
> This could be parsed (by GRAMPS?) as the description:
>
> * Event: Marriage
> * Surname: Jones
> * Firstname: Mary Angus
> * Surname: Williams
> * Firstname: Matthew
> * Date: 2nd Jan, 1923
> * Note: William angus to the right of mary
>
> or the text:
>
> Mary Angus Jones and Matthew Williams, marriage 2nd Jan 1923. (William
> angus to the right of mary)
>
> Comments?
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Gramps-users mailing list
> Gramps-users@...
> https://lists.sourceforge.net/lists/listinfo/gramps-users
>

--
Martin Ewing, AA6E
Branford, CT

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[steve_geo]

Duncan Lithgow wrote:
> 2008/7/21 Brad Rogers <brad@...>:
>  
>> Hyphen & $ have uses in various command line environments.  Both are
>> best avoided in file names.
>>    
> Could one of you find a reference for me on that? Especially about the hyphen...
>  
http://www.portfoliofaq.com/pfaq/FAQ00352.htm
Have a look at notes #2 and #14

Steve

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[Duncan Lithgow-2]

2008/7/21 Martin, AA6E <martin.s.ewing@...>:
> Duncan,
>
> You asked for thoughts.  Let me share a few.
>
> It seems your intention is to use the filesystem's namespace in what
> looks (to me) to be a strained way.  That is, the name is a long
> string that tries to summarize the context if not the content of a
> file.   There are redundant elements from one file name to the next.
What do you mean by 'redundant elements'?

> (Maybe I don't understand your intent, but it seems you value "human
> readable" filenames more than I might.)
That could be right, read on...

> I can think of at least two other approaches.  The simplest, though
> less friendly, would be to name the files in a simple way: g00001,
> g00002, etc.
Certainly the easiest approach.

> Then provide an index file with as much explanatory
> information as needed.  You could have a simple gramps filebrowser
> application that would let users see everything together in full
> glory.  (Well, that's what gramps *is*, right? ;)
Well, no, that's not really what gramps is. The connection between
GRAMPS and the media files is pretty weak in my opinion. It doesn't
take much to break it and end up with a meaningful database which has
lost it's links to the files, and a bunch of files which have lost
their meaning.

_If_ GRAMPS began storing some information in EXIF or other meta data
I would relax a bit more. Or any one of many other similar approaches.
As it stands I'm nervous about the stability of the link between
GRAMPS and my 500 and growing media files.

> A different approach is to use folders to categorize files.  So you
> would have key1/.../keyN/filename.  But maybe you prefer a "flat" set
> of files?  Of course you would wrap everything in a tar or zip file
> for easy transport.
It occured to me earlier today - why doesn't GRAMPS wrap all linked
media into a .tar file itself? Then it could assign meta-data and
who-knows-what without being dependent on the file systems restriction
on meta data.

> Even if transport is not an issue, it's hard to have a single naming
> or folder scheme that covers all cases.  What about a jpeg that has
> two cousins in it? ;)
I haven't hit that problem as much as I thought. If there are really a
lot of people I use only the surnames represented. Still too many?
Drop the ones not in my database.

> The file system is an elementary database.  Mapping a complex db into
> it is tricky.  And, the lowest common denominator (like isofs) will
> have the hardest names thing to work with.
I'm not trying to map the whole GRAMPS data set. That would indeed be
foolish. I just want to be able to send my files to people so they can
see what they are, find my files without having to start gramps and
things like that. I want to be able to send a folder of files about a
specific surname to someone without them having to install gramps to
know what the files contain...

> I don't know which is best, but I'm not sure about long filenames . As
> you point out, some OSes don't much like them either!

Thanks for your comments, keep them coming. You'll see I've started a
new thread.

Duncan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[Martin Ewing]

Duncan,

A naive question, perhaps, but why are there any auxiliary files at
all?  Why shouldn't everything be incorporated in the database (jpegs,
audios, PhD theses, what have you.)?  Is it a technical limitation
(size?), or is there a philosophical issue?  An integrated db would
reduce the "file placement" problem.

One reason that occurs to me is that much of my research data is
tentative and possibly even contradictory, so I wouldn't want it to be
presented on an equal level with the better verified information.
Still, there should be a way to handle that within the db, using some
kind of "quality" tiering.

Also, if "media" files are separate, it's easier to work on them with
OS-specific tools.

Martin

On Mon, Jul 21, 2008 at 6:57 PM, Duncan Lithgow <dlithgow@...> wrote:
> 2008/7/21 Martin, AA6E <martin.s.ewing@...>:
>> Duncan,
>>
>> You asked for thoughts.  Let me share a few.
>>


--
Martin Ewing, AA6E
Branford, CT

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users

[Duncan Lithgow-2]

2008/7/22 Martin, AA6E <martin.s.ewing@...>:

> Duncan,
>
> A naive question, perhaps, but why are there any auxiliary files at
> all?  Why shouldn't everything be incorporated in the database (jpegs,
> audios, PhD theses, what have you.)?  Is it a technical limitation
> (size?), or is there a philosophical issue?  An integrated db would
> reduce the "file placement" problem.
>
> One reason that occurs to me is that much of my research data is
> tentative and possibly even contradictory, so I wouldn't want it to be
> presented on an equal level with the better verified information.
> Still, there should be a way to handle that within the db, using some
> kind of "quality" tiering.
>
> Also, if "media" files are separate, it's easier to work on them with
> OS-specific tools.

I'm not really the person to ask. I'm just trying to deal with the
current reality. I could see large problems with pulling all data into
the database - but into a TAR file? That might  be do-able...

Duncan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Gramps-users mailing list
Gramps-users@...
https://lists.sourceforge.net/lists/listinfo/gramps-users