re groupware, lotus notes

[bdogodev]

discuss-admin@... wrote on 13/06/2008 05:26:20:

Apologies to the list if this is all too long or too off-topic.  Hopefully
it provides some information about Notes.

> There also seems to be a powerful bias against Notes in some circles.
> Personally,  I've never run a production Notes server - I just have one
> here to poke at,  and I've read some documentation.  But I also read
> TechNet articles relating to Exchange.  None of these products are
> "simple".

I'm not sure where the hatred of Notes comes from, but you are right it is
widespread.  Partly it may be because visually the Notes client is unlike
other applications - at one point Notes was cross-platform (unix, mac,
windows, os/2, even DOS), and the interface didn't really conform to the
look-and-feel of  any particular platform.  Partly the hostility may be
because the Notes client is a complex piece of software, and that
complexity is hard to hide.  (A few people have advocated that there needs
to be the ability to strip it down to have the option to hide un-needed
complexity from users.)   IBM has long marketed Notes as being an email
system, yet if that is how it is being perceived/used by end users, then
there is definitely too much complexity in the client.  I don't believe
Notes itself is over-complex, since it can do so much.  But most end-users
probably don't need to see that complexity.  Maybe the way Notes is used
is to 'manage' people and working practices leads to people transferring
their hostility to the tool rather than the process itself.   Another
point is that it is probable that the majority of Notes developers were
never developers before they started using Notes - that was certainly true
of where I worked with Notes.  Until about  6 or 7 years ago, every Notes
client was also an IDE, and since the @formula language and the free-form
structure of Notes databases makes it very easy to start getting something
going, it was not unusual for someone to 'drift' into Notes development.
Consequently, many departments/businesses were able to get useful/needed
applications developed, often by people with no knowledge of programming
but with direct line-of-business knowledge, bypassing either their IT
department or even external contractors.  These applications don't even
require modification when the server or client is upgraded - apps
developed using the tools available in 1993 can be used in the latest
clients.  The really weird thing is that by starting the HTTP process of a
Notes server, applications that were built before the web can be served as
web applications with no modification (OK, but of course they look
rubbish).  Anyway, it is not unusual for large companies to have hundreds
of Notes applications, making it very hard for them to move off Notes even
if they stop using it for email.

But there's no doubt about it, Notes is a bizarre mixture of technologies
- a kind of swiss-army knife.  There are currently 5 programming languages
that can be used for various (over-lapping) areas: javascript, @formulas,
lotusscript (vb), java, and c.  And to do any development in Notes you
really have to know at least the @formulas.  So, many 'serious' programmer
will never take Notes seriously, because at some point they are going to
have to start doing the kind of development that was done by
non-programmers in things like Excel or 1-2-3.  But it is very powerful.
Besides the normal things you'd expect of a groupware application, I've
used Notes to write online payment systems (including integrating with
Paypal), software deployment/update systems, SQL replication, server admin
tools like Nagios, scraping and archiving of remote web sites.

Notes has a very flexible built-in cron like system, which will run code
not only at specific times, dates, periods etc. but also run it as
specific users, or on specific servers.  And not only can databases be
encrypted so that only a specific server or only a specific user can
access them, data can even be optionally encrypted so that only specific
fields are encrypted whilst the rest is unencrypted.  And it supports
about 5 different networking protocols (although these days there are few
uses for most except tcp/ip - although there are uses for e.g. having 1
notes server communicate with another via SPX or IPX, or even by Notes own
networking protocol where one server can dial up another modem-to-modem
without crossing the intenet at all).  The granularity of access control
is extraordinary: control extends down to the level of paragraphs, fields,
and even GUI controls such as section-expansion controls, and since the
directory is hierarchical, one can even use wildcards in specifying access
control e.g.allow editor access to everyone in  '*/finance/US/SomeCompany'
and author access to '*/US/SomeCompany'.  The administration of Notes is
also surprisingly easy - large enterprises can have hundreds of Notes
servers all interacting across the globe, yet administered by only a
couple of people.  And because of the quality of the replication system,
in over 10 years of working with Notes I've never taken a backup, even in
production environments.  After Sept 11th, I had responsibility for
building a disaster recovery system for my employers main Oracle based
application.  With Notes we just installed additional servers in the DR
site, and included them in the replication cycle.

Over the last 7 years or so IBM seems to have been mostly going in the
wrong direction with Notes.  They basically indicated that it was being
de-emphasized in place of Websphere Application Server, and even the
datastore was moving in the direction of being DB2-based rather than
free-form (yet here we are with the web development abuzz with the idea of
using non-relational databases such as CouchDB or Amazon SimpleDB).  IBM
de-emphasized all languages except Java, when even Java developers would
need to know at least the @formulas.  And now they have made the client
cross-platform again, by making the 'standard' configuration be
Eclipse-based, which means it needs about 1-2gb of RAM on a client OS to
get reasonable performance.  Bearing in mind that probably the majority of
Notes users use it for mail and calendaring, that's pretty onerous.

So even though Notes is second-nature to me, I don't want to work with
something where the people in control of the technology seem to have so
little understanding of the strengths of the technology, and so clueless
about its direction.  I couldn't be more passionate about the need for
RAD, replication, access control, encryption, etc.  But not if the only
way to get those things is through Eclipse on the client and J2EE on the
server.

> while it has more users certainly, doesn't seem to have nearly the
> active and open developer community that notes does
> <http://www.openntf.org/internal/home.nsf>

Actually, the open source, community spirit in Notes developemnt goes way
back before OpenNTF.  For probably 15 years or so Lotus hosted a "sandbox"
where developers contributed samples and useful pieces of code or even
whole applications (
http://www-10.lotus.com/ldd/sandbox.nsf/ByDate?OpenView&Start=849).  It is
not used so much in the last few years (people host code on their own
sites now, or through openNTF).  But you can see from that URL that there
are approx. 850 entries in the sandbox, the oldest of which is about 10
years old.  Because the code is included in the database, it is open
source (in Notes you have to go through an extra step to actually hide the
code in your application).

> > I think
> > that all businesses need groupware providing at the very minimum the
> > features of OGO.  The businesses just don't know it :-)
>
> "  I think that all businesses need groupware providing at the very
> minimum the features of OGO.  The businesses just don't know it :-)"
>
> Man, how many times have I said that during a pitch/presentation! :)
>
> Inevitably you'll have at least one of these <http://www.jwz.org/
> doc/groupware.html> in your audience.

I can't take that seriously as a critique of the irrelevance of a whole
area of application development. I grew tired of nightclubs at about the
same time that he grew tired of software development (
http://www.jwz.org/hacks/).  Seems from the article you referenced that
his major interest is getting laid.  Maybe he'll finally end up as
glamorous and successful as Peter Stringfellow (
http://en.wikipedia.org/wiki/Peter_Stringfellow).  I think that putting
'getting laid' as your number one motive can only appeal to those who
never got laid.  I'm assuming that no-one has ever seriously cited that as
a critique of groupware ;-)

The important thing is being able to demonstrate useful functionality to
users rather than explain to them what 'groupware' is.  The fact is that
after 25 years or so since Notes was created, most Notes users would still
probably just say that it is 'email' rather than 'groupware'.  Yet there
are other web applications that they might well use on a daily basis,
without realizing that Notes has been shipping with exactly those kinds of
applications since before the web existed (I'm thinking of blogs cf. the
Notes 'personal journal' app, and wikis cf. the Notes 'discussion
database').  Notes was almost predicated on the grounds of secure,
signable B2B communications via email and inter-networked apps. Slowly the
internet is building applications and functionality that existed in Notes
at least a decade before.  The whole business with tagging makes me laugh
- for more than a decade the Notes client has had the facility for users
to tag any document in any application with categories.  The irony is that
IBM often only sees something as useful when it is used on the internet,
and then wants to add it to Notes, apparently not realizing that Notes
already has those features, and that they were allowed to languish because
IBM had lost their vision for their own technology.

> Ohloh actually offers some convenient and easy to use tools.  Especially
> the ability to check if anyone is committing code to a project anymore.
> It also has some flaws such as really underestimating the activity of
> some projects depending on how they are structured.

I can see how the idea is useful in principle, but it's a pity if their
system for estimating activity is so flawed.  Although really, the metric
should not just be how much active committing is going on in a project,
but also the feature set, longevity, etc.  In terms of what OGO offers
there are very few open source competitors.

Bernard

--
OpenGroupware.org Discussion discuss@...
http://mail.opengroupware.org/mailman/listinfo/discuss

[Adam Tauno Williams-2]

>> Ohloh actually offers some convenient and easy to use tools.  Especially
>> the ability to check if anyone is committing code to a project anymore.
>> It also has some flaws such as really underestimating the activity of
>> some projects depending on how they are structured.
> I can see how the idea is useful in principle, but it's a pity if their
> system for estimating activity is so flawed.

But it may be the best/available possible;  code repositories and  
commit logs can, after all, only tell you so much.  The biggest flaw  
is that if N number of people create pataches, etc.. and they all  
committed by one or two people (gatekeepers) then Ohloh only sees one  
or two active participants.  They also, of course, can't take into  
account any subprojects, etc.., in orbit around a project.

> Although really, the metric
> should not just be how much active committing is going on in a project,
> but also the feature set, longevity, etc.

Ohloh takes those into account to the extend that they are available.  
But if someone moves their code repository from X to Y then Ohloh  
legitimately has no way of knowing that it existing for four years on  
X before it was listed from Y.

> In terms of what OGO offers there are very few open source competitors.

Yep.

--
OpenGroupware.org Discussion discuss@...
http://mail.opengroupware.org/mailman/listinfo/discuss

[Adam Tauno Williams-2]

> Apologies to the list if this is all too long or too off-topic.  Hopefully
> it provides some information about Notes.
> > There also seems to be a powerful bias against Notes in some circles.
> > Personally,  I've never run a production Notes server - I just have one
> > here to poke at,  and I've read some documentation.  But I also read
> > TechNet articles relating to Exchange.  None of these products are
> > "simple".
> Another point is that it is probable that the majority of Notes developers
> were never developers before they started using Notes - that was certainly
> true of where I worked with Notes.  Until about  6 or 7 years ago, every
> Notes client was also an IDE, and since the @formula language and the
> free-form structure of Notes databases makes it very easy to start getting
> something going, it was not unusual for someone to 'drift' into Notes
> development.

Much the same effect plagued the enterprise with the advent of Microsoft
Access and each department's PC jockey knocking together their own
little apps.   I spent years warring with Access apps and stomping it
out wherever it reared its pimply little head.  Fortunately those days
seem to have passed to some degree,  maybe because it just isn't so very
"cool" to be a PC jockey anymore.

> Consequently, many departments/businesses were able to get useful/needed
> applications developed, often by people with no knowledge of programming
> but with direct line-of-business knowledge, bypassing either their IT
> department or even external contractors.

>From an inside-IT perspective I felt the other side of this freedom.
When "the app" is broken or never backed up or the change not
documented...  That might explain some of the Notes bashing I hear from
IT people.

That, and at least once upon a time,  it apparently had stability and
performance issues.  At least according to people I know.  But I've
mostly assumed that was a historical fact carried forward although it
was no longer applicable.

> These applications don't even
> require modification when the server or client is upgraded

A real accomplishment on their part.

> rubbish).  Anyway, it is not unusual for large companies to have hundreds
> of Notes applications, making it very hard for them to move off Notes even
> if they stop using it for email.

Which is one of the advantages of making a development platform from
your app! :)  Maybe it isn't vendor lock-in, but is sure is
inertial-lock-in.

> Notes has a very flexible built-in cron like system, which will run code
> not only at specific times, dates, periods etc. but also run it as
> specific users, or on specific servers.  

Yea, OGo doesn't have that but I'd often thought it would be nice.
Allow the user to send e-mail / fax, etc.. starting at HH:MM, etc...
But without Kerberos or some such mechanism doing run-as-user is a very
hard thing to do in a secure fashion.

> And not only can databases be
> encrypted so that only a specific server or only a specific user can
> access them, data can even be optionally encrypted so that only specific
> fields are encrypted whilst the rest is unencrypted.

Having compression (and possibly encryption) in OGo would be nice.
Especially since document access is almost always pretty low volume.
But currently the DocumentAPI is one very scary chunk of code so I
haven't ventured into there.

> and author access to '*/US/SomeCompany'.  The administration of Notes is
> also surprisingly easy - large enterprises can have hundreds of Notes
> servers all interacting across the globe, yet administered by only a
> couple of people.  And because of the quality of the replication system,
> in over 10 years of working with Notes I've never taken a backup, even in
> production environments.

Interesting.  That is almost the exact opposite story told by Notes
haters.

> Over the last 7 years or so IBM seems to have been mostly going in the
> wrong direction with Notes.  They basically indicated that it was being
> de-emphasized in place of Websphere Application Server, and even the
> datastore was moving in the direction of being DB2-based rather than
> free-form (yet here we are with the web development abuzz with the idea of
> using non-relational databases such as CouchDB or Amazon SimpleDB).

They are currently in love with Websphere,  no doubt about that.  But in
their defense today's DB2 isn't yesterday's DB2.   Support for complex
datatypes and even non-structured database is much easier than it once
was;  the engines even support XML & XPath.  [We are an Informix/DB2
shop].  On the other-hand, if I had to build an app like OGo from
scratch today I'd probably use Db4o <http://www.db4o.com/>.  It has been
a real pleasure to work with.  The only thing that really worries me
about the non-RDBMS solutions is what happens when it is upgrade time,
and how smoothly they can deal with schema changes (or changes to the
object prototypes).   Currently I'm just using Db4o for caching so I
always have the option of just deleting the db.

> Eclipse-based, which means it needs about 1-2gb of RAM on a client OS to
> get reasonable performance.  Bearing in mind that probably the majority of
> Notes users use it for mail and calendaring, that's pretty onerous.

I've used eclipse,  I was pretty shocked by the Eclipse-as-a-client
direction.

> So even though Notes is second-nature to me, I don't want to work with
> something where the people in control of the technology seem to have so
> little understanding of the strengths of the technology, and so clueless
> about its direction.  I couldn't be more passionate about the need for
> RAD, replication, access control, encryption, etc.  But not if the only
> way to get those things is through Eclipse on the client and J2EE on the
> server.

What is used on the server side is pretty immaterial to me - I'll deal
with it,  I've already had to learn something like a dozen environments
[to varying degrees].   Just give me a platform neutral way to talk the
server and I'm happy.

> > > I think
> > > that all businesses need groupware providing at the very minimum the
> > > features of OGO.  The businesses just don't know it :-
> > "  I think that all businesses need groupware providing at the very
> > minimum the features of OGO.  The businesses just don't know it :-)"
> > Man, how many times have I said that during a pitch/presentation! :)
> > Inevitably you'll have at least one of these <http://www.jwz.org/
> > doc/groupware.html> in your audience.
> I can't take that seriously as a critique of the irrelevance of a whole
> area of application development.

I don't take is seriously intellectually,  but I have no choice but to
take it seriously politically.   You have to very explicitly make it
clear how scheduling / workflow / etc... is going to help *specifically*
THEM.  Because that is what (IMO) just rants are really about,  they are
narcissism in the corporate space - users who won't work with a system
that requires one iota of effort more than is absolutely necessary to do
the task they are directly held accountable for.   Facilitating your
co-workers job(s), data retention, etc... all mean nothing to that set
of users.  [ But I won over one of my most powerful anti-collaboration
users last week!  I showed him how to automatically make the system
harass people holding tasks from him.  He loved that and is now a
fervent advocate. :) ]

> The important thing is being able to demonstrate useful functionality to
> users rather than explain to them what 'groupware' is.  The fact is that
> after 25 years or so since Notes was created, most Notes users would still
> probably just say that it is 'email' rather than 'groupware'.  Yet there
> are other web applications that they might well use on a daily basis,
> without realizing that Notes has been shipping with exactly those kinds of
> applications since before the web existed (I'm thinking of blogs cf. the
> Notes 'personal journal' app, and wikis cf. the Notes 'discussion
> database').

But it isn't Web 2.0!!! :)

OGo lacks journal support which is, feature-wise, its biggest limitation
currently. IMHO.  The project system can almost be a Wiki itself.  I've
toyed around with planting Wicked in front of OGo
<http://www.horde.org/wicked/> projects.  But I haven't had the time to
flesh it out.  

Horde <http://www.horde.org/> is one of the nicest webmail-ish
interfaces available.  Horde works in a stable fashion with even the
most tortured e-mail messages which is an area where many webmail
interfaces fail quite badly.   I have a zOGI sub-project HordOGo
<http://code.google.com/p/zogi/wiki/HordOGo> where I'm slowly
refactoring my old XML-RPC code to the zOGI API.  I hope to get it
included into Horde itself at some point (they seem open to that, they
already support Kolab).  The advantage is that (a) Horde is more pretty
than the OGo WebUI and (b) it provides a MUCH reduced interface for the
users who just need mail and an address book.

> The whole business with tagging makes me laugh

Yes, but tagging is Web 2.0!!! :)

--
OpenGroupware.org Discussion discuss@...
http://mail.opengroupware.org/mailman/listinfo/discuss