Code scanning issues

[Jess Holle]

We have numerous projects, some of which are very large.  Some of these involve not only large jars, but also large loose codebase directories in their classpaths.

With this reality (this is not going to change), NetBeans has in one respect continuously gotten worse despite its numerous improvements: the IDE is ever more insistent on scanning all of these directories before letting you do real work -- even in 6.1 where this is purported not to be the case.  Also in cases where one rebuilding the output jar in a project upon which many, many other open projects depend the IDE will decide that their sources needs to be rescanned in their entirety in cases.  This is *not* a recipe for large code set scalability.

Perhaps we're alone in this boat -- or perhaps others in it have already jumped ship to Eclipse with its incremental compilation logic, but I wanted to raise the issue for community discussion.

In my book I think the IDE needs to provide more control over when code scanning is done at a bare minimum -- especially for those unfortunate enough to have loose codebase directories.  It should also likely provide some control over the priority the background code scanning thread is given -- as this thread has been aggressive enough to render the remainder of the IDE barely usable.  There's also got to be a better way than rescanning all downstream projects in their entirety when you rebuild a project upon which many others depend.  Finally, I'd like to see the IDE live up to the promises in the 6.1 release notes and allow me to do "Go to Type", etc, with existing code completion information without waiting for all scanning to complete.

On top of all this, I really think "Go to Type", etc, should still be faster, though I certainly recognize progress in this regard.  That said, as a point of comparison I've used NetBeans' classfile module to analyze all our class files, store cross-references in a relational database (with appropriate indexing, of course), and then fronted this with a web app.  The results of "Go to Type" type lookups (and many others) are very fast in all cases.  I guess I should integrate that with the IDE, but I'd think the IDE should do this sort of thing better than my somewhat ad hoc web app!  [Also I limit the fidelity of my Find Usages to which class is using another and don't record/divulge which lines the usages occur upon.]

Am I alone in having issues in this regard?

--
Jess Holle

[Giampaolo Melis]

I agree with you. We have had similar problems with our projects. The
main is about 7000 files considering only the core modules.
We have a big source tree starting at the top of a Clearcase
repository. Many improvements have been done since 5.0 (the first I've
really tested) but with 6.1 we're still experiencing problems with
projects using external source directories and file filters.
I've tried with snapshot view (a faster clearcase repository view) and
also copying files on local disk but sometimes the IDE freezes for
several seconds, probably to refresh his class cache.
In the 6.1 I've also found a bug creating projects with external
sources: The editor does not consider the sources like if they were in
the classpath. No code completion is available and lots of error are
signaled but the build operation (the ant task) goes well.
I will probably submit an issue for this 'cause I've not had answers
to the regular post.

Regards.

Giampaolo

On Tue, May 13, 2008 at 4:24 AM, Jess Holle <jessh@...> wrote:

> We have numerous projects, some of which are very large.  Some of these
> involve not only large jars, but also large loose codebase directories in
> their classpaths.
>
> With this reality (this is not going to change), NetBeans has in one respect
> continuously gotten worse despite its numerous improvements: the IDE is ever
> more insistent on scanning all of these directories before letting you do
> real work -- even in 6.1 where this is purported not to be the case.  Also
> in cases where one rebuilding the output jar in a project upon which many,
> many other open projects depend the IDE will decide that their sources needs
> to be rescanned in their entirety in cases.  This is *not* a recipe for
> large code set scalability.
>
> Perhaps we're alone in this boat -- or perhaps others in it have already
> jumped ship to Eclipse with its incremental compilation logic, but I wanted
> to raise the issue for community discussion.
>
> In my book I think the IDE needs to provide more control over when code
> scanning is done at a bare minimum -- especially for those unfortunate
> enough to have loose codebase directories.  It should also likely provide
> some control over the priority the background code scanning thread is given
> -- as this thread has been aggressive enough to render the remainder of the
> IDE barely usable.  There's also got to be a better way than rescanning all
> downstream projects in their entirety when you rebuild a project upon which
> many others depend.  Finally, I'd like to see the IDE live up to the
> promises in the 6.1 release notes and allow me to do "Go to Type", etc, with
> existing code completion information without waiting for all scanning to
> complete.
>
> On top of all this, I really think "Go to Type", etc, should still be
> faster, though I certainly recognize progress in this regard.  That said, as
> a point of comparison I've used NetBeans' classfile module to analyze all
> our class files, store cross-references in a relational database (with
> appropriate indexing, of course), and then fronted this with a web app.  The
> results of "Go to Type" type lookups (and many others) are very fast in all
> cases.  I guess I should integrate that with the IDE, but I'd think the IDE
> should do this sort of thing better than my somewhat ad hoc web app!  [Also
> I limit the fidelity of my Find Usages to which class is using another and
> don't record/divulge which lines the usages occur upon.]
>
> Am I alone in having issues in this regard?
>
> --
> Jess Holle
>
>