Status update: modularization

[Braden McDaniel]

As I mentioned in a previous update, the primary feature I want to have
in place for 0.17 is the fragmentation of libopenvrml into dynamically
loaded modules. I've started some work on this.

There will be two kinds of modules: node components and scripting
engines. I've started with the latter because it is more obvious to me
how scripting engine modules should work; node component modules are
going to require an XML descriptor and inter-component dependencies and
it's not yet clear to me what the desirable mechanics are for that.

At this point I have successfully ported the JavaScript scripting
backend to a dynamically loaded module; it seems to be working just
fine. I would like to take this opportunity to fix the long-ailing Java
scripting engine. I'm just now getting an idea of what I'm in for.

It hasn't been compiling for a while; but that's not the worst of the
problems. It is currently written assuming that all Java scripts in a
world would share the same VM. I don't think that's desirable; and I'm
even less sure that it's workable: it's my understanding that each
vrml.node.Script subclass will need to have its base URI added to the
classpath; and surely different Scripts need to run in isolation from
one another.

So, proceeding with the assumption that we need a VM-per-Script node,
the JNI documentation says that you can have no more than one VM per
thread. That means that the openvrml::script subclass in the Java script
  engine needs to spawn its own thread and function more-or-less as an
active object. This is nontrivial code, none of which is in place.

I'm not throwing my hands up in dismay; but getting
Java-in-the-Script-node working is going to take longer than I'd
anticipated. So while I'd hoped to check in JavaScript and Java
scripting engines at the same time, I'm now leaning toward just checking
in the JavaScript one for now. The Java stuff will most likely be going
away for a little while so I can rework it. Given the time that's
elapsed since anyone tried to compile this, I doubt it will be missed.

Braden

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
openvrml-develop mailing list
openvrml-develop@...
https://lists.sourceforge.net/lists/listinfo/openvrml-develop

[Reed Hedges]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Braden McDaniel wrote:

>
> So, proceeding with the assumption that we need a VM-per-Script node,
> the JNI documentation says that you can have no more than one VM per
> thread. That means that the openvrml::script subclass in the Java script
>   engine needs to spawn its own thread and function more-or-less as an
> active object. This is nontrivial code, none of which is in place.
>
> I'm not throwing my hands up in dismay; but getting
> Java-in-the-Script-node working is going to take longer than I'd
> anticipated. So while I'd hoped to check in JavaScript and Java
> scripting engines at the same time, I'm now leaning toward just checking
> in the JavaScript one for now. The Java stuff will most likely be going
> away for a little while so I can rework it. Given the time that's
> elapsed since anyone tried to compile this, I doubt it will be missed.
>

Have you talked to Peter Amstutz about any of this?  He's working on
nailing down some issues like this in VOS right now.

Reed
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFOqvaFK83gN8ItOQRAkfEAJwOSyqbxXHekG5g8WZ4/SdJy1m6HgCeJ/y1
Fc6l3xclLLNhI77b+Gb+uF4=
=F5yZ
-----END PGP SIGNATURE-----

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
openvrml-develop mailing list
openvrml-develop@...
https://lists.sourceforge.net/lists/listinfo/openvrml-develop

[Josip Almasi]

Braden McDaniel wrote:

> At this point I have successfully ported the JavaScript scripting
> backend to a dynamically loaded module; it seems to be working just
> fine. I would like to take this opportunity to fix the long-ailing Java
> scripting engine. I'm just now getting an idea of what I'm in for.
>
> It hasn't been compiling for a while; but that's not the worst of the
> problems. It is currently written assuming that all Java scripts in a
> world would share the same VM. I don't think that's desirable; and I'm
> even less sure that it's workable: it's my understanding that each
> vrml.node.Script subclass will need to have its base URI added to the
> classpath; and surely different Scripts need to run in isolation from
> one another.

Oh I'm not that sure about that classpath thing. IIRC from the spec,
each script needs to have access to full usual java stuff. Like,
standard J2SE API.
The catch is, other components, 'optional' packages may be available
too. Like, JMF, J3D, JavaComm etc. We usually enable them by placing
them into jre/lib/ext or adding to runtime by java -Djava.ext.dirs=...
So that's why I call them 'usual' - whatever this specific JVM has
installed.
And you can use that to your advantage - just exec jvm with
-Djava.ext.dirs=scripts dir(s). IIRC it has to be defined during JVM
bootstrap.
As for isolating scripts from one another, I think this could be handled
by java security manager/policy. Like, you can forbid reflection so an
object can't inspect others.

> So, proceeding with the assumption that we need a VM-per-Script node,
> the JNI documentation says that you can have no more than one VM per
> thread. That means that the openvrml::script subclass in the Java script
>   engine needs to spawn its own thread and function more-or-less as an
> active object. This is nontrivial code, none of which is in place.

I say drop VM-per-Script idea.
One interpreter to host scripts, that's ok.
I'll gladly help you with that, in fact I think we can use VRSpaceLoader
for starters. That's what it does, loads classes in runtime...
As for thread-per-script model... sure, it has to be done that way.
So, say, you spawn a jvm and send/receive events thru a pipe or...?
Suppose you send events like 'script <id> <eventname> <eventvalue>'.
There's more vrspace code to handle it, but it's around already.
And yes it's not trivial...;)

Regards...

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
openvrml-develop mailing list
openvrml-develop@...
https://lists.sourceforge.net/lists/listinfo/openvrml-develop

[Braden McDaniel]

On Sat, 2006-10-21 at 19:23 -0400, Reed Hedges wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Braden McDaniel wrote:
> >
> > So, proceeding with the assumption that we need a VM-per-Script node,
> > the JNI documentation says that you can have no more than one VM per
> > thread. That means that the openvrml::script subclass in the Java script
> >   engine needs to spawn its own thread and function more-or-less as an
> > active object. This is nontrivial code, none of which is in place.
> >
> > I'm not throwing my hands up in dismay; but getting
> > Java-in-the-Script-node working is going to take longer than I'd
> > anticipated. So while I'd hoped to check in JavaScript and Java
> > scripting engines at the same time, I'm now leaning toward just checking
> > in the JavaScript one for now. The Java stuff will most likely be going
> > away for a little while so I can rework it. Given the time that's
> > elapsed since anyone tried to compile this, I doubt it will be missed.
> >
>
>
> Have you talked to Peter Amstutz about any of this?  He's working on
> nailing down some issues like this in VOS right now.

I've talked a bit with Peter on IRC. And I've talked to some folks who
know more about Java than I do.

I've come around to the conclusion that each vrml.node.Script does not
need its own VM instance after all. Though there's still plenty of work
to be done to get things working, though. I have toyed with the idea of
going ahead and adding the machinery to support a VM-per-Script;
however, I think I'd end up with a code path that would barely be
used--and I don't want that.

--
Braden McDaniel                           e-mail: <braden@...>
<http://endoframe.com>                    Jabber: <braden@...>



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
openvrml-develop mailing list
openvrml-develop@...
https://lists.sourceforge.net/lists/listinfo/openvrml-develop

[Braden McDaniel]

On Mon, 2006-10-23 at 00:30 +0200, Josip Almasi wrote:

> Braden McDaniel wrote:
> > At this point I have successfully ported the JavaScript scripting
> > backend to a dynamically loaded module; it seems to be working just
> > fine. I would like to take this opportunity to fix the long-ailing Java
> > scripting engine. I'm just now getting an idea of what I'm in for.
> >
> > It hasn't been compiling for a while; but that's not the worst of the
> > problems. It is currently written assuming that all Java scripts in a
> > world would share the same VM. I don't think that's desirable; and I'm
> > even less sure that it's workable: it's my understanding that each
> > vrml.node.Script subclass will need to have its base URI added to the
> > classpath; and surely different Scripts need to run in isolation from
> > one another.
>
> Oh I'm not that sure about that classpath thing. IIRC from the spec,
> each script needs to have access to full usual java stuff. Like,
> standard J2SE API.

I see no obvious way to avoid access to the Java standard libraries--and
as you note, I wouldn't want that anyway.

You have the opportunity to set the classpath when starting the VM via
JNI. The semantics of this are the same as the -cp command line argument
to the java runtime executable; i.e., it overrides the CLASSPATH
environment variable.

As such, I'll get the value of CLASSPATH from the environment and
prepend it to the setting passed to the VM.

> The catch is, other components, 'optional' packages may be available
> too. Like, JMF, J3D, JavaComm etc. We usually enable them by placing
> them into jre/lib/ext or adding to runtime by java -Djava.ext.dirs=...
> So that's why I call them 'usual' - whatever this specific JVM has
> installed.
> And you can use that to your advantage - just exec jvm with
> -Djava.ext.dirs=scripts dir(s). IIRC it has to be defined during JVM
> bootstrap.
> As for isolating scripts from one another, I think this could be handled
> by java security manager/policy. Like, you can forbid reflection so an
> object can't inspect others.

A quick experiment demonstrates for me that applets on the same Web page
run in the same VM. While I question the soundness of that design, I
know (at least one reason) why they did it: the VM has historically been
a pig and it's only relatively recently that Sun has begun to optimize
concurrently running VMs to share resources.

> > So, proceeding with the assumption that we need a VM-per-Script node,
> > the JNI documentation says that you can have no more than one VM per
> > thread. That means that the openvrml::script subclass in the Java script
> >   engine needs to spawn its own thread and function more-or-less as an
> > active object. This is nontrivial code, none of which is in place.
>
> I say drop VM-per-Script idea.
> One interpreter to host scripts, that's ok.
> I'll gladly help you with that, in fact I think we can use VRSpaceLoader
> for starters. That's what it does, loads classes in runtime...
> As for thread-per-script model... sure, it has to be done that way.
> So, say, you spawn a jvm and send/receive events thru a pipe or...?
> Suppose you send events like 'script <id> <eventname> <eventvalue>'.
> There's more vrspace code to handle it, but it's around already.
> And yes it's not trivial...;)

At the moment, I'm just planning on running the VM in OpenVRML's main
(rendering) thread. I could give the VM its own thread if I need to--and
in that case I'd simply use something along the lines of an active
object pattern to transmit data--no separate process, so no pipes.

As for loading the classes, I'm under the impression that I can just use
URLClassLoader for that. I think I can construct one of these via JNI
and tell it to construct an instance of the vrml.node.Script subclass.

--
Braden McDaniel                           e-mail: <braden@...>
<http://endoframe.com>                    Jabber: <braden@...>



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
openvrml-develop mailing list
openvrml-develop@...
https://lists.sourceforge.net/lists/listinfo/openvrml-develop

[Josip Almasi]

Braden McDaniel wrote:
>
> You have the opportunity to set the classpath when starting the VM via
> JNI. The semantics of this are the same as the -cp command line argument
> to the java runtime executable; i.e., it overrides the CLASSPATH
> environment variable.
>
> As such, I'll get the value of CLASSPATH from the environment and
> prepend it to the setting passed to the VM.

OK this should do it.

> A quick experiment demonstrates for me that applets on the same Web page
> run in the same VM. While I question the soundness of that design, I
> know (at least one reason) why they did it: the VM has historically been
> a pig and it's only relatively recently that Sun has begun to optimize
> concurrently running VMs to share resources.

Right, and AFAIK apple did it in their jvm earlier.
In the meantime, there's a number of methods that more or less suck.
But it turns out it's not that unsafe design. These applets can't access
each other just like that, nor hang each other just like that.
After all, each of them can easy hang the browser so it doesn't matter;)

> At the moment, I'm just planning on running the VM in OpenVRML's main
> (rendering) thread. I could give the VM its own thread if I need to--and
> in that case I'd simply use something along the lines of an active
> object pattern to transmit data--no separate process, so no pipes.

IOW JNI method calls.
Well then a nasty script could hang the main thread... or, say, a
TimeSensor routed to it?
... no big deal, now that I think of it some more.
Once you start a jvm it can start its own threads, in fact even java
scripts can start their own threads.
So there's no point in trying to prevent eventual mess a script can do.
Some reasonable strategy I use in somewhat simillar case is startup in
multiple threads, so they can't hang main, and main doesn't have to wait
for initialization.
Applied to vrml java scripts, like,
if (method initialize() exists) {
   (new Thread( script )).start()
} else {
   try { script.run() } catch ( Throwable t ) {...}
It's probably overkill for scripts...

> As for loading the classes, I'm under the impression that I can just use
> URLClassLoader for that. I think I can construct one of these via JNI
> and tell it to construct an instance of the vrml.node.Script subclass.

Right, though I'd suggest extending URLClassLoader with some extras.
Nothing specific so far, but that's why I mention VRSpaceLoader - it
extends URLClassLoader too. As do classloaders of tomcat and others.

Regards...

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
openvrml-develop mailing list
openvrml-develop@...
https://lists.sourceforge.net/lists/listinfo/openvrml-develop

[Braden McDaniel]

On Mon, 2006-10-23 at 02:36 +0200, Josip Almasi wrote:
> Braden McDaniel wrote:

[snip]

> > At the moment, I'm just planning on running the VM in OpenVRML's main
> > (rendering) thread. I could give the VM its own thread if I need to--and
> > in that case I'd simply use something along the lines of an active
> > object pattern to transmit data--no separate process, so no pipes.
>
> IOW JNI method calls.
> Well then a nasty script could hang the main thread... or, say, a
> TimeSensor routed to it?
> ... no big deal, now that I think of it some more.

Right. Scripts block--period. It doesn't matter if they're in another
thread--all that means is that a request is dispatched to the script
thread and the rendering thread has to wait on it to be processed.

> Once you start a jvm it can start its own threads, in fact even java
> scripts can start their own threads.
> So there's no point in trying to prevent eventual mess a script can do.
> Some reasonable strategy I use in somewhat simillar case is startup in
> multiple threads, so they can't hang main, and main doesn't have to wait
> for initialization.
> Applied to vrml java scripts, like,
> if (method initialize() exists) {
>    (new Thread( script )).start()
> } else {
>    try { script.run() } catch ( Throwable t ) {...}
> It's probably overkill for scripts...

Maybe not. Or not quite, anyway.

I ought to spawn a new thread to download the script code, at the very
least; I don't want to have the rendering thread block on that. I
*could* just keep that thread running and run the script in it and do
the active object thing.

I'll have to play around with how all this works. I'm starting to get to
far ahead of myself without having gotten some basic code running.

> > As for loading the classes, I'm under the impression that I can just use
> > URLClassLoader for that. I think I can construct one of these via JNI
> > and tell it to construct an instance of the vrml.node.Script subclass.
>
> Right, though I'd suggest extending URLClassLoader with some extras.
> Nothing specific so far, but that's why I mention VRSpaceLoader - it
> extends URLClassLoader too. As do classloaders of tomcat and others.

I've had a brief look; nothing there jumped out at me as being something
I needed. But I'm still pretty naive about a lot of this. Though it has
occurred to me that if I ever want to do something like display download
progress for this stuff, I'll presumably need to implement my own
URLStreamHandler. Joy.

--
Braden McDaniel                           e-mail: <braden@...>
<http://endoframe.com>                    Jabber: <braden@...>



-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
openvrml-develop mailing list
openvrml-develop@...
https://lists.sourceforge.net/lists/listinfo/openvrml-develop

[Josip Almasi]

Braden McDaniel wrote:
>
> Right. Scripts block--period. It doesn't matter if they're in another
> thread--all that means is that a request is dispatched to the script
> thread and the rendering thread has to wait on it to be processed.

Right. AFAIK this is just as other browsers do.

> Maybe not. Or not quite, anyway.
>
> I ought to spawn a new thread to download the script code, at the very
> least; I don't want to have the rendering thread block on that. I
> *could* just keep that thread running and run the script in it and do
> the active object thing.

Right.

> I've had a brief look; nothing there jumped out at me as being something
> I needed. But I'm still pretty naive about a lot of this. Though it has
> occurred to me that if I ever want to do something like display download
> progress for this stuff, I'll presumably need to implement my own
> URLStreamHandler. Joy.

:)
I think you can safelly assume that scripts are small and there's no
need to display progress bar.

Regards...

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
openvrml-develop mailing list
openvrml-develop@...
https://lists.sourceforge.net/lists/listinfo/openvrml-develop

[Braden McDaniel]

On Fri, 2006-10-20 at 17:40 -0400, Braden McDaniel wrote:
> As I mentioned in a previous update, the primary feature I want to have
> in place for 0.17 is the fragmentation of libopenvrml into dynamically
> loaded modules. I've started some work on this.

Well, obviously that didn't quite happen...

> There will be two kinds of modules: node components and scripting
> engines. I've started with the latter because it is more obvious to me
> how scripting engine modules should work; node component modules are
> going to require an XML descriptor and inter-component dependencies and
> it's not yet clear to me what the desirable mechanics are for that.
>
> At this point I have successfully ported the JavaScript scripting
> backend to a dynamically loaded module; it seems to be working just
> fine. I would like to take this opportunity to fix the long-ailing Java
> scripting engine. I'm just now getting an idea of what I'm in for.

[snip]

Damn... It's hard for me to believe that it's been more than a year
since I started on this--before getting sidetracked with creating the
stand-alone player and replacing the parsers.

I have resurrected this work and put it in svn. It can be found at

        <http://openvrml.svn.sourceforge.net/svnroot/openvrml/branches/script-modules>

I'm pressing forward with bringing the Script node Java stuff online.

--
Braden McDaniel                           e-mail: <braden@...>
<http://endoframe.com>                    Jabber: <braden@...>



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
openvrml-develop mailing list
openvrml-develop@...
https://lists.sourceforge.net/lists/listinfo/openvrml-develop