Michael Bien wrote:
> Gregg Wonderly wrote:
>> Michael Bien wrote:
>>> Ben Manes wrote:
>>>> How far along is this project?
>>>>
>>>> I wrote a distributed master/slave framework and (for fun) have a
>>>> map-reduce abstraction prototyped. I was hoping to add fork-join,
>>>> similar to your message, when we moved to Java-6 (and could use the
>>>> jsr lib). So far my framework has been in production for 2 years,
>>>> gone through performance reviews, etc. I've always wanted to open
>>>> source it, but never got the powers that be to approve.
>>>>
>>>> If your interested, we can discuss such things off-line.
>>> For sure we can discuss this off-line if you like. Feel free to
>>> contact me directly.
>>> But keep in mind all the motivation to build FishFarm was actually
>>> doing distribution without introducing a new Framework ;) The idea is
>>> to make grid computing optional in case you are using the
>>> fj-framework anyway.
>>>
>>> FishFarm is around 6 month old and is a freetime project, it was
>>> never in production since I would never introduce a project which is
>>> based on a fork of a jsr ;-)
>> I hope that everyone working in this direction with Java, remembers to
>> pay attention to all the work that has gone into Jini to make this
>> possible with security and all the other tools such as leases and
>> transactions, already implemented and working to help track
>> distributed resources effectively, including the security features of
>> Java, extended in the JERI stack to help eliminate the impact of DOS
>> attacks.
>>
>> The current version of Jini is now the River Podling in the Apache
>> Incubator.
>>
>> Gregg Wonderly
>>
> Hello Gregg,
>
> There are many distribution systems around and each of them has its use
> case. Java Spaces (and its commercial forks), Grid Gain, Terracotta,
> Apache River... and even more specialized and therefore not mainstream
> projects like Helios for Sunflow ray tracing tasks. Almost each of them
> introduced its own framework or lets say it in other words - IS a
> framework. (Terracotta is the exception which is a layer on top of the JVM)
>
> FishFarm is no full featured distribution system, it concentrates
> currently only on the distribution of ForkJoinPools and has no framework
> at all (jsr166y is its framework). Again, it simple works if you
> exchange "new ForkJoinPool()" with "new DistributedForkJoinPool()" and
> put the jars in the classpath.
One the more interesting issues is that Remote computational tasks have a unique
remote communications requirement that has to be accounted for in some way.
Partial failure and security usually become issues faster than most think, and
as you start to deal with those issues, you end up reinventing the wheel because
of all the work already done to deal with those issues.
In the end, it comes down to APIs and some simple algorithmic choices. All of
the issues that tool sets such as Jini deal with are real and important to
manage at some level. The JERI stack, finally provides a unique opportunity to
optimize and tune the RPC stack, dynamically, at deployment time, instead of in
your code. The Jini Configuration interface and the ConfigurationFile
implementation allow you to change the Exporter instance you deploy with for
example, so that you can put TCP vs SSL, No-Auth vs X.500 vs Kerbos
authentication and security constraints all into a configuration step instead of
a programming/reprogramming task.
Many people still look at Jini as "tied" to "RMI". It is tied to the RMI
programming model, and through the use of JERI, you can separate most language
issues for remote interaction (HTTP endpoint with an XML invocation layer
factory for example).
> It has currently no security at all since
> we haven't got the requirement yet to install it outside closed networks
> which are pretty common for clusters. We even decided to try it without
> transactions (there is no JEE in it just simple messaging based on Shoal
> which is based on JXTA) because the master simple does not care if
> worker succeed with the computation or got its requested work or not.
> The master simple puts all of its "stolen" work into a backup and copies
> it back into the pool when the pool is empty - it still behaves like a
> ForkJoinPool.
Simple is easy to start with. There was a study done on creating a JXTA
endpoint for the JERI stack. I've not used it. The transactional services in
Jini are based on a simple 2PC model that is implemented by the Mahalo service
in the River podling (and the previously released Jini Technology Starter Kit v2.1).
The ability to get dependable and predictable results and see progress, seems
like an important thing to me. So I'm not sure that I understand how this
framework can support that.
The Javaspaces master/worker pattern is pretty well proven as a distributed
system for massive scaling. Creating a worker that uses the ForkJoin stuff to
further divide work locally would be an added mechanism for further exploiting
the power available.
> But my initial question was pretty general even FishFarm independent and
> could be used in any framework you like:
> Is there interest in making ForkJoinTasks Serializeable and providing a
> mechanism for querying not yet executed tasks from a ForkJoinPool?
I think these two things have some merit. Serializable implies a number of
issues about the underlying references as well though.
Gregg Wonderly
_______________________________________________
Concurrency-interest mailing list
Concurrency-interest@...
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Some parts of this message have been removed.
Learn more about Nabble's security policy.
