Re: multithreaded Content Aggregator migration from 2.1 to 2.2

[Imran Pariyani]

Hello,

we have come across this issue .. i posted it on user mailing list and
some one from there suggested me to post it here since it seems to be
bug in cocoon 2.2 .. its a very crucial issue for us.. we cant upgrade
to 2.2  unless we  some how solve or find a work around for this issue ..

we have multithreaded Content Aggregator which is based on the
CIncludeTransformer and it also implements CacheableProcessingComponent
for caching .. it used to work fine for cocoon 2.1.x but with cocoon 2.2
it gives strange error .. it is actually a generator and it gives error
in the generate method ..

the code where it gives error

// generate...
           for (Part part : this.parts) {
               if (this.manager.hasService(IncludeCacheManager.ROLE)) {
                   part.uri = this.cacheManager.load(part.uri,
this.cachingSession);
               } else {
                   this.getLogger().error(
                           "The ContentAggregator: aggregator cannot
find the IncludeCacheManager");
               }
           }

           // aggregate...
           StreamPipe streamPipe = new StreamPipe(this.contentHandler);
           for (Part part : this.parts) {
               streamPipe.firstElement(part.element, this.rootElement,
part.stripRootElement);
               try {
                   System.out.println("ParallelContentAgg.part.uri:" +
part.uri);
                   this.cacheManager.stream(part.uri,
this.cachingSession, streamPipe);
               } finally {
                   streamPipe.lastElement(part.element);
               }
           }

it gives the following error

Caused by: org.springframework.beans.factory.BeanCreationException:
Error creating bean with name
'scopedTarget.org.apache.cocoon.el.objectmodel.ObjectModel': Scope
'request' is not active for the current thread; consider defining a
scoped proxy for this bean if you intend to refer to it from a
singleton; nested exception is java.lang.IllegalStateException: No
thread-bound request found: Are you referring to request attributes
outside of an actual web request? If you are actually operating within a
web request and still receive this message,your code is probably running
outside of DispatcherServlet/DispatcherPortlet: In this case, use
RequestContextListener or RequestContextFilter to expose the current
request.
       at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:293)

       at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)

       at
org.springframework.aop.target.SimpleBeanTargetSource.getTarget(SimpleBeanTargetSource.java:33)

       at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:184)

       at $Proxy2.cleanupLocalContext(Unknown Source)
       at
org.apache.cocoon.components.source.impl.SitemapSource.toSAX(SitemapSource.java:382)

       at
org.apache.cocoon.components.source.util.SourceUtil.toSAX(SourceUtil.java:111)

       at
org.apache.cocoon.components.source.util.SourceUtil.toSAX(SourceUtil.java:170)

       at
org.apache.cocoon.components.source.SourceUtil.toSAX(SourceUtil.java:63)
       at
org.apache.cocoon.transformation.helpers.DefaultIncludeCacheManager$LoaderThread.run(DefaultIncludeCacheManager.java:415)

       at
org.apache.cocoon.environment.CocoonRunnable.doRun(CocoonRunnable.java:64)
       at
org.apache.cocoon.environment.internal.EnvironmentHelper$AbstractCocoonRunnable.run(EnvironmentHelper.java:453)

       at
EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)
       ... 1 more

Grzegorz Kossakowski wrote:

> Imran Pariyani pisze:
>> Hi
>
> Hi Imran,
>
>> last night i wasted 5 hrs overs this without any success .. i guess
>> it has something to do with passing the RequestContextListener to the
>> spring context .. i have initialized the listener in my web.xml and
>> also added the config in applicationcontext.xml .. i checked
>> everything on my end .. everything seems to be fine .. is something
>> wrong with the cachemanager present in cocoon ?
>>
>> any help will be greatly appreciated !! ..
>
> Unfortunately, it looks like you stumbled across rather complicated
> (and obscure) bug. It's related to the fact how Spring's request scope
> works (objects are tied to the thread). If you run multithreaded
> aggregation then child threads don't have an access to beans defined
> with 'request' scope because they are visible only in parent thread.
>
> More details below...
>
> <snip/>
>
>>> org.apache.cocoon.components.source.impl.SitemapSource.toSAX(SitemapSource.java:382)
>>
>
> The failing code is:
>             if (touchedOM) {
>                 //Because of complicated flow of this source it must
> maintain the cleaness of OM on its own
>                 ObjectModel newObjectModel;
>                 try {
>                     newObjectModel =
> (ObjectModel)manager.lookup(ObjectModel.ROLE);
>                 } catch (ServiceException e) {
>                     throw new SAXException("Couldn't look up Object
> Model", e);
>                 }
>                 newObjectModel.cleanupLocalContext();
>                 touchedOM = false;
>             }
> Here, manager.lookup fails because ObjectModel is a bean with request
> scope.
>
> To be honest, I don't have an idea how this should be fixed. I guess
> it's a bug of CocoonRunnable class but I don't know how to give child
> threads access to parent thread variables in a safe way.
>
> This would require more investigation that I have no time for at the
> moment, unfortunately.
>
> I hope it helped you a little to understand the problem. Maybe you
> will find someone on dev@ mailing list that has an idea how to fix it.
>

[Joerg Heinicke]

InheritableThreadLocal [1] might be one solution. But being not able to
clean up a thread is always a problem in web environment. IIRC Spring
used to use InheritableThreadLocal in RequestContextListener, but they
changed it to standard ThreadLocal for that reason.

I don't know if CocoonRunnable mentioned by Grek is another point for
potential changes.

Joerg

[1]
http://java.sun.com/j2se/1.4.2/docs/api/java/lang/InheritableThreadLocal.html

On 05.06.2008 09:21, Imran Pariyani wrote:

> Hello,
>
> we have come across this issue .. i posted it on user mailing list and
> some one from there suggested me to post it here since it seems to be
> bug in cocoon 2.2 .. its a very crucial issue for us.. we cant upgrade
> to 2.2  unless we  some how solve or find a work around for this issue ..
>
> we have multithreaded Content Aggregator which is based on the
> CIncludeTransformer and it also implements CacheableProcessingComponent
> for caching .. it used to work fine for cocoon 2.1.x but with cocoon 2.2
> it gives strange error .. it is actually a generator and it gives error
> in the generate method ..
>
> the code where it gives error
>
> // generate...
>           for (Part part : this.parts) {
>               if (this.manager.hasService(IncludeCacheManager.ROLE)) {
>                   part.uri = this.cacheManager.load(part.uri,
> this.cachingSession);
>               } else {
>                   this.getLogger().error(
>                           "The ContentAggregator: aggregator cannot find
> the IncludeCacheManager");
>               }
>           }
>
>           // aggregate...
>           StreamPipe streamPipe = new StreamPipe(this.contentHandler);
>           for (Part part : this.parts) {
>               streamPipe.firstElement(part.element, this.rootElement,
> part.stripRootElement);
>               try {
>                   System.out.println("ParallelContentAgg.part.uri:" +
> part.uri);
>                   this.cacheManager.stream(part.uri,
> this.cachingSession, streamPipe);
>               } finally {
>                   streamPipe.lastElement(part.element);
>               }
>           }
>
> it gives the following error
>
> Caused by: org.springframework.beans.factory.BeanCreationException:
> Error creating bean with name
> 'scopedTarget.org.apache.cocoon.el.objectmodel.ObjectModel': Scope
> 'request' is not active for the current thread; consider defining a
> scoped proxy for this bean if you intend to refer to it from a
> singleton; nested exception is java.lang.IllegalStateException: No
> thread-bound request found: Are you referring to request attributes
> outside of an actual web request? If you are actually operating within a
> web request and still receive this message,your code is probably running
> outside of DispatcherServlet/DispatcherPortlet: In this case, use
> RequestContextListener or RequestContextFilter to expose the current
> request.
>       at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:293)
>
>       at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
>
>       at
> org.springframework.aop.target.SimpleBeanTargetSource.getTarget(SimpleBeanTargetSource.java:33)
>
>       at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:184)
>
>       at $Proxy2.cleanupLocalContext(Unknown Source)
>       at
> org.apache.cocoon.components.source.impl.SitemapSource.toSAX(SitemapSource.java:382)
>
>       at
> org.apache.cocoon.components.source.util.SourceUtil.toSAX(SourceUtil.java:111)
>
>       at
> org.apache.cocoon.components.source.util.SourceUtil.toSAX(SourceUtil.java:170)
>
>       at
> org.apache.cocoon.components.source.SourceUtil.toSAX(SourceUtil.java:63)
>       at
> org.apache.cocoon.transformation.helpers.DefaultIncludeCacheManager$LoaderThread.run(DefaultIncludeCacheManager.java:415)
>
>       at
> org.apache.cocoon.environment.CocoonRunnable.doRun(CocoonRunnable.java:64)
>       at
> org.apache.cocoon.environment.internal.EnvironmentHelper$AbstractCocoonRunnable.run(EnvironmentHelper.java:453)
>
>       at
> EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)
>       ... 1 more
>
> Grzegorz Kossakowski wrote:
>> Imran Pariyani pisze:
>>> Hi
>>
>> Hi Imran,
>>
>>> last night i wasted 5 hrs overs this without any success .. i guess
>>> it has something to do with passing the RequestContextListener to the
>>> spring context .. i have initialized the listener in my web.xml and
>>> also added the config in applicationcontext.xml .. i checked
>>> everything on my end .. everything seems to be fine .. is something
>>> wrong with the cachemanager present in cocoon ?
>>>
>>> any help will be greatly appreciated !! ..
>>
>> Unfortunately, it looks like you stumbled across rather complicated
>> (and obscure) bug. It's related to the fact how Spring's request scope
>> works (objects are tied to the thread). If you run multithreaded
>> aggregation then child threads don't have an access to beans defined
>> with 'request' scope because they are visible only in parent thread.
>>
>> More details below...
>>
>> <snip/>
>>
>>>> org.apache.cocoon.components.source.impl.SitemapSource.toSAX(SitemapSource.java:382)
>>>
>>>
>>
>> The failing code is:
>>             if (touchedOM) {
>>                 //Because of complicated flow of this source it must
>> maintain the cleaness of OM on its own
>>                 ObjectModel newObjectModel;
>>                 try {
>>                     newObjectModel =
>> (ObjectModel)manager.lookup(ObjectModel.ROLE);
>>                 } catch (ServiceException e) {
>>                     throw new SAXException("Couldn't look up Object
>> Model", e);
>>                 }
>>                 newObjectModel.cleanupLocalContext();
>>                 touchedOM = false;
>>             }
>> Here, manager.lookup fails because ObjectModel is a bean with request
>> scope.
>>
>> To be honest, I don't have an idea how this should be fixed. I guess
>> it's a bug of CocoonRunnable class but I don't know how to give child
>> threads access to parent thread variables in a safe way.
>>
>> This would require more investigation that I have no time for at the
>> moment, unfortunately.
>>
>> I hope it helped you a little to understand the problem. Maybe you
>> will find someone on dev@ mailing list that has an idea how to fix it.
>>
>

[Grzegorz Kossakowski-2]

Joerg Heinicke pisze:
> InheritableThreadLocal [1] might be one solution. But being not able to
> clean up a thread is always a problem in web environment. IIRC Spring
> used to use InheritableThreadLocal in RequestContextListener, but they
> changed it to standard ThreadLocal for that reason.
>
> I don't know if CocoonRunnable mentioned by Grek is another point for
> potential changes.

The problem with inheritance from parent thread could be solved with one or another way. The only
problem I can see is synchronization and keeping consistency of data.

E.g. if two threads share the same (inherited) instance of ObjectModel all nasty side-effects of
multi-threading should be expected like parallel modification and loosing of data consistency kept
in OM.
How Cocoon used to solve these kind of problems in the past?

--
Grzegorz Kossakowski

[Grzegorz Kossakowski-2]

Imran Pariyani pisze:

> Hello,
>
> we have come across this issue .. i posted it on user mailing list and
> some one from there suggested me to post it here since it seems to be
> bug in cocoon 2.2 .. its a very crucial issue for us.. we cant upgrade
> to 2.2  unless we  some how solve or find a work around for this issue ..
>
> we have multithreaded Content Aggregator which is based on the
> CIncludeTransformer and it also implements CacheableProcessingComponent
> for caching .. it used to work fine for cocoon 2.1.x but with cocoon 2.2
> it gives strange error .. it is actually a generator and it gives error
> in the generate method ..

Imran, have you tried replacing cocoon: protocol usage with servlet: one? I don't think this will
solve all problems but since it's much more limited in functionality and more focused on one task it
would be easier to resolve any further problems.


--
Grzegorz Kossakowski

[Joerg Heinicke]

On 06.06.2008 12:51, Grzegorz Kossakowski wrote:

> E.g. if two threads share the same (inherited) instance of ObjectModel
> all nasty side-effects of multi-threading should be expected like
> parallel modification and loosing of data consistency kept in OM.
> How Cocoon used to solve these kind of problems in the past?

I don't see that much of a problem here - and it always used to be that
way. If it is a distinctive object model or just the request object does
not really make a difference to me. Also in other frameworks where you
only work with request or session objects you have to be aware of
multi-threading.

Is the multi-threading processing a feature one has to switch on
intentionally?

Joerg

[Grzegorz Kossakowski-2]

Joerg Heinicke pisze:

> On 06.06.2008 12:51, Grzegorz Kossakowski wrote:
>
>> E.g. if two threads share the same (inherited) instance of ObjectModel
>> all nasty side-effects of multi-threading should be expected like
>> parallel modification and loosing of data consistency kept in OM.
>> How Cocoon used to solve these kind of problems in the past?
>
> I don't see that much of a problem here - and it always used to be that
> way. If it is a distinctive object model or just the request object does
> not really make a difference to me. Also in other frameworks where you
> only work with request or session objects you have to be aware of
> multi-threading.

I agree in general but there is a subtle difference here. I think there are plenty of people that
agree on a statement that one doesn't need to be a Java programmer in order to use Cocoon. There are
many success stories proving that.

In Cocoon you can have complicated scenarios that you introduce only by using standard Cocoon
components and pipelines all glued by sitemap. In such scenarios, when one does not write any line
of custom java code she should be protected from tricky troubles like the one Imran ran into.

The whole issue comes down to finding an answer to crucial question that appeared here already many
times:
How much do we want to inherit from original request (more generally environment) to subrequests?

The ideal situation would be if we could inherit nothing thus avoiding any tricky problems but such
solution is rather unacceptable because it brings another sort of problems.

Another idea I can think of is to have a clearly and narrowly defined set of things (objects, data)
we want to inherit in subrequests and pay as much attention as we can to assuring that these objects
are protected from any side-effects including running in multi-threaded environments.

At the moment I can think only about two kind of data that we should inherit, it is:
a) request data (uploaded files, attributes, etc.)
b) ObjectModel (the one defined in cocoon-expression-api that stores for example data coming from
flowscript)

I think the only way we can avoid side-effects is by _cloning_ objects that are inherited so
subrequests do not affect each other and original request (and their environments). Actually, that
not a new idea, see EnvironmentHelper.AbstractCocoonRunnable[1]:
         public AbstractCocoonRunnable() {
             // Clone the environment stack of the calling thread.
             // We'll use it in run() below
             Object stack = EnvironmentHelper.environmentStack.get();
             if (stack != null) {
                 this.parentStack = ((EnvironmentStack)stack).clone();
             }
         }

Therefore I would like to propose introduction of a new scope called "request-inheritable" which
would allow beans (like ObjectModel) to be inherited but by cloning.

What do you think?

> Is the multi-threading processing a feature one has to switch on
> intentionally?

Yes, but even if you enable it intentionally you still don't have be a Java programmer so all above
applies here as well.

[1]
http://svn.eu.apache.org/viewvc/cocoon/trunk/core/cocoon-sitemap/cocoon-sitemap-impl/src/main/java/org/apache/cocoon/environment/internal/EnvironmentHelper.java?view=markup

--
Grzegorz Kossakowski

[Christoph Gaffga (triplemind.com)]

Grzegorz Kossakowski wrote:
> Therefore I would like to propose introduction of a new scope called
> "request-inheritable" which would allow beans (like ObjectModel) to be
> inherited but by cloning.
>
> What do you think?

hmm, just tried to solve that problem and found that in
IncludeTransformer it works with threading and parallel processing. Only
for the DefaultIncludeCacheManager we are using in our replacement for
the aggregator to do parallel aggregation it is broken since version 2.2.

So I decided to patch the DefaultIncludeCacheManager to work in the same
way as the IncludeTransformer does, by setting up the child threads with
the same context/environment:

   // Setup this thread's environment
   RequestContextHolder.setRequestAttributes(
                             this.session.getRequestAttributes());
   EnvironmentHelper.enterProcessor(this.session.getProcessor(),
                             this.session.getEnvironment());

I'll test this out, and if it work's I send in the patch.

Christoph

[Christoph Gaffga (triplemind.com)]

hi,

we found the problem in DefaultIncludeCacheManager and fixed it. I just
submitted a patch together with some more information on the problem.

see:
https://issues.apache.org/jira/browse/COCOON-2216

thanks for your help.

regards,
Christoph


Joerg Heinicke wrote:

> InheritableThreadLocal [1] might be one solution. But being not able to
> clean up a thread is always a problem in web environment. IIRC Spring
> used to use InheritableThreadLocal in RequestContextListener, but they
> changed it to standard ThreadLocal for that reason.
>
> I don't know if CocoonRunnable mentioned by Grek is another point for
> potential changes.
>
> Joerg
>
> [1]
> http://java.sun.com/j2se/1.4.2/docs/api/java/lang/InheritableThreadLocal.html 
>
>
> On 05.06.2008 09:21, Imran Pariyani wrote:
>> Hello,
>>
>> we have come across this issue .. i posted it on user mailing list and
>> some one from there suggested me to post it here since it seems to be
>> bug in cocoon 2.2 .. its a very crucial issue for us.. we cant upgrade
>> to 2.2  unless we  some how solve or find a work around for this issue ..
>>
>> we have multithreaded Content Aggregator which is based on the
>> CIncludeTransformer and it also implements
>> CacheableProcessingComponent for caching .. it used to work fine for
>> cocoon 2.1.x but with cocoon 2.2 it gives strange error .. it is
>> actually a generator and it gives error in the generate method ..
>>
>> the code where it gives error
>>
>> // generate...
>>           for (Part part : this.parts) {
>>               if (this.manager.hasService(IncludeCacheManager.ROLE)) {
>>                   part.uri = this.cacheManager.load(part.uri,
>> this.cachingSession);
>>               } else {
>>                   this.getLogger().error(
>>                           "The ContentAggregator: aggregator cannot
>> find the IncludeCacheManager");
>>               }
>>           }
>>
>>           // aggregate...
>>           StreamPipe streamPipe = new StreamPipe(this.contentHandler);
>>           for (Part part : this.parts) {
>>               streamPipe.firstElement(part.element, this.rootElement,
>> part.stripRootElement);
>>               try {
>>                   System.out.println("ParallelContentAgg.part.uri:" +
>> part.uri);
>>                   this.cacheManager.stream(part.uri,
>> this.cachingSession, streamPipe);
>>               } finally {
>>                   streamPipe.lastElement(part.element);
>>               }
>>           }
>>
>> it gives the following error
>>
>> Caused by: org.springframework.beans.factory.BeanCreationException:
>> Error creating bean with name
>> 'scopedTarget.org.apache.cocoon.el.objectmodel.ObjectModel': Scope
>> 'request' is not active for the current thread; consider defining a
>> scoped proxy for this bean if you intend to refer to it from a
>> singleton; nested exception is java.lang.IllegalStateException: No
>> thread-bound request found: Are you referring to request attributes
>> outside of an actual web request? If you are actually operating within
>> a web request and still receive this message,your code is probably
>> running outside of DispatcherServlet/DispatcherPortlet: In this case,
>> use RequestContextListener or RequestContextFilter to expose the
>> current request.
>>       at
>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:293)
>>
>>       at
>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
>>
>>       at
>> org.springframework.aop.target.SimpleBeanTargetSource.getTarget(SimpleBeanTargetSource.java:33)
>>
>>       at
>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:184)
>>
>>       at $Proxy2.cleanupLocalContext(Unknown Source)
>>       at
>> org.apache.cocoon.components.source.impl.SitemapSource.toSAX(SitemapSource.java:382)
>>
>>       at
>> org.apache.cocoon.components.source.util.SourceUtil.toSAX(SourceUtil.java:111)
>>
>>       at
>> org.apache.cocoon.components.source.util.SourceUtil.toSAX(SourceUtil.java:170)
>>
>>       at
>> org.apache.cocoon.components.source.SourceUtil.toSAX(SourceUtil.java:63)
>>       at
>> org.apache.cocoon.transformation.helpers.DefaultIncludeCacheManager$LoaderThread.run(DefaultIncludeCacheManager.java:415)
>>
>>       at
>> org.apache.cocoon.environment.CocoonRunnable.doRun(CocoonRunnable.java:64)
>>
>>       at
>> org.apache.cocoon.environment.internal.EnvironmentHelper$AbstractCocoonRunnable.run(EnvironmentHelper.java:453)
>>
>>       at
>> EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown
>> Source)
>>       ... 1 more
>>
>> Grzegorz Kossakowski wrote:
>>> Imran Pariyani pisze:
>>>> Hi
>>>
>>> Hi Imran,
>>>
>>>> last night i wasted 5 hrs overs this without any success .. i guess
>>>> it has something to do with passing the RequestContextListener to
>>>> the spring context .. i have initialized the listener in my web.xml
>>>> and also added the config in applicationcontext.xml .. i checked
>>>> everything on my end .. everything seems to be fine .. is something
>>>> wrong with the cachemanager present in cocoon ?
>>>>
>>>> any help will be greatly appreciated !! ..
>>>
>>> Unfortunately, it looks like you stumbled across rather complicated
>>> (and obscure) bug. It's related to the fact how Spring's request
>>> scope works (objects are tied to the thread). If you run
>>> multithreaded aggregation then child threads don't have an access to
>>> beans defined with 'request' scope because they are visible only in
>>> parent thread.
>>>
>>> More details below...
>>>
>>> <snip/>
>>>
>>>>> org.apache.cocoon.components.source.impl.SitemapSource.toSAX(SitemapSource.java:382)
>>>>
>>>>
>>>>
>>>
>>> The failing code is:
>>>             if (touchedOM) {
>>>                 //Because of complicated flow of this source it must
>>> maintain the cleaness of OM on its own
>>>                 ObjectModel newObjectModel;
>>>                 try {
>>>                     newObjectModel =
>>> (ObjectModel)manager.lookup(ObjectModel.ROLE);
>>>                 } catch (ServiceException e) {
>>>                     throw new SAXException("Couldn't look up Object
>>> Model", e);
>>>                 }
>>>                 newObjectModel.cleanupLocalContext();
>>>                 touchedOM = false;
>>>             }
>>> Here, manager.lookup fails because ObjectModel is a bean with request
>>> scope.
>>>
>>> To be honest, I don't have an idea how this should be fixed. I guess
>>> it's a bug of CocoonRunnable class but I don't know how to give child
>>> threads access to parent thread variables in a safe way.
>>>
>>> This would require more investigation that I have no time for at the
>>> moment, unfortunately.
>>>
>>> I hope it helped you a little to understand the problem. Maybe you
>>> will find someone on dev@ mailing list that has an idea how to fix it.
>>>
>>
>

--
Triplemind Internet OHG
Berlinerstr. 2
DE-63065 Offenbach
Germany

http://www.triplemind.com
Mail info@...
Tel  [phone]
Fax  [phone]

Unternehmensdaten: Triplemind OHG
Geschaeftsfuehrung: Christoph Gaffga, Mathias Ziegler
Bankverbindung: Postbank Frankfurt,  BLZ: 50010060, Kto: 563581608,
IBAN: DE28 5001 0060 0563 5816 08, BIC: PBNKDEFF
Eingetragen im HR Offenbach am Main 40202, USt-IdNr.: DE210080643

[Christoph Gaffga (triplemind.com)]

sorry, was to quick :((

 > So I decided to patch the DefaultIncludeCacheManager to work in the same
 > way as the IncludeTransformer does, by setting up the child threads with
 > the same context/environment:
 >
 >   // Setup this thread's environment
 >   RequestContextHolder.setRequestAttributes(
 >                             this.session.getRequestAttributes());
 >   EnvironmentHelper.enterProcessor(this.session.getProcessor(),
 >                             this.session.getEnvironment());
 >
 > I'll test this out, and if it work's I send in the patch.

it does not help. It's the way it's meant to work, but it doesn't.
Somehow the ObjectModels in the parallel running JXTemplateGenerators
affect each other and we have strange side affects on most of our request.

we are digging around, but no ideas any more how to fix it...

any help would be appreciated.

Christoph

[Imran Pariyani]

Hello all

Grzegorz as u wrote...

Therefore I would like to propose introduction of a new scope called
"request-inheritable" which
would allow beans (like ObjectModel) to be inherited but by cloning.

can you please elaborate on that ... coz thats where we are stuck ...  
if u have a look at the AbstractInterpreter.java file

public void forwardTo(String uri, Object bizData,
                          WebContinuation continuation,
                          Redirector redirector)
    throws Exception {
        if (SourceUtil.indexOfSchemeColon(uri) == -1) {
            uri = "cocoon:/" + uri;
            final Map objectModel =
this.processInfoProvider.getObjectModel();
            FlowHelper.setWebContinuation(objectModel, newObjectModel,
continuation);
            FlowHelper.setContextObject(objectModel, newObjectModel,
bizData);
            if (redirector.hasRedirected()) {
                throw new IllegalStateException("Pipeline has already
been processed for this request");
            }
            // this is a hint for the redirector
            objectModel.put("cocoon:forward", "true");
            redirector.redirect(false, uri);
        } else {
            throw new Exception("uri is not allowed to contain a scheme
(cocoon:/ is always automatically used)");
        }
    }

so the instance of newObjectModel is the one from the bean factory ...
and if u have multithreading then all the threads change that
objectmodel simultaneously and then finally the objectmodel coming to
the JXTemplategenerator(Injected by spring) is not thread specific ...

in the setup method of the generator class we replace the objectmodel
injected by spring with the one passed with that method ... after doing
this the errors are not that frequent .. but they are still there :( ..
its still being modified by some other threads ...


Could you please let us know how do u intend to implement ur proposed
solution

regards

Imran

Christoph Gaffga (triplemind.com) wrote:

> sorry, was to quick :((
>
> > So I decided to patch the DefaultIncludeCacheManager to work in the
> same
> > way as the IncludeTransformer does, by setting up the child threads
> with
> > the same context/environment:
> >
> >   // Setup this thread's environment
> >   RequestContextHolder.setRequestAttributes(
> >                             this.session.getRequestAttributes());
> >   EnvironmentHelper.enterProcessor(this.session.getProcessor(),
> >                             this.session.getEnvironment());
> >
> > I'll test this out, and if it work's I send in the patch.
>
> it does not help. It's the way it's meant to work, but it doesn't.
> Somehow the ObjectModels in the parallel running JXTemplateGenerators
> affect each other and we have strange side affects on most of our
> request.
>
> we are digging around, but no ideas any more how to fix it...
>
> any help would be appreciated.
>
> Christoph

[Grzegorz Kossakowski-2]

imran pisze:

> Hello all
>
> Grzegorz as u wrote...
>
> Therefore I would like to propose introduction of a new scope called
> "request-inheritable" which
> would allow beans (like ObjectModel) to be inherited but by cloning.
>
> can you please elaborate on that ... coz thats where we are stuck ...  
> if u have a look at the AbstractInterpreter.java file
>
> public void forwardTo(String uri, Object bizData,
>                          WebContinuation continuation,
>                          Redirector redirector)
>    throws Exception {
>        if (SourceUtil.indexOfSchemeColon(uri) == -1) {
>            uri = "cocoon:/" + uri;
>            final Map objectModel =
> this.processInfoProvider.getObjectModel();
>            FlowHelper.setWebContinuation(objectModel, newObjectModel,
> continuation);
>            FlowHelper.setContextObject(objectModel, newObjectModel,
> bizData);
>            if (redirector.hasRedirected()) {
>                throw new IllegalStateException("Pipeline has already
> been processed for this request");
>            }
>            // this is a hint for the redirector
>            objectModel.put("cocoon:forward", "true");
>            redirector.redirect(false, uri);
>        } else {
>            throw new Exception("uri is not allowed to contain a scheme
> (cocoon:/ is always automatically used)");
>        }
>    }
> so the instance of newObjectModel is the one from the bean factory ...
> and if u have multithreading then all the threads change that
> objectmodel simultaneously and then finally the objectmodel coming to
> the JXTemplategenerator(Injected by spring) is not thread specific ...
> in the setup method of the generator class we replace the objectmodel
> injected by spring with the one passed with that method ... after doing
> this the errors are not that frequent .. but they are still there :( ..
> its still being modified by some other threads ...
>
>
> Could you please let us know how do u intend to implement ur proposed
> solution

Just wanted to let you know that I know this stuff really well (I've worked on ObjectModel last
summer) so surely can give you some tips and precise instructions.

This won't happen now as I'm leaving but I'll try to explain my thoughts tomorrow after I check
everything against Cocoon's source code.

--
Best regards,
Grzegorz Kossakowski

[Grzegorz Kossakowski-2]

imran pisze:

> Hello all
>
> Grzegorz as u wrote...
>
> Therefore I would like to propose introduction of a new scope called
> "request-inheritable" which
> would allow beans (like ObjectModel) to be inherited but by cloning.
>
> can you please elaborate on that ... coz thats where we are stuck ...  
> if u have a look at the AbstractInterpreter.java file
>
> public void forwardTo(String uri, Object bizData,
>                          WebContinuation continuation,
>                          Redirector redirector)
>    throws Exception {
>        if (SourceUtil.indexOfSchemeColon(uri) == -1) {
>            uri = "cocoon:/" + uri;
>            final Map objectModel =
> this.processInfoProvider.getObjectModel();
>            FlowHelper.setWebContinuation(objectModel, newObjectModel,
> continuation);
>            FlowHelper.setContextObject(objectModel, newObjectModel,
> bizData);
>            if (redirector.hasRedirected()) {
>                throw new IllegalStateException("Pipeline has already
> been processed for this request");
>            }
>            // this is a hint for the redirector
>            objectModel.put("cocoon:forward", "true");
>            redirector.redirect(false, uri);
>        } else {
>            throw new Exception("uri is not allowed to contain a scheme
> (cocoon:/ is always automatically used)");
>        }
>    }
> so the instance of newObjectModel is the one from the bean factory ...
> and if u have multithreading then all the threads change that
> objectmodel simultaneously and then finally the objectmodel coming to
> the JXTemplategenerator(Injected by spring) is not thread specific ...

It's not thread-specific but request-specific but here request means the one coming from browser and
handled by servlet container and not those internal requests that Cocoon is doing when cocoon: or
servlet: protocols are being used.

> in the setup method of the generator class we replace the objectmodel
> injected by spring with the one passed with that method ... after doing
> this the errors are not that frequent .. but they are still there :( ..
> its still being modified by some other threads ...
>
>
> Could you please let us know how do u intend to implement ur proposed
> solution

The idea is to introduce a new implementation of a Spring scope that will work in a following way:
1. if requested object (like OM) exists in current context then just return it
2. if requested object does not yet exist in current context then find out if this context is
derived from root one:
    a) if the context is derived then ask the root for the object, clone it and store cloned version
in current context
    b) if the context is not derived one so itself is a root then just ask Spring to create a new
instance of requested object and again store it in current context

The introduction of new contexts should happen only in two cases:
1. new thread is being invoked (for multi-thread scenarios)
2. new internal request is being invoked (in case of using servlet/cocoon protocols)


Even if this may sound scary, the implementation of this idea should be very simple. It's only a few
lines of code are needed to handle everything. Of course, the problem is where to inject this code.
Actually, today I was so busy with other work that I hadn't enough time to look into Cocoon itself
but tomorrow I'll do so and give you more detailed instructions how to implement this.

Hopefully, proposed solution should fix this problem once and for ever (at least I fail to imagine
any situation when this wouldn't work)

--
Best regards,
Grzegorz Kossakowski

[Grzegorz Kossakowski-2]

Grzegorz Kossakowski pisze:

>
> It's not thread-specific but request-specific but here request means the
> one coming from browser and handled by servlet container and not those
> internal requests that Cocoon is doing when cocoon: or servlet:
> protocols are being used.
>
>> in the setup method of the generator class we replace the objectmodel
>> injected by spring with the one passed with that method ... after
>> doing this the errors are not that frequent .. but they are still
>> there :( .. its still being modified by some other threads ...
>>
>>
>> Could you please let us know how do u intend to implement ur proposed
>> solution
>
> The idea is to introduce a new implementation of a Spring scope that
> will work in a following way:
> 1. if requested object (like OM) exists in current context then just
> return it
> 2. if requested object does not yet exist in current context then find
> out if this context is derived from root one:
>    a) if the context is derived then ask the root for the object, clone
> it and store cloned version in current context
>    b) if the context is not derived one so itself is a root then just
> ask Spring to create a new instance of requested object and again store
> it in current context
>
> The introduction of new contexts should happen only in two cases:
> 1. new thread is being invoked (for multi-thread scenarios)
> 2. new internal request is being invoked (in case of using
> servlet/cocoon protocols)
>
>
> Even if this may sound scary, the implementation of this idea should be
> very simple. It's only a few lines of code are needed to handle
> everything. Of course, the problem is where to inject this code.
> Actually, today I was so busy with other work that I hadn't enough time
> to look into Cocoon itself but tomorrow I'll do so and give you more
> detailed instructions how to implement this.
>
> Hopefully, proposed solution should fix this problem once and for ever
> (at least I fail to imagine any situation when this wouldn't work)

Imran, after taking a closer look at our code I can see that there are little bit more issues that
need to be fixed before I can fix this one.

I guess it's going to be much easier if I take care of a whole work because it's really non-trivial
stuff and you really need to know all the internals of Cocoon in order to properly fix the problem
you have.

Therefore I suggest that you provide me a simple application that exhibits this problem so I can
test my fixes and track the whole bug. I suggest creation of two different scenarios where one will
use cocoon: protocol and another will use servlet: protocol.

The most preferred way of providing me this test-cases would be to prepare two ITs as it's done in
http://svn.eu.apache.org/viewvc/cocoon/trunk/blocks/cocoon-it/
http://svn.eu.apache.org/viewvc/cocoon/trunk/core/cocoon-webapp/src/test/java/org/apache/cocoon/it/

In the first location you add sitemap entries and other resources that you usually at to your block,
and in the webapp you create a test-case that tests expected behavior.

As soon as you provide me such test-cases (as a patch in JIRA) I'll start working on this problem.

--
Best regards,
Grzegorz Kossakowski