[jira] Created: (COCOON-2216) IncludeCacheManager can not perfom parallel includes

IncludeCacheManager can not perfom parallel includes
----------------------------------------------------

                 Key: COCOON-2216
                 URL: https://issues.apache.org/jira/browse/COCOON-2216
             Project: Cocoon
          Issue Type: Bug
          Components: - Components: Sitemap
    Affects Versions: 2.2-dev (Current SVN)
            Reporter: Christoph Gaffga


Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.

We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
see also: http://www.mail-archive.com/users@.../msg42173.html

I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...

But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.

We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...



--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/COCOON-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Christoph Gaffga updated COCOON-2216:
-------------------------------------

    Attachment: ParallelInclusionProblem-cocoon_TRUNK.patch

patch fixing DefaultIncludeCacheManager so it can include in parallel from cocoon-pipelines

> IncludeCacheManager can not perfom parallel includes
> ----------------------------------------------------
>
>                 Key: COCOON-2216
>                 URL: https://issues.apache.org/jira/browse/COCOON-2216
>             Project: Cocoon
>          Issue Type: Bug
>          Components: - Components: Sitemap
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Christoph Gaffga
>         Attachments: ParallelInclusionProblem-cocoon_TRUNK.patch
>
>
> Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.
> We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
> see also: http://www.mail-archive.com/users@.../msg42173.html
> I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...
> But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
> and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.
> We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/COCOON-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12613580#action_12613580 ]

Christoph Gaffga commented on COCOON-2216:
------------------------------------------

the patch fixes the DefaultIncludeCacheManager, so it can do parallel includions, but their still is an issue with the ObjectModel that does not support multi-threading. Resulting in random strange exceptions. Still need to be fixed

> IncludeCacheManager can not perfom parallel includes
> ----------------------------------------------------
>
>                 Key: COCOON-2216
>                 URL: https://issues.apache.org/jira/browse/COCOON-2216
>             Project: Cocoon
>          Issue Type: Bug
>          Components: - Components: Sitemap
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Christoph Gaffga
>         Attachments: ParallelInclusionProblem-cocoon_TRUNK.patch
>
>
> Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.
> We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
> see also: http://www.mail-archive.com/users@.../msg42173.html
> I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...
> But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
> and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.
> We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/COCOON-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

imran pariyani updated COCOON-2216:
-----------------------------------

    Attachment: test-webapp.zip

webapp block

> IncludeCacheManager can not perfom parallel includes
> ----------------------------------------------------
>
>                 Key: COCOON-2216
>                 URL: https://issues.apache.org/jira/browse/COCOON-2216
>             Project: Cocoon
>          Issue Type: Bug
>          Components: - Components: Sitemap
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Christoph Gaffga
>         Attachments: cocoon-trunk.patch, ParallelInclusionProblem-cocoon_TRUNK.patch, test-block.zip, test-webapp.zip
>
>
> Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.
> We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
> see also: http://www.mail-archive.com/users@.../msg42173.html
> I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...
> But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
> and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.
> We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/COCOON-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

imran pariyani updated COCOON-2216:
-----------------------------------

    Attachment: test-block.zip

test block

> IncludeCacheManager can not perfom parallel includes
> ----------------------------------------------------
>
>                 Key: COCOON-2216
>                 URL: https://issues.apache.org/jira/browse/COCOON-2216
>             Project: Cocoon
>          Issue Type: Bug
>          Components: - Components: Sitemap
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Christoph Gaffga
>         Attachments: cocoon-trunk.patch, ParallelInclusionProblem-cocoon_TRUNK.patch, test-block.zip, test-webapp.zip
>
>
> Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.
> We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
> see also: http://www.mail-archive.com/users@.../msg42173.html
> I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...
> But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
> and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.
> We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/COCOON-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

imran pariyani updated COCOON-2216:
-----------------------------------

    Attachment: cocoon-trunk.patch

for running the testcases u will have to first apply this patch file

> IncludeCacheManager can not perfom parallel includes
> ----------------------------------------------------
>
>                 Key: COCOON-2216
>                 URL: https://issues.apache.org/jira/browse/COCOON-2216
>             Project: Cocoon
>          Issue Type: Bug
>          Components: - Components: Sitemap
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Christoph Gaffga
>         Attachments: cocoon-trunk.patch, ParallelInclusionProblem-cocoon_TRUNK.patch, test-block.zip, test-webapp.zip
>
>
> Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.
> We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
> see also: http://www.mail-archive.com/users@.../msg42173.html
> I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...
> But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
> and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.
> We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/COCOON-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615481#action_12615481 ]

imran pariyani commented on COCOON-2216:
----------------------------------------

have attached the test case to reproduce the error .. i have attached three files .. test-block is the main block .. webapp is for running that block .. before that u will have to apply the patch cocoon-trunk.patch to the cocoon other wise it wont work at all ...
for both the block perform mvn install .. and then for from inside the webapp block mvn jetty:run

so the page

http://localhost:8888/test-block/index.html

is generated using the parallel generator .. if ur lucky and dont get the error the first time then just refresh and am sure the error will be there .. there are five components included in that page ... out of which 4 are normal jx components and the fifth one is cforms component ... all the jx component have been assigned different instance of the same object but still the properties displayed on the page shows the same value(they should be different) .. if u execute query for these components individually then they appear without any error .. links for querying them individually

[jx components .. shows xml output .. forgot to add the pipeline to serialize it to html.. u can add one]
http://localhost:8888/test-block/showjx
http://localhost:8888/test-block/showjx1
http://localhost:8888/test-block/showjx2
http://localhost:8888/test-block/showjx3

[cforms component]
http://localhost:8888/test-block/form2bean.flow

most of the time we get the error
Caused by: org.apache.commons.jxpath.JXPathException: No value for xpath: $cocoon/continuation/id
its because the objectmodel is being changed by all the components simultaneously ...the error is there in the log file not on the page .. on the page it will just ignore that component

> IncludeCacheManager can not perfom parallel includes
> ----------------------------------------------------
>
>                 Key: COCOON-2216
>                 URL: https://issues.apache.org/jira/browse/COCOON-2216
>             Project: Cocoon
>          Issue Type: Bug
>          Components: - Components: Sitemap
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Christoph Gaffga
>         Attachments: cocoon-trunk.patch, ParallelInclusionProblem-cocoon_TRUNK.patch, test-block.zip, test-webapp.zip
>
>
> Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.
> We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
> see also: http://www.mail-archive.com/users@.../msg42173.html
> I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...
> But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
> and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.
> We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/COCOON-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Grzegorz Kossakowski reassigned COCOON-2216:
--------------------------------------------

    Assignee: Grzegorz Kossakowski

> IncludeCacheManager can not perfom parallel includes
> ----------------------------------------------------
>
>                 Key: COCOON-2216
>                 URL: https://issues.apache.org/jira/browse/COCOON-2216
>             Project: Cocoon
>          Issue Type: Bug
>          Components: - Components: Sitemap
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Christoph Gaffga
>            Assignee: Grzegorz Kossakowski
>         Attachments: cocoon-trunk.patch, ParallelInclusionProblem-cocoon_TRUNK.patch, test-block.zip, test-webapp.zip
>
>
> Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.
> We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
> see also: http://www.mail-archive.com/users@.../msg42173.html
> I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...
> But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
> and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.
> We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/COCOON-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622964#action_12622964 ]

Grzegorz Kossakowski commented on COCOON-2216:
----------------------------------------------

Sorry for big delay but I was busy with other tasks which got a higher priority. Finally I'm finished and I have some free time to spend on this issue which (to be honest) is not the easiest one.

Now I'll at least try to reproduce your problem and try to estimate how much of work is needed to fix this in proper way.

> IncludeCacheManager can not perfom parallel includes
> ----------------------------------------------------
>
>                 Key: COCOON-2216
>                 URL: https://issues.apache.org/jira/browse/COCOON-2216
>             Project: Cocoon
>          Issue Type: Bug
>          Components: - Components: Sitemap
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Christoph Gaffga
>            Assignee: Grzegorz Kossakowski
>         Attachments: cocoon-trunk.patch, ParallelInclusionProblem-cocoon_TRUNK.patch, test-block.zip, test-webapp.zip
>
>
> Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.
> We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
> see also: http://www.mail-archive.com/users@.../msg42173.html
> I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...
> But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
> and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.
> We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/COCOON-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623150#action_12623150 ]

Grzegorz Kossakowski commented on COCOON-2216:
----------------------------------------------

After applying your patch I've managed to reproduce your problem.

I have some plan how to fix problems with ObjectModel but this will involve:
1. Enforcing that OM implementation implements Cloneable interface (change to API)
2. Implementing custom FactoryBean for OM in cocoon-sitemap-impl that will be aware of issues related to threads (but not only to threads, there some other, even more tricky issues waiting for someone to discover).
3. Making this custom FactoryBean aware of servlet: calls is crucial as well.

I've already created a branch in my Git repository for this issue and started to play with code. If nothing unplanned happens, I'll be able to give you some patches for testing upcoming week.

> IncludeCacheManager can not perfom parallel includes
> ----------------------------------------------------
>
>                 Key: COCOON-2216
>                 URL: https://issues.apache.org/jira/browse/COCOON-2216
>             Project: Cocoon
>          Issue Type: Bug
>          Components: - Components: Sitemap
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Christoph Gaffga
>            Assignee: Grzegorz Kossakowski
>         Attachments: cocoon-trunk.patch, ParallelInclusionProblem-cocoon_TRUNK.patch, test-block.zip, test-webapp.zip
>
>
> Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.
> We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
> see also: http://www.mail-archive.com/users@.../msg42173.html
> I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...
> But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
> and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.
> We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/COCOON-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623350#action_12623350 ]

imran pariyani commented on COCOON-2216:
----------------------------------------

great !! .. i will wait for the patches .. thanks :)

> IncludeCacheManager can not perfom parallel includes
> ----------------------------------------------------
>
>                 Key: COCOON-2216
>                 URL: https://issues.apache.org/jira/browse/COCOON-2216
>             Project: Cocoon
>          Issue Type: Bug
>          Components: - Components: Sitemap
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Christoph Gaffga
>            Assignee: Grzegorz Kossakowski
>         Attachments: cocoon-trunk.patch, ParallelInclusionProblem-cocoon_TRUNK.patch, test-block.zip, test-webapp.zip
>
>
> Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.
> We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
> see also: http://www.mail-archive.com/users@.../msg42173.html
> I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...
> But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
> and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.
> We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/COCOON-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624370#action_12624370 ]

Grzegorz Kossakowski commented on COCOON-2216:
----------------------------------------------

Hi Imran,

I worked on this issue for a few hours but this turned out to be even more complicated than I thought so I have no working code yet and tomorrow I won't be able to work on this. :(

As I said, the idea is to create custom bean factory, I've started to write a prototype of it:
package org.apache.cocoon.components.pipeline.spring;

import java.util.Stack;

public class ObjectModelFactoryBean implements
                FactoryBean {
   
    private InheritableThreadLocal<Stack<ObjectModel>> objectModelStack = new InheritableThreadLocal<Stack<ObjectModel>>() {
       
        protected Stack<ObjectModel> initialValue() {
            Stack<ObjectModel> newStack = new Stack<ObjectModel>();
            newStack.push(createObjectModel());
            return newStack;
        };
       
        protected Stack<ObjectModel> childValue(Stack<ObjectModel> parentValue) {
            Stack<ObjectModel> newStack = new Stack<ObjectModel>();
            try {
                newStack.push((ObjectModel)parentValue.peek().clone());
            } catch (CloneNotSupportedException e) {
                // TODO Have to add some logging
                e.printStackTrace();
            }
            return newStack;
        };
    };

        public Object getObject() throws Exception {
            synchronizeStack();
                return objectModelStack.get().peek();
        }

        public Class<ObjectModel> getObjectType() {
                return ObjectModel.class;
        }

        public boolean isSingleton() {
                return false;
        }
       
        /**
         * This method is heart of this class.
         * It's purpose is to synchronize its own internal stack with stack maintained by Servlet Service Framework
         * for servlet: calls and stack maintained by Cocoon Core for cocoon: calls.
         */
        protected void synchronizeStack() {
            //no implementation at the moment
        }
       
        protected ObjectModel createObjectModel() {
            //here we'll need to create ObjectModelImpl instance
            //and inject all initial values
            return null;
        }
}

Only once I started to write it I realized that there is a problem with cyclic dependencies caused by this class. The point is that, synchronizeStack must have an access to both Cocoon Core and Servlet Service Framework classes and itself must be placed in Cocoon Core.

But, by design, Cocoon core cannot depend on SSF. If I decide to put it somewhere then cyclic dependencies are starting to appear. I have some ideas how to create two different object factories and design whole thing in layered why so layers can be independent but this will take me some time.

On the other hand, this issue has got a higher priority for me because my company that I'm working at want to have it fixed as well so at least you can be sure that it will get fixed sooner or later.

I'll continue my work on this issue on monday and I'll have to fix it until Friday because later on I'll be busy with completely Cocoon-unrelated stuff.

> IncludeCacheManager can not perfom parallel includes
> ----------------------------------------------------
>
>                 Key: COCOON-2216
>                 URL: https://issues.apache.org/jira/browse/COCOON-2216
>             Project: Cocoon
>          Issue Type: Bug
>          Components: - Components: Sitemap
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Christoph Gaffga
>            Assignee: Grzegorz Kossakowski
>         Attachments: cocoon-trunk.patch, ParallelInclusionProblem-cocoon_TRUNK.patch, test-block.zip, test-webapp.zip
>
>
> Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.
> We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
> see also: http://www.mail-archive.com/users@.../msg42173.html
> I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...
> But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
> and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.
> We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/COCOON-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624395#action_12624395 ]

imran pariyani commented on COCOON-2216:
----------------------------------------

Hi Grzegorz,

I appreciate your efforts Thanks .. would like to see the implementation of the method synchronizeStack() .. hope to see it by next friday :)

> IncludeCacheManager can not perfom parallel includes
> ----------------------------------------------------
>
>                 Key: COCOON-2216
>                 URL: https://issues.apache.org/jira/browse/COCOON-2216
>             Project: Cocoon
>          Issue Type: Bug
>          Components: - Components: Sitemap
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Christoph Gaffga
>            Assignee: Grzegorz Kossakowski
>         Attachments: cocoon-trunk.patch, ParallelInclusionProblem-cocoon_TRUNK.patch, test-block.zip, test-webapp.zip
>
>
> Since we migrated from cocoon 2.1 to 2.2 a generator that merges multiple sources from other cocoon pipelines into one (similar to the aggregator) is not working anymore.
> We also posted our problem to the mailing list, got little feedback but it brought us on the right way...
> see also: http://www.mail-archive.com/users@.../msg42173.html
> I found out that it's a problem with the DefaultIncludeCacheManager, that can not do parallel inclusion of cocoon-pipelines anymore. I checked several classes where inclusion is used. In the aggregator parallel inclusion is not an option anymore, in CIncludeTransformer the IncludeCacheManager is used, but it can't do parallel inclusion. In the new IncludeTransfomer parallel inclusion is supported, but it does not use caching as it does not use the IncludeCacheManager...
> But we needed caching AND parallel processing, so I tried to find out what's broken in the DefaultIncludeCacheManager:
> and it seems that the ThreadLocal variables are not initialized for the child threads that do the inclusion. Neither the spring context nor the old environment stuff was initialized. And all the source resolving was done outside the child thread and that way using the wrong thread context.
> We were able to fix that issue by small changes to DefaultIncludeCacheManager and IncludeCacheManagerSession. It would be great if somebody could apply this patch so we don'T have to patch every cocoon version again and again...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.