|
View:
New views
3 Messages
—
Rating Filter:
Alert me
|
|
|
Cocoon 2.1.10 Application FreezingHi, I am sort of going to get a heart attack in
the next hours, if I can’t resolve this quickly. We recently deployed a new Version of a
intranet-application of a large logistic company here in Germany. It is
completely written in cocoon and performs extraordinary nicely. Until at one
point the System starts thrashing. CPU Usage climbs from 12% to 80% and no
request are served anymore. Here a little more details on the Setup: -
Windows 2003 Server -
4 GB Ram … 1,5 GB reserved
for the appserver -
Bea Weblogic 9.2 -
JRockit 1.5 VM -
MySQL 5.1 DB Backend During normal operation a thread dump is
about 30k big … during thrashing this increases to 1,5MB. Yesterday we had similar Problems which
were unfortunately related to too much logging (100 Threads blocked because
they were locked from the Cocoon Log system). After removing a lot of the Log
Output the system performed nicely even under heavier load. Until the whole
thing blew up again. This time the Thread Dump showed a lot of Threads blocked
at: "[ACTIVE] ExecuteThread: '3' for
queue: 'weblogic.kernel.Default (self-tuning)'" id=35 idx=0x80 tid=5092
prio=5 alive, in native, blocked, daemon -- Blocked trying to get lock:
java/lang/Object@0x011A8D38[unlocked] at
jrockit/vm/Threads.waitForUnblockSignal()V(Native Method) at
jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1630)[optimized] at
jrockit/vm/Locks.lockFat(Locks.java:1731)[optimized] at
jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1277)[optimized] at
org/apache/avalon/excalibur/pool/InstrumentedResourceLimitingPool.put(InstrumentedResourceLimitingPool.java:411)[inlined] at
org/apache/avalon/excalibur/component/PoolableComponentHandler.doPut(PoolableComponentHandler.java:212)[optimized] at
org/apache/avalon/excalibur/component/ComponentHandler.put(ComponentHandler.java:425)[inlined] at
org/apache/avalon/excalibur/component/ExcaliburComponentSelector.release(ExcaliburComponentSelector.java:307)[inlined] at
org/apache/cocoon/components/ExtendedComponentSelector.release(ExtendedComponentSelector.java:301)[optimized] at
org/apache/cocoon/components/pipeline/AbstractProcessingPipeline.recycle(AbstractProcessingPipeline.java:743)[optimized] at
org/apache/avalon/excalibur/pool/InstrumentedResourceLimitingPool.put(InstrumentedResourceLimitingPool.java:407)[inlined] at
org/apache/avalon/excalibur/component/PoolableComponentHandler.doPut(PoolableComponentHandler.java:212)[optimized] at
org/apache/avalon/excalibur/component/ComponentHandler.put(ComponentHandler.java:425)[inlined] at
org/apache/avalon/excalibur/component/ExcaliburComponentSelector.release(ExcaliburComponentSelector.java:307)[inlined] at
org/apache/cocoon/components/ExtendedComponentSelector.release(ExtendedComponentSelector.java:301)[optimized] at org/apache/cocoon/components/treeprocessor/InvokeContext.dispose(InvokeContext.java:283) at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:179)[inlined] at
org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcessor.java:253)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/MountNode.invoke(MountNode.java:118)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNode.java:108)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(PipelineNode.java:143)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(PipelinesNode.java:93)[optimized] at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:235)[optimized] at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:177)[inlined] at
org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcessor.java:253)[inlined] at org/apache/cocoon/components/treeprocessor/sitemap/MountNode.invoke(MountNode.java:118)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/ActTypeNode.invoke(ActTypeNode.java:139)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] at org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNode.java:108)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] at org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(PipelineNode.java:143)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] at org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(PipelinesNode.java:93)[optimized] at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:235)[optimized] at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:177)[inlined] at
org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcessor.java:253)[optimized] at
org/apache/cocoon/Cocoon.process(Cocoon.java:699)[optimized] at
org/apache/cocoon/servlet/CocoonServlet.service(CocoonServlet.java:1154)[optimized] at
javax/servlet/http/HttpServlet.service(HttpServlet.java:856)[optimized] at
weblogic/servlet/internal/StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)[optimized] at
weblogic/servlet/internal/StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)[inlined] at
weblogic/servlet/internal/ServletStubImpl.execute(ServletStubImpl.java:283)[optimized] at weblogic/servlet/internal/TailFilter.doFilter(TailFilter.java:26)[optimized] at
weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[optimized] at
de/transa/disco/filter/SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:104)[optimized] at
weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[optimized] at
de/transa/disco/database/hibernate/CloseHibernateSessionFilter.doFilter(CloseHibernateSessionFilter.java:25)[optimized] at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[inlined] at
weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3242)[optimized] at
weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)[optimized] at
weblogic/security/service/SecurityManager.runAs(SecurityManager.java:121)[inlined] at
weblogic/servlet/internal/WebAppServletContext.securedExecute(WebAppServletContext.java:2010)[inlined] at weblogic/servlet/internal/WebAppServletContext.execute(WebAppServletContext.java:1916)[optimized] at
weblogic/servlet/internal/ServletRequestImpl.run(ServletRequestImpl.java:1366)[optimized] at
weblogic/work/ExecuteThread.execute(ExecuteThread.java:209)[optimized] at
weblogic/work/ExecuteThread.run(ExecuteThread.java:181) at jrockit/vm/RNI.c2java(IIIII)V(Native
Method) -- end of trace As far as I understand the thing, the
problem are pool limitations to some of the sitemap components. In general this
should not be a problem. Unfortunately my application uses JavaFlow and a lot
of Flows have to send Data to other Systems. They all use internal pipelines to
generate this output. My guess now is, that if a Flow is executed to produce
output, it blocks some resources. Maybe some resources it needs itself to
finish the request, causing a deadlock. This would be the case in the following
Thread-Dump: "[ACTIVE] ExecuteThread: '4' for
queue: 'weblogic.kernel.Default (self-tuning)'" id=36 idx=0x84 tid=5100
prio=5 alive, in native, native_blocked, daemon at
jrockit/vm/Allocator.allocLargeObjectOrArray(IIIZ)Ljava/lang/Object;(Native
Method) at
jrockit/vm/Allocator.allocLargerThanFreeTLA(Allocator.java:831)[inlined] at
jrockit/vm/Allocator.allocSlowCaseInner(Allocator.java:947)[inlined] at
jrockit/vm/Allocator.allocSlowCase(Allocator.java:793)[optimized] at
org/apache/cocoon/util/BufferedOutputStream.<init>(BufferedOutputStream.java:63)[optimized] at
org/apache/cocoon/util/BufferedOutputStream.<init>(BufferedOutputStream.java:46) at
org/apache/cocoon/environment/AbstractEnvironment.getOutputStream(AbstractEnvironment.java:490) at
org/apache/cocoon/environment/http/HttpEnvironment.getOutputStream(HttpEnvironment.java:263) at
org/apache/cocoon/environment/wrapper/EnvironmentWrapper.getOutputStream(EnvironmentWrapper.java:311) at
org/apache/cocoon/components/pipeline/AbstractProcessingPipeline.processXMLPipeline(AbstractProcessingPipeline.java:577)[optimized] at
org/apache/cocoon/components/pipeline/AbstractProcessingPipeline.process(AbstractProcessingPipeline.java:481)[optimized] at
org/apache/cocoon/components/treeprocessor/sitemap/SerializeNode.invoke(SerializeNode.java:121)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNode.java:108)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(PipelineNode.java:143)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(PipelinesNode.java:93)[optimized] at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:235)[optimized] at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.handleCocoonRedirect(ConcreteTreeProcessor.java:299)[optimized] at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.access$000(ConcreteTreeProcessor.java:48)[inlined] at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor$TreeProcessorRedirector.cocoonRedirect(ConcreteTreeProcessor.java:340)[inlined] at
org/apache/cocoon/environment/ForwardRedirector.redirect(ForwardRedirector.java:60)[optimized] at
org/apache/cocoon/components/flow/java/AbstractContinuable.sendPage(AbstractContinuable.java:99)[optimized] at
de/transa/disco/flow/im/InstantMessengerFlow.doCheckMessageClosed(InstantMessengerFlow.java:88) at
sun/reflect/GeneratedMethodAccessor1853.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Unknown
Source) at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)[optimized] at
java/lang/reflect/Method.invoke(Method.java:585)[optimized] at
org/apache/cocoon/components/flow/java/JavaInterpreter.callFunction(JavaInterpreter.java:173)[optimized] at
org/apache/cocoon/components/treeprocessor/sitemap/CallFunctionNode.invoke(CallFunctionNode.java:139)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNode.java:108)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] at org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(PipelineNode.java:143)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] at org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(PipelinesNode.java:93)[optimized] at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:235)[optimized] at org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:177)[inlined] at
org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcessor.java:253)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/MountNode.invoke(MountNode.java:118)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNode.java:108)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(PipelineNode.java:143)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(PipelinesNode.java:93)[optimized] at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:235)[optimized] at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:177)[inlined] at
org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcessor.java:253)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/MountNode.invoke(MountNode.java:118)[optimized] at org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/ActTypeNode.invoke(ActTypeNode.java:139)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNode.java:108)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(PipelineNode.java:143)[optimized] at
org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] at
org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(PipelinesNode.java:93)[optimized] at org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:235)[optimized] at
org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:177)[inlined] at org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcessor.java:253)[optimized] at
org/apache/cocoon/Cocoon.process(Cocoon.java:699)[optimized] at
org/apache/cocoon/servlet/CocoonServlet.service(CocoonServlet.java:1154)[optimized] at javax/servlet/http/HttpServlet.service(HttpServlet.java:856)[optimized] at
weblogic/servlet/internal/StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)[optimized] at
weblogic/servlet/internal/StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)[inlined] at
weblogic/servlet/internal/ServletStubImpl.execute(ServletStubImpl.java:283)[optimized] at
weblogic/servlet/internal/TailFilter.doFilter(TailFilter.java:26)[optimized] at
weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[optimized] at
de/transa/disco/filter/SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:104)[optimized] at
weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[optimized] at
de/transa/disco/database/hibernate/CloseHibernateSessionFilter.doFilter(CloseHibernateSessionFilter.java:25)[optimized] at
weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[inlined] at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3242)[optimized] at
weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)[optimized] at
weblogic/security/service/SecurityManager.runAs(SecurityManager.java:121)[inlined] at
weblogic/servlet/internal/WebAppServletContext.securedExecute(WebAppServletContext.java:2010)[inlined] at
weblogic/servlet/internal/WebAppServletContext.execute(WebAppServletContext.java:1916)[optimized] at weblogic/servlet/internal/ServletRequestImpl.run(ServletRequestImpl.java:1366)[optimized] at
weblogic/work/ExecuteThread.execute(ExecuteThread.java:209)[optimized] at
weblogic/work/ExecuteThread.run(ExecuteThread.java:181) at jrockit/vm/RNI.c2java(IIIII)V(Native
Method) -- end of trace Causing the System to immediately run out
of resources. The only solution I can think of would be
to turn off pooling of components. Where and how can I do this? Is this simply
done by setting the pool-max properties in the core.properties to 0 or do I
have to do more? We currently have 4 CPUs and 4GB Ram, but
this could be extended if we need to (Virtual environment). Regards, Chris [ C h
r i s t o f e r D u t z ] FA Darmstadt: 07 813 60581 |
|
|
Re: Cocoon 2.1.10 Application FreezingA couple of thoughts.
1. In my experience turning off pooling will make matters worse, not better. 2. You show threads blocked, but they aren't using CPU. What is using all the CPU? Look at the threads that are actually running. Chances are the clue to the bottleneck will be there. The two threads you have shown say "blocked" so I doubt either of them is the cause. 3. If the bottleneck can't be alleviated then you need to increase your pool size to allow for that. Christofer.Dutz@... wrote: > > Hi, > > I am sort of going to get a heart attack in the next hours, if I can’t > resolve this quickly. > > We recently deployed a new Version of a intranet-application of a > large logistic company here in Germany. It is completely written in > cocoon and performs extraordinary nicely. Until at one point the > System starts thrashing. CPU Usage climbs from 12% to 80% and no > request are served anymore. > > Here a little more details on the Setup: > > - Windows 2003 Server > > - 4 GB Ram … 1,5 GB reserved for the appserver > > - Bea Weblogic 9.2 > > - JRockit 1.5 VM > > - MySQL 5.1 DB Backend > > During normal operation a thread dump is about 30k big … during > thrashing this increases to 1,5MB. > > Yesterday we had similar Problems which were unfortunately related to > too much logging (100 Threads blocked because they were locked from > the Cocoon Log system). After removing a lot of the Log Output the > system performed nicely even under heavier load. Until the whole thing > blew up again. This time the Thread Dump showed a lot of Threads > blocked at: > > "[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default > (self-tuning)'" id=35 idx=0x80 tid=5092 prio=5 alive, in native, > blocked, daemon > > -- Blocked trying to get lock: java/lang/Object@0x011A8D38[unlocked] > > at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method) > > at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1630)[optimized] > > at jrockit/vm/Locks.lockFat(Locks.java:1731)[optimized] > > at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1277)[optimized] > > at > org/apache/avalon/excalibur/pool/InstrumentedResourceLimitingPool.put(InstrumentedResourceLimitingPool.java:411)[inlined] > > at > org/apache/avalon/excalibur/component/PoolableComponentHandler.doPut(PoolableComponentHandler.java:212)[optimized] > > at > org/apache/avalon/excalibur/component/ComponentHandler.put(ComponentHandler.java:425)[inlined] > > at > org/apache/avalon/excalibur/component/ExcaliburComponentSelector.release(ExcaliburComponentSelector.java:307)[inlined] > > at > org/apache/cocoon/components/ExtendedComponentSelector.release(ExtendedComponentSelector.java:301)[optimized] > > at > org/apache/cocoon/components/pipeline/AbstractProcessingPipeline.recycle(AbstractProcessingPipeline.java:743)[optimized] > > at > org/apache/avalon/excalibur/pool/InstrumentedResourceLimitingPool.put(InstrumentedResourceLimitingPool.java:407)[inlined] > > at > org/apache/avalon/excalibur/component/PoolableComponentHandler.doPut(PoolableComponentHandler.java:212)[optimized] > > at > org/apache/avalon/excalibur/component/ComponentHandler.put(ComponentHandler.java:425)[inlined] > > at > org/apache/avalon/excalibur/component/ExcaliburComponentSelector.release(ExcaliburComponentSelector.java:307)[inlined] > > at > org/apache/cocoon/components/ExtendedComponentSelector.release(ExtendedComponentSelector.java:301)[optimized] > > at > org/apache/cocoon/components/treeprocessor/InvokeContext.dispose(InvokeContext.java:283) > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:179)[inlined] > > at > org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcessor.java:253)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MountNode.invoke(MountNode.java:118)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNode.java:108)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(PipelineNode.java:143)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(PipelinesNode.java:93)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:235)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:177)[inlined] > > at > org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcessor.java:253)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MountNode.invoke(MountNode.java:118)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/ActTypeNode.invoke(ActTypeNode.java:139)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNode.java:108)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(PipelineNode.java:143)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(PipelinesNode.java:93)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:235)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:177)[inlined] > > at > org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcessor.java:253)[optimized] > > at org/apache/cocoon/Cocoon.process(Cocoon.java:699)[optimized] > > at > org/apache/cocoon/servlet/CocoonServlet.service(CocoonServlet.java:1154)[optimized] > > at javax/servlet/http/HttpServlet.service(HttpServlet.java:856)[optimized] > > at > weblogic/servlet/internal/StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)[optimized] > > at > weblogic/servlet/internal/StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)[inlined] > > at > weblogic/servlet/internal/ServletStubImpl.execute(ServletStubImpl.java:283)[optimized] > > at > weblogic/servlet/internal/TailFilter.doFilter(TailFilter.java:26)[optimized] > > at > weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[optimized] > > at > de/transa/disco/filter/SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:104)[optimized] > > at > weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[optimized] > > at > de/transa/disco/database/hibernate/CloseHibernateSessionFilter.doFilter(CloseHibernateSessionFilter.java:25)[optimized] > > at > weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[inlined] > > at > weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3242)[optimized] > > at > weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)[optimized] > > at > weblogic/security/service/SecurityManager.runAs(SecurityManager.java:121)[inlined] > > at > weblogic/servlet/internal/WebAppServletContext.securedExecute(WebAppServletContext.java:2010)[inlined] > > at > weblogic/servlet/internal/WebAppServletContext.execute(WebAppServletContext.java:1916)[optimized] > > at > weblogic/servlet/internal/ServletRequestImpl.run(ServletRequestImpl.java:1366)[optimized] > > at weblogic/work/ExecuteThread.execute(ExecuteThread.java:209)[optimized] > > at weblogic/work/ExecuteThread.run(ExecuteThread.java:181) > > at jrockit/vm/RNI.c2java(IIIII)V(Native Method) > > -- end of trace > > As far as I understand the thing, the problem are pool limitations to > some of the sitemap components. In general this should not be a > problem. Unfortunately my application uses JavaFlow and a lot of Flows > have to send Data to other Systems. They all use internal pipelines to > generate this output. My guess now is, that if a Flow is executed to > produce output, it blocks some resources. Maybe some resources it > needs itself to finish the request, causing a deadlock. This would be > the case in the following Thread-Dump: > > "[ACTIVE] ExecuteThread: '4' for queue: 'weblogic.kernel.Default > (self-tuning)'" id=36 idx=0x84 tid=5100 prio=5 alive, in native, > native_blocked, daemon > > at > jrockit/vm/Allocator.allocLargeObjectOrArray(IIIZ)Ljava/lang/Object;(Native > Method) > > at > jrockit/vm/Allocator.allocLargerThanFreeTLA(Allocator.java:831)[inlined] > > at jrockit/vm/Allocator.allocSlowCaseInner(Allocator.java:947)[inlined] > > at jrockit/vm/Allocator.allocSlowCase(Allocator.java:793)[optimized] > > at > org/apache/cocoon/util/BufferedOutputStream.<init>(BufferedOutputStream.java:63)[optimized] > > at > org/apache/cocoon/util/BufferedOutputStream.<init>(BufferedOutputStream.java:46) > > at > org/apache/cocoon/environment/AbstractEnvironment.getOutputStream(AbstractEnvironment.java:490) > > at > org/apache/cocoon/environment/http/HttpEnvironment.getOutputStream(HttpEnvironment.java:263) > > at > org/apache/cocoon/environment/wrapper/EnvironmentWrapper.getOutputStream(EnvironmentWrapper.java:311) > > at > org/apache/cocoon/components/pipeline/AbstractProcessingPipeline.processXMLPipeline(AbstractProcessingPipeline.java:577)[optimized] > > at > org/apache/cocoon/components/pipeline/AbstractProcessingPipeline.process(AbstractProcessingPipeline.java:481)[optimized] > > at > org/apache/cocoon/components/treeprocessor/sitemap/SerializeNode.invoke(SerializeNode.java:121)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNode.java:108)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(PipelineNode.java:143)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(PipelinesNode.java:93)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:235)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.handleCocoonRedirect(ConcreteTreeProcessor.java:299)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.access$000(ConcreteTreeProcessor.java:48)[inlined] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor$TreeProcessorRedirector.cocoonRedirect(ConcreteTreeProcessor.java:340)[inlined] > > at > org/apache/cocoon/environment/ForwardRedirector.redirect(ForwardRedirector.java:60)[optimized] > > at > org/apache/cocoon/components/flow/java/AbstractContinuable.sendPage(AbstractContinuable.java:99)[optimized] > > at > de/transa/disco/flow/im/InstantMessengerFlow.doCheckMessageClosed(InstantMessengerFlow.java:88) > > at > sun/reflect/GeneratedMethodAccessor1853.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Unknown > Source) > > at > sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)[optimized] > > at java/lang/reflect/Method.invoke(Method.java:585)[optimized] > > at > org/apache/cocoon/components/flow/java/JavaInterpreter.callFunction(JavaInterpreter.java:173)[optimized] > > at > org/apache/cocoon/components/treeprocessor/sitemap/CallFunctionNode.invoke(CallFunctionNode.java:139)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNode.java:108)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(PipelineNode.java:143)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(PipelinesNode.java:93)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:235)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:177)[inlined] > > at > org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcessor.java:253)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MountNode.invoke(MountNode.java:118)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNode.java:108)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(PipelineNode.java:143)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(PipelinesNode.java:93)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:235)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:177)[inlined] > > at > org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcessor.java:253)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MountNode.invoke(MountNode.java:118)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/ActTypeNode.invoke(ActTypeNode.java:139)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNode.java:108)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(PipelineNode.java:143)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(PipelinesNode.java:93)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:235)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:177)[inlined] > > at > org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcessor.java:253)[optimized] > > at org/apache/cocoon/Cocoon.process(Cocoon.java:699)[optimized] > > at > org/apache/cocoon/servlet/CocoonServlet.service(CocoonServlet.java:1154)[optimized] > > at javax/servlet/http/HttpServlet.service(HttpServlet.java:856)[optimized] > > at > weblogic/servlet/internal/StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)[optimized] > > at > weblogic/servlet/internal/StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)[inlined] > > at > weblogic/servlet/internal/ServletStubImpl.execute(ServletStubImpl.java:283)[optimized] > > at > weblogic/servlet/internal/TailFilter.doFilter(TailFilter.java:26)[optimized] > > at > weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[optimized] > > at > de/transa/disco/filter/SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:104)[optimized] > > at > weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[optimized] > > at > de/transa/disco/database/hibernate/CloseHibernateSessionFilter.doFilter(CloseHibernateSessionFilter.java:25)[optimized] > > at > weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[inlined] > > at > weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3242)[optimized] > > at > weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)[optimized] > > at > weblogic/security/service/SecurityManager.runAs(SecurityManager.java:121)[inlined] > > at > weblogic/servlet/internal/WebAppServletContext.securedExecute(WebAppServletContext.java:2010)[inlined] > > at > weblogic/servlet/internal/WebAppServletContext.execute(WebAppServletContext.java:1916)[optimized] > > at > weblogic/servlet/internal/ServletRequestImpl.run(ServletRequestImpl.java:1366)[optimized] > > at weblogic/work/ExecuteThread.execute(ExecuteThread.java:209)[optimized] > > at weblogic/work/ExecuteThread.run(ExecuteThread.java:181) > > at jrockit/vm/RNI.c2java(IIIII)V(Native Method) > > -- end of trace > > Causing the System to immediately run out of resources. > > The only solution I can think of would be to turn off pooling of > components. Where and how can I do this? Is this simply done by > setting the pool-max properties in the core.properties to 0 or do I > have to do more? > > We currently have 4 CPUs and 4GB Ram, but this could be extended if we > need to (Virtual environment). > > Regards, > > Chris > > [ C h r i s t o f e r D u t z ] > > C-Ware IT-Service > Inhaber > Dipl. Inf. Christofer Dutz > Karlstraße. 104, 64285 Darmstadt > > fon: 0 61 51 / 27315 - 61 > fax: 0 61 51 / 27315 - 64 > mobil: 0171 / 7 444 2 33 > email: christofer.dutz@... <mailto:christofer.dutz@...> > http://www.c-ware.de <http://www.c-ware.de/> > > FA Darmstadt: 07 813 60581 > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... |
|
|
AW: [english 84%] Re: Cocoon 2.1.10 Application FreezingHi,
well I turned off the pooling and the system is behaving nicely. The problem I was having, was that tons of threads were blocked, which all blocked resources themselves. All resources were blocked by threads, that were blocked and hereby could not release them. I didn't turn off the pooling of DB-Connections and so on, but I turned it off for all Sitemap Components. In theory modern Garbage Collections work a lot better than pooling for small and cheap objects. In Spring they even suggest leaving it away completely. Since Yesterday I have to confirm this. My application is now surviving load-peaks it wouldn't have survived yesterday and the general average system load dropped about 5%. Regards, Chris -----Ursprüngliche Nachricht----- Von: Ralph Goers [mailto:Ralph.Goers@...] Gesendet: Donnerstag, 3. Juli 2008 05:53 An: users@... Betreff: [english 84%] Re: Cocoon 2.1.10 Application Freezing A couple of thoughts. 1. In my experience turning off pooling will make matters worse, not better. 2. You show threads blocked, but they aren't using CPU. What is using all the CPU? Look at the threads that are actually running. Chances are the clue to the bottleneck will be there. The two threads you have shown say "blocked" so I doubt either of them is the cause. 3. If the bottleneck can't be alleviated then you need to increase your pool size to allow for that. Christofer.Dutz@... wrote: > > Hi, > > I am sort of going to get a heart attack in the next hours, if I cant > resolve this quickly. > > We recently deployed a new Version of a intranet-application of a > large logistic company here in Germany. It is completely written in > cocoon and performs extraordinary nicely. Until at one point the > System starts thrashing. CPU Usage climbs from 12% to 80% and no > request are served anymore. > > Here a little more details on the Setup: > > - Windows 2003 Server > > - 4 GB Ram 1,5 GB reserved for the appserver > > - Bea Weblogic 9.2 > > - JRockit 1.5 VM > > - MySQL 5.1 DB Backend > > During normal operation a thread dump is about 30k big during > thrashing this increases to 1,5MB. > > Yesterday we had similar Problems which were unfortunately related to > too much logging (100 Threads blocked because they were locked from > the Cocoon Log system). After removing a lot of the Log Output the > system performed nicely even under heavier load. Until the whole thing > blew up again. This time the Thread Dump showed a lot of Threads > blocked at: > > "[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default > (self-tuning)'" id=35 idx=0x80 tid=5092 prio=5 alive, in native, > blocked, daemon > > -- Blocked trying to get lock: java/lang/Object@0x011A8D38[unlocked] > > at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method) > > at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1630)[optimized] > > at jrockit/vm/Locks.lockFat(Locks.java:1731)[optimized] > > at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1277)[optimized] > > at > mentedResourceLimitingPool.java:411)[inlined] > > at > org/apache/avalon/excalibur/component/PoolableComponentHandler.doPut(Poolabl eComponentHandler.java:212)[optimized] > > at > org/apache/avalon/excalibur/component/ComponentHandler.put(ComponentHandler. java:425)[inlined] > > at > org/apache/avalon/excalibur/component/ExcaliburComponentSelector.release(Exc aliburComponentSelector.java:307)[inlined] > > at > org/apache/cocoon/components/ExtendedComponentSelector.release(ExtendedCompo nentSelector.java:301)[optimized] > > at > org/apache/cocoon/components/pipeline/AbstractProcessingPipeline.recycle(Abs tractProcessingPipeline.java:743)[optimized] > > at > org/apache/avalon/excalibur/pool/InstrumentedResourceLimitingPool.put(Instru mentedResourceLimitingPool.java:407)[inlined] > > at > org/apache/avalon/excalibur/component/PoolableComponentHandler.doPut(Poolabl eComponentHandler.java:212)[optimized] > > at > org/apache/avalon/excalibur/component/ComponentHandler.put(ComponentHandler. java:425)[inlined] > > at > org/apache/avalon/excalibur/component/ExcaliburComponentSelector.release(Exc aliburComponentSelector.java:307)[inlined] > > at > org/apache/cocoon/components/ExtendedComponentSelector.release(ExtendedCompo nentSelector.java:301)[optimized] > > at > org/apache/cocoon/components/treeprocessor/InvokeContext.dispose(InvokeConte xt.java:283) > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(Con creteTreeProcessor.java:179)[inlined] > > at > org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcess or.java:253)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MountNode.invoke(MountNod e.java:118)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNod e.java:108)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(Pipel ineNode.java:143)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(Pipe linesNode.java:93)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(Con creteTreeProcessor.java:235)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(Con creteTreeProcessor.java:177)[inlined] > > at > org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcess or.java:253)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MountNode.invoke(MountNod e.java:118)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/ActTypeNode.invoke(ActTyp eNode.java:139)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNod e.java:108)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(Pipel ineNode.java:143)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(Pipe linesNode.java:93)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(Con creteTreeProcessor.java:235)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(Con creteTreeProcessor.java:177)[inlined] > > at > org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcess or.java:253)[optimized] > > at org/apache/cocoon/Cocoon.process(Cocoon.java:699)[optimized] > > at > org/apache/cocoon/servlet/CocoonServlet.service(CocoonServlet.java:1154)[opt imized] > > at javax/servlet/http/HttpServlet.service(HttpServlet.java:856)[optimized] > > at > weblogic/servlet/internal/StubSecurityHelper$ServletServiceAction.run(StubSe curityHelper.java:227)[optimized] > > at > weblogic/servlet/internal/StubSecurityHelper.invokeServlet(StubSecurityHelpe r.java:125)[inlined] > > at > weblogic/servlet/internal/ServletStubImpl.execute(ServletStubImpl.java:283)[ optimized] > > at > weblogic/servlet/internal/TailFilter.doFilter(TailFilter.java:26)[optimized] > > at > weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[ optimized] > > at > de/transa/disco/filter/SetCharacterEncodingFilter.doFilter(SetCharacterEncod ingFilter.java:104)[optimized] > > at > weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[ optimized] > > at > de/transa/disco/database/hibernate/CloseHibernateSessionFilter.doFilter(Clos eHibernateSessionFilter.java:25)[optimized] > > at > weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[ inlined] > > at > weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(W ebAppServletContext.java:3242)[optimized] > > at > weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubjec t.java:321)[optimized] > > at > weblogic/security/service/SecurityManager.runAs(SecurityManager.java:121)[in lined] > > at > weblogic/servlet/internal/WebAppServletContext.securedExecute(WebAppServletC ontext.java:2010)[inlined] > > at > weblogic/servlet/internal/WebAppServletContext.execute(WebAppServletContext. java:1916)[optimized] > > at > weblogic/servlet/internal/ServletRequestImpl.run(ServletRequestImpl.java:136 6)[optimized] > > at weblogic/work/ExecuteThread.execute(ExecuteThread.java:209)[optimized] > > at weblogic/work/ExecuteThread.run(ExecuteThread.java:181) > > at jrockit/vm/RNI.c2java(IIIII)V(Native Method) > > -- end of trace > > As far as I understand the thing, the problem are pool limitations to > some of the sitemap components. In general this should not be a > problem. Unfortunately my application uses JavaFlow and a lot of Flows > have to send Data to other Systems. They all use internal pipelines to > generate this output. My guess now is, that if a Flow is executed to > produce output, it blocks some resources. Maybe some resources it > needs itself to finish the request, causing a deadlock. This would be > the case in the following Thread-Dump: > > "[ACTIVE] ExecuteThread: '4' for queue: 'weblogic.kernel.Default > (self-tuning)'" id=36 idx=0x84 tid=5100 prio=5 alive, in native, > native_blocked, daemon > > at > > Method) > > at > jrockit/vm/Allocator.allocLargerThanFreeTLA(Allocator.java:831)[inlined] > > at jrockit/vm/Allocator.allocSlowCaseInner(Allocator.java:947)[inlined] > > at jrockit/vm/Allocator.allocSlowCase(Allocator.java:793)[optimized] > > at > :63)[optimized] > > at > org/apache/cocoon/util/BufferedOutputStream.<init>(BufferedOutputStream.java :46) > > at > org/apache/cocoon/environment/AbstractEnvironment.getOutputStream(AbstractEn vironment.java:490) > > at > org/apache/cocoon/environment/http/HttpEnvironment.getOutputStream(HttpEnvir onment.java:263) > > at > org/apache/cocoon/environment/wrapper/EnvironmentWrapper.getOutputStream(Env ironmentWrapper.java:311) > > at > org/apache/cocoon/components/pipeline/AbstractProcessingPipeline.processXMLP ipeline(AbstractProcessingPipeline.java:577)[optimized] > > at > org/apache/cocoon/components/pipeline/AbstractProcessingPipeline.process(Abs tractProcessingPipeline.java:481)[optimized] > > at > org/apache/cocoon/components/treeprocessor/sitemap/SerializeNode.invoke(Seri alizeNode.java:121)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNod e.java:108)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(Pipel ineNode.java:143)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(Pipe linesNode.java:93)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(Con creteTreeProcessor.java:235)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.handleCocoo nRedirect(ConcreteTreeProcessor.java:299)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.access$000( ConcreteTreeProcessor.java:48)[inlined] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor$TreeProcess orRedirector.cocoonRedirect(ConcreteTreeProcessor.java:340)[inlined] > > at > org/apache/cocoon/environment/ForwardRedirector.redirect(ForwardRedirector.j ava:60)[optimized] > > at > org/apache/cocoon/components/flow/java/AbstractContinuable.sendPage(Abstract Continuable.java:99)[optimized] > > at > de/transa/disco/flow/im/InstantMessengerFlow.doCheckMessageClosed(InstantMes sengerFlow.java:88) > > at > sun/reflect/GeneratedMethodAccessor1853.invoke(Ljava/lang/Object;[Ljava/lang /Object;)Ljava/lang/Object;(Unknown > Source) > > at > sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl .java:25)[optimized] > > at java/lang/reflect/Method.invoke(Method.java:585)[optimized] > > at > org/apache/cocoon/components/flow/java/JavaInterpreter.callFunction(JavaInte rpreter.java:173)[optimized] > > at > org/apache/cocoon/components/treeprocessor/sitemap/CallFunctionNode.invoke(C allFunctionNode.java:139)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNod e.java:108)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(Pipel ineNode.java:143)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(Pipe linesNode.java:93)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(Con creteTreeProcessor.java:235)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(Con creteTreeProcessor.java:177)[inlined] > > at > org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcess or.java:253)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MountNode.invoke(MountNod e.java:118)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNod e.java:108)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(Pipel ineNode.java:143)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(Pipe linesNode.java:93)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(Con creteTreeProcessor.java:235)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(Con creteTreeProcessor.java:177)[inlined] > > at > org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcess or.java:253)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MountNode.invoke(MountNod e.java:118)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/ActTypeNode.invoke(ActTyp eNode.java:139)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:47)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/MatchNode.invoke(MatchNod e.java:108)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelineNode.invoke(Pipel ineNode.java:143)[optimized] > > at > org/apache/cocoon/components/treeprocessor/AbstractParentProcessingNode.invo keNodes(AbstractParentProcessingNode.java:69)[inlined] > > at > org/apache/cocoon/components/treeprocessor/sitemap/PipelinesNode.invoke(Pipe linesNode.java:93)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(Con creteTreeProcessor.java:235)[optimized] > > at > org/apache/cocoon/components/treeprocessor/ConcreteTreeProcessor.process(Con creteTreeProcessor.java:177)[inlined] > > at > org/apache/cocoon/components/treeprocessor/TreeProcessor.process(TreeProcess or.java:253)[optimized] > > at org/apache/cocoon/Cocoon.process(Cocoon.java:699)[optimized] > > at > org/apache/cocoon/servlet/CocoonServlet.service(CocoonServlet.java:1154)[opt imized] > > at javax/servlet/http/HttpServlet.service(HttpServlet.java:856)[optimized] > > at > weblogic/servlet/internal/StubSecurityHelper$ServletServiceAction.run(StubSe curityHelper.java:227)[optimized] > > at > weblogic/servlet/internal/StubSecurityHelper.invokeServlet(StubSecurityHelpe r.java:125)[inlined] > > at > weblogic/servlet/internal/ServletStubImpl.execute(ServletStubImpl.java:283)[ optimized] > > at > weblogic/servlet/internal/TailFilter.doFilter(TailFilter.java:26)[optimized] > > at > weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[ optimized] > > at > de/transa/disco/filter/SetCharacterEncodingFilter.doFilter(SetCharacterEncod ingFilter.java:104)[optimized] > > at > weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[ optimized] > > at > de/transa/disco/database/hibernate/CloseHibernateSessionFilter.doFilter(Clos eHibernateSessionFilter.java:25)[optimized] > > at > weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42)[ inlined] > > at > weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(W ebAppServletContext.java:3242)[optimized] > > at > weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubjec t.java:321)[optimized] > > at > weblogic/security/service/SecurityManager.runAs(SecurityManager.java:121)[in lined] > > at > weblogic/servlet/internal/WebAppServletContext.securedExecute(WebAppServletC ontext.java:2010)[inlined] > > at > weblogic/servlet/internal/WebAppServletContext.execute(WebAppServletContext. java:1916)[optimized] > > at > weblogic/servlet/internal/ServletRequestImpl.run(ServletRequestImpl.java:136 6)[optimized] > > at weblogic/work/ExecuteThread.execute(ExecuteThread.java:209)[optimized] > > at weblogic/work/ExecuteThread.run(ExecuteThread.java:181) > > at jrockit/vm/RNI.c2java(IIIII)V(Native Method) > > -- end of trace > > Causing the System to immediately run out of resources. > > The only solution I can think of would be to turn off pooling of > components. Where and how can I do this? Is this simply done by > setting the pool-max properties in the core.properties to 0 or do I > have to do more? > > We currently have 4 CPUs and 4GB Ram, but this could be extended if we > need to (Virtual environment). > > Regards, > > Chris > > [ C h r i s t o f e r D u t z ] > > C-Ware IT-Service > Inhaber > Dipl. Inf. Christofer Dutz > Karlstraße. 104, 64285 Darmstadt > > fon: 0 61 51 / 27315 - 61 > fax: 0 61 51 / 27315 - 64 > mobil: 0171 / 7 444 2 33 > email: christofer.dutz@... <mailto:christofer.dutz@...> > http://www.c-ware.de <http://www.c-ware.de/> > > FA Darmstadt: 07 813 60581 > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... |
| Free Forum Powered by Nabble | Forum Help |