The Woes of CFThread

[Ian Skinner-3]

Is anybody using <cfthread...> functionality?  Is it working well for you?

I have been fighting for most of a month with <cfthread...>
functionality and it is driving me batty.

Run the application one time and it does all the processing, but does
not output the results.

Run it a second time and the system locks up and ColdFusion services
must be cycled.

Error trapping seems to be a crap shoot.

If anything goes wrong in any of the spawned threads and all is lost.  
Again ColdFusion has to be cycled.

UHHG


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308674
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[Raymond Camden]

On Mon, Jul 7, 2008 at 1:06 PM, Ian Skinner <HOF@...> wrote:
> Is anybody using <cfthread...> functionality?  Is it working well for you?

Yep.

> I have been fighting for most of a month with <cfthread...>
> functionality and it is driving me batty.

cfthread is a VERY powerful tool. It also takes a lot of care to use correctly.

>
> Run the application one time and it does all the processing, but does
> not output the results.

That is expected. The output is stored in the thread scope for the
thread itself. If you use the JOIN action (Which basically means, wait
for these threads to end), you can get the output and look at it. This
should be documented. (And let me add - be sure you have read the docs
COMPLETELY!)


> Run it a second time and the system locks up and ColdFusion services
> must be cycled.

Most likely it is something in your code.

> Error trapping seems to be a crap shoot.

Nope it isn't. Again, if you look at the thread data (if you use JOIN)
you can determine if an error was thrown. Or you can use try/catch to
log any errors.

It is your responsibility to monitor what happens in the threads.

> If anything goes wrong in any of the spawned threads and all is lost.
> Again ColdFusion has to be cycled.

Nope. See above. :)

--
===========================================================================
Raymond Camden, VP of Software Dev, Broadchoice

Email : ray@...
Blog : www.coldfusionjedi.com
AOL IM : cfjedimaster

Keep up to date with the community: http://www.coldfusionbloggers.org

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308686
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[Ian Skinner-3]

Raymond Camden wrote:
>
> That is expected. The output is stored in the thread scope for the
> thread itself. If you use the JOIN action (Which basically means, wait
> for these threads to end), you can get the output and look at it. This
> should be documented. (And let me add - be sure you have read the docs
> COMPLETELY!)
>  

Just to quickly address this point.  The expected output is not from the
threads.  It's basically a simple 'I'm Done' statement after the thread
loop and join.  The individual threads output is files written to the
server.  What is wierd is that sometimes all the threads have processed
as evidanced that all the expected files have been created and there are
no threads shown in the CF Monitor.  But the output at the end of the
file is not shown and the browser is still in waiting mode.



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308697
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[Raymond Camden]

So then look in your code thats running in the thread. Is there
something there that can hang? Do you have a deadlock of some sort?

On Mon, Jul 7, 2008 at 3:21 PM, Ian Skinner <HOF@...> wrote:

> Raymond Camden wrote:
>>
>> That is expected. The output is stored in the thread scope for the
>> thread itself. If you use the JOIN action (Which basically means, wait
>> for these threads to end), you can get the output and look at it. This
>> should be documented. (And let me add - be sure you have read the docs
>> COMPLETELY!)
>>
>
> Just to quickly address this point.  The expected output is not from the
> threads.  It's basically a simple 'I'm Done' statement after the thread
> loop and join.  The individual threads output is files written to the
> server.  What is wierd is that sometimes all the threads have processed
> as evidanced that all the expected files have been created and there are
> no threads shown in the CF Monitor.  But the output at the end of the
> file is not shown and the browser is still in waiting mode.
>
>
>
>

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308702
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[S. Isaac Dealey]

> So then look in your code thats running in the thread. Is there
> something there that can hang? Do you have a deadlock of some sort?

I was thinking about that and knowing that he said the threads were
writing files I was wondering about the number of threads he's spawning,
the number of files being written, the size of those files, what
webserver is in use (and whether or not it performs any file-locking and
whether or not that file-locking might be invoked, even though I doubt
it) and -- the relationship of all that information to the speed of the
write-head on the drive. I wouldn't expect an overabundance of
file-writing to cause the CF server to seize up completely and need to
be restarted, but then I also don't know the whole configuration, am not
an expert on server monitoring or hardware resources, etc. There just
seem to be a large number of variables potentially affecting it.

--
s. isaac dealey  ^  new epoch
 isn't it time for a change?
     ph: 617.365.5732

http://onTap.riaforge.org/blog



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308714
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[Ian Skinner-3]

Well I fail.

Can anybody point out what is fundamentally wrong with this code.  Such
that it has at least a 50% chance of complete failure resulting in a
non-responsive ColdFusion server that must be restarted.  What gets me
is that sometimes it works completely correct and produces all the
desired files.  The next time, with the exact same code and data, it
fails and hangs the server, but generates no exceptions or errors.  At
least none that I have been able to find.

<cfsetting requesttimeout="1800">
<cfinclude template = "scripts.cfm">
<cfset Application.DSN = LIC_DSN>

<h1>PROCESSING</h1>

<cfset variables.pathDate = #dateFormat(now(),'yyyy')#/>
<cfset variables.pIssueDate = #dateFormat(now(),'mm/dd/yy')#>

<!--- GET RENEWAL ID --->
<cfstoredproc procedure="REPORT_UTILS.getBatchRenewals"
  dataSource = "#LIC_DSN#">
  <cfprocresult name = "variables.ResultFirms">
</cfstoredproc>

<cfset variables.threads = "">
<ul>
<cfloop index = "i" from="1" to="#ResultFirms.recordcount#">
    <cfif i GT 10><cfbreak></cfif><!--- Development throttle ... only
process first 15 rows --->
    <cfset variables.threads =
listAppend(variables.threads,'batchRenew_thread#i#')>
    <li><cfoutput>batchRenew_thread#i#</cfoutput></li>
   
    <cfthread name="batchRenew_thread#i#" threadIndex="#i#"
threadPathDate="#pathDate#" action="run">
    <cfset var vRenewalReport = "">

    <cftry>
       
        <!--- GET RENEWAL ID --->
        <cfstoredproc procedure="REPORT_UTILS.getRenewalSequenceId"
            dataSource = "#LIC_DSN#">
            <cfprocparam type="out" variable="thread.vRenewalSeq"
CFSQLType="CF_SQL_VARCHAR">
        </cfstoredproc>
       
        <!--- PATH --->
        <cfset thread.vOutputFilePath = variables.RENEWALS_FILE_PATH &
               variables.pathDate & "\" &
                     Trim(variables.ResultFirms["firmno"][threadIndex])
& "\"/>
               
        <!--- CREATE OUTPUT PATH --->
        <cfif DirectoryExists(thread.vOutputFilePath) is False>
            <cfdirectory action="create"
directory="#thread.vOutputFilePath#"/>
        </cfif>
         
        <!--- FILE NAME --->
        <cfset thread.vOutputFilePath &= GetFileName()/>
       
        <!--- BUILD REPORT --->
        <cfreport template="ren.cfr" format="pdf"
name="vRenewalReport">    
            <cfreportparam name="pFirmNo"
value="#variables.ResultFirms['firmno'][threadIndex]#">
            <cfreportparam name="pIssueDate"
value="#DateFormat(variables.pIssueDate, 'mm/dd/yy')#">
            <cfreportparam name="pRenewalId" value="#thread.vRenewalSeq#">
        </cfreport>
       
        <cfpdf action="write" source="vRenewalReport"
destination="#thread.vOutputFilePath#" overwrite="yes">
       
        <cfset vRenewalReport = "">
       
        <!--- INSERT RENEWAL --->
        <cfstoredproc procedure="REPORT_UTILS.insertRenewal"
            dataSource = "#LIC_DSN#">
            <cfprocparam type="in" value="#thread.vRenewalSeq#"
CFSQLType="CF_SQL_NUMERIC">
            <cfprocparam type="in"
value="#variables.ResultFirms['firmno'][threadIndex]#"
CFSQLType="CF_SQL_NUMERIC">
            <cfprocparam type="in" value="#threadPathDate#"
CFSQLType="CF_SQL_NUMERIC">
        </cfstoredproc>
       
        <cfstoredproc procedure="REPORT_UTILS.insertRenewalProductsForFirm"
            dataSource = "#LIC_DSN#">
            <cfprocparam type="in" value="#thread.vRenewalSeq#"
CFSQLType="CF_SQL_NUMERIC">
            <cfprocparam type="in"
value="#ResultFirms['firmno'][threadIndex]#" CFSQLType="CF_SQL_NUMERIC">
        </cfstoredproc>
       
        <cfcatch type="any">
            <cflock name="treadLogLock" timeout="5" type="exclusive">
                    <cfset thread.cfcatch = duplicate(cfcatch)>
            </cflock>
        </cfcatch>
    </cftry>
    </cfthread>

    <!--- Pause every x threads to allow garbage collection.  --->
    <cfif i MOD 10 EQ 0>
        <cfthread action="join" name="#variables.threads#" timeout="2500"/>
       
        <!---  Loop over threads and kill any hanging threads --->
        <cfloop collection="#cfThread#" item="th">
            <cfif cfThread[th].status NEQ 'COMPLETED'>
                <cfthread action="terminate" name="#th#"/>
            </cfif>
        </cfloop>
<h2>SLEEPING
<cfoutput>#timeFormat(now(),"hh:mm:ss:l")#</cfoutput></h2>      
        <cfset sleep(10000)>
        <cfset variables.threads = "">
        </ul><ul>
    </cfif>
</cfloop>
</ul>

<h2>DONE</h2>
<cfthread action="join" name="#structKeyList(cfThread)#" timeout="10000" />

<!---  Loop over threads and kill any hanging threads --->
<cfloop collection="#cfThread#" item="th">
    <cfif cfThread[th].status NEQ 'COMPLETED'>
        <cfthread action="terminate" name="#th#"/>
    </cfif>
</cfloop>

<cfoutput>
<ul>
<cfloop collection="#cfThread#" item="th">
    <li style="margin-top: .5em;">#th#
        <ul>
        <cfloop collection="#cfThread[th]#" item="key">
           
            <li>#key#:   <cfif
isSimpleValue(cfThread[th][key])>#cfThread[th][key]#<cfelse><cfdump
var="#cfThread[th][key]#"></cfif></li>
        </cfloop>
        </ul>
    </li>
</cfloop>
</ul>
</cfoutput>

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308739
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[Ian Skinner-3]

Hello is this thing on?

Raymond Camden wrote:
> On Mon, Jul 7, 2008 at 1:06 PM, Ian Skinner <HOF@...> wrote:
>  
>> Is anybody using <cfthread...> functionality?  Is it working well for you?
>>    
>
> Yep.
>  
How are you using it?  Are there praticle limits on how many threads to
create, how long thread generating code can be running?  Other factors
to 'over using' this thread tool?

>  
>> I have been fighting for most of a month with <cfthread...>
>> functionality and it is driving me batty.
>>    
>
> cfthread is a VERY powerful tool. It also takes a lot of care to use correctly.
>  
Any pointers on what these cares might be?

>  
>> Run the application one time and it does all the processing, but does
>> not output the results.
>>    
>
> That is expected. The output is stored in the thread scope for the
> thread itself. If you use the JOIN action (Which basically means, wait
> for these threads to end), you can get the output and look at it. This
> should be documented. (And let me add - be sure you have read the docs
> COMPLETELY!)
>  

I have read the documents several times.  I understood the relationship
between threads and output.  The output I was refering to was after the
join statement and consisted of a "Done" string and a dump of the
'cfThread' structure.  I freely admit I do not completely understand all
the ins-and-outs which is why I am experimenting.  But I have been very
frustrated in that my attempts do not seem to yield predictable and
repeatable results.

>
>  
>> Run it a second time and the system locks up and ColdFusion services
>> must be cycled.
>>    
>
> Most likely it is something in your code.
>
>  
>> Error trapping seems to be a crap shoot.
>>    
>
> Nope it isn't. Again, if you look at the thread data (if you use JOIN)
> you can determine if an error was thrown. Or you can use try/catch to
> log any errors.
>
> It is your responsibility to monitor what happens in the threads.
>
>  
>> If anything goes wrong in any of the spawned threads and all is lost.
>> Again ColdFusion has to be cycled.
>>    
>
> Nope. See above. :)
>  

What could I add to the following thread to prevent it from generating
hung threads that often require a service restart to clear?

    <cfthread name="batchRenew_thread#i#" threadIndex="#i#"
threadPathDate="#pathDate#" action="run">
    <cfset var vRenewalReport = "">

    <cftry>
        <cfoutput>#threadIndex#: #threadPathDate#</cfoutput>
       
        <cfset thread.foobar = threadIndex & ": " & threadPathDate>
   
        <cfcatch type="any">
            <cflock name="treadLogLock" timeout="5" type="exclusive">
                    <cfset thread.cfcatch = duplicate(cfcatch)>
            </cflock>
        </cfcatch>
    </cftry>
    </cfthread>



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308744
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[S. Isaac Dealey]

Just glancing at this code snippet... I haven't done any work with
cfthread recently, but I have implemented it in the past... I wouldn't
expect the var keyword at the top to work. The thread does its own
encapsulation that's more similar to a custom tag, so I would expect the
var keyword to produce an error. I believe that's a parser error, which
happens before compiling the template, so the template with the cfthread
in it shouldn't even run. But then I've never tested this either.

Also I wouldn't lock that catch - it doesn't need a lock around it as
far as I can tell and if it did, then I suspect the lock would need to
be outside the try tags, because putting it inside the catch will mean
you still have a race condition. But I don't think a race condition
exists there in the first place because of the cfthread tag's
encapsulation. You also shouldn't need to duplicate the cfcatch. I don't
think it'll hurt anything, but it shouldn't be necessary.

hth,
ike

>     <cfthread name="batchRenew_thread#i#" threadIndex="#i#"
> threadPathDate="#pathDate#" action="run">
>     <cfset var vRenewalReport = "">
>
>     <cftry>
>         <cfoutput>#threadIndex#: #threadPathDate#</cfoutput>
>        
>         <cfset thread.foobar = threadIndex & ": " & threadPathDate>
>    
>         <cfcatch type="any">
>             <cflock name="treadLogLock" timeout="5" type="exclusive">
>                     <cfset thread.cfcatch = duplicate(cfcatch)>
>             </cflock>
>         </cfcatch>
>     </cftry>
>     </cfthread>
>

--
s. isaac dealey  ^  new epoch
 isn't it time for a change?
     ph: 781.769.0723

http://onTap.riaforge.org/blog



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308804
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[Rupesh Kumar]

Well, Did you check what is the thread doing when you say the Server is hung? Can you take a thread dump and post that here? You can take the thread dump using Server monitor or you can refer to my post http://coldfused.blogspot.com/2005/11/thread-dumps.html

however, I see few potential problems in the code
- In the loop, when you fire 10th threads, you join with timeout of 2.5 seconds, and if by that time, all the threads are not finished, you kill them all. So it might happen that threads would not have finished by that time

- If you see the docs and if you see my presentation, everywhere it is said that you should use 'terminate' only when absolutely necessary. Don't overuse it as it might lead the thread in some inconsistent state. You should code it in such way that you don't need to terminate the threads most of the time

- I don't think you should duplicate the catch. get the message from catch and set it to the thread. For test purpose you can remove catch and see what is the error that you get.

Always remember that the thread does not write any output or error to the response. It will be available in the thread scope.

Hope that helps.
Rupesh

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308844
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[Rupesh Kumar]

Well, Did you check what is the thread doing when you say the Server is hung? Can you take a thread dump and post that here? You can take the thread dump using Server monitor or you can refer to my post http://coldfused.blogspot.com/2005/11/thread-dumps.html

however, I see few potential problems in the code
- In the loop, when you fire 10th threads, you join with timeout of 2.5 seconds, and if by that time, all the threads are not finished, you kill them all. So it might happen that threads would not have finished by that time

- If you see the docs and if you see my presentation, everywhere it is said that you should use 'terminate' only when absolutely necessary. Don't overuse it as it might lead the thread in some inconsistent state. You should code it in such way that you don't need to terminate the threads most of the time

- I don't think you should duplicate the catch. get the message from catch and set it to the thread. For test purpose you can remove catch and see what is the error that you get.

Always remember that the thread does not write any output or error to the response. It will be available in the thread scope.

Hope that helps.
Rupesh

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308845
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[Dan G. Switzer, II]

Ian,

>Can anybody point out what is fundamentally wrong with this code.  Such
>that it has at least a 50% chance of complete failure resulting in a
>non-responsive ColdFusion server that must be restarted.  What gets me
>is that sometimes it works completely correct and produces all the
>desired files.  The next time, with the exact same code and data, it
>fails and hangs the server, but generates no exceptions or errors.  At
>least none that I have been able to find.
>
> [... clip ...]
>
>        <!--- BUILD REPORT --->
>        <cfreport template="ren.cfr" format="pdf"
>name="vRenewalReport">
>            <cfreportparam name="pFirmNo"
>value="#variables.ResultFirms['firmno'][threadIndex]#">
>            <cfreportparam name="pIssueDate"
>value="#DateFormat(variables.pIssueDate, 'mm/dd/yy')#">
>            <cfreportparam name="pRenewalId" value="#thread.vRenewalSeq#">
>        </cfreport>
>
>        <cfpdf action="write" source="vRenewalReport"
>destination="#thread.vOutputFilePath#" overwrite="yes">
>
> [... clip ...]

I wonder if the problem isn't with <cfthread /> but actually with heavy
usage of either <cfreport /> or <cfpdf /> tags.

Are you creating large reports? It could be a memory issue your running in
to. How much RAM is available to the JVM?

I would try simplifying things. Does it always work if you comment out the
<cfreport /> and <cfpdf /> tags?

If it does, what happens if you comment out the <cfreport /> tag and just
write a very simply PDF?

I could see how if you're generating some large PDFs, that without enough
RAM it could drive the JVM nuts.

-Dan


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308847
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[Rupesh Kumar]

Excellent point Dan!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308851
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[Brian Kotek]

Also, Ian, how many threads are you creating here based on the query
results? 10? 10,000?


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308855
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[Ian Skinner-3]

Brian Kotek wrote:
> Also, Ian, how many threads are you creating here based on the query
> results? 10? 10,000?
>  
For this process 1,371.  That is my basic question at this time -- What
are the practical limits of the <cfthread...>

Here is my simplest test case to date.  I seems to show that there is a
limit, but I really don't know where it might be.

<cfloop from="1" to="1500" index="i">
    <cfthread action="run" name="th_#i#" threadID="#i#">
        <cfset thread.foobar = threadID>
        <cfset thread.calc = threadID * randRange(1,10)>
    </cfthread>    
</cfloop>

<cfthread action="join" name="#structKeyList(cfThread)#" timeout="10000"/>

<cfdump var="#cfThread#" format="text" expand="no">


This code will frequently cause hung threads that NEVER end until the
server is restarted.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308856
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4

[Ian Skinner-3]

Dan G. Switzer, II wrote:

> Ian,
> I wonder if the problem isn't with <cfthread /> but actually with heavy
> usage of either <cfreport /> or <cfpdf /> tags.
>
> Are you creating large reports? It could be a memory issue your running in
> to. How much RAM is available to the JVM?
>
> I would try simplifying things. Does it always work if you comment out the
> <cfreport /> and <cfpdf /> tags?
>
> If it does, what happens if you comment out the <cfreport /> tag and just
> write a very simply PDF?
>
> I could see how if you're generating some large PDFs, that without enough
> RAM it could drive the JVM nuts.
>
> -Dan

The memory usage of the PDF report generation is defiantly a problem.  
It is what we where trying to address with the usage of the thread tag.  
The idea being that by splitting the process into separate threads and
throttle them down would allow the server time to clean up memory used
in previous iterations.

That is a topic for a seperate discussion.  How does <cfreport...> work
in large iterations.  We found that no matter what we did, the memory
used by the report tag would just climb and climb each iteration unless
we broke up the process into completely separate HTTP requests.

By creating a template that would process a small number of reports per
request and then send a page to the browser with a META refresh to
re-request the page in one second.  This works fine.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date
Get the Free Trial
http://ad.doubleclick.net/clk;203748912;27390454;j

Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:308857
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=17837.14401.4