Multiple Targets Queue Policy?
|
View:
New views
10 Messages
—
Rating Filter:
Alert me
|
 |
Multiple Targets Queue Policy?
by MICHAELMCGRADY
::
Rate this Message:
Is there a Java code construct (queue?) available that has the
following policy: a queue that allows a specified number of multiple threaded users to use an item before that item is discarded? This is to avoid the obvious sorts of out-of-memory problems that can arise when multiple users have different processing speeds. Thanks for any assistance. Mike Michael McGrady Senior Engineer mmcgrady@... 1 (253) 720-3365 _______________________________________________ Concurrency-interest mailing list Concurrency-interest@... http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest |
|
|
Re: Multiple Targets Queue Policy?
by David Holmes-3
::
Rate this Message:
Hi Michael,
It is unclear to me what you are looking for. With any of the Java collection classes you, the programmer, are in control of the items in the collection: if you add it then it's there; while it's there you can access it; if you remove it then it's no longer in the collection. David Holmes > -----Original Message----- > From: concurrency-interest-bounces@... > [mailto:concurrency-interest-bounces@...]On Behalf Of Michael > McGrady > Sent: Wednesday, 4 June 2008 11:14 AM > To: concurrency-interest@... > Subject: [concurrency-interest] Multiple Targets Queue Policy? > > > Is there a Java code construct (queue?) available that has the > following policy: a queue that allows a specified number of multiple > threaded users to use an item before that item is discarded? This is > to avoid the obvious sorts of out-of-memory problems that can arise > when multiple users have different processing speeds. > > Thanks for any assistance. > > Mike > > Michael McGrady > Senior Engineer > mmcgrady@... > 1 (253) 720-3365 > > > > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest@... > http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest _______________________________________________ Concurrency-interest mailing list Concurrency-interest@... http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest |
|
|
Re: Multiple Targets Queue Policy?
by Jed Wesley-Smith
::
Rate this Message:
I recently wrote something that sounds similar to your needs (it is for
multiplexing Input/OutputStream pairs) where a single producer thread would have multiple consumer threads. It is implemented with a blocking queue for each consumer thread and calls on the producer thread just add a element to each consumer's queue. The code is here for SVN: https://svn.atlassian.com/svn/public/atlassian/atlassian-stream-multiplexer/trunk or for browse via FishEye http://svn.atlassian.com/fisheye/browse/public/atlassian/atlassian-stream-multiplexer/trunk/ where most of the interesting concurrency stuff is here http://svn.atlassian.com/fisheye/browse/public/atlassian/atlassian-stream-multiplexer/trunk/src/main/java/com/atlassian/multiplexer/concurrent please note this project is still very young and doesn't have a release yet. Some or all of it will probably go into various Apache Commons projects at some point. cheers, jed. Michael McGrady wrote: > Is there a Java code construct (queue?) available that has the > following policy: a queue that allows a specified number of multiple > threaded users to use an item before that item is discarded? This is > to avoid the obvious sorts of out-of-memory problems that can arise > when multiple users have different processing speeds. > _______________________________________________ Concurrency-interest mailing list Concurrency-interest@... http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest |
|
|
Re: Multiple Targets Queue Policy?
by MICHAELMCGRADY
::
Rate this Message:
I can see that my question makes it look like I am interested in a
collection class. My mistake. I was not. Thanks to those who assumed otherwise. I appreciate the responses. I think the decoration of a concurrent queue sounds like the ticket1 MG Michael McGrady Senior Engineer mmcgrady@... 1 (253) 720-3365 On Jun 3, 2008, at 9:32 PM, David Holmes wrote: > Hi Michael, > > It is unclear to me what you are looking for. With any of the Java > collection classes you, the programmer, are in control of the items > in the > collection: if you add it then it's there; while it's there you can > access > it; if you remove it then it's no longer in the collection. > > David Holmes > >> -----Original Message----- >> From: concurrency-interest-bounces@... >> [mailto:concurrency-interest-bounces@...]On Behalf Of >> Michael >> McGrady >> Sent: Wednesday, 4 June 2008 11:14 AM >> To: concurrency-interest@... >> Subject: [concurrency-interest] Multiple Targets Queue Policy? >> >> >> Is there a Java code construct (queue?) available that has the >> following policy: a queue that allows a specified number of multiple >> threaded users to use an item before that item is discarded? This is >> to avoid the obvious sorts of out-of-memory problems that can arise >> when multiple users have different processing speeds. >> >> Thanks for any assistance. >> >> Mike >> >> Michael McGrady >> Senior Engineer >> mmcgrady@... >> 1 (253) 720-3365 >> >> >> >> >> _______________________________________________ >> Concurrency-interest mailing list >> Concurrency-interest@... >> http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest > _______________________________________________ Concurrency-interest mailing list Concurrency-interest@... http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest |
|
|
Re: Multiple Targets Queue Policy?
by David Holmes-3
::
Rate this Message:
In what way is a "concurrent queue" not a collection class ?
I'm afraid I'm none the wiser as to what you are actually looking for. David Holmes > -----Original Message----- > From: concurrency-interest-bounces@... > [mailto:concurrency-interest-bounces@...]On Behalf Of Michael > McGrady > Sent: Wednesday, 4 June 2008 8:21 PM > To: dholmes@... > Cc: concurrency-interest@... > Subject: Re: [concurrency-interest] Multiple Targets Queue Policy? > > > I can see that my question makes it look like I am interested in a > collection class. My mistake. I was not. Thanks to those who > assumed otherwise. I appreciate the responses. I think the > decoration of a concurrent queue sounds like the ticket1 > > MG > > > Michael McGrady > Senior Engineer > mmcgrady@... > 1 (253) 720-3365 > > > > > On Jun 3, 2008, at 9:32 PM, David Holmes wrote: > > > Hi Michael, > > > > It is unclear to me what you are looking for. With any of the Java > > collection classes you, the programmer, are in control of the items > > in the > > collection: if you add it then it's there; while it's there you can > > access > > it; if you remove it then it's no longer in the collection. > > > > David Holmes > > > >> -----Original Message----- > >> From: concurrency-interest-bounces@... > >> [mailto:concurrency-interest-bounces@...]On Behalf Of > >> Michael > >> McGrady > >> Sent: Wednesday, 4 June 2008 11:14 AM > >> To: concurrency-interest@... > >> Subject: [concurrency-interest] Multiple Targets Queue Policy? > >> > >> > >> Is there a Java code construct (queue?) available that has the > >> following policy: a queue that allows a specified number of multiple > >> threaded users to use an item before that item is discarded? This is > >> to avoid the obvious sorts of out-of-memory problems that can arise > >> when multiple users have different processing speeds. > >> > >> Thanks for any assistance. > >> > >> Mike > >> > >> Michael McGrady > >> Senior Engineer > >> mmcgrady@... > >> 1 (253) 720-3365 > >> > >> > >> > >> > >> _______________________________________________ > >> Concurrency-interest mailing list > >> Concurrency-interest@... > >> http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest > > > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest@... > http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest Concurrency-interest mailing list Concurrency-interest@... http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest |
|
|
Re: Multiple Targets Queue Policy?
by MICHAELMCGRADY
::
Rate this Message:
I will try again, David. Thanks for your call for clarity.
The present code passes off each incoming data instance to multiple instances of a PooledExecutor wrapper that uses a LinkedQueue. The problem is that the processing times of the different wrappers is different, leading to data piling up in one of the wrappers and throwing an out of memory exception. What is needed is some way to throttle the data flow so that the slowest consumer queue does not blow up. MG Michael McGrady Senior Engineer mmcgrady@... 1 (253) 720-3365 On Jun 4, 2008, at 4:55 AM, David Holmes wrote: > In what way is a "concurrent queue" not a collection class ? > > I'm afraid I'm none the wiser as to what you are actually looking for. > > David Holmes > >> -----Original Message----- >> From: concurrency-interest-bounces@... >> [mailto:concurrency-interest-bounces@...]On Behalf Of >> Michael >> McGrady >> Sent: Wednesday, 4 June 2008 8:21 PM >> To: dholmes@... >> Cc: concurrency-interest@... >> Subject: Re: [concurrency-interest] Multiple Targets Queue Policy? >> >> >> I can see that my question makes it look like I am interested in a >> collection class. My mistake. I was not. Thanks to those who >> assumed otherwise. I appreciate the responses. I think the >> decoration of a concurrent queue sounds like the ticket1 >> >> MG >> >> >> Michael McGrady >> Senior Engineer >> mmcgrady@... >> 1 (253) 720-3365 >> >> >> >> >> On Jun 3, 2008, at 9:32 PM, David Holmes wrote: >> >>> Hi Michael, >>> >>> It is unclear to me what you are looking for. With any of the Java >>> collection classes you, the programmer, are in control of the items >>> in the >>> collection: if you add it then it's there; while it's there you can >>> access >>> it; if you remove it then it's no longer in the collection. >>> >>> David Holmes >>> >>>> -----Original Message----- >>>> From: concurrency-interest-bounces@... >>>> [mailto:concurrency-interest-bounces@...]On Behalf Of >>>> Michael >>>> McGrady >>>> Sent: Wednesday, 4 June 2008 11:14 AM >>>> To: concurrency-interest@... >>>> Subject: [concurrency-interest] Multiple Targets Queue Policy? >>>> >>>> >>>> Is there a Java code construct (queue?) available that has the >>>> following policy: a queue that allows a specified number of >>>> multiple >>>> threaded users to use an item before that item is discarded? >>>> This is >>>> to avoid the obvious sorts of out-of-memory problems that can arise >>>> when multiple users have different processing speeds. >>>> >>>> Thanks for any assistance. >>>> >>>> Mike >>>> >>>> Michael McGrady >>>> Senior Engineer >>>> mmcgrady@... >>>> 1 (253) 720-3365 >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Concurrency-interest mailing list >>>> Concurrency-interest@... >>>> http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest >>> >> >> _______________________________________________ >> Concurrency-interest mailing list >> Concurrency-interest@... >> http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest _______________________________________________ Concurrency-interest mailing list Concurrency-interest@... http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest |
|
|
|
|
|
Re: Multiple Targets Queue Policy?
by MICHAELMCGRADY
::
Rate this Message:
Thanks, will check this out.
Mike Michael McGrady Senior Engineer mmcgrady@... 1 (253) 720-3365 On Jun 4, 2008, at 9:46 AM, Alex Miller wrote: > Micheal McGrady said: > >> I will try again, David. Thanks for your call for clarity. >> >> The present code passes off each incoming data instance to multiple >> instances of a PooledExecutor wrapper that uses a LinkedQueue. The >> problem is that the processing times of the different wrappers is >> different, leading to data piling up in one of the wrappers and >> throwing an out of memory exception. What is needed is some way to >> throttle the data flow so that the slowest consumer queue does not >> blow up. >> >> MG > > That sounds shocking familiar to the work-stealing stuff that the > upcoming fork-join framework does (except much more primitive :). > Maybe you could make use of it to recast your code in terms of fork- > join? Probably would be a lot simpler. To point at a relatively > arbitrary starting point: http://www.infoq.com/news/2008/03/fork_join > > Alex Miller > Terracotta _______________________________________________ Concurrency-interest mailing list Concurrency-interest@... http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest |
|
|
Re: Multiple Targets Queue Policy?
by tpeierls
::
Rate this Message:
Be careful. The fork-join framework is only appropriate for use with pure computational tasks. If tasks block for I/O, it won't work nicely.
--tim On Wed, Jun 4, 2008 at 12:46 PM, Alex Miller <alexdmiller@...> wrote:
_______________________________________________ Concurrency-interest mailing list Concurrency-interest@... http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest |
|
|
Re: Multiple Targets Queue Policy?
by Robert Nicholson-3
::
Rate this Message:
So do you have an unbounded queue?
If that's the case you might want to make it a finite bounds and add code to handle rejections On Jun 4, 2008, at 9:54 AM, Michael McGrady wrote: > I will try again, David. Thanks for your call for clarity. > > The present code passes off each incoming data instance to multiple > instances of a PooledExecutor wrapper that uses a LinkedQueue. The > problem is that the processing times of the different wrappers is > different, leading to data piling up in one of the wrappers and > throwing an out of memory exception. What is needed is some way to > throttle the data flow so that the slowest consumer queue does not > blow up. > > MG > > > Michael McGrady > Senior Engineer > mmcgrady@... > 1 (253) 720-3365 > > > > > On Jun 4, 2008, at 4:55 AM, David Holmes wrote: > >> In what way is a "concurrent queue" not a collection class ? >> >> I'm afraid I'm none the wiser as to what you are actually looking >> for. >> >> David Holmes >> >>> -----Original Message----- >>> From: concurrency-interest-bounces@... >>> [mailto:concurrency-interest-bounces@...]On Behalf Of >>> Michael >>> McGrady >>> Sent: Wednesday, 4 June 2008 8:21 PM >>> To: dholmes@... >>> Cc: concurrency-interest@... >>> Subject: Re: [concurrency-interest] Multiple Targets Queue Policy? >>> >>> >>> I can see that my question makes it look like I am interested in a >>> collection class. My mistake. I was not. Thanks to those who >>> assumed otherwise. I appreciate the responses. I think the >>> decoration of a concurrent queue sounds like the ticket1 >>> >>> MG >>> >>> >>> Michael McGrady >>> Senior Engineer >>> mmcgrady@... >>> 1 (253) 720-3365 >>> >>> >>> >>> >>> On Jun 3, 2008, at 9:32 PM, David Holmes wrote: >>> >>>> Hi Michael, >>>> >>>> It is unclear to me what you are looking for. With any of the Java >>>> collection classes you, the programmer, are in control of the items >>>> in the >>>> collection: if you add it then it's there; while it's there you can >>>> access >>>> it; if you remove it then it's no longer in the collection. >>>> >>>> David Holmes >>>> >>>>> -----Original Message----- >>>>> From: concurrency-interest-bounces@... >>>>> [mailto:concurrency-interest-bounces@...]On Behalf Of >>>>> Michael >>>>> McGrady >>>>> Sent: Wednesday, 4 June 2008 11:14 AM >>>>> To: concurrency-interest@... >>>>> Subject: [concurrency-interest] Multiple Targets Queue Policy? >>>>> >>>>> >>>>> Is there a Java code construct (queue?) available that has the >>>>> following policy: a queue that allows a specified number of >>>>> multiple >>>>> threaded users to use an item before that item is discarded? >>>>> This is >>>>> to avoid the obvious sorts of out-of-memory problems that can >>>>> arise >>>>> when multiple users have different processing speeds. >>>>> >>>>> Thanks for any assistance. >>>>> >>>>> Mike >>>>> >>>>> Michael McGrady >>>>> Senior Engineer >>>>> mmcgrady@... >>>>> 1 (253) 720-3365 >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Concurrency-interest mailing list >>>>> Concurrency-interest@... >>>>> http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest >>>> >>> >>> _______________________________________________ >>> Concurrency-interest mailing list >>> Concurrency-interest@... >>> http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest@... > http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest _______________________________________________ Concurrency-interest mailing list Concurrency-interest@... http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest |
| Free Forum Powered by Nabble | Forum Help |