ThreadPoolExecutor goes below corePoolSize

_______________________________________________
Concurrency-interest mailing list
Concurrency-interest@...
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest

Moran,

Moran Avigdor wrote:
> I searched a bit on bugs opened related to corePoolSize, but did not
> find a related post regarding corePoolThreads being removed from the
> cache, although they shouldn't.

Probably http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6458662

This can be seen in a debugger - threads expire regardless of corePoolSize
and allowCoreThreadTimeout being false. In JDK6 this happens every time.

Interestingly, the backport library gets it right and keeps min
(corePoolSize) threads running; it reliably lets your test pass.

Holger
_______________________________________________
Concurrency-interest mailing list
Concurrency-interest@...
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest

Lots of ThreadPoolExecutor bugs got fixed in very recent JDKs
(including OpenJDK 6 and 7).
The changes were too extensive to backport to JDK 5.

http://bugs.sun.com/bugdatabase/search.do?process=1&category=java&bugStatus=10%2C11&subcategory=classes_util_concurrent&type=bug&keyword=ThreadPoolExecutor+%22Release+Fixed%2C+7%22

Martin

On Mon, Jun 2, 2008 at 3:21 AM, Moran Avigdor <moran@...> wrote:
> Hi,
>
> I searched a bit on bugs opened related to corePoolSize, but did not find a
> related post
>
> regarding corePoolThreads being removed from the cache, although they
> shouldn't.
_______________________________________________
Concurrency-interest mailing list
Concurrency-interest@...
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest

If you restrict your usage of ThreadPoolExecutor to an unbounded  
queue. It's generally more reliable. That said you need to consider  
the memory footprint of running an application with an bounded queue.

It's already proven in 1.5 that with a bounded queue if you submit N  
tasks it will execute only < N tasks.

Consider the following code that is a modified version from one of the  
bug submissions

import java.util.concurrent.*;
import java.util.concurrent.atomic.*;

public class RecycledTPETask {
      public static void main(String[] args) throws Exception {
          final int nTasks = 1000;
          final AtomicInteger nRun = new AtomicInteger(0);
          final AtomicInteger nRejections = new AtomicInteger(0);
          final Runnable recycledTask = new Runnable() {
                  public void run() {
                      nRun.getAndIncrement();
                  } };
          final ThreadPoolExecutor p =
              new ThreadPoolExecutor(1, 30, 60, TimeUnit.SECONDS,
                                     new ArrayBlockingQueue(30));
          try {
              for (int i = 0; i < nTasks; ++i) {
                  for (;;) {
                      try {
                          p.execute(recycledTask);
                          break;
                      }
                      catch (RejectedExecutionException ignore) {
                          nRejections.getAndIncrement();
                      }
                  }
              }
              Thread.sleep(10000); // enough time to run all tasks

              System.out.println(nRejections.intValue() + "  
rejections");

              if (nRun.get() < nTasks)
                  throw new Error("Started " + nTasks +
                                  " Ran " + nRun.get());
          } catch(Exception ex) {
              ex.printStackTrace();
          } finally {
              p.shutdown();
          }
      }
}

init:
deps-jar:
compile-single:
run-single:
3013 rejections
Exception in thread "main" java.lang.Error: Started 1000 Ran 971
         at RecycledTPETask.main(RecycledTPETask.java:33)
Java Result: 1
BUILD SUCCESSFUL (total time: 10 seconds)

import java.util.concurrent.*;
import java.util.concurrent.atomic.*;

public class RecycledTPETask {
      public static void main(String[] args) throws Exception {
          final int nTasks = 1000;
          final AtomicInteger nRun = new AtomicInteger(0);
          final AtomicInteger nRejections = new AtomicInteger(0);
          final Runnable recycledTask = new Runnable() {
                  public void run() {
                      nRun.getAndIncrement();
                  } };
          final ThreadPoolExecutor p =
              new ThreadPoolExecutor(1, 30, 60, TimeUnit.SECONDS,
                                     new LinkedBlockingQueue());
          try {
              for (int i = 0; i < nTasks; ++i) {
                  for (;;) {
                      try {
                          p.execute(recycledTask);
                          break;
                      }
                      catch (RejectedExecutionException ignore) {
                          nRejections.getAndIncrement();
                      }
                  }
              }
              Thread.sleep(10000); // enough time to run all tasks

              System.out.println(nRejections.intValue() + "  
rejections");

              if (nRun.get() < nTasks)
                  throw new Error("Started " + nTasks +
                                  " Ran " + nRun.get());
          } catch(Exception ex) {
              ex.printStackTrace();
          } finally {
              p.shutdown();
          }
      }
}

init:
deps-jar:
compile-single:
run-single:
0 rejections
BUILD SUCCESSFUL (total time: 11 seconds)

On Jun 2, 2008, at 3:13 PM, Martin Buchholz wrote:

_______________________________________________
Concurrency-interest mailing list
Concurrency-interest@...
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest

 

I remember there was a problem in 6.0, but I wasn't aware of problems in 1.5 in this regard.

Thank you for the replies – this obviously answers my question.

 

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6458662

 

 

---

From: Moran Avigdor
Sent: Monday, June 02, 2008 1:22 PM
To: concurrency-interest@...
Subject: [concurrency-interest] ThreadPoolExecutor goes below corePoolSize

 

Hi,

I searched a bit on bugs opened related to corePoolSize, but did not find a related post

regarding corePoolThreads being removed from the cache, although they shouldn't.

 

My test case below simulates a ThreadPool, which is constructed with a min corePoolSize and a max pool size.

The idle wait time is set to 1 ms. I block my threads with a CountDownLatch to ensure that threads scale from min to max (and are not reused).

By releasing the latch I free up all the threads to complete their work, and check that we did not go below core pool size.

 

This test sometimes fails on the assertion that we are below corePoolSize:

junit.framework.AssertionFailedError: Expected minimum pool size. expected:<2> but was:<1>

 

Is anyone familiar with such a scenario?

 

I am using JDK 1.5.0_12.

Regards,

Moran Avigdor

 

 

 

    public void testScaleDown() throws Exception {

        final int min = 2;

        final int max = 4;

        final CountDownLatch active = new CountDownLatch(max +1);

       

        ThreadPoolExecutor pool = (ThreadPoolExecutor)Executors.newCachedThreadPool();

        pool.setCorePoolSize(min);

        pool.setMaximumPoolSize(max);

        pool.setKeepAliveTime(1, TimeUnit.MILLISECONDS);

 

        for (int i=0; i<max; ++i) {

            pool.execute( new Runnable() {

                public void run()

                {

                    try {

                        active.countDown();

                        active.await();

                    }catch (InterruptedException e) {

                        e.printStackTrace(); //ignore

                    }

                }

            });

            //wait time to execute task by pool

            Thread.sleep(100);

        }

       

        Assert.assertEquals("Expected all threads to block. ", 1, active.getCount());

        Assert.assertEquals("Expected active. ", max, pool.getActiveCount());

        Assert.assertEquals("Expected max pool size. ", max, pool.getPoolSize());

 

        //release the latch

        active.countDown();

        active.await();

       

        //wait around for one second for state to settle

        Thread.sleep(1000);

        Assert.assertEquals("Expected completion of all tasks. ", max, pool.getCompletedTaskCount());

        Assert.assertEquals("Expected completion of all threads. ", 0, pool.getActiveCount());

        Assert.assertEquals("Expected minimum pool size. ", min, pool.getPoolSize());

    }

 

 

_______________________________________________
Concurrency-interest mailing list
Concurrency-interest@...
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest