Java Concurrency: Understanding Thread Pool and Executors

This Java Concurrency tutorial helps you get started with the high-level concurrency API in the java.util.concurrent package that provides utility classes commonly useful in concurrent programming such as executors, threads pool management, scheduled tasks execution, the Fork/Join framework, concurrent collections, etc.

Throughout this tutorial, you will learn how thread pool works, and how to use different kinds of thread pools via executors.

Table of Content:

1. Understanding Thread Pool

2. Understanding Executors

3. Simple Executor and ExecutorService Examples

4. Cached Thread Pool Executor Example

5. Fixed Thread Pool Executor Example

6. Single-threaded Pool Executor Example

7. Creating a Custom Thread Pool Executor

1. Understanding Thread Pool in Java

In terms of performance, creating a new thread is an expensive operation because it requires the operating system allocates resources need for the thread. Therefore, in practice thread pool is used for large-scale applications that launch a lot of short-lived threads in order to utilize resources efficiently and increase performance.

Instead of creating new threads when new tasks arrive, a thread pool keeps a number of idle threads that are ready for executing tasks as needed. After a thread completes execution of a task, it does not die. Instead it remains idle in the pool waiting to be chosen for executing new tasks.

You can limit a definite number of concurrent threads in the pool, which is useful to prevent overload. If all threads are busily executing tasks, new tasks are placed in a queue, waiting for a thread becomes available.

The Java Concurrency API supports the following types of thread pools:

• Cached thread pool: keeps a number of alive threads and creates new ones as needed.

• Fixed thread pool: limits the maximum number of concurrent threads. Additional tasks are waiting in a queue.

• Single-threaded pool: keeps only one thread executing one task at a time.

• Fork/Join pool: a special thread pool that uses the Fork/Join framework to take advantages of multiple processors to perform heavy work faster by breaking the work into smaller pieces recursively.

That’s basically how thread pool works. In practice, thread pool is used widely in web servers where a thread pool is used to serve client’s requests. Thread pool is also used in database applications where a pool of threads maintaining open connections with the database.

Implementing a thread pool is a complex task, but you don’t have to do it yourself. As the Java Concurrency API allows you to easily create and use thread pools without worrying about the details. You will learn how in the next section.

2. Understanding Executors in Java

An Executoris an object that is responsible for threads management and execution of Runnable tasks submitted from the client code. It decouples the details of thread creation, scheduling, etc from the task submission so you can focus on developing the task’s business logic without caring about the thread management details.

That means, in the simplest case, rather than creating a thread to execute a task like this:

Thread t = new Thread(new RunnableTask());
t.start();

You submit tasks to an executor like this:

Executor executor = anExecutorImplementation;
executor.execute(new RunnableTask1());
executor.execute(new RunnableTask2());

The Java Concurrency API defines the following 3 base interfaces for executors:

• Executor: is the super type of all executors. It defines only one method execute(Runnable).

• ExecutorService: is an Executor that allows tracking progress of value-returning tasks (Callable) via Future object, and manages the termination of threads. Its key methods include submit() and shutdown().

• ScheduledExecutorService: is an ExecutorService that can schedule tasks to execute after a given delay, or to execute periodically. Its key methods are schedule(), scheduleAtFixedRate() and scheduleWithFixedDelay().

You can create an executor by using one of several factory methods provided by the Executors utility class. Here’s to name a few:

• newCachedThreadPool(): creates an expandable thread pool executor. New threads are created as needed, and previously constructed threads are reused when they are available. Idle threads are kept in the pool for one minute. This executor is suitable for applications that launch many short-lived concurrent tasks.

• newFixedThreadPool(int n): creates an executor with a fixed number of threads in the pool. This executor ensures that there are no more than n concurrent threads at any time. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread becomes available. If any thread terminates due to failure during execution, it will be replaced by a new one. The threads in the pool will exist until it is explicitly shutdown. Use this executor if you and to limit the maximum number of concurrent threads.

• newSingleThreadExecutor(): creates an executor that executes a single task at a time. Submitted tasks are guaranteed to execute sequentially, and no more than one task will be active at any time. Consider using this executor if you want to queue tasks to be executed in order, one after another.

• newScheduledThreadPool(int corePoolSize): creates an executor that can schedule tasks to execute after a given delay, or to execute periodically. Consider using this executor if you want to schedule tasks to execute concurrently.

• newSingleThreadScheduleExecutor(): creates a single-threaded executor that can schedule tasks to execute after a given delay, or to execute periodically. Consider using this executor if you want to schedule tasks to execute sequentially.

In case the factory methods do not meet your need, you can construct an executor directly as an instance of either ThreadPoolExecutor or ScheduledThreadPoolExecutor, which gives you additional options such as pool size, on-demand construction, keep-alive times, etc.

For creating a Fork/Join pool, construct an instance of the ForkJoinPool class.

3. Java Simple Executor and ExecutorService Examples

Let’s see a couple of quick examples showing how to create an executor to execute a Runnable task and a Callable task.

The following program shows you a simple example of executing a task by a single-threaded executor:

import java.util.concurrent.*;

/**
 * SimpleExecutorExample.java
 * This program demonstrates how to create a single-threaded executor
 * to execute a Runnable task.
 * @author www.codejava.net
 */
public class SimpleExecutorExample {

	public static void main(String[] args) {
		ExecutorService pool = Executors.newSingleThreadExecutor();

		Runnable task = new Runnable() {
			public void run() {
				System.out.println(Thread.currentThread().getName());
			}
		};

		pool.execute(task);

		pool.shutdown();
	}
}

As you can see, a Runnable task is created using anonymous-class syntax. The task simply prints the thread name and terminates. Compile and run this program and you will see the output something like this:

pool-1-thread-1

Note that you should call shutdown() to destroy the executor after the thread completes execution. Otherwise, the program is still running afterward. You can observe this behavior by commenting the call to shutdown.

And the following program shows you how to submit a Callable task to an executor.  A Callable task returns a value upon completion and we use the Future object to obtain the value. Here’s the code:

import java.util.concurrent.*;

/**
 * SimpleExecutorServiceExample.java
 * This program demonstrates how to create a single-threaded executor
 * to execute a Callable task.
 * @author www.codejava.net
 */
public class SimpleExecutorServiceExample {

	public static void main(String[] args) {
		ExecutorService pool = Executors.newSingleThreadExecutor();

		Callable<Integer> task = new Callable<Integer>() {
			public Integer call() {
				try {
					// fake computation time
					Thread.sleep(5000);
				} catch (InterruptedException ex) {
					ex.printStackTrace();
				}

				return 1000;
			}
		};

		Future<Integer> result = pool.submit(task);

		try {

			Integer returnValue = result.get();

			System.out.println("Return value = " + returnValue);

		} catch (InterruptedException | ExecutionException ex) {
			ex.printStackTrace();
		}

		pool.shutdown();
	}
}

Note that the Future’s get() method blocks the current thread until the task completes and returns the value. Run this program and you will see the following output after 5 seconds:

Return value = 1000

For more details about executing tasks with Callable and Future, see Java Concurrency: Executing Value-Returning Tasks with Callable and Future.

Let’s see a more complex example in which I show you how to execute multiple tasks using different kinds of executors.

4. Java Cached Thread Pool Executor Example

The following example shows you how to create a cached thread pool to execute some tasks concurrently. Given the following class:

/**
 * CountDownClock.java
 * This class represents a coutdown clock.
 * @author www.codejava.net
 */
public class CountDownClock extends Thread {
	private String clockName;

	public CountDownClock(String clockName) {
		this.clockName = clockName;
	}

	public void run() {
		String threadName = Thread.currentThread().getName();

		for (int i = 5; i >= 0; i--) {

			System.out.printf("%s -> %s: %d\n", threadName, clockName, i);

			try {
				Thread.sleep(1000);
			} catch (InterruptedException ex) {
				ex.printStackTrace();
			}
		}
	}
}

This class represents a countdown clock that counts a number from 5 down to 0, and pause 1 second after every count. Upon running, it prints the current thread name, follows by the clock name and the count number.

Let’s create an executor with a cached thread pool to execute 4 clocks concurrently. Here’s the code:

import java.util.concurrent.*;

/**
 * MultipleTasksExecutorExample.java
 * This program demonstrates how to execute multiple tasks
 * with different kinds of executors.
 * @author www.codejava.net
 */
public class MultipleTasksExecutorExample {

	public static void main(String[] args) {

		ExecutorService pool = Executors.newCachedThreadPool();

		pool.execute(new CountDownClock("A"));
		pool.execute(new CountDownClock("B"));
		pool.execute(new CountDownClock("C"));
		pool.execute(new CountDownClock("D"));

		pool.shutdown();

	}
}

Compile and run this program, you will see that there are 4 threads executing the 4 clocks at the same time:

Modify this program to add more tasks e.g. add more 3 clocks. Recompile and run the program again, you will see that the number of threads is as equal as the number of submitted tasks. That’s the key behavior of a cached thread pool: new threads are created as needed.

5. Java Fixed Thread Pool Executor Example

Next, update the statement that creates the executor to use a fixed thread pool:

ExecutorService pool = Executors.newFixedThreadPool(2);

Here, we create an executor with a pool of maximum 2 concurrent threads. Keep only 4 task (4 clocks) submitted to the executor. Recompile and run the program you will see that there are only 2 threads executing the clocks:

The clocks A and B run first, while the clocks C and D are waiting in the queue. After A and B completes execution, the 2 threads continue executing the clocks C and D. That’s the key behavior of a fixed thread pool: limiting the number of concurrent threads and queuing additional tasks.

6. Java Single-threaded Pool Executor Example

Let’s update the program above to use a single-threaded executor like this:

ExecutorService pool = Executors.newSingleThreadExecutor();

Recompile and run the program, you will see that there’s only one thread executing the 4 clocks sequentially:

That’s the key behavior of a single-threaded executor: queue tasks to execute in order, one after another.

7. Creating a Custom Thread Pool Executor

In case you want to have more control over the behaviors of a thread pool, you can create a thread pool executor directly from the ThreadPoolExecutorclass instead of the factory methods of the Executors utility class.

For example, the ThreadPoolExecutor has a general purpose constructor as follows:

public ThreadPoolExecutor(int corePoolSize,
                  int maximumPoolSize,
                  long keepAliveTime,
                  TimeUnit unit,
                  BlockingQueue<Runnable> workQueue)

You can tweak the parameters to suit your need, as long as you really understand what they mean:

• corePoolSize: the number of threads to keep in the pool.

• maximumPoolSize: the maximum number of threads to allow in the pool.

• keepAliveTime: if the pool currently has more than corePoolSize threads, excess threads will be terminated if they have been idle for more than keepAliveTime.

• unit: the time unit for the keepAliveTime argument. Can be NANOSECONDS, MILLISECONDS, SECONDS, MINUTES, HOURS and DAYS.

• workQueue: the queue used for holding tasks before they are executed. Default choices are SynchronousQueue for multi-threaded pools and LinkedBlockingQueue for single-threaded pools.

Let’s see an example. The following code creates a cached thread pool that keeps minimum of 10 threads and allow maximum of 1,000 threads, and idle threads are kept in the pool for 120 seconds:

int corePoolSize = 10;
int maxPoolSize = 1000;
int keepAliveTime = 120;
BlockingQueue<Runnable> workQueue = new SynchronousQueue<Runnable>();

ThreadPoolExecutor pool = new ThreadPoolExecutor(corePoolSize,
						 maxPoolSize,
						 keepAliveTime,
						 TimeUnit.SECONDS,
						 workQueue);
pool.execute(new RunnableTask());

You can see that when corePoolSize = maxPoolSize = 1, we have a single-threaded pool executor.

API References:

• Executor Interface Javadoc
• ExecutorService Interface Javadoc
• ScheduledExecutorService Interface Javadoc
• Executors Class Javadoc
• Callable Interface Javadoc
• Future Interface Javadoc
• ThreadPoolExecutor Class Javadoc

Related Tutorials:

• Executing Value-Returning Tasks with Callable and Future
• Scheduling Tasks to Execute After a Given Delay or Periodically

Other Java Concurrency Tutorials:

• How to use Threads in Java (create, start, pause, interrupt and join)
• Java Synchronization Tutorial
• Understanding Deadlock, Livelock and Starvation with Code Examples in Java
• Understanding Java Fork-Join Framework with Examples
• Understanding Atomic Variables in Java

About the Author:

Nam Ha Minh is certified Java programmer (SCJP and SCWCD). He began programming with Java back in the days of Java 1.4 and has been passionate about it ever since. You can connect with him on Facebook and watch his Java videos on YouTube.