Multithreading & Concurrency

Thread Concept

• Thread states

• Thread lifecycle

• A thread contains:

  • Stack (memory region)

  • Instruction Pointer (for next instruction to be executed)

• Threads in a process share

  • Heap

  • Opened files

  • Metadata

• Multithreaded architecture is good when

  • Lots of sharing data

  • Lots of tread creation / destruction

  • Lots of context swtich

• Multi-Process architecture is good when

  • Not sharing data

  • Tasks are not related

Concurrency

• Maps

  • HashTable: using synchronized to lock entire data set.

  • ConcurrentHashMap

    • before Java 1.8

      • using ReentrantLock to lock segments with put method API

      • read with volatile, but no need to lock

    • Java 1.8 and up

      • replace the lock with syncrhozied

      • using red-black tree instead of linked list

• List

  • ConcurrentModificationException

    • Thrown when attempting to remove an element from an ArrayList using the remove() method while traversing it using an enhanced for loop or an Iterator

    • Can happen either in single thread or multi-threads

    • Solutions

      • If traverse with Iterator, use Iterator.remove() rather than list.remove(element);

      • Use for loop of list size to traverse, and remove with list.remove(index)

      • Consider Stream API filter()

      • Remove after Traversal

      • Replacing ArrayList with CopyOnWriteArrayList when change is not frequent.

      • Using synchronized

• Native Java API

  • Extends from Thread class

  • Implements Runnalbe interface

    • new Thread(aRunnableInstance);

  • Termination / interruption

    • Exception handling with InterruptedException

    • Checking boolean of Thread.currentThread.isInterrupted()

    • Using Daemon thread to exit when main thread is terminated. thread.setDaemon(true);

  • Coordination

    • Wait (for a given period of time) with join() API

• Executor

  • Staic Factory - Executors

    • SingleThreadExecutor()

    • FixedThreadPool(Runtime.getRuntime().availableProcessors())

    • CachedThreadPool()

      • For huge amounts of small executions.

    • ScheduledExecutor(int)

  • Runnable / Callable

  • Pool management methods

    • shutdown()

    • boolean service.awaitTermination(int, TimeUnit)

    • shutdownNow()

  • Future

    • get() / get(int, TimeUnit)

  • Thread pool size

    • Need real measurement, but can start from:

      • Runtime.getRuntime().availableProcessors() * (1 + wait time / process time)

    • Or in a range between:

      • CPU intensive: CPU numbers

      • IO intensive: CPU numbers * 2

Locking

• synchronized

  • Key Concepts

    • Monitor (for a class / an instance)
    • All syncrhonized methods (blocks) can be run with only one thread at a time.

    • Not interruptible when waiting for the monitor. Interruptible when the monitor is got.

  • Lock types

    • Instance method: lock per instance

    • Static method: lock per class

    • Code block

      • synchronized(this) // lock of the class

      • synchronized(object) // lock per object

  • Unlock when

    • Finished the execution in the synchronized block

    • Invoking lock.wait() // would wait indefinitely

      • Need to recover the thread by

        • Other thread(s) to call lock.notify() to recover the waiting thread

        • Specify timeout to recover automaitically.

  • Optimization

    • Multiple options by specifying JVM arguments, lock level can be upgraded, but not be downgraded.

    • Optimistic Locking - CAS (compare and swap)

      • Good for read-intensive scenarios.

      • Types:

        • Spin Lock (enabled by default)

          • Good for low lock competition and quick operations. (Consuming CPU but avoid blocking and re-invoking of another thread.)

        • Bias Lock

          • Good for almost no lock competition. (To improve effeciency for one thread operation. If lock competition was met, upgrade lock level to Lightweight Lock.)

          • deprecated in Java 15

        • Lightweight Lock

    • Pessimistic Locking

      • Used for write-intensive scenarios, fetch lock for reading / writing.

      • Type:

        • Heavyweight Lock

• ReentrantLock

  • Difference from keyword synchronized, supports:

    • Lock interruption

      • unlock()

      • lockInterruptibly() with InterruptException handling

    • Test lock

      • tryLock() // return boolean and acquire the lock if available

      • isLocked()

      • getQueuedThreads()

      • getOwner()

      • isHeldByCurrentThread()

    • Fair lock

      • Synchonized only provides unfair mechanism, while ReentrantLock provides both.

  • Remember to unlock in finally block.

  • Use tryLock() instead of lock() in real time applications

• ReentrantReadWriteLock

  • Avoid race conditions by complete mutual exclusion

  • For read-intensive operations

• BlockingQueue

  • If queue is empty, the consumer thread would be blocked until queue receives a new object.

  • If queue is full, the producer thread would be blocked until there's more capacity.

  • A good fit for producer / consumer model.

• Semaphore

  • To restrict a given number of accesses to the resource

    • Not having the notion of owner thread. One thread can call acquire many times, so it's not reentrant.

    • Can call release even if not having acquired it.

    • APIs

      • semaphore.acquire(NUMBER);

      • semaphore.release(NUMBER);

    • Usage: Producer / Consumer

Producer / Consumer with Semaphore and ReentrantLock

• Condition Variable

  • lock.lock() // lock

  • lock.unlock() // unlock

  • condition.await() // suspend the thread, and wait for a state change.

  • condition.signal() / Condition.signalAll() // wake up waiting thread(s).

  • condition.awaitUninterruptibly() // more flexibility than Object Signaling

  • condition.awaitUntil(Date deadline) // more flexibility than Object Signaling

• Object Signaling

  • synchronized(object) { // lock

  • } // release

  • wait() // causes the current thread to wait until gets waked up by another thread.

  • notify() / notifyAll() // wake up waiting thread(s)

  • To call wait, notify, notifyAll APIs, need to acquire the monitor of the object (use synchronized on that object)

• ThreadLocal

  • For a non-thread-safe object to be get / set by each thread.

  • If threads are from a thread pool, remember to call remove otherwise it would still be kept.

  public class Impl {
      private static String name;
      private static ThreadLocal<String> nameLocal = new ThreadLocal<>();
  
      public static void main(String[] args) {
          for (int i = 0; i < 10; i++) {
              int finalI = i;
              Thread th = new Thread(new Runnable() {
                  @Override
                  public void run() {
                      name = String.valueOf(finalI);
                      nameLocal.set(String.valueOf(finalI));
                      try {
                          Thread.sleep(100);
                      } catch (InterruptedException e) {
                          e.printStackTrace();
                      }
                      System.out.println("name = " + name + ", nameLocal = " + nameLocal.get());
                  }
              });
              th.start();
          }
      }
  }

• Deadlocks

  • Necessary conditions

    • Mutual exclusive (a resource is not shareable)

    • Hold and wait

    • No preemption (the resource can be released when the thread is done using it)

    • Circular wait

  • Solution

    • Avoid to cause "circular wait" by a strict lock acqusition order.