Looming Changes in Concurrent Programming

TOPICS

We will discuss the following points:

1. Fibers, erm, virtual threads
2. Blocking is cheap
3. Can have millions (=>blocking no more needed)
4. Charming API --just like j.l.Thread, j.u.concurrent
5. No more async
6. Structured concurrency -- no more goto with thread
7. Make concurrency easy again?
8. How should we teach this stuff?
9. awaitTermination

CONCURRENCY ON THE JAVA PLATFORM

*1995: Java language has thread support

*1997: Java web server (predecessor of Tomcat) runs each web request in a new thread

1. =>Amazing in 1997: thousand of concurrent requests

*2004: Java 5 java.util.concurrent

1. =>ReentrantLock, ConcurrentHashMap, Executor, Future

*2008: Java 8

1. =>Parallel streams, CompletableFuture

LOOM

*Thread are expensive

1. =>But what can you do when one blocks?

*Asynchronous programming

1. =>Callback hell, Futures, Async/await

*Loom: What if blocking wasn't expensive?

1. =>Millions of concurrent fibers
2. =>Each thread runs many fibers
3. =>Creating, switching between fibers cheap
4. =>Blocking is virtually free
5. =>VM, API park, unpark blocking fibers

MAKE CONCURRENCY EASY AGAIN

*Not so fast

*More than one reason for concurrency

*User interfaces: UI components not threadsafe

1. =>Single UI thread serializes operations
2. =>Fibers won't help
3. =>Keep using AsyncTaskSwingWorker

*What about parallel streams -- the previous promise to "make concurrency easy again?"

1. =>Works great for non-blocking workloads...
2. =>...on splittable data structures
3. =>Nevertheless parallel streams is not a universal solution

*Fibers don't add value for computationally-intensive tasks (like cryptography)

*Sweet spot:

1. =>Many more tasks than threads...
2. =>... that mostly block

VIRTUAL THREADS

*Tasks run in fiber, now called "virtual thread"

*Virtual threads mapped onto "carrier" threads

*When a virtual thread blocks, it is "parked" and the carrier thread executes another virtual thread

*Didn't we have all that with "green threads" way back in Java 1.0?

1. =>When a green thread blocked, it blocked the carrier thread

*Naming is hard

1. =>Originally, "lightweight" threads were called "fibers" --hence "loom"
2. =>Fiber API converged to thread API
3. =>"new" or "lightweight" not so good when something newer or lighter weight comes along

*"Virtual" supposed to evoke virtual memory that is mapped to actual RAM

1. =>Do not think "virtual function"

CONSTRUCTING VIRTUAL THREADS

*Quick one-off (for demos etc.)

*Builder API:

*But why are you constructing your own thread? Use an executor

USING AN EXECUTOR SERVICE

*Submit runnables and callables in the usual way:

*Can customize with thread factory:

*Don't mix with cached thread pool!

KICK THE TIRES

*Download binaries from http://jdk.java.net/loom/

*Run a million fibers:

*Try it with threads

1. =>On my laptop, out of memory after about 10 000 threads

STATE OF THE PROJECT

*API implementations are being made fiber friendly

1. =>Thread.sleep
2. =>j.u.c locks
3. =>NIO, sockets
4. =>JSSE implementation of TLS

*Reimplementations already in JDK 11,12,13

*Can't yet block on monitors (ReentrantLock is ok)

*Working on debugger, monitoring support

*Lots of "instabilities"

*Performance nowhere near where it needs to be

*Need feedback on API

*Need help with testing

STRUCTURE CONCURRENCY

*Nathaniel Smith: "Start and forget" is like goto

*1960s: Structured programming replaces goto with branches, loops, functions

*Structured concurrency: Should do the same with concurrent tasks

1. =>It waits for tasks to complete together.

STRUCTURING WITH EXECUTOR

*exec.close() blocks until all tasks are done

1. =>Remember: blocking is cheap

*Executor is autocloseable:

*With callables, use invokeAny, invokeAll

*To cancel overdue tasks, use a deadline:

CANCELLATION

*Cooperative cancellation

*Has never been fun in Java:

*Poor match for thread pools

*Inconsistently used

1. =>ExecutorService.invokeAny cancels remaining tasks
2. =>CompletableFutire.anyOf doesn't

*Loom flirted with a different cancellation model

*Now back to interruption

*Interrupting blocked virtual thread is cheap (no cost)

*ExecutorService.shutdownNow, expired deadline interrupts remaining tasks

THREAD LOCALS #underConstruction #maybeChange

*Thread.Builder methods noThreadLocals(), noInheritableThreadLocals()

*Exploratory work on "lightweight thread locals"

*Bound to a scope (such as an executor), not the lifetime of the thread

*Binding is immutable

*This worked in build 16-loom+9-316 (but not in build 17-loom+2-42)

*Why no tie more closely to executor? Other arrangements are plausible:

HEINZ KABUTZ'EXAMPLE

*At JCrete, Heinz Kabutz gave a puzzler with a program that loaded thousand of Dilbert cartoon images, one per day

*For each image,

1. =>Load page such as https://dilbert.com/strip/2011-06-05
2. =>Find image URL in strip
3. =>Load image from that URL
4. =>Display or save image

*It was a mess of completable futures, somewhat like:

GOOD USE OF FIBERS?

*With virtual threads:

*No callback code in load

1. =>Blocking call to semaphore for throttling request
2. =>Blocking read of the page
3. =>blocking read of the image

*No win to use fibers -- gating factor is number of concurrent request to Wikimedia

*Actually, there is a win:

1. =>No-guilt call to semaphore for throttling concurrent requests
2. =>Explicitly tuning instead of through the size of the thread pool

*The article by Jetty veterans is dubious

1. =>If the millions of virtual threads consume nontrivial resources, those will be the gating factor
2. =>Mixed approach: kernel threads for the parts that need tight control, virtual threads for code that shouldn't have to struggle with async

HOW DO WE TEACH CONCURRENCY

*Case study: The Java Tutorials Lesson: Concurrency

*Thread.start, Thread.sleep

*Thread.interrupted, InterruptedException

*Thread.join

*Race conditions

*synchronized methods, synchronized blocks

*volatile

*Deadlocks, starvation, livelocks

*wait, notifyAll

*This is so wrong

TASKS, NOT THREADS

*Let's rethink concurrency

*A Runnable is a task (presumably with a side effect)

*A Callable delivers a result (hopefully without side effects)

*An executor executes tasks:

*With callables, execution yields a future:

*With Loom, we don't care that result.get() blocks.

TASK COORDINATION

*Typically, a task decomposed into subtasks

*Is it worth executing them concurrently?

1. =>To keep cores busy
2. =>If they spend a lot of time blocking

*"Cores busy" is messy because it depends on the number of cores

1. =>Parallel streams
2. =>Recursive tasks (fork-join)

*Loom works well for blocking tasks

1. =>ExecutorService.invokeAll/invokeAny for composing results
2. =>Async should be left to experts (sorry CompletableFuture)

WHAT CAN POSSIBLY GO WRONG?

*Java is not done

*Major structural changes get a lot of attention by very smart people

*But the interest of system/application programmers are not always represented

*They need you!

*Read those JEPs and project pages

*Try early builds

*Give feedback

*Don't get bamboozled by the surface claims ("millions of fibers")

1. =>The payoff for Loom is a better programming model
2. =>Same as with j.u.stream

*Data is gold (measurements feedbacks)