Following-up on my last post, we found a second problem. When you initialize a CherryPy HTTP server, you may pass either or both of two options (among many others of course):
- thread_pool, which is documented as "The number of worker threads to start up in the pool."
- thread_pool_max: "The maximum size of the worker-thread pool. Use -1 to indicate no limit."
This rather strongly suggests that the size of the server's threadpool is variable. We were not configuring the latter option, and so foolishly believed that the server would spin up as many more threads as it thought it needed. Given that our CPU usage was tiny, and that we had no other indicia of excessive context switching, we were confused.
Again, we resorted to simply… reading the code. It was easy to locate the grow() & shrink() methods… which are called nowhere in the codebase. The threadpool remains forever at its initial size. A teammate quickly turned up this issue, "Dynamic resizing of cheroot Threadpool", opened in… 2019. And closed as "stale" (whatever that means) in 2020.
We should be willing to just read the code more often. Too often, we view third-party libraries as black boxes that are implicitly assumed to work. In this case, it wasn't my service: I was called-in to help on a production issue. The team is firmly on-board with getting off of CherryPy at this point. Now to wean them off of Python altogether…