[Beginner] Multithreading C++ Socket

Question

In C++ (well I think in every programming language) what is better to do when dealing with multithreading socket?

1. Cap the number of threads it can execute the requests so that you always have 1/2 thread available for accepting new connections? In this case I can see a situation where the threads might be I/O bottlenecked so they just wait instead of actually doing something else.
2. Don't cap it? In this case can't you get in a situation where all threads are busy and a new connection isn't accepted so the client doesn't connect to the socket server?

I know it is a beginner question but I didn't found a truly definitive answer (except I guess if you ask Gemini/ChatGPT, but I would like an answer from someone that uses them, and not scraping the internet and maybe even getting wrong answers but answering like it is 100% sure it is right). For example you have a processor with 32 logical cores (16 physical with SMT / Hyper-Threading). Is it better to cap it to 31/30 threads for execution and keep 1/2 for only accepting connections, or leave it uncapped?

Answer 1

Hi,

The question is too broad. So, it depends on app design.

First, consider whether multithreading is needed, else use asynchronous io sockets.

https://stackoverflow.com/questions/4653984/designing-scalable-multi-threaded-socket-push-server-in-net

Multithreaded sockets can introduce race conditions and deadlocks, make it difficult to access shared data securely.

Various application designs adopt different multithreaded socket models.

For example:
https://stackoverflow.com/questions/71572056/multithreaded-server-c-socket-programming

netIOManager (could be a ThreadPool) that will only receive net data and hand it over to a MainManager(also another ThreadPool) to process that data and then immediately return to its main job which is receiving net data.

The MainManager may have further database tasks(also another ThreadPool) and then after processing the data it will deliver the response to the netIoManager so it will send the response to the client.

Best regards,

Minxin Yu

---

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 2

it depends on app design. no threads are required to support multiple connections. for example node.js only uses a single thread for all sockets, but because it uses asynchronous I/O it still very fast.

only one thread receives the connection. it can process the connection directly, create a new thread to process. or in a application like IIS, there is a receive thread pool, a send thread pool and process thread pool.

as receiving and sending and are I/O bound, the the thread is often in a wait state (not using cpu). therefore its practical to have many more I/O threads than cores (hundred to thousands). the processing of the request uses cpu, but typical is often I/O bound using disk, databases or network calls, so again more threads than core makes a lot of sense.