Concurrency, part 7 - Why would you ever want to use concurrency in your application?

So I've spent a great deal of time talking about concurrency issues, but one thing I've avoided mentioning until now is when do you worry about concurrency.

The first (and most common) time that concurrency matters occurs when your code lives in a DLL.  If your code is in a DLL, then you've got to worry about concurrency (unless you have some external contract that guarantees your code is only called on a single thread (like being a COM apartment model object)).  This is an inescapable rule, if you're in a DLL, you have no control over how your code is called, and you must assume a worst case scenario.  And there are times in a DLL when it's necessary to do work on separate threads - as I mentioned before, you can't use COM objects in a DLL, which means that all COM calls in a DLL have to be done on another thread (unless your code is living in a COM DLL, in which case you can safely assume that your caller's called CoInitialize for you).

But if you're writing an application, you still might want to use multiple threads.  The biggest reason is for convenience.  There are times when it's just useful to be able to kick off a chunk of work onto another thread.  This is especially true for applications that interact with the user.  Even though those applications spend 99% of their time idle, it's critically important that the application be immediately responsive to the user.  So any tasks that could conceivably take a long time (like opening a file) should be performed on a thread that's not interacting with the user.  That's why my copy of FrontPage has 7 threads running.  In fact, on my machine, the only application with only one thread is cmd.exe - all the other processes have at least two threads.  Outlook has 37 threads on my machine right now :).

And, of course, the third reason for writing for concurrency is for performance.  If your machine has only one CPU core, then adding multiple threads won't actually improve your performance (actually they'll hurt your performance due to the time spent waiting for the system to switch from one thread to another), but on a machine with more than one processor, if you have multiple CPU-bound operations that can be overlapped, then running them on separate threads can dramatically improve your scalability.

But if you take the latter tack (adding multiple threads to resolve performance bottlenecks), it's critical to realize that you're potentially walking into a minefield.  All the stuff I've talked about so far is pretty straightforward, and applies to all sorts of applications.  But when you start trying to adopt your code for high performance computing, it opens up a whole new world of potential bottlenecks and issues.

And that's what I'll spend some time talking about next - a rough primer on concurrency for scalability, and some of the issues associated with it.

Comments

• Anonymous
  February 25, 2005
  Multiple threads on a single CPU system can greatly improve performance if the tasks can be split up and the threads are not CPU bound and making them multithreaded improves CPU usage. This assumes that adding more threads won't increase the stress on other subsystems that are already swamped. (i.e. adding more disk IO threads in a program that is already disk IO bound.)
  
  Oh course you can get into other problems such as training where one thread has to wait on another thread thus in effect causing all threads to be serialized in such a way as to only have one thread ready/current at any one time.
  
  Concurrency is a bear.
• Anonymous
  February 25, 2005
  The comment has been removed
• Anonymous
  February 26, 2005
  Absolutely true Niclas. Your example (creating a new thread to do work) simply uses principle #1. As another example, it's relatively hard (not impossible, but relatively hard) to do file I/O asynchronously (even though it's usually faster) because of all the state you need to manage. But if you dispatch to a worker thread, you can keep your state on the stack without having to worry about concurrency issues - you ensure that only one thread can access the data because its on the stack.
  
• Anonymous
  February 26, 2005
  The comment has been removed
• Anonymous
  February 26, 2005
  Artemis,
  Microsoft cares HIGHLY about concurrency and scalability. That's why we've got to such lengths to get the AD to scale evenly to (I believe) 12+ CPUs (making apps scale to more than somewhere around 3 CPUs is HARD, and gets exponentially harder the more CPUs you want to support).
  
  Clusters aren't about concurrency and scalability, not really. They're about load balancing and fault tolerance.
  
  Clusters can also be done to host distributed computing projects, but that's a different type of problem. For Microsoft's clustering solution, see: http://www.microsoft.com/windowsserver2003/hpc/default.mspx - I know some of the people working on the project, and they absolutely know what they're doing.
  
  And the rumour that Windows builds are compiled on BSD is a complete lie. Windows (and MS-DOS) builds have NEVER been on BSD.
  
  Back in 1984, MS-DOS builds were done on Sun workstations running Xenix, and Microsoft's languages were done on a Decsystem-20 running TOPS-20, but since 1987 or so, all Microsoft development's been done on a PC based platform (OS/2 originally, then Windows NT).
  
• Anonymous
  February 27, 2005
  The comment has been removed
• Anonymous
  February 27, 2005
  The "outlook locks up with exchange" problem's been fixed since Outlook XP, and it should be impossible to see it with Outlook 2003.
  
  And yup, OS/2 had one message queue, it was one of the huge improvements put in for Win32.
  
  I'm surprised you can do serial I/O synchronously - I thought you couldn't get above about 9600 baud with synchronous I/O.
  
  
  The Threads and window controls issue is directly related to the per-thread window issue, btw.
  
• Anonymous
  February 27, 2005
  It's good of you to point out that even a simple "File > Open" ought to take place in another thread, since it might involve sluggish networking, name resolution, etc.
  Speaking as a software developer, the reason why we don't do things correctly, and treat OpenFile() as if it were a fast operation, is that spawning things off into another thread automatically doubles the complexity of the user interface. We now have to have a UI state that says "Yes, thank you for your request to open a file, one day we'll get back to you but for the moment, just wait".
  All in all, an hourglass and a temporarily unresponsive application seems a friendlier solution!
• Anonymous
  February 28, 2005
  The comment has been removed
• Anonymous
  February 28, 2005
  Larry, my Outlook XP regulary freezes its UI for around a minute every single time it connects to the Exchange server. It asks for my credentials right away, then it pops up a timeout window and when I click Retry it freezes for a very long time. It eventually displays my folders and home page and freezes again for about 10 to 20 seconds, simetimes timing out loading my tasks/reminders whatever. After this horrible startup it works fine, with ocassional exchange timeouts when resolving names (which btw causes the message to be saved to my Sent Items without recipients even though it sends the e-mail out fine).
• Anonymous
  February 28, 2005
  The comment has been removed
• Anonymous
  February 28, 2005
  O2K3 freezes after it displays the home page/inbox. There's a few second delay (5 seconds or so) when it's responding and then it hangs for tens of seconds and then comes back. As I said, my guess is that it's loading my tasks and meetings, because sometimes it comes back after a very long wait saying that it was unable to load my tasks and/or free/busy data (I don't remember the exact wording of the error message). Once that is over it works great, with the exception of resolving names (Alt+K), that is still done on the UI thread, or at least it blocks the UI thread.
• Anonymous
  February 28, 2005
  Mark Mulin:
  Larry pointed to the "Free lunch is over" article in the first article in this serie (http://blogs.msdn.com/larryosterman/archive/2005/02/14/372508.aspx), and earlier here http://weblogs.asp.net/larryosterman/archive/2005/01/03/345889.aspx).
  
  Not 100% of Win32 is available from the .NET v1.x classes. But, have you checked out System.Diagnostics.ProcessThread.ProcessorAffinity property? A nice article is [1] which shows the mapping from Win32 APIs to the .NET classes.
  
  [1] http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndotnet/html/win32map.asp
• Anonymous
  June 16, 2009
  PingBack from http://lowcostcarinsurances.info/story.php?id=4382