When tuning messaging engine threads per cpu lower from defaults can help performance
Say you have a large flatfile in an application that does messaging only. You drop a single file and you notice elevated cpu for the receive host for several minutes. What’s happening in the host is that a single messaging thread is using the ffdasm to disassemble the flat file. This is a very cpu intensive activity
Now you drop 100 large flatfiles. Since messaging engine threads per cpu is set to 20, if you have a two processors, then there will be 40 messaging engine threads created and operating on 40 inbound flatfiles. cpu still goes to 100%, but it takes longer for the first flatfile to be processed because other threads are taking valuable cpu resources.
In testing on a two processor system I was able to get the best end to end performance by specifying that only one messaging engine thread per cpu be used on the receive host when performance testing with 100 files. Tuning messaging engine threads per cpu to a lower value results in faster processing of the first file through the system and additionally faster overall processing.
1 messaging engine thread per cpu on the receive host about 4 minutes to process the first file. 1 hour and 30 minutes for all files
10/21/2011 02:46 PM set messaging engine threads to 1 for full test.txt
10/21/2011 02:50 PM 2,752,368 {FC0AD608-1F76-43A2-B7D5-44DDB66E66DF}.xml
10/21/2011 04:16 PM 2,752,368 {F7241E83-C7F1-428D-A5DD-2582CF8ABE20}.xml
2 messaging engine threads per cpu on the receive host about 4 minutes to process the first file. 1 hour 42 minutes to finish all approx..
10/19/2011 04:29 PM set messaging engine threads to 2 for full test.txt
10/19/2011 04:33 PM 2,752,368 {C29DB82A-98E8-4B41-B917-093DDC708D21}.xml
10/19/2011 05:02 PM 2,752,368 {E9AFFEC2-B60B-446C-9D63-1BBBB2B2FD12}.xml 33rd file in 34 minutes. Had to stop test early summary
extrapolated.
4 messaging engine threads per cpu on the receive host 10 minutes time to first message and 1 hour and 39 minutes total processing time.
10/19/2011 06:06 PM set messaging engine threads to 4 for full test.txt
10/19/2011 06:16 PM 2,752,368 {E96CD77C-8D24-4BB9-A111-871594B06F3F}.xml
10/19/2011 07:45 PM 2,752,368 {2F2EE1F9-C9F0-44A9-B2D1-1A02BC9F7EAD}.xml
20 messaging engine threads per cpu on the receive host (the default setting) 18 minutes. 1 hour and 42 minutes for all files to be processed.
10/20/2011 09:35 AM set messaging engine threads to 20 for full test.txt
10/20/2011 09:53 AM 2,752,368 {9BA6A42F-A634-4FEF-A29E-F16A774D823F}.xml
10/20/2011 11:17 AM 2,752,368 {262F3035-CACF-423A-BDFA-5EA817D2639B}.xml
50 messaging engine threads per cpu on the receive host 23 minutes before first file is processed. 1 hour 54 minutes for all files to be processed.
10/20/2011 02:56 PM set messaging engine threads to 50 for full test.txt
10/20/2011 03:19 PM 2,752,368 {40BF75AF-3935-4DC1-99C4-048CD61421D1}.xml
10/20/2011 04:50 PM 2,752,368 {5B72CC56-FB61-4A33-9186-3A3F954AC38A}.xml