DirectEventStream vs BufferedEventStream

Which EventStream to use:

DirectEventStream (DES) vs BufferedEventStream (BES)

Use DES when you don't want any latency of data appearing in BAMPrimaryImport database.   Since this is synchronous call, any error will be thrown back to the calling application.

Use BES if your concern is throughput.  This is particularly suited for scenarios with spurts of large chuck of events.  A small penalty for using this is that there will be small latency between when the event is sent and the event appearing in BAMPrimaryImport database.

For more info on EventStream: https://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdk/htm/frlrfmicrosoftbiztalkbameventobservationeventstreamclasstopic.asp

Comments

• Anonymous
  February 01, 2006
  I have a client who is concerned about the latency and would really like to use the DES for the application that does pre-processing of the file before it is handed to BizTalk. I am actually against this approach, as I would rather have a guaranteed delivery using the BES. Once the application is through its processing, and actually hands the file off to BizTalk, BizTalk must capture the file and check BAM to make sure that no records were lost. If I have implemented the BES inside of my application, would there be a possibility of entries to the BAMPrimaryImport database not being committed before a BizTalk orchestration checks. If I were to use the BES and flushing the instance in the application before creating the file, is there ANY possibility of that entry into the database not being there when BTS actually picks it up?
  
  This brings me to another question, what approach does Microsoft recommends when an application hands a message (in this case a file) to BizTalk, and to ensure that nothing is lost, needs balancing designed into the two processes. In BizTalk 2004, is there an automated way to check the BAM database to verify particular information, filename/row count has been satisfied before continue processing?

• Anonymous
  February 01, 2006
  I am assuming that you would have to use the IBAMDataExtractor.GetValue Method in the BizTalk orchestration. If I have my application write the filename and rowcount to the BAM database, once BTS picks up the file, it would get the rowcount and filename and then invoke the GetValue Method to retrieve the rowcount for this filename from BAM I would assume. Alas, I am unable to find an example of how this would be done though.
  
  Or even better yet, would it be possible to use the GetValue method to capture information not explicitly defined in the database. Example is that my applicaion is writing informaiton about each row in the file to BAM, however when BTS picks up the file, is there a way to sum up the number of rows for that file using this method?

• Anonymous
  February 02, 2006
  The comment has been removed

• Anonymous
  February 02, 2006
  "I am interested to know why would the customer want to write to BAM first ourside of the Biztalk pipelines and orchestrations, and then submit the file to Biztalk? Unless there is a good reason, the writing to BAM is better done from within Biztalk."
  
  We are recieving a 45mb file from a mainframe process, and to improve performance, we have a VB app that calls a stored proc and parses out only the records that are to be directed to BizTalk. If BTS were to consume the entire 45mb file, it then becomes a process that would take the flat file pipeline 20 minutes to dis-assemble, before we would even start interrogating where split the records and we only have a 30 minute window to process the file.
  
  We would like a central repository to be able to go to, to look at the activity to look at our KPI's.

• Anonymous
  February 02, 2006
  Interesting...my turn to do ask some questions :) to help me better understand your scenario, could you elaborate on the process that takes place. What do you want to capture while the big file is being split up? Do you want to find KPI's such as how long does it take to split up the 45 Mb, each individual files, how many good and bad files? Or can I assume always all good or if any bad considered all bad?
  
  Are these individual files then sent to Biztalk orchestration, e.g. into a in folder/directory for Biztalk to pickup? What kind of processing takes place in the orchestration? Where do they uliimately end up? And during this processing, what KPI's would you like to capture?
  
  Thanks,
  Keith
  
  Will be back tomorrow.
  

• Anonymous
  February 02, 2006
  "On checking if any specific information is fulfilled before continuing, you should design your orchestration accordingly with the correct logic, e.g. to take a different branch or to suspend or to send request, e.g. to custom asking for additional missing info. Think of BAM at this point as just to collect data."
  
  If I were to use the DirectEventStream in the application then, I would know that the records were successfully inserted before I continued. I then could test, in the application, that the record was inserted and then confidently continue with my file processing. Otherwise, the VB app can have error checking logic and fail if the BAM insert fails.
  
  I really don't want to create a seperate datastore location to do this 'balancing' when it seems like everything I want to track would be in the BAM database already.
  
  So there are no examples of using the IBAMDataExtractor?

• Anonymous
  February 02, 2006
  The comment has been removed

• Anonymous
  March 23, 2006
  What about TPE? Does it use the DirectEventStream or the BufferedEventStream

• Anonymous
  March 24, 2006
  Internally, TPE uses an EventStream very similar to BufferedEventStream.  
  
  

• Anonymous
  March 29, 2006
  
  
  While working on a BizTalk POC last week, I needed to track Business Activity Monitoring (BAM) data...

• Anonymous
  March 29, 2006
  The comment has been removed

• Anonymous
  May 07, 2006
  Hey Keith, that was a trackback from my post, not a question ;)

• Anonymous
  September 10, 2006
  "On IBAMDataExtractor interface, this is really only if you want to write your own custom interceptor. I can tell you that TPE internally uses this."
  
  Can "Interceptor" capture BAM messages (milestones & data) by TPE?
  In other word, can I customise the TPE's interceptor?

• Anonymous
  May 08, 2007
  Hi, I have one question about when event stream is used behind the scene when we map using tracking profile editor instead of using BAM API. Is it one of the asynchronous eventsreams or DirectEventStream. I tried serching for this but could not get much info on this. Thanks Rajeshwar Polaty

• Anonymous
  May 08, 2007
  Hi Rajeshwar, The EventStream used behind the scene when mapped using TPE, of of asynchronous type similar to BufferedEventStream. Thanks, Keith p/s I will be posting on BAM WCF Interceptor very soon.