Your feedback on web and load testing

As you can guess from my recent lack of posts, the Ocracoke team has been extremely busy trying to finish up our last major coding milestone.  We're nearly done with the new features, but we still have several weeks of bug fixing and stabilization ahead of us before the November CTP release of Visual Studio Team System.  A few of the things I've been working on are an overhaul of the Web Test Recorder, automatic hidden field tracking (including but not limited to ViewState), file upload support, and the ability to put comments in web tests.

Even though this is our last major coding milestone, there's still time for us to incorporate your feedback into the final product.  Please leave a comment with ANY suggestions and complaints you've come up with while using the current CTP release.  I'm especially interested in any comments and feature requests related to coded web tests.  I want you to be able to handle any web testing scenario imaginable from a coded web test, so let me know what scenarios are impossible or just too hard to implement.  This is your chance to directly affect the final product, so please speak up and let us know what you think.

Comments

• Anonymous
  September 16, 2004
  The comment has been removed
• Anonymous
  September 16, 2004
  I did not find a way how can I create dynamic scripts. There is a way to generate code from recorded test, but what should I do with generated file? I do not see any options in a Test project where I can specify that file. Did I miss it?
• Anonymous
  September 16, 2004
  another one :)
  Even tough you have gave me the brief description how can I store testing results, it was not easy. Frankly, I could not follow it. I did not find a way to save the result (TRX file).
  I hope that November version will make it easier, as well as display and compare several results on one graph (like with ACT you could select a few results and see the difference between them).
• Anonymous
  September 16, 2004
  Josh,
  something I just noticed that I'm hoping you guys have or will fix. When you add a coded test (via the button) it doesn't show up in the test explorer. I thought it was a bug at first. Tried shutting down/restarting. The solution was to simply recompile the project, but some notification that a recompile was required would be nice.
  
  Great tools. Looking forward to the next CTP release.
• Anonymous
  September 16, 2004
  just a funny thing, if you look for Ocracoke + Visual Studio 2005, Josh's blog would be the first one in the list.
  http://www.google.com/search?hl=en&ie=UTF-8&q=ocracoke+visual+studio+2005
  
  btw, Josh, a feedback, sometimes, requires the answers ;)
• Anonymous
  September 16, 2004
  Alex is getting anxious, so here it goes..
  
  Your comment about changing the URL is a good scenario that we have been mulling over for several weeks. We've discussed allowing you to override the server name using a context variable or just providing a button or context menu for "Change Server Name".
  
  About creating coded web tests.. I believe coded web tests should show up in Test View immediately in the next CTP release. There was a bug where they weren't being detected until after the project was built.
  
  Comparing runs will be possible in the next CTP release with our reporting features. We'll be including reports for comparing runs and you can easily create your own custom reports that access all of the data we captured during a load test.
  
  
  Tim,
  
  I'm glad you like our product.. The coded web tests will be automatically detected in future releases and running them should trigger a build if needed. Any feedback on the coded web test object model?
• Anonymous
  September 16, 2004
  Josh,
  
  Can you use web test or coded web test to verify if the content on the page (field labels, header note, items in the data grid, contents of Dropdown list etc) against a predefined set. If not, adding this capability will be of huge advantage for testing teams, who at the present are manually doing screen scrapping.
  
  Regards
  Niv
  
  
• Anonymous
  September 17, 2004
  Nivesh,
  Sorry for the delay today, our building's network has been down!
  
  We do have the concept of validation rules that can perform validation of an HTTP response. We'll be including some basic rules such as verifying the contents of an html tag or finding text on a page, but it'll also be very easy to create your own validation rules. The event args that a validation rule gets include all the HTTP headers, response timing stats, and a collection of HTML tags to make it pretty easy to validate pages.
  
  Validation rules can be used in web tests and coded web tests, but coded web tests can also just write code directly to handle the validation event.
  
  Which validation rules do you think we need to include in the product? If you had to pick only a few of them which ones would they be?
• Anonymous
  September 22, 2004
  here's a validation rule I'd love : xpath expression. If the html is XHTML compliant, I should be able to use an xpath expression as a validator. If not, my xpath expression would fail, and the validation fails. simple? probably not.
  
  Along the same vein, how about adding an xmldocument as a arg to custom validation rules? if the html can't be parsed into a dom, them that arg is null. At the very least, the raw html so we can parse it ourselves.
• Anonymous
  October 04, 2004
  Can someone point me to installation instructions, I'm keep getting a File Not found exception when trying to install:
  Visual Studio 2005 Beta 1 Refresh with Visual Studio 2005 Team System - Full DVD Install - (English)
  
  Thanks.
  
• Anonymous
  June 03, 2008
  Josh Christie (Visual Studio Team System) is