다음을 통해 공유

SQL Server Modeling Services Usability Study – System.Runtime Model/Loader

  • 아티클

One of the favorite part of my job as a Program Manager in product engineering groups at Microsoft is seeking customer feedback for products we build. It is really energizing for me to let people take our under-development products for a test run and observe them to see what was hard/easy for them to use. In past I have observed people participate in usability studies and folks from product groups observe them behind a glass door. I recently had my first experience with an “agile usability study” we bring in 5-8 participants from within Microsoft to a conference room for couple of hours and give them a set of tasks. The study was organized by our user experience group. In addition to the satisfaction of making a difference in a product, each participant also gets a $25 gift card!

The session format was:

· 10 minute introduction (5 minute study intro and 5 minute product context)

· 1 hour and 30 minute of actual study

· 20 minute wrap up

 

Folks from product groups (PM, SDE, SDET) in the room observe the participants during the study, write the feedback on huge stickies and put them on the whiteboard. We categorize the feedback during the session into high level categories and then we discuss the most relevant category in the post-study 20-minute wrap up. I found this study format to be great as it was low overhead and makes it quite interactive. We provided virtual machines with pre-loaded bits to the participants – all they had to do was bring their own laptops. At recruitment, we had specifically asked for .NET developers with SQL experience for this study.

 

In my 5 minute context (why) about SQL Server Modeling Services, I didn’t mention “M” or “Quadrant” at all. Our usability study for System.Runtime domain had the following three primary evaluation goals:

 

1) User comprehension of the System.Runtime database model (tables, views, table value functions)

2) Usability of LoadAssembly.exe tool

3) Usability of writing T-SQL queries over the model

 

The scenario, sample application and task list is available at: System.Runtime Model/Loader Usability Task List.

 

The sessions like these make us (folks from product groups) get out of our cocoon and see the ‘truth’. J

 

I will do another post with analysis of the feedback once we have compiled it all, but here were my high level takeaways:

1) Don’t make me write T-SQL queries, provide me a tool ( seems i am the only one who likes writing T-SQL queries ;-) )

2) It is very difficult to navigate the structure of the schema in SQL Server Management Studio (SSMS)

3) Provide an EDM / object model head over the schema so I can then write queries in code

 

It was apparent in this session, that unless our target customer is an ISV developer who is building metadata-aware tools on top of the SQL Server Modeling Services database, an enterprise developer or architect will not write T-SQL queries for analysis. If our value proposition is ‘impact analysis over codebase’ then we need to provide a targeted tool for those users (and they need not worry about the database structure).

What are your thoughts? Can you benefit by having your .NET applications metadata in a database? How much metadata do you really need in there? Are you interested in participating in our agile usability studies (i.e. if you are in the Seattle area)?

Comments

  • Anonymous
    March 19, 2010
    I'm not sure having .NET applications metadata in a database is necessary? I don't know how to get any benefit from that. I haven't seen anything from Microsoft to show me how I would benefit from that. Perhaps if their were more examples of how that information could be used

  • Anonymous
    March 23, 2010
    Happy to provide more samples. If you're only browsing one solution, there may be limited value. You can use Arch Explorer in VS10. The place where things get interesting is when you have multiple solutions, and multiple versions of solutions in the database, and multiple solutions to which you only have the built artifacts, not source. You can start using the database for predicting the impact of change, or determining the root cause of breaking changes. Which solution takes a dependency on this service? Any? Can I retire the service? Who am I going to break? What did I just break? It's all about getting access to the data. UI is nice, but the power's in the data...