Attributes and API usability (again!)
I've been running another usability study on an API that makes heavy use of attributes and have made similar observations to a previous study (see https://weblogs.asp.net/stevencl/archive/2004/05/12/130826.aspx#FeedBack)
This time we've been looking at the Indigo APIs. Indigo makes heavy use of attributes throughout. For example, attributes are used to identify an interface as representing a service contract or a method inside that interface as a service operation.
In this study, we've observed similar issues to those observed before. One of the issues that has resurfaced is the apparent reluctance of participants to consider the attributes sprinkled throughout their code as potential causes of unexpected application behavior. As in the previous study, we've seen developers consistently neglecting to consider modifying any of the attributes that they have applied to their code when attempting to modify the behavior of their code. In many cases, the desired effect (changing the lifetime of a service for example) is done by simply changing a property of a particular attribute. But instead, many participants spend time investigating their own code and even changing their own code to try to achieve the change in behavior they are looking for. It seems too that this behavior isn't necessarily isolated to the usability labs. One of the Indigo team members told me of a recent example where some folks from the product team were attempting to debug some customer code. They spent a lot of time checking the IIS configuration, the customer code etc and overlooked the attributes in the code. It turned out that the wrong attribute had been used.
I think one factor in this is the low visibility of attributes. For example, one participant in the study this week was stepping through his code in the debugger when he noticed some unexpected behavior at some point during the excecution of his code. He was focused on a particular block of code and concentrated his efforts on understanding how that block of code might have caused the behavior he had just observed. The cause for that behavior was due to an attribute that had been applied to the class that defined the method the participant was stepping through. Thus when he was reading his code, the attribute was well out of his focus of attention.
However, such issues are typically an issue for the first few times only - once you learn the effect of an attribute, it's more likely that you'll remember that attribute and will revisit it if the same issue occurs again. And indeed, this is what we observed during the study.
Another factor involved is when there are relationships between two or more attributes. The problem gets worse when the change in application behavior necessitates modifying more than one attribute at a time. The problem is that the relationship between multiple attributes is often unclear. One of the tasks in the study I am running this week requires that participants apply an attribute to a method inside an interface declaration and another attribute to the class that implements that interface. Participants who have worked on this task have been unaware of this requirement and have not been able to complete the task without significant help. There is nothing about the design of the attributes that makes this relationship clear.
Once participants have been told about the multiple attributes and the requirement to modify them all in conjunction their reaction is always the same - "that's kludgy!". Having seen this in multiple studies on different APIs, I'm pretty much at the point where I would strongly recommend API teams avoid designing APIs that require developers to modify multiple attributes to achieve specific functionality. Instead, I'd recommend examining ways that the same functionality could be achieved through the use of one attribute or at the very least, consider naming the multiple attributes in such a way that the relationship between the attributes is much clearer.
It would be interesting to hear anyone's feedback on this issue. Is this something you have experienced yourself when using an attribute heavy API? How big of an issue do you think it is? What would be your preferred solution?
Comments
- Anonymous
October 08, 2004
The comment has been removed - Anonymous
October 08, 2004
The comment has been removed - Anonymous
October 08, 2004
I think it's awesome that you guys are studying API usability. Programmers are terrible at usability, and guess who designs most APIs? ;)
If you ever want a good laugh (and a case study in the above) check out the PHP API. It's unintentionally hilarious.
As for attributes: I agree, their use needs to be strictly limited-- it's really a special purpose tool, not a general one. The lack of locality problem is tremendous: you have an exception in code block (x) but the cause is an attribute way, way up in the function declaration. - Anonymous
October 09, 2004
The comment has been removed - Anonymous
October 10, 2004
I'm particularly interested in what people think of as alternatives. Here is a snippet of Indigo code that illustrates all of the attributes needed in Steven's study:
[ServiceContract]
public class Conversation
{
[ServiceOperation]
public string Greeting (string Hello);
public string Parting (string Goodbye);
}
Conversation is exposed as a service. Greeting() is part of the service contract; Parting() is not.
There are families of alternatives to these attributes that move away from a class-oriented approach. You could imagine, say, a contract container to which you add delegates keyed by their action. Microsoft has a long history of products (DCOM, Enterprise Services, .NET Remoting, ASP.NET) where you implement a service by implementing methods on a class you then expose as a service. We would like to bring these customers forward. Things like contract containers have seemed complicated to us.
Two alternatives to [ServiceContract] are a required base class or an empty marker interface. Our experience with .NET Remoting argues that required bases classes are not acceptable to users (suppose you want to put a service contract on a WinForm). Our experience with Indigo says that people are confused by empty interfaces. Steven's experiments seem to indicate that people are confused by interfaces altogether.
I cannot think of an alternative to [ServiceOperation]. For the sake of consistency and security, we don't want to light up all methods on a service just because it's a service. We're in a "private by default" world. - Anonymous
October 11, 2004
Actually, the attributes that caused the most problems were those used in handling and reporting errors from the service to the client, not those used to mark a method as being part of the contract.
The [ErrorHandling] attribute defines how a service handles errors. The [ErrorReporting] attribute defines how errors are reported to the client. Lastly, the [KnownFault] attribute identifes the exceptions that a particular [ServiceOperation] can throw.
The values of each of these attributes has an effect on the behavior of the other attributes, to the extent that a table needs to be drawn in the documentation to demonstrate the effect.
Two of the attributes are set on the service implementation, while the [KnownFault] attribute is set either on the [ServiceOperation] or the [ServiceContract]. The relationships between these three attributes were hard for participants to discover and to control.
The issues above with [ServiceOperation] were largely that many participants forgot to add this attribute to an interface that had been marked as [ServiceContract] but this is one of those problems that seems to be a one time only hit. - Anonymous
October 11, 2004
I think I agree with you that [ErrorHandling] and [ErrorReporting] are bad. I think they're too complex. I think the interdependence between them is unacceptable. They're probably trying to do more than attributes should be asked to do.
[KnownFault] is a different beast. In WSDL, you can assign faults to INOUT operations. We use the attribute to do this. If you want to expose the fault on all operations on a contract, you putit to the contract; if you want to expose the fault on a particular attribute, you put it on the operation. We need to be able to discover this operation/fault linkage when we're reflecting on the type in order to auto-generate the contract metadata, and of course at runtime when we catch exceptions and generate faults.
What are alternative ways of associating fault types with operations? - Anonymous
October 11, 2004
The comment has been removed - Anonymous
October 13, 2004
Perhaps if attributes were specified as [System(attribute, attribute, ...)] it might be clearer what attributes are associated with what system, and my varying the constructor, you could specify 'schema' of what attributes are applicable.
Steve Schwartz, when you put a [ServiceContract] on a class, you are saying that each method of that class can optionally have a [ServiceOperation] attribute put on it, but that 'schema' is expressed exactly nowhere. If there was a way to specify the relationship between [ServiceContract] and [ServiceOperation] wrt the relationship between class and method, then intellisense could display a list of attribute options for each method. - Anonymous
October 14, 2004
How is the use of attributes in Steve Swartz' scenario different from cargo cult programming? The Indigo team seems to want developers to ignore the "man behind the curtain". Thus, Steve presents us with a couple of completely false alternatives to the use of attributes. The correct answer is pretty obvious from Steven Clarke's analysis. The code that connects your application to the wire is no minor detail. It is an essential facet of a distributed application's design. It shouldn't be hidden behind an impenetrable wall of attributes and codegen.