UML, DSLs and software factories: let those analogies flow...
I typed this entry a few days ago, but then managed to lose it through a set of circumstances I'm too embarrassed to tell you about. It's always better second time around in any case.
Anyway, reading this recent post from Simon Johnston prompted a few thoughts that I'd like to share. In summary, Simon likens UML to a craftsman's toolbox, that in the hand of a skilled craftsman can produce fine results. He then contrasts this with the domain specific language approach and software factories, suggesting that developers are all going to be turned into production-line workers - no more craftsman. The argument goes something like this: developers become specialized in a software factory to work on only one aspect of the product through a single, narrowly focussed domain specific language (DSL); they do their work in silos, without any awareness of what the others are doing; this may increase productivity of the individual developers, but lead to a less coherent solution.
Well, I presume this is a veiled reference to the recent book on Software Factories, written by Jack Greenfield and Keith Short, architects in my product group at Microsoft, and to which I contributed a couple of chapters with Steve Cook. The characterization of sofware factories suggested by Simon is at best an over-simplification of the vision presented in this book.
I trained as a mathematician. When constructing a proof in mathematics there are two approaches. Go back to the original definitions, the first principles, and work out your proof from there; or build on top of theorems already proven by others. The advantage of the first approach is that all you have to learn is the first principles and then you can set your hand to anything. The problem, is that it will take you a very long to time to prove all but the simplest theorems, and you'll continually be treading over ground you've trod many times before. The problem with the second approach is that you have to learn a lot more, including new notations (dare I say DSLs) and inevitably end up becoming a specialist in a particular branch of the subject; but in that area you'll be a lot more productive. And it is not unknown for different areas of mathematics to combine to prove some of the more sophisticated theorems.
With software factories we're saying that to become more productive we need to get more domain specific so that we can provide more focused tooling that cuts out the grunt work and let's us get on with the more challenging and exciting parts of the job. As with mathematics, the ability to invent domain specific notations, and, in our case, the automated tools to support them, is critical to this enterprise. And sophisticated factories (that is, most of them) will combine expertise from different domains, both horizontal and vertical, to get the job done, just as different branches of mathematics can combine to tackle tricky problems.
So our vision of software factories is closer to the desirable situation described by Simon towards the end of his article, where he talks about the need for a "coherent set of views into the problem". Each DSL looks at the problem, the software system being built or maintained, from a particular perspective. These perspectives need to be combined with the other views to give a complete picture. If developers specialize in one perspective or another, then so be it, but that doesn't mean that they can sit in silos and not communicate with the others in the team. There are always overlaps between views and work done by one will impact the work of another. But, having more specialized tooling should avoid a lot of error-prone grunt work, and will make the team as a whole far more productive as a result.
So what about UML in all this? To return to Simon's toolbox analogy (and slightly toungue-in-cheek) UML is like having a single hand drill in the toolbox, which we've got to try and use to drill all sizes of hole (for large holes you drill a number of small holes close together), and in all kinds of material; some materials you won't be able to drill into at all. DSLs, on the other hand, is like having a toolbox full of drill bits of all different sizes, each designed to drill into a particular material. And in a software factory, you support your DSLs with integrated tooling, which is like providing the electric hammer-drill: you'll be a lot more productive with these specialist tools, and even do things you couldn't manage before, like drill holes in concrete.
So I don't see UML as a central part of the software factory/DSL story. I see it first and foremost as a language for (sketching) the design of object-oriented programs - at least this is its history and its primary use to date. Later versions of UML, in particular the upcoming UML 2, have tried to extend its reach by adding to the bag of notations that it includes. At best, this bag is useful inspiration in the development of some DSLs, but I doubt very much that they'll get used exactly as specified in the standard - as far as conformance against the standard can be checked that is...
Comments
- Anonymous
September 29, 2004
Every time some describes software factories I find myself struggling to understand how the term factory came to be used. Nothing that you or Jack Greenfield or Keith Short describes sounds anything like a factory in the normal sense of the word. Using that word generates a lot of hostility from the very people you're trying to persuade to adopt your ideas. Wouldn't it be better to come up with a name that more accurately reflects the fundamental ideas you expound and doesn't come loaded with negative connotations? - Anonymous
September 29, 2004
So UML is limited because it's actually a (set of) DSLs that are a pretty low level of abstraction. But there is also the MOF which is aimed at allowing you to define DSLs (since it's for so-called meta-modelling), so how are "Software Factories" different from the MOF/MDA story (rather than the UML/MDA story)? - Anonymous
September 29, 2004
John, the term is now in print so it's too late to change. I was asked to contribute to the book after the 'Software Factory' term had already been chosen. So I've forwarded your comment to Jack and Keith. Hopefully one of them will find the time to explain the choice here. You could also ask Keith directly on his blog. - Anonymous
September 29, 2004
Michael, great question.
First let's tackle MDA. The software factory vision is much broader then MDA. MDA tends to focus on generation of platform specific models from platform independent models, and mandates the use of OMG modelling standards. The platform specific/platform independent aspect is just one, and IMO not very important and in many cases unachievable, benefit of abstracting out views of your software system into models. See my earlier posting for more on this: http://blogs.msdn.com/stuart_kent/archive/2004/09/14/229415.aspx. Software factories is much broader in its outlook: to create a software factory you need to integrate and tool up languages and development processes whose focus is on producing applications within a specific domain (and I have discussed what is meant by the term domain in http://blogs.msdn.com/stuart_kent/articles/228917.aspx). It's not primarily about trying to abstract out a model which you can then use to generate the same application but working on two different platforms, though this can sometimes be a side-effect.
Now MOF. You are right to observe that MOF is aimed at allowing you to define DSLs. But I think it only takes us a small step in that direction. Defining a DSL, or more accurately a domain specific modelling tool, has a lot more to it than just defining a metamodel. What about notation, including behavioral aspects of a tool which are dependent on the notation, model explorer, persistance story, well-formedness constraints, including how they get exposed to the user when working in a tool, and so on. And once you can edit your models, what about all the automation tools you need to build that consume them: the model animators, the model transformers, the model synchronizers, the code generators and so on. Technology is emerging to help us build these cost effectively, but there are gaps and we need a lot more experience under our belts before we will be more sure about what works and what doesn't. So my position on MOF is that it is a step in the right direction, but suffers from the syndrome of premature standardization (see my earlier posting: http://blogs.msdn.com/stuart_kent/archive/2004/09/14/229089.aspx). - Anonymous
September 29, 2004
Michael,<br><br>Thanks for asking. We've had this question a lot, so it's a good idea to answer it here, where the answer can be easily seen and referenced. <br><br>The analogies a reader draws from the term factory necessarily depend on what the reader thinks a factory is. A reader who thinks a factory is a 19th century sweatshop that mass produces identical goods using cheap human labor might understandably find the analogy unappealing. Modern factories are nothing like that image, of course. They are highly automated facilities populated largely by robots, and they produce a wide variety of goods from shared parts. This was the image we had in mind.<br><br>Perhaps it would also help to explain that I long ago realized that productive developers are both bright and lazy in good measure at the same time. Such a developer dislikes rote and menial tasks, and builds tools that do them on his behalf, so that he can spend his time on the more creative aspects of development.<br><br>When thinking about software factories, such a developer should not think of himself as a laborer in a 19th century sweatshop. Instead, he should think of himself as an engineer who builds tools and organizes them into factories that raise productivity and reduce risk by doing rote and menial but necessary tasks very efficiently. - Anonymous
September 29, 2004
John
Thanks for asking. We've had this question a lot, so it's a good idea to answer it here, where the answer can be easily seen and referenced.
The analogies a reader draws from the term factory necessarily depend on what the reader thinks a factory is. A reader who thinks a factory is a 19th century sweatshop that mass produces identical goods using cheap human labor might understandably find the analogy unappealing. Modern factories are nothing like that image, of course. They are highly automated facilities populated largely by robots, and they produce a wide variety of goods from shared parts. This was the image we had in mind.
Perhaps it would also help to explain that I long ago realized that productive developers are both bright and lazy in good measure at the same time. Such a developer dislikes rote and menial tasks, and builds tools that do them on his behalf, so that he can spend his time on the more creative aspects of development.
When thinking about software factories, such a developer should not think of himself as a laborer in a 19th century sweatshop. Instead, he should think of himself as an engineer who builds tools and organizes them into factories that raise productivity and reduce risk by doing rote and menial but necessary tasks very efficiently. - Anonymous
October 02, 2004
The depth of the social, cultural, and historical naivete in your (Jack Greenfield) answer is truly mind-boggling. There is a strongly held cultural belief (founded in historical truth) that the purpose of factories is to replace skilled workers with unskilled workers. This is true across a wide range of industries, locations, and centuries (especially the 18th - 20th). The key to the early success of the factory in most industries is to distill the complex activities of a craft into an easily repeatable process that can be performed by unskilled labor. If you didn't mean to convey the idea that you want replace skilled craft workers with unskilled menial labor, you picked the wrong term. This has nothing to do with modern factories vs. 19th century sweatshops. The vast majority of factories today use unskilled labor. Only in those places where the cost of unskilled labor is higher than the cost of robotic machinery do skilled workers dominate the factory floor. Of course, no industry goes directly from skilled craftsmen to robotic factories. That's just not economically or practically feasible.<br><br>It's truly unfortunate that this concept, which may be really useful, is saddled with such an inappropriate and misleading name. The name will always interfere with people's ability to understand what you're trying to convey.<br><br>