共用方式為

The purpose of an operating system, redux

  • 發行項

Something changed in the stress mix last week, and I've been swamped with stress failures, that's what caused the lack of posts last week.  But I've spent a fair amount of time while driving into work thinking about my last post (on the purpose of an operating system).  I've not actually read most of the comments (I've really been busy), so if I'm echoing comments made on the other post, forgive me for poaching your ideas, it really is a coincidence.  I'm a little surprised it's taken me this long to have this idea gel in it's current form - it's blindingly obvious, but I never put the pieces together in this way before.

I still stand by my original premise.  The purpose of an operating system is to shield the application from the hardware.

But the thing that you purchase in the store (or buy with your new computer, or download from RedHat) ISN'T an operating system.  It's a platform.  

And a platform does more than just isolate the user from the hardware, a platform is something on which you build applications.  This is a key distinction that it seems many people have a really hard time making.

Let me take Linux as an example, even though I'm far from a Linux expert.  The Linux operating system exists to isolate the user from the hardware.  It hosts drivers and presents a fundamental API layer on which to build systems.  But someone who receiving a machine with just copy of Linux would be rather upset, because it's not really very useful.  What makes Linux useful is that it's a platform on which to build applications.  In the FOSS community, the "platform" is called a "distribution", it contains a set of tools (Apache, Perl, etc).  Those tools make up the platform on which you write applications.  Btw, RMS has been saying this for years now in insisting that people call the "OS" known as Linux "GNU/Linux" - he's explicitly making the differentiation between Linux the operating system and GNU/Linux the platform.

 

Similarly, Windows isn't an operating system.  Windows is a platform.  Nowadays, the Windows platform runs on the NT operating system, in previous years, the Windows platform ran on the Windows VXD operating system, and before that it ran on the MS-DOS operating system.  OSX is also a platform, it runs on an operating system that is (I believe) a derivative of the Mach OS, running with a BSD personality layer (I may be wrong on this, I'm not enough of an OSX expert to know the subtleties of its implementation).

For convenience sake, we refer to the Windows platform as an operating system, just like people refer to OSX or Linux as operating systems, it's simply easier to present it that way to users. 

If you make the paradigm shift to considering the "operating system" as an operating system plus a development platform, it makes a heck of a lot more sense why the platform contains things like an HTML renderer, a sockets layer, a TCP/IP stack, an HTTP server, a directory service, a network filesystem, etc.  These aren't things that shield an application from the hardware, but they ARE things that provide value to applications that want to run on that platform. 

As an easy example, consider an application like the game Neverwinter Nights.  Because the developers of Neverwinter Knights knew that there was an HTML renderer built-into the platform, it meant that they could leverage that renderer in their launcher application (or their update application, I forget which of them used the MSHTML control).  Because they knew that the platform contained a multimedia stack with WAV file rendering, they didn't have to build in a WAV renderer into the game.  Because they knew the platform had support built-in for video rendering, they didn't have to include a video renderer.  They might have had to include a video codec along with their application, because the platform didn't necessarily include that, but it's orders of magnitude easier to write (or license) a video codec than it is to write or license an entire multimedia pipeline.

A rich platform means that applications can depend on that platform, which, in turn makes the platform more attractive to the applications.  Everything that the platform does that an application doesn't have to do is one less thing an application needs to worry about.  Of course, the challenge when enhancing the platform is to ensure that the platform provides the right level of capabilities, and the right ease of use for those capabilities, otherwise applications won't use the platform's implementation, but will chose to roll their own.

Comments

  • Anonymous
    April 10, 2006
    "Of course, the challenge when enhancing the platform is to ensure that the platform provides the right level of capabilities, and the right ease of use for those capabilities, otherwise applications won't use the platform's implementation, but will cho[o]se to roll their own."

    I definitely don't envy your job, trying to define functionality that won't be mainstream for years. No matter how well you think it out, there's a good chance it won't work as you intended once it's in wide distribution. I'm not placing blame here, just expressing frustration.

    Those of us who write apps for the installed base are saddled with those 1990s APIs and techniques that didn't quite work out as intended, but they're present on every Windows system so they're the best bet for compatibility. Sure I could use some new Vista/W2003 API but I still have to roll my own for the 95% of the market that doesn't have them. So the new APIs just make the job more complex until they're mainstream.

    The same goes for .NET, I'd love to use it but I can't justify the hassle of delivering the 2.0 framework to users, who will blame my app for being bloated. Plus, I want to see Microsoft eating some dogfood there before I take the jump. I would prefer that Microsoft finish its compatibility with XHTML and CSS 2.1 (specs that were nailed down almost half a decade ago). That way, in about five years I will finallly be able to build standard browser-based apps that work across platforms.


  • Anonymous
    April 10, 2006
    Dave, actually CSS 2.1 still hasn't been finalized after 8 years. And that's absolutely ridiculous and why I don't trust the W3C. Not only that but 2.1 exists simply to correct the huge number of errata in 2.0.  I don't blame the IE team for being hesistent to implement the recommendation.

    I agree about the .NET framework though. The fact that the Windows shell team gave up on it halfway through makes it clear that it didn't deliver the performance it was supposed to in terms of runtime overhead.  Indeed it can be hard convincing your boss you need to deploy a .NET application to do some trivial thing yet it uses 20 megs of RAM on every machine.

  • Anonymous
    April 10, 2006
    I agree with Dave on sometimes MS is a bit behind set standards, or sometimes their standards are different than the set standard hence forcing us to roll our own. However don't get me wrong they are getting much much better.

    Me on the other hand love .net and I think that a huge part of the it's getting better I see. I am able to do so much more and so much faster in a much more secure way with .net than ever before. The Crux is the Framework install. Yet still it is far superior. Now my question though for Larry to think about is the .net a platform. Hence it could be considered Windows actually hosts several platforms hence the scripting engines, some call Direct X and platform, etc or is .net just another part of windows. We can go right into the windows API's with unsafe codeblocks after all and a lot of the framework does that as well.

  • Anonymous
    April 10, 2006
    > Everything that the platform does that an application doesn't have to do is one less thing an application needs to worry about

    And conversely, every decision the platform makes for the application is one less thing an application can decide for itself.

  • Anonymous
    April 10, 2006
    Sweet.  Or should I say "suite."  

    This is very clean.  

    Also, the prospect of different hostings, posted by Maurits (e.g., .NET versus WinApps versus ...) is also something to sort out.  There do seem to be multiple platforms that emerge from the various stacks.  (No wonder VC++ newbies are so confused.)

  • Anonymous
    April 10, 2006
    Dave wrote:
    > XHTML and CSS 2.1 (specs that were nailed down almost half a decade ago)

    XHTML I'll give you (XHTML 1.0 became a recommendation in 2000, 1.1 in 2001), but CSS 2.1 is in fact not yet a recommendation at all.

    Jeff: my view on this is that you can build platforms on top of existing platforms. The IBM compatible PC and the x86 architecture are also platforms, on top of which the Windows platform is built. .Net is a platform that builds on the Windows platform, and occasionally such newer platforms become part of the base platform, like what happened to DirectX before and .Net with the introduction of Windows Server 2003 and now Vista.

  • Anonymous
    April 10, 2006
    Maurits, I don't follow that - applications can choose to opt-into platform features.  Firefox doesn't HAVE to use the Windows HTML renderer, it can write its own.

    Similarly, an application doesn't HAVE to use the Windows TCP/IP implementation, it can write its own and deploy it if it finds the Windows implementation insufficient.

  • Anonymous
    April 10, 2006
    > Firefox doesn't HAVE to use the Windows HTML renderer

    Indeed (and of course, it doesn't.)

    Suppose an application has some rather primitive HTML rendering needs: in particular, <b>, <i>, and <blink>.

    Windows HTML renderer made the choice not to support <blink> (by popular demand.  Neither does Firefox's.)

    Now the app has to look elsewhere for its HTML rendering needs.  It either has to license another engine (such as Gecko) or write one from scratch.

    This is unfortunate.  The platform has made a decision that is not compatible with the application's needs.

    This happens all the time, to a greater or lesser extent, in any platform.  This is an impetus to platform evolution... future versions of Windows HTML might respond to this need by allowing a "SupportBlink" option.

    But if you're the application developer, it's a bit of a pain.  It's almost enough to make you redesign your app to not make use of <blink>.

  • Anonymous
    April 10, 2006
    So what?  If there were no MSHTML you'd have to do implement it yourself anyway.

  • Anonymous
    April 10, 2006
    "I agree about the .NET framework though. The fact that the Windows shell team gave up on it halfway through makes it clear that it didn't deliver the performance it was supposed to in terms of runtime overhead.  Indeed it can be hard convincing your boss you need to deploy a .NET application to do some trivial thing yet it uses 20 megs of RAM on every machine."

    Performance wasn't the primary issue why .Net isn't used in the shell. I believe the primary problem was in terms of shell extensions, since you can only have one version of the .net framework per process. If someone wanted to run the shell with an extension written in 1.1, and another in 2.0 (or some future version of .Net) the two wouldn't be able to run in the same process.

  • Anonymous
    April 10, 2006
    The comment has been removed

  • Anonymous
    April 10, 2006
    Larry Osterman is a well known and very respected developer at Microsoft. Recently he has been writing...

  • Anonymous
    April 10, 2006
    I think it is natural for an operating system to evolve over time so that it will eventually encompass other application functions.

    The OS is really just a big toolkit for applications to take advantage of. Once an application gains wide acceptance it seems natural to add it to the toolkit.

  • Anonymous
    April 10, 2006
    Monday, April 10, 2006 7:15 PM by Maurits
    [Larry Osterman:]
    >> Firefox doesn't HAVE to use the Windows HTML renderer
    >
    > Indeed (and of course, it doesn't.)

    IE lovers will be glad to see the following parallel:

    Firefox doesn't have to use the Windows dial-up dialog.
    And of course it doesn't.
    And it doesn't come close to providing an adequate dial-up dialog for dial-up users who have two ISPs (or more).
    I had to recommend to my parents that it would be easiest for them to return from Firefox to Internet Explorer.

    Of course, that's no excuse for what Microsoft did to Netscape, or to Real, etc.

  • Anonymous
    April 10, 2006
    Maurits: I'm not sure where you're trying to get here.

    Are you sugesting that the platform shouldn't provide "extra services" because the chosen implementation might not match some developers' needs?

  • Anonymous
    April 10, 2006
    Great blogpost. Although you were indeed a bit off on the OSX thing. It has a hybrid kernel. Basicly it has huge parts of mach 3.0 and huge parts of (Free)BSD. All of the VM code for example is mach code while other parts like the TCP/IP stack are (Free)BSD code. Both the mach and (Free)BSD system calls are available for OSX (alsthough for some all the functionality got hacked out and a call to the other (be it mach or BSD) syscall is made).

  • Anonymous
    April 10, 2006
    Though people claim you are an OS expert, you still don't seem to realize that the whole platform need not be built into the kernel to provide a developer a platform to work on.

    With people like you developing Windows no wonder the security is at the level it is and the product delays and feature sets are so part of Windows and you need to steal ideas from OSX.

  • Anonymous
    April 10, 2006
    Dave, did I EVER say the platform was in kernel mode?  Please show me where I said that and I'll fix it.

    Parts of the platform need to run in kernel mode, other parts (the vast majority of them) don't.

  • Anonymous
    April 10, 2006
    The comment has been removed

  • Anonymous
    April 11, 2006
    >Let me take Linux as an example, even though I'm far from a Linux expert.  
    Me neither, but, let me correct you: Linux is just a KERNEL. Nothing less, nothing more.
    In order to become POSIX-conformant operating system, it needs a lot of stuff: C runtime library (glibc), shell (bash), and a number of utility packages just to make it usable (i.e. UNIX-like).

    When it comes to "platform", as a "thing where applications are running" both Linux and Windows look miserable. Say, I want to run Eclipse (Java app). In both Windows and Linux it's neccessary to install JRE from Sun. The same thing is for BitTorrent, which is Python application. Or, Source Navigatgor (TCL app). The same thing is for Perl, Ruby, LUA, whatever apps.

    So, I think, a term platform better matches "a thing which isolates application from operating system differences". Namely, these Pythons, Javas, Rubys and .NETs are platforms. Windows is still operating system or, even better, "Windows distribution" in Linux terms :-))).

  • Anonymous
    April 11, 2006
    "This is an impetus to platform evolution... future versions of Windows HTML might respond to this need by allowing a "SupportBlink" option."

    Actually, I think that the Microsoft WebBrowser object already provides all the hooks to add a blink tag (shudder):

    http://msdn.net/workshop/author/dhtml/overview/customtags.asp

    That's a good example of the way a platform should work. It lets you take advantage of all the good infrastructure but still build and integrate the minor missing pieces you need to finish the job. (For the blink example in particular, you could easily implement some Javascript and CSS to make the tag blink without creating any custom tags at all--that's the approach I'd take.)

    OTOH, I understand why Mozilla doesn't use a lot of the Windows infrastructure. They are writing a browser that needs to run on multiple platforms that don't offer equivalent functionality. They chose a roll-your-own approach to solve that problem. It simplifies debugging (fewer divergent code paths) and makes the user experience consistent across platforms, but may not give the best experience on a particular platform.

  • Anonymous
    April 11, 2006
    > Are you sugesting that the platform shouldn't provide "extra services" because the chosen implementation might not match some developers' needs?

    I'm saying that bundling any particular implementation of a given platform feature with a given operating system has an opportunity cost associated with it.

  • Anonymous
    April 11, 2006
    The comment has been removed

  • Anonymous
    April 11, 2006
    > I'm saying that bundling any particular implementation of a given platform feature with a given operating system has an opportunity cost associated with it.

    From the point of view of someone building an application on the platform, I'm not clear on what that opportunity cost is.

    Case 1: Platform provides an HTML renderer.  In the best case, it does what I need and I can put it to use.  In the worst case, it doesn't do what I need and I have to roll my own (or include someone else's HTML renderer with my distribution).

    Case 2: Platform doesn't provide an HTML renderer.  In the best case, I have to roll my own (or include someone else's).  The best case is now the same as the worst case.

    So how is Case 2 better for me the developer?

  • Anonymous
    April 12, 2006
    Vassili: "Linux is just a KERNEL. Nothing less, nothing more."

    Yeah - what part of "[...] just copy of Linux [is] not really very useful.  What makes Linux useful is that it's a platform on which to build applications.  In the FOSS community, the "platform" is called a "distribution"" did you not understand?

    "When it comes to "platform", as a "thing where applications are running" both Linux and Windows look miserable."

    Really? My preferred distribution (platform) - Debian GNU/Linux - comes with Python[0], TCL[1], Perl[2], Ruby[3], LUA[4], etc... (Java support is still a bit iffy due to proprietary Java's non-freeness, free Java's non-maturity, and the DFSG[5], but the DFSG is what attracts me to Debian). As far as I'm aware, all the other Linux distros are similarly featureful. Which distro have you been using that doesn't have any of this?

    Adam

    [0] http://packages.debian.org/testing/python/python
    [1] http://packages.debian.org/testing/interpreters/tcl8.4
    [2] http://packages.debian.org/testing/perl/perl
    [3] http://packages.debian.org/testing/interpreters/ruby
    [4] http://packages.debian.org/testing/interpreters/lua5.1
    [5] http://www.debian.org/social_contract#guidelines

  • Anonymous
    April 12, 2006
    Vassili: "Linux is just a KERNEL. Nothing less, nothing more."

    What part of "[just Linux is] not really very useful. What makes Linux useful is that it's a platform on which to build applications.  In the FOSS community, the "platform" is called a "distribution" [...]" did you not understand? Read that whole paragraph again - I think Larry gets what Linux is and is not.

    And, while I'd have to agree with you that a platform is a particular environment/set of libraries, I wouldn't make the distinction that POSIX and Win32 aren't platforms. Both abstract away the kernel to enough of a degree that they are both implementable on each other (see Cygwin/Wine).

    So, while some platforms are more easily implementable in terms of other platforms (e.g. dotnet is being implemented on POSIX (as mono) for Linux, but on Win32 for MSs version), I see no reason why that's a requirement. Mono could be written call directly into the Linux kernel if the developers wanted to, but why bother?

    As for how miserable Linux is, my distro (Debian Etch) comes with at least the following platforms: POSIX, Win32 (wine), dotNet (mono), Python, Perl, Ruby, TCL, Lua, PHP, etc, etc, etc... Most do these days. What distro are you using that doesn't have these platforms included?

  • Anonymous
    April 12, 2006
    "The OS is really just a big toolkit for applications to take advantage of. Once an application gains wide acceptance it seems natural to add it to the toolkit."

    Indeed. Just as a naive developer might find it natural to keep adding public methods to an object. It's not such a big deal when you do it just once or twice, but if that's how you do everything then yer cruisin' fer a bruisin'.

    That's not to say that MS is being naive or anything, but their business model means that Windows has dependancies that other platforms might not have. The business model that drives Windows development will have to change when (not if) Moore's Law ceases to apply to hardware development, for instance. Under such a circumstance, slim'n'sassy will be the norm, and the rebels will probably be the ones packing in the extra features.

  • Anonymous
    April 12, 2006
    Maurits, I'm not quite sure I follow the logic of your criticism:

    Scenario 1:  The OS provides certain HTML rendering capabilities -> apps that require functions it does not provide must source their own, but apps that only require the functionality provided can use it.

    Scenario 2:  The OS provide no HTML rendering capabilities.  All apps must now source their own HTML libraries.


    Your seem to be arguing that because Scenario 1 disadvantages a small proportion of applications, that it is inferior to Scenario 2 and should not be pursued.  I'm not quite sure I follow...

  • Anonymous
    April 13, 2006
    The comment has been removed

  • Anonymous
    April 13, 2006
    The cost to the application developers who can't use the features of the platform never changes.  Before Windows provided a renderer without blink support, the application would have to include its own renderer.  After Windows provided a renderer without blink support, the application still would have to include its own.  There's no trade-off here if the application absolutely must have blink support.

    However the cost to developers who can live without blink support decreases dramatically.

  • Anonymous
    April 14, 2006
    IMHO what you call an operating system is the kernel. I.e. the kernel is responsible for shielding the hardware from the userland applications. The operating system is the kernel + all the tools needed to make the system usable.
    So Linux is a kernel and GNU/Linux is an operating system which uses the Linux kernel and the GNU tools.

  • Anonymous
    April 16, 2006
    >>I still stand by my original premise.  The purpose of an operating system is to shield the application from the hardware.<<

    I don't think that tells the whole story at all. I would say something more like "the purpose of an operating system is to provide abstract models of the hardware that are as useful as and general as possible."

    For example, to access a sector on disk without and OS, I would need to know the details of the disk controller, do INs and OUTs to the controller out the gazoo, and so on. By your description, what an OS should then do is, for example, provide a simple call by which I could access sector N on the disk. That is really all that is required to shield the application from the hardware.

    But the OS should, and does, do something better. It invents an abstraction called "files" and another abstraction called "(sub)directories,"   which is quite different from accessing sectors. The abstraction is a good one in that, with this abstraction, but with the ability to directly access sectors not provided, the abstraction is good enough that it is really all I need for virtually anything I might want to do, and the inability to access sectors directly is no significant loss.

    For each peice of hardware, there is not only the job of isolating the apps from the "details" of how it is controlled and accessed, there is also the job of inventing the abstract model by which the device will be seen by applications. Inventing the abstract model is quite difficult, because a major job in inventing the abstract model is to "predict the future" to some extent, so the the abstract model not only handles todays devices, but, if well done, the model will be able to serve without change in handling future devices.

    Doing all of this, I submit, is the job of any basic OS, not the job of some larger entity we call a platform.

  • Anonymous
    April 17, 2006
    Larry,

    Your comparison with Linux (indeed any *ix) has one issue; the fact that in Windows, unlike its open-source counterpart, you can't seperate, replace, improve or otherwise completly remove ANY part of it.

    It's not only a technical issue, remember the DOJ anti-trust case... this was a big deal, and still is. It's called CHOICE.

    Everything you said is true, only it's true in the sense of marketing, not in terms of flexibility, extensibility or choice.

    So while you (and if you haven't noticed, ALL of MS management AND, coincidentialy, the legal team too) sound like making a logical statement to the user's benefit, it really is an advertisment for why Windows needs all that, and what MS is doing for the customers.

    Granted, it's useful in (possibly) many cases, but not admitting its drawbacks, is an ommision that can't be easly overlooked, especially if the statement is made by someone as knowledgable as you. You know better!

    -Ash

  • Anonymous
    April 18, 2006
    Ashod, I'm talking about the terminology.  Nothing more, nothing less.  FOSS is irrelevant to the discussion, the terms I use apply to FOSS and CSS, so any discussion about replacing/improving/whatever is out of scope IMHO.

  • Anonymous
    April 18, 2006
    Wow, everyone's disagreeing with me. :)

    Let me restate my point slightly.

    Every developer-hour Microsoft spends on the development platform (HTML renderer, video renderer, etc.) is a developer-hour taken away from the core OS (WinFS, anyone?)

    Development platforms are a fundamental need and I don't dispute that.  But is licensing another HTML renderer such a bad thing?

  • Anonymous
    April 18, 2006
    I would agree, that Windows is a platform, not just an OS. But the problem is that platform is not a word from user's dictionary. Platform is something, what developer needs. Why should I, as a user, pay for a platform? Maybe I like Firefox. Why should I invest into IE development? Why should I pay out huge investment into platform, if I just want OS to make my favorite apps running? We can say that platform is free, included into OS price, but it is a some sort of commercial lie, which everyone is used to understand.

    So here we have a conflict, which is quite visible now. There are obviously two target groups, but MS do not have an interest to target them separately.

  • Anonymous
    April 18, 2006
    Maurits, but now you're assuming that a developer who works on the developement platform (HTML renderer, video renderer, etc) is completely replaceable with one who works on the core OS, which is patently false. Someone who is able to write a great HTML renderer isn't nessecarily going to be able to write a great thread scheduling algorithm or file system driver (or vice versa).

    So just because there are people at Microsoft working on IE7 doesn't mean those same people could instead be making the release of WinFS any quicker.

  • Anonymous
    April 19, 2006
    Larry,

    I agree with the terminology point you are making. I think few people would disagree. In fact, I'm saying that what you are saying is something that has been advertised many times in many forms and shapes.

    What I disagree with (and what I wrote to voice out) is the reasoning you give for the need to have a platform.

    "These aren't things that shield an application from the hardware, but they ARE things that provide value to applications that want to run on that platform."

    And...

    "Everything that the platform does that an application doesn't have to do is one less thing an application needs to worry about."

    These are marketing statements more than anything. And the value of something is ONLY in the eye of the beholder. As others have noted on this post, sometimes we have to roll our own exactly because the existing OS-based one has some issue. In fact, your last statement also means that everything that the platfrom does, takes away some flexibility from the application.

    And no, I'm not taking issue because you compared FOSS with MS Windows, but because you argued that a platform is a must-have and can't do without. While doing that you ignored other factors, such as flexibility and user choice, which I think passes for stepping into marketing turf.

    Keep up the thought-provoking posts though!
    -Ash

  • Anonymous
    April 23, 2006
    The comment has been removed

  • Anonymous
    April 25, 2006
    The comment has been removed

  • Anonymous
    April 26, 2006
    Monday, April 24, 2006 2:10 AM by Troy Campbell
    > Why reinvent the wheel?

    Hey!  That used to be one of my Usenet signatures!

    More precisely:
    Why are programmers criticised for reinventing the wheel,
    when car manufacturers are praised for it?

  • Anonymous
    October 12, 2007
    PingBack from http://www.imbrandon.com/2007.10.13/why-all-the-name-calling.html

  • Anonymous
    October 12, 2007
    PingBack from http://www.linuxindex.com/2007/10/13/brandon-holtsclaw-why-all-the-name-calling/

  • Anonymous
    October 14, 2007
    PingBack from http://www.linuxindex.com/2007/10/14/brandon-holtsclaw-why-all-the-name-calling-2/

  • Anonymous
    June 07, 2009
    PingBack from http://besteyecreamsite.info/story.php?id=1156

  • Anonymous
    June 15, 2009
    PingBack from http://mydebtconsolidator.info/story.php?id=21437