通过

Pico Process Overview

  • 项目

Posted on behalf of Nick Judge.

Overview

This post discusses pico processes, the foundation of WSL.  It explains how pico processes work in Windows and goes into the history of how they came to be, the abstractions we decided to implement and the various use cases beyond WSL that emerged. Armed with this context, the next series of posts will dive into specific areas of how exactly WSL works.

Drawbridge

The pico process concept originated in MSR as part of the Drawbridge project. A goal of this project was to implement a lightweight way to run an application in an isolated environment, with the application’s OS dependencies decoupled from the underlying host OS (e.g., XP application running on Windows 10).  Normally, this would be done by running the application and OS in a virtual machine, but this comes with significant resource overhead. Instead, the Drawbridge project aimed to run the target application and OS entirely within the user-mode address space of a single process on the host OS. With its reduced overhead, when compared to a VM, this approach allows for greater density of application workloads on a single host while still providing much of the same isolation and compatibility guarantees. Check out the official Drawbridge project page for more detailed information for how they pulled this off.

In Drawbridge, the “library OS” was the target OS of the application workload.  To support a library OS that differs from the underlying host OS in addition to running the library OS in user-mode, the MSR folks needed the host OS to get out of the way and stop trying to manage the user-mode address space inside this process. They coined this type of process a “pico process” on the host side to indicate that it is a smaller version of a normal host process. A kernel-mode driver was responsible for supporting the pico process and acting as the broker between the host OS kernel and the library OS in user-mode.

Of course, proper support for this on top of a publicly released and supported version of Windows would require core Windows kernel changes to add the facilities necessary for pico processes. The MSR team brought this idea to us over in the kernel team toward the beginning of 2013, and we all agreed it would be great feature to add. In fact, we realized that this approach aligned well with some internal discussion we had been having around longer-term strategy to bring back the idea of subsystems to facilitate future architectural changes within Windows. This initial support for pico processes first appeared in Windows 8.1 and Windows Server 2012R2 but was limited to Drawbridge. Pico process support was later expanded to other Windows features.

Minimal Process

As we started implementing official support for pico processes, we decided to split the abstraction into two layers:

  • Minimal process: This is the most rudimentary type of process. Specifically, a process marked as a minimal process tells the rest of the host to get out of the way and not manage it. From the Windows kernel point of view, it is simply an empty user-mode address space.
  • Pico process: This is a minimal process with an associated pico provider kernel-mode driver to manage that empty user-mode address space.

picoProcess

Unlike traditional NT processes, when creating a minimal process, the user-mode address space is untouched and no threads are created to run in that process. Various locations in the kernel were surgically updated to skip user-mode address space setup, including:

  • The user-mode binary ntdll.dll is not mapped into the process by default.
  • The process environment block (PEB) is not created.
  • There is no initial thread created, and thread environment blocks (TEBs) are not automatically created whenever a thread is created for the pico process.
  • The shared user data section is not mapped into the process. This is a block of memory mapped read-only into all user-mode address space for efficient retrieval of common system-wide information.
  • Various places that assumed a process would always have a PEB and/or TEBs were updated to be able to handle processes without them.

While the Windows kernel does not actively manage a minimal process, it still provides all of the underlying OS support you would expect – thread scheduling, memory management, etc.

You may be wondering – why did we separate the notion of “minimal” and “pico” processes? The idea of an empty minimal process seemed useful on its own, separate from anything related to supporting a pico process. While we had nothing specific in mind around this time back in 2013, eventually a couple of scenarios did surface in Windows 10 that are now using minimal processes directly:

  • Memory Compression : Memory compression is a Windows feature that compresses unused memory to keep more data resident in RAM. It also reduces the amount of data written to and read from the pagefile, thus improving performance. Windows memory compression utilizes the user-mode address space of a minimal process.
  • Virtualization based Security   (VBS) : Using underlying virtualization capabilities, VBS isolates the user-mode address space of critical user-mode processes from the rest of the OS to prevent tampering, even from the kernel or kernel-mode drivers. A minimal process is created to indicate to management tools (e.g., Task Manager) that VBS is running.

Pico Processes and Providers

A pico process is simply a minimal process that is associated with a pico provider kernel-mode driver. This pico provider surfaces the entire kernel interface as far as the user-mode portion of the process is concerned. The Windows kernel passes all system calls and exceptions that originate from the user-mode portion of a pico process to the pico provider to handle as it sees fit. This allows the pico provider to model a different user/kernel contract separate from what Windows would normally provide.

Early during boot, a kernel-mode driver registers with the Windows kernel as a pico provider, and the kernel and provider exchange a set of interfaces specific to the needs of a pico provider. For example, the pico provider provides function pointers for the kernel to call when dispatching a user-mode system call or exception, and the kernel provides function pointers for creating pico processes and threads.

Regardless of what behaviors and abstractions the pico provider exposes to user-mode, it ultimately will rely on the Windows kernel for underlying support of thread scheduling, memory management and I/O. Of course, portions of the Windows kernel had to be updated to support new scenarios where needed.

Windows Kernel Changes

Later posts will detail the Windows kernel changes in more detail, but here is a quick sampling:

  • Improved fork support: Yes – the Windows kernel has supported “fork” for a long time (going back to earlier POSIX and SFU application support), but it is not exposed in the Win32 programming model that the rest of Windows is programmed against. We have improved the fork implementation to meet some new requirements as part of the WSL work.
  • Fine-grained memory management: Windows normally manages the user-mode address space in 64KB chunks, but was updated to allow management at single-page 4KB granularity for pico processes.
  • Case-sensitive file names: Again, yes – the Windows kernel and NTFS have long supported case-sensitive file names, but it is disabled by default and not exposed in the Win32 programming model. Changes were made to allow individual threads to opt-in to case-sensitivity operations to support a broader range of WSL scenarios.

 

Nick Judge and Seth Juarez discuss the underlying architecture that enables Pico Processes.

Comments

  • Anonymous
    May 24, 2016
    Typo:Unlike traditional NT processes, when creating a minimal process, the user-mode address space is untouched...Missing: is
    • Anonymous
      May 24, 2016
      Fixed. Thanks!
  • Anonymous
    May 24, 2016
    Are there any APIs available to provide minimal process/pico process support to third parties, or is this limited to internal OS support?
    • Anonymous
      June 06, 2016
      There are no APIs available at this time.
  • Anonymous
    May 24, 2016
    What about colon char in filenames?
  • Anonymous
    May 25, 2016
    That should probably be "Virtualization-based Security" instead of "Visualization based Security", though I would love to learn more about the latter if it existed.
  • Anonymous
    May 25, 2016
    Thank you!If I understand it correctly, there may be several subsystems based on bare NT (like OS2, POSIX and so on). So why WSL hasn't been implemeneted that way? Is it becase you need PE to have "subsystem" field in header and linux bins. are ELFs? If WSL were implemented using subsystem and not Pico it would be possible to use CreateProcess to launch Linux binaries and not Psp.. functions which are not documented yet.So, what was the reason to choose Pico instead of "subsystem"?
    • Anonymous
      May 27, 2016
      Part of the system is a subsystem, but if you are aware of how the old POSIX subsystem worked, you needed to actually recompile to get things to work. Pico is what makes it possible to run unmodified Linux binaries.
    • Anonymous
      May 31, 2016
      One of the reasons to implement WSL in the manner we did was to enable it to interleave with Windows and offer a more integrated experience. This allows Linux apps to run alongside Windows apps within the same session, sharing the same networking stack, etc. This way, one can view a Ruby website running in Bash from edge/IE/Chome/etc. by viewing http://localhost[:port]/[path]
  • Anonymous
    May 25, 2016
    There's a typo : "Visualization based Security (VBS) ..." should be "Virtualization based Security (VBS) ..."From a big *nix enthusiast point of view, I always felt that Windows was a big monolithic thing but the more I read about its kernel and how flexible it is, the more I think that I was wrong. This series of article is awesome for anybody curious enough about Windows kernel and OS internals in general o/
  • Anonymous
    May 25, 2016
    You should probably start the article by explaining what WSL stands for.
  • Anonymous
    May 25, 2016
    also what is SFU? :)
  • Anonymous
    June 09, 2016
    The comment has been removed
  • Anonymous
    June 10, 2016
    The comment has been removed
  • Anonymous
    June 16, 2016
    Reading this, and being reasonably familiar with Drawbridge/Graphene, why didn't you go full-out library OS and just have NT expose the Drawbridge PAL API (<50 calls) and provide the hundreds of Linux system calls in user space?
  • Anonymous
    June 19, 2016
    The presentation mentions that when the NT kernel receives a syscall, it first checks if the process in question is a pico process or e.g., a Win32 process. Do you happen to know some internal details behind this step? E.g., which kernel data structure is being examined?Thanks!
  • Anonymous
    July 01, 2016
    You could simple name them nano & pico while minimal process should be called pico instead jus for consistency
  • Anonymous
    July 19, 2016
    What is SFU?
  • Anonymous
    August 25, 2016
    Typo: "Visualization based Security" => "Virtualization based Security"
    • Anonymous
      August 25, 2016
      Thanks for pointing this out.
  • Anonymous
    January 26, 2017
    Thanks for the series of articles. I'm using build:Windows 10 Kernel Version 15007 MP (2 procs) Free x64Product: WinNt, suite: TerminalServer SingleUserTSBuilt by: 15007.1000.amd64fre.rs_prerelease.170107-1846When I call ZwQueryInformationProcess with ProcessImageFileName for a WSL Pico Process, I get back an empty string. I see the same empty string in the EPROCESS:1: kd> dt _EPROCESS 0xffffbe0af9cc0080 nt!_EPROCESS +0x000 Pcb : _KPROCESS... +0x450 ImageFilePointer : (null) +0x458 ImageFileName : [15] ""Is this a bug in the preview build or am I supposed to get the image name another way?
    • Anonymous
      January 26, 2017
      The best way to get help on this would be by submitting an issue to our github.
      • Anonymous
        January 27, 2017
        Will do, thanks!
  • Anonymous
    August 13, 2017
    Is it even possible that I can load and run a version of the Linux Which I customized using LFS ...I mean run that one as sub-system and then Develop and test for that ?
    • Anonymous
      September 21, 2017
      No - WSL adds a Linux-kernel compatible layer to Windows and provides a set of syscalls which are compatible with Linux. When running a distro on WSL, it runs atop WSL and the NT kernel, and doesn't contain/use any Linux kernel whatsoever.
    • Anonymous
      November 13, 2017
      Yes, to a certain degree, you can do that.The reason I think this should be possible is that I successfully replaced the Ubuntu ontop of WSL in my installation with an Arch Linux. I used these instructions as inspiration, although they aren't completely up to date: https://wiki.archlinux.org/index.php/Install_on_WSLI would suggest you try to replace the system files of the default Ubuntu with the ones of your LFS setup. Then cross your fingers and hope for the best :)
  • Anonymous
    July 05, 2018
    Will Microsoft release the details on how to create a pico process / subsystem? It may not be for everyone, but this would address a need my client have. I have already provided a solution with Hyper-V, but that is a lot of overhead and my clients are not the most sophisticated. Converting my solution to a custom subsystem would definitely be an improvement.