Developing Web Applications using ASP.NET 5 (beta 7) on Ubuntu Linux 14.04.2 LTS Part 3 – Understanding DNX Architecture
By: Nestor Guadarrama
| NOTE: This article will be based on ASP.NET RC1 version, released on November 2015. In the first two (2) articles of these series, I used ASP.NET beta 7. From an architectural point of view, there are too few change that I’ll be pointing here. |
In the first article Developing Web Applications using ASP.NET 5 (beta 7) on Ubuntu Linux 14.04.2 LTS. Part 1 – Installing & Configuring, I explained that every ASP.NET 5 project in fact is a DNX Project. Let’s take a look on how DNX projects create the basic required structures for build and host ASP.NET 5 applications.
DNX ARCHITECTURE
Figure 1. DNX Project – Libraries and infrastructure components.
From DNX documentation on GitHub, main DNX framework is divided by layers:
a) Layer 0: Native Process
This is a very thin layer whose responsibility is to find and call the CLR Native host, passing arguments given to the process to the native host to be used by the rest of the stack. An example of this layer is DNX.exe included in the DNX package. DNX.exe would be used for scenarios such as self-host or building and running from the command line.
b) Layer 1: CLR Native Host
This layer specific to the version of the CLR that you are using and has two main responsibilities:
a. Boot the CLR, how this is achieved depends on the version of the CLR. For Core CLR the process involves loading coreclr.dll, configuring and starting the runtime, and creating the AppDomain that all managed code will run in.
b. Calling the Managed Entry Point, Layer 2. When the entry point of the Native Host returns this process will then cleanup and shutdown the CLR. i.e unload the app domain and stop the runtime.
c) Layer 2: Managed Entry Point
This layer is the first layer that is written in managed code, it is responsible for:
a. Creating the LoaderContainer that will contain the required ILoaders. An ILoader is responsible for loading an assembly by name. When the CLR asks for an assembly to be resolved the LoaderContainer will resolve the required assembly using its ILoaders.
b. Provide the root ILoader that will load assemblies, and satisfy dependencies, from the provided --lib provided when running the native process. This is usually the DNX package itself.
c. Call the main entry point of the provided program.
d) Layer 3: Application host / Application
If a user compiles their entire application to assemblies on disk in the libpath then this layer is your application. To do this you pass the name of the assembly containing your applications entry point in the [ProgramName] argument and layer 2 will invoke it directly. However, in all other scenarios you would use an application host to resolve app dependencies and run your app. Microsoft.Net.ApplicationHost is the application host provided in the runtime, and has a few responsibilities:
a. Walks the dependencies in the project.json and builds up the closure of dependencies the app will use.
b. Adds an ILoader to the LoaderContainer that can load assemblies from various sources, NuGet, Roslyn, etc.
c. Calls the entry point of the assembly whose name is given as the next argument to the native process
e) Layer 4: Application
As the name says, this is the developer’s application code when running on the application host.
As you remember in our project testproject, I created two (2) basic files: a) project.json, and b) startup.cls. Both files creates the basic structure of a DNX Project. There are two mandatory conditions for a project. The project.json file must contain valid json, where brackets { } are used inside the file, and that your startup.cs file contains valid C# (at this moment, there is no way to use VB code for DNX).
The presence of a project.json file defines a DNX project. It is the project.json file that contains all the information that DNX needs to run and package your project. You can find project.json structure getting access to https://json.schemastore.org/project. Actually, the ongoing documentation for this structure is located in https://github.com/aspnet/Home/wiki/Project.json-file. project.json file will let us decide how can we accomplish next assignments:
· What framework do I want to use for compile and deploy my web app?
· Which dependencies (server and client) do I need for my web app?
· Which specific assemblies/references do I need to add to my web app?
· Which commands do I need to execute for my web app?
· Which files will I include/exclude from my web app during compilation time?
· Which scripts do I need to set for my web app before/after compilation time?
· Where do I want to publish my web app?
From project.json structure, I’d like to bring it up some interesting sections/properties that will let’s understand how to accomplish these assignments:
· Properties: This section defines most of all required individual properties related to the package/project. Some interesting properties are:
o Version: As its name says, this property describes the version of the project/package. Examples: 1.2.3, 1.2.3-beta, 1.2.3-*:
| { "version": “1.2.3-*" } |
o Authors: Property for documenting developers and/or team project related to this file (array type).
| { "authors": ["Nestor Guadarrama"] } |
o Compile: Specify which files will be compiled. Default value is "**/*.cs".
| { "compile": "*.cs" } |
o WebRoot: Specifying the webroot property in the project.json file specifies the web server root (aka public folder). In Visual Studio, this folder will be used to root IIS. Static files should be put in here.
| { "webroot": "wwwroot" } |
o Exclude: Basically, this property is a list (array type) of folders that will be excluded during pre-compilation time. Example: [ \"Folder1/*.ext\", \"Folder2/*.ext\" ]",
| { "exclude": ["wwwroot", “node_modules”, “bower_components”] } |
o PublishExclude: List of extensions files that will be excluded during pre-compilation time.
| { "publishExclude": ["**.user", “**.vspscc”] } |
Documentation from ASP.NET, has a list of common include/exclude properties:
| Name | Default value |
| compile | |
| compileExclude | |
| content | **/* |
| contentExclude | |
| preprocess | compiler/preprocess/**/*.cs |
| preprocessExclude | |
| resource | compiler/preprocess/resources/**/* |
| resourceExclude | |
| shared | compiler/shared/**/*.cs |
| sharedExclude | |
| publishExclude | bin/**;obj/**;**/.*/** |
| exclude |
Table 1. List of include/exclude properties
o Dependencies: Dependencies section lists all the dependencies of your application. These are defined by name and version, the runtime loaders will determine what should be loaded. NuGet package, source code, etc.
| { "dependencies": { "Microsoft.AspNet.IISPlatformHandler": "1.0.0-beta*", "Microsoft.AspNet.Server.Kestrel": "1.0.0.-beta*", "Microsoft.AspNet.Diagnostic": "1.0.0.-beta*" } } |
According with GitHub documentation, on final version, you'll be able to have different types of references:
§ Private - Don't expose this dependency to intellisense or compilation transitively.
§ Bago (Build and go away) - After compiling this reference will be compiled into the target project
| { "dependencies": { "Microsoft.AspNet.ConfigurationModel": { "version": "0.1-alpha-*", "options": "private", "FakeToolingPackage" : {"version": "0.1", "options": "dev"} } } |
o Command: When running dnx.exe you can pass the name of a command to execute it. In this snippet you could run "dnx . web" to self host the app on Windows or "dnx . kestrel" to run your app on OS X/Linux. In our case, because I defined “web” command for using kestrel, if I run “dnx web”, I’ll launch Kestrel for self-host m web app:
| { "commands": { "kestrel":"Microsoft.AspNet.Server.Kestrel server.urls=https://localhost:6001", "web": "Microsoft.AspNet.Hosting server.name=Microsoft.AspNet.Server.WebListener server.urls=https://localhost:6005", “gen”: "Microsoft.Framework.CodeGeneration", “ef”: "EntityFramework.Commands" } } |
o Frameworks: Target frameworks that will be built, and dependencies that are specific to the configuration. Next sample, will build for Desktop (dnx451) or Core CLR (dnxcore50). Core CLR has many extra dependencies as the packages that make up the BCL need to be referenced.
| { "frameworks": { "dnx451": {}, "dnxcore50": { “dependencies”: { "System.Collections": "4.0.0.0", "System.Collections.Concurrent": "4.0.0.0", "System.ComponentModel": "4.0.0.0", "System.Linq": "4.0.0.0", "System.Reflection": "4.0.10.0", "System.Runtime": "4.0.20.0", "System.Runtime.InteropServices": "4.0.10.0", "System.Threading": "4.0.0.0", "System.Threading.Tasks": "4.0.0.0" { } } |
o Configurations: Configurations are named groups of compilation settings. There are 2 defaults built into the runtime, Debug and Release. You can override these (or add more) by adding to the configurations section in the project.json.
| { "configurations": { "Debug": { "compilationOptions": { "define": ["DEBUG", "TRACE"] } }, "Release": { "compilationOptions": { "define": ["RELEASE", "TRACE"], "optimize": true } } } } |
When building a DNX based application, such as by using dnu build or via pack/publish with dnu pack or dnu publish, you can pass --configuration <configuration> to have DNX use the named configuration. This is an example of how to build “Release” set project for both dnx451 and dnx5 (using dnu build with no switches will compile project for all configuration set on project.json file):
| dnu build –configuration “Release” … … … Using Package dependency System.Threading.Thread 4.0.0-beta-23409 Source: /home/nguada/.dnx/packages/System.Threading.Thread/4.0.0-beta-23409 File: ref/dotnet/System.Threading.Thread.dll Using Package dependency System.Threading.ThreadPool 4.0.0-beta-23409 Source: /home/nguada/.dnx/packages/System.Threading.ThreadPool/4.0.0-beta-23409 File: ref/dotnet/System.Threading.ThreadPool.dll Using Package dependency System.Threading.ThreadTimer 4.0.0-beta-23409 Source: /home/nguada/.dnx/packages/System.Threading.ThreadTimer/4.0.0-beta-23409 File: ref/dotnet/System.Threading.ThreadTimer.dll Build succeeded. 0 Warning(s) 0 Error(s) Time elapsed 00:00:01.8216115 Total build time elapsed: 00:00:01.8634398 Total projects built: 1 |
Figure 2. “Release” project folder output after dnu build command
o Scripts: The scripts section of the project.json allows you to hook into events that happen as you work on your application:
| { "scripts": { "prebuild": "executed before building", "postbuild": "executed after building", "prepack": "executed before packing", "postpack": "executed after packing", "prepublish": "executed before publish", "postpublish": "executed after publish", "prerestore": "executed before restoring packages", "postrestore": "executed after restoring packages", "prepare": "After postrestore but before prepublish" } } |
Most of these are fairly self-explanatory and each matches an explicit command in the DNU. Except for prepare. Prepare runs both after a restore and before publishing and is intended to be used to make sure everything is ready for either development or publishing. For example, you often need to make sure that you run all of your gulp tasks after you restore packages, to make sure you get things like css copied from new bower packages, and you also want to make sure that gulp is run before you publish so that you are publishing the latest code generated from your tasks.
The values of the scripts are commands that will be run in your environment as if you had opened a terminal and run them:
| { "scripts": { "postrestore": [ "npm install", "bower install" ], "prepare": [ "gulp copy" ] } } |
DNU & DNX UTILITIES
As you noticed, both command utilities dnu and dnx are critical components of all these configurations take place in our ASP.NET 5 projects. We already used dnu restore and dnx web commands for restore all dependency packages and self-host our ASP.NET 5 project. Let’s take a look how to use these utilities through examples. First, help documentation for dnu:
| root@UBUNTUDESKTOP:~/git/testasp5# dnu –help Microsoft .NET Development Utility Mono-x64-1.0.0-beta8-15858 Usage: dnu [options] [command] Options: -v|--verbose Show verbose output -?|h|--help Show help information --version Show version information Commands: build Produce assemblies for the project in given directory clear-http-cache Clears the package cache commands Commands related to managing application commands (install, uninstall) feeds Commands related to managing package feeds currently in use install Install the given dependency list Print the dependencies of a given project pack Build NuGet packages for the project in given directory packages Commands related to managing local and remote package folders publish Publish application for deployment restore Restore packages wrap Wrap a csproj/assembly into a project.json, wich can be referenced by project.json files |
Because I’m using an ASP.NET 5 MVC project, template already has some dependencies included into project.json file. However, I’ll assume that I need for my project a server-side dependency for EntityFramework.Commands and EntityFramework.SQLite packages. What I need to do is just using dnu for install my package and restore it into web app cache:
Figure 3. dnu install command used for add EntityFramework server-side packages to our ASP.NET 5 project
However, if you try to build (dnu build) your web app, you will receive compilation errors because this version of EntityFramework package doesn’t support .NET Core 5:
Figure 4. dnu build compilation error for using an unsupported version of a specific package.
In order to check which versions of .NET Core are installed on my environment, I can use dnvm utility (.NET Version Manager):
| root@UBUNTUDESKTOP:~/git/testasp5# dnvm list -detailed Active Version Runtime Architecture OperatingSystem Alias Location ------ ------- ------- ------------ --------------- ----- -------- 1.0.0-beta7 mono Linux/osx ~/.dnx/runtimes 1.0.0.-beta8 mono Linux/osx ~/.dnx/runtimes * 1.0.0-rc1-update1 mono Linux/osx default ~/.dnx/runtimes |
I’ve not RC final packages installed on my environment. What I’ll do is downgrade versions of EntityFramework package that I need for testing my app. So, I’ll use 7.0.0-beta8 instead and I’ll come back to previous .NET version:
Figure 5. project.json file updated with previous version of EntityFramework packages
| root@UBUNTUDESKTOP:~/git/testasp5# dnvm use 1.0.0-beta8 Adding /home/nguada/.dnx/runtimes/dnx-mono.1.0.0-beta8/bin to process PATH root@UBUNTUDESKTOP:~/git/testasp5# dnvm list –detailed Active Version Runtime Architecture OperatingSystem Alias Location ------ ------- ------- ------------ --------------- ----- -------- 1.0.0-beta7 mono Linux/osx ~/.dnx/runtimes * 1.0.0.-beta8 mono Linux/osx ~/.dnx/runtimes 1.0.0-rc1-update1 mono Linux/osx default ~/.dnx/runtimes root@UBUNTUDESKTOP:~/git/testasp5# dnu restore Microsoft .NET Development Utility Mono-x64-1.0.0-beta8-15858 CACHE https://api.nuget.org/v3/index.json Restoring pacakges for /home/nguada/git/testasp5/Project.json Writing lock file /home/nguada/git/testasp5/project.lock.json Restore complete, 16389ms elapsed root@UBUNTUDESKTOP:~/git/testasp5# dnu build … … … Build succeeded. 0 Warning(s) 0 Error(s) Time elapsed 00:00:03.6556967 Total build time elapsed: 00:00:03.6787158 Total projects built: 1 |
You also has been noticed that every time that we execute dnu restore command, process create a new files named project.lock.json. When doing a package restore, DNU builds up information about the dependencies of your application, this information is persisted to disk in this file.
| root@UBUNTUDESKTOP:~/git/testasp5# dnu restore Microsoft .NET Development Utility Mono-x64-1.0.0-beta8-15858 CACHE https://api.nuget.org/v3/index.json Restoring pacakges for /home/nguada/git/testasp5/Project.json Writing lock file /home/nguada/git/testasp5/project.lock.json Restore complete, 16389ms elapsed |
DNX reads the lock file when running your application instead of rebuilding all the information that the DNU already generated. To understand the reason for that, imagine what DNX has to do without the lock file:
1. Find each dependency listed in the project.json file.
2. Open the nuspec of each package and get all of their dependencies.
3. Repeat step 2 for each dependency until it has the entire graph.
4. Load all the dependencies.
By using the lock file, this process is reduced to:
1. Read the lock file.
2. Load all the dependencies.
There is significantly less disk IO involved in the second list.
The lock file ensures that after you run dnu restore, you have a fixed set of packages that you are referencing. When restoring, the DNU generates the lock file which specifies the exact versions that your project will use. This way, versions only get modified when you run dnu restore, not during run-time. Restoring also ends up improving performance at run-time since DNX no longer has to probe the packages directory to find the right version to use, DNX just does what the lock file instructs DNX to do.
| NOTE: The primary advantage of the lock file is to prevent the application from be affected by someone else installing a package into your global install directory. For this reason, the lock file is mandatory to run. If you do not have a lock file, DNX will fail to load your application. |
There is a field in the lock file, locked, which can be set to true either manually or via dnu restore —lock. Setting this field to true specifies that dnu restore will just download the versions specified in the lock file and will not do any dependency graph walking or version selection. You can run dnu restore —lock to generate a locked lock file. Future restores will not change your installed version, unless you use dnu restore --unlock to remove the lock. You could lock your lock file and check it in on a release branch to ensure that you always get the exact version you expect, but leave it unlocked ()and ignored by source control on development branch(es):
Figure 6. project.lock.json file showing locked field already set in false
Now, dnx help:
| root@UBUNTUDESKTOP:~/git/testasp5# dnu –help Microsoft .NET Execution environment Mono-x64-1.0.0-beta8-16231 Usage: dnx [options] Options: --project|-p <PATH> Path to the project.json file of the application folder. Defaults to the current folder if not provided. --appbase <PATH> Application base directory path --lib <LIB_PATHS> Paths used for library look-up --debug Waits for the debugger to attach before beginning execution --framework <FRAMEWORK_ID> Set the framework version to use when running (i.e. dnx541, dnx452, dnx46, …) -?|-h|--help Show help information --version Show version information --watch Watch file changes --packages <PACKAGE_DIR> Directory containing packages --configuration <CONFIGURATION> The configuration to run under --port <PORT> The port to the compilation server |
There are several interesting options to test. However, our main focus will be compile and execute our web app using –configuration option:
| root@UBUNTUDESKTOP:~/git/testasp5# dnx web –configuration “Debug” info : [Microsoft.Framework.DependencyInjection,DataProtectionServices] User profile is available. Using ‘home/nguada/.local/share/ASP.NET/DataProtection-Keys’ as a key repository; key will not be encrypted at rest. Hosting environment: Production Now listening on: https://localhost:5000 Application started. Press Ctrl+C to shut down. |
In order to play a little bit more with this configuration, I added Logging features into my ASP.NET 5 MVC app. Basically, I wanted to write a basic message every time that a controller is hit by browsing the app, so in order to do this, I modify my HomeController class and used Dependency Injection for using ILogger<T> interface:
Figure 7. HomeController.cs class implementing Microsoft.Framework.Logging ILogger interface
After restore packages, compiling this modification and running, this is the desired result:
Figure 8. Output from ILogger<T> interface for logging
On next articles, I’ll be discussing ASP.NET 5 Architecture, other tools beside Visual Studio for creating ASP.NET 5 projects, client dependency features and more. Stay tuned!
‘till next time!