Writing FormUrlEncoded data with ASP.NET Web APIs

The code for this post can be downloaded in the MSDN Code Gallery.

The FormUrlEncodedMediaTypeFormatter class shipped with the ASP.NET Web APIs beta is one of the default formatters in the Web APIs and can be used to support incoming data from the application/x-www-form-urlencoded media type. This is the default format used for HTML form submission, and it has always been supported in ASP.NET MVC (but not in WCF, which was a constant feature request). This is also the default format used by jQuery when submitting objects, which makes supporting this format even more important, given the almost ubiquity of that library.

So ASP.NET Web APIs continues with the tradition of ASP.NET MVC and provides a formatter which supports consuming forms data natively. It also supports the complex objects format which is used by jQuery, in which the objects are “flattened” and all values are sent as key-value pairs as forms data. For example, this JavaScript object

1. var data = {
2.     name: 'John',
3.     age: 33,
4.     luckyNumbers: [3, 7],
5.     children: [
6.         { name: 'Jack', age: 6 },
7.         { name: 'Jane', age: 4 }
8.     ]
9. };

when sent as the body of a POST request (or other requests with a body) is encoded as follows (line breaks added for clarity)

name=John&age=33&luckyNumbers[]=3&luckyNumbers[]=7&
children[0][name]=Jack&children[0][age]=6&
children[1][name]=Jane&children[1][age]=4

The FormUrlEncodedMediaTypeFormatter, however, can only read form-urlencoded data; it cannot produce them, because the main scenario for which this formatter was created was to consume such data. There was one request for writing data on this format (for an API which needs to invoke an existing service which doesn’t support JSON input, and supports only forms data instead), so this post will show a simple media type formatter which can both read and write forms encoded data.

A small warning before going on: I haven’t been able to find any formal specification for the format used by jQuery when encoding complex types, so this is what I was able to find out from passing many different object types and observing what their wire representation was. In other words, it works on my machine. If you know of any formal specification of that format, please let me know.

The existing form encoded formatter already does all the reading part, so we can inherit from the type and only override the writing methods. Below is the virtual methods of the class, nothing new here – we only support JsonObject (which provides a good abstraction for JavaScript complex types), but supporting other types (such as JSON.NET’s JObject) should be trivial. When writing the type, start a new factory which first flattens the object in a list of key/value pairs, then writes them to the stream separated by ‘&’.

1. class ReadWriteFormUrlEncodedMediaTypeFormatter : FormUrlEncodedMediaTypeFormatter
2. {
3.     private readonly static Encoding encoding = new UTF8Encoding(false);
4.  
5.     protected override bool CanWriteType(Type type)
6.     {
7.         return base.CanWriteType(type) || type == typeof(JsonObject);
8.     }
9.  
10.     protected override Task OnWriteToStreamAsync(Type type, object value, Stream stream, HttpContentHeaders contentHeaders, FormatterContext formatterContext, TransportContext transportContext)
11.     {
12.         if (type == typeof(JsonObject))
13.         {
14.             return Task.Factory.StartNew(() =>
15.             {
16.                 List<string> pairs = new List<string>();
17.                 Flatten(pairs, value as JsonObject);
18.                 byte[] bytes = encoding.GetBytes(string.Join("&", pairs));
19.                 stream.Write(bytes, 0, bytes.Length);
20.             });
21.         }
22.         else
23.         {
24.             return base.OnWriteToStreamAsync(type, value, stream, contentHeaders, formatterContext, transportContext);
25.         }
26.     }
27. }

Flattening the object means that for each of the object’s keys, we’ll push them to a stack and start flattening their values. At the end of the operation, the stack should be empty (otherwise some error happened), so we check that for debugging sake.

1. private void Flatten(List<string> pairs, JsonObject input)
2. {
3.     List<object> stack = new List<object>();
4.     foreach (var key in input.Keys)
5.     {
6.         stack.Add(key);
7.         Flatten(pairs, input[key], stack);
8.         stack.RemoveAt(stack.Count - 1);
9.         if (stack.Count != 0)
10.         {
11.             throw new InvalidOperationException("Something went wrong");
12.         }
13.     }
14. }

The main logic of this formatter happens in the recursive Flatten method. Here we check all the possible types of objects which can be written out. jQuery doesn’t write out any null values, so we’re doing the same here. For arrays and objects, the method pushes the key into the stack and calls itself recursively for the value. Finally, for “primitive” values (numbers, strings, Boolean), we create the key by traversing the stack, separating them with square brackets. One thing which I noticed is that for arrays, the last element doesn’t have the index in the serialized format, as shown in the “luckyNumbers” member in the example above, so this code also accounts for that.

1. private static void Flatten(List<string> pairs, JsonValue input, List<object> indices)
2. {
3.     if (input == null)
4.     {
5.         return; // null values aren't serialized
6.     }
7.  
8.     switch (input.JsonType)
9.     {
10.         case JsonType.Array:
11.             for (int i = 0; i < input.Count; i++)
12.             {
13.                 indices.Add(i);
14.                 Flatten(pairs, input[i], indices);
15.                 indices.RemoveAt(indices.Count - 1);
16.             }
17.  
18.             break;
19.         case JsonType.Object:
20.             foreach (var kvp in input)
21.             {
22.                 indices.Add(kvp.Key);
23.                 Flatten(pairs, kvp.Value, indices);
24.                 indices.RemoveAt(indices.Count - 1);
25.             }
26.  
27.             break;
28.         default:
29.             string value = input.ReadAs<string>();
30.             StringBuilder name = new StringBuilder();
31.             for (int i = 0; i < indices.Count; i++)
32.             {
33.                 var index = indices[i];
34.                 if (i > 0)
35.                 {
36.                     name.Append('[');
37.                 }
38.  
39.                 if (i < indices.Count - 1 || index is string)
40.                 {
41.                     // last array index not shown
42.                     name.Append(index);
43.                 }
44.  
45.                 if (i > 0)
46.                 {
47.                     name.Append(']');
48.                 }
49.             }
50.  
51.             pairs.Add(string.Format("{0}={1}", Uri.EscapeDataString(name.ToString()), Uri.EscapeDataString(value)));
52.             break;
53.     }
54. }

That’s it. In the project from code gallery I’ll include a test project which shows some examples of the conversions this formatter can do.

[Code in this post]