Complex types and Azure Mobile Services

About a year ago I posted a series of entries in this blog about supporting arbitrary types in Azure Mobile Services. Back then, the managed client SDK was using a custom serializer which only supported a very limited subset of simple types. To be use other types in the CRUD operations, one would need to decorate the types / properties with a special attribute, and implement an interface which was used to convert between those types to / from a JSON representation. That was cumbersome for two reasons – the first was that even for simple types, one would need to define a converter class; the other was that the JSON representation was different for the supported platforms (JSON.NET for Windows Phone; Windows.Data.Json classes for Windows Store) – and the OM for the WinStore JSON representation was, frankly, quite poor.

With the changes made on the client SDK prior to the general release, the SDK for all managed platforms started using a unified serializer – JSON.NET for all platforms (in the context of this post, I mean all platforms using managed code). It also started taking more advantage of the extensibility features of that serializer, so that whatever JSON.NET could do on its own the mobile services SDK itself wouldn’t need to do anything else. That by itself gave the mobile services SDK the ability to serialize, in all supported platforms all primitive types which JSON.NET supported natively, so there was no need to implement custom serialization for things such as TimeSpan, enumerations, Uri, among others which weren’t supported in the initial version of the SDK.

What that means is that, for simple types, all the code which I wrote on the first post with the JSON converter isn’t required anymore. You can still change how a simple type is serialized, though, if you really want to, by using a JsonConverter (and decorating the member with the JsonConverterAttribute). For complex types, however, there’s still some work which needs to be done – the serializer on the client will happily serialize the object with its complex members, but the runtime won’t know what to do with those until we tell it what it needs to do.

Let’s look at an example – my app adds and display movies, and each movie has some reviews associated with it.

1. public class Movie
2. {
3.     [JsonProperty("id")]
4.     public int Id { get; set; }
5.     [JsonProperty("title")]
6.     public string Title { get; set; }
7.     [JsonProperty("year")]
8.     public int ReleaseYear { get; set; }
9.     [JsonProperty("reviews")]
10.     public MovieReview[] Reviews { get; set; }
11. }
12.  
13. public class MovieReview
14. {
15.     [JsonProperty("stars")]
16.     public int Stars { get; set; }
17.     [JsonProperty("comment")]
18.     public string Comment { get; set; }
19. }

Now let’s try to insert one movie into the server:

1. try
2. {
3.     var movieToInsert = new Movie
4.     {
5.         Title = "Pulp Fiction",
6.         ReleaseYear = 1994,
7.         Reviews = new MovieReview[] { new MovieReview { Stars = 5, Comment = "Best Movie Ever!" } }
8.     };
9.     var table = MobileService.GetTable<Movie>();
10.     await table.InsertAsync(movieToInsert);
11.     this.AddToDebug("Inserted movie {0} with id = {1}", movieToInsert.Title, movieToInsert.Id);
12. }
13. catch (Exception ex)
14. {
15.     this.AddToDebug("Error: {0}", ex);
16. }

The movie object is serialized without problems to the server, as can be seen in Fiddler (many headers omitted):

POST https://MY-SERVICE-NAME.azure-mobile.net/tables/Movie HTTP/1.1
Content-Type: application/json; charset=utf-8
Host: MY-SERVICE-NAME.azure-mobile.net
Content-Length: 89
Connection: Keep-Alive

{"title":"Pulp Fiction","year":1994,"reviews":[{"stars":5,"comment":"Best Movie Ever!"}]}

The complex type was properly serialized as expected. But the server responds saying that it was a bad request:

HTTP/1.1 400 Bad Request
Cache-Control: no-cache
Content-Length: 112
Content-Type: application/json
Server: Microsoft-IIS/8.0
Date: Thu, 22 Aug 2013 22:10:11 GMT

{"code":400,"error":"Error: The value of property 'reviews' is of type 'object' which is not a supported type."}

Since the runtime doesn’t know which column type to insert non-primitive types. So, as I mentioned on the original posts, there are two ways to solve this issue – make the data, on the client side, of a type which the runtime understands, or “teach” the runtime to understand non-primitive types, by defining scripts for the table operations. Let’s look at both alternatives.

Client-side data manipulation

For the client side, in the original post we converted the complex types into simple types by using a data member JSON converter. That interface doesn’t exist anymore, so we just use the converters from JSON.NET to do that. Below would be one possible implementation of such converter, which “flattens” the reviews array into a single string.

1. public class Movie_ComplexClientSide
2. {
3.     [JsonProperty("id")]
4.     public int Id { get; set; }
5.     [JsonProperty("title")]
6.     public string Title { get; set; }
7.     [JsonProperty("year")]
8.     public int ReleaseYear { get; set; }
9.     [JsonProperty("reviews")]
10.     [JsonConverter(typeof(ReviewArrayConverter))]
11.     public MovieReview[] Reviews { get; set; }
12. }
13.  
14. class ReviewArrayConverter : JsonConverter
15. {
16.     public override bool CanConvert(Type objectType)
17.     {
18.         return objectType == typeof(MovieReview[]);
19.     }
20.  
21.     public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
22.     {
23.         var reviewsAsString = serializer.Deserialize<string>(reader);
24.         return reviewsAsString == null ?
25.             null :
26.             JsonConvert.DeserializeObject<MovieReview[]>(reviewsAsString);
27.     }
28.  
29.     public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
30.     {
31.         var reviewsAsString = JsonConvert.SerializeObject(value);
32.         serializer.Serialize(writer, reviewsAsString);
33.     }
34. }

This approach has the advantage which it is fairly simple – the converter implementation, as seen above, is trivial, and there’s no need to change server scripts. However, this has the drawback that we’re essentially denormalizing the relationship between the movie and its comments. In this case, this actually shouldn’t be a big deal (since a review is inherently tied to a movie), but in other scenarios the loss of normalization may lead to other issue. For example, we can’t (easily) query the database for which movie has the most reviews, or which movie has more 5-star reviews. Also, if we want to use the same data in different platforms (such as JavaScript, Android, iOS, etc.) we’ll need to do this manipulation on those platforms as well.

Server-side data manipulation

Another alternative is to not do anything on the client, and deal with the complex data on the server-side itself. At the server-side, we have two more options – denormalize the data (using a similar technique as we did at the client side), or keep it normalized (in two different tables). The scripts at the second post of the original series can still be used for the denormalization technique, so I won’t repeat them here.

To keep the table normalized (i.e., to implement a 1:n relationship between the Movie and the new MovieReview tables) we need to add some scripts at the server side to deal with that data. The last post on the original series talked about that, but with a different scenario. For completeness sake, I’ll add the scripts for this scenario (movies / reviews) here as well.

First, inserting data. When the data arrives at the server, we first remove the complex type (which the runtime doesn’t know how to handle), and after inserting the movie, we iterate through the reviews to insert them with the associated movie id as the “foreign key”.

1. function insert(item, user, request) {
2.     var reviews = item.reviews;
3.     if (reviews) {
4.         delete item.reviews; // will add in the related table later
5.     }
6.  
7.     request.execute({
8.         success: function () {
9.             var movieId = item.id;
10.             var reviewsTable = tables.getTable('MovieReview');
11.             if (reviews) {
12.                 item.reviews = [];
13.                 var insertNextReview = function (index) {
14.                     if (index >= reviews.length) {
15.                         // done inserting reviews, respond to client
16.                         request.respond();
17.                     } else {
18.                         var review = reviews[index];
19.                         review.movieId = movieId;
20.                         reviewsTable.insert(review, {
21.                             success: function () {
22.                                 item.reviews.push(review);
23.                                 insertNextReview(index + 1);
24.                             }
25.                         });
26.                     }
27.                 };
28.  
29.                 insertNextReview(0);
30.             } else {
31.                 // no need to do anythin else
32.                 request.respond();
33.             }
34.         }
35.     });
36. }

Reading is similar – first read the movies themselves, then iterate through them and read their reviews from the associated table.

1. function read(query, user, request) {
2.     request.execute({
3.         success: function (movies) {
4.             var reviewsTable = tables.getTable('MovieReview');
5.             var readReviewsForMovie = function (movieIndex) {
6.                 if (movieIndex >= movies.length) {
7.                     request.respond();
8.                 } else {
9.                     reviewsTable.where({ movieId: movies[movieIndex].id }).read({
10.                         success: function (reviews) {
11.                             movies[movieIndex].reviews = reviews;
12.                             readReviewsForMovie(movieIndex + 1);
13.                         }
14.                     });
15.                 }
16.             };
17.  
18.             readReviewsForMovie(0);
19.         }
20.     });
21. }

That’s it. I wanted to have an updated post so I could add a warning in the original ones that some of their content was out-of-date. The code for this project can be found in GitHub at https://github.com/carlosfigueira/blogsamples.

Comments

• Anonymous
  October 04, 2013
  Wow, you probably posted this a while back and I'm late to the game but great tutorial man. Just implemented something similar a second ago based off of your code.Nice, thanks.

• Anonymous
  October 20, 2013
  After reading your post, I'm not second guessing my decision to go with Azure Mobile Services. I had a belief that AMS was supposed to make my life easier, but having to write custom serialization to handle basic  collection/array properties on my objects is ridiculous. This seems to be useful for only the most BASIC apps out there.

• Anonymous
  October 27, 2013
  The code quickly becomes messy if you need to fetch data from multiple tables. To simplify, use a chaining pattern.For instance, say you want to fetch a list of movie posters for each movie. I like to do it like this (see: stackoverflow.com/.../139388 ):function read(query, user, request) {   request.execute({       success: function (movies) {           var reviewsTable = tables.getTable('MovieReview');           var readExtraDataForMovie = function (movieIndex) {               if (movieIndex >= movies.length) {                   request.respond();               } else {                   var movie = campaigns[index];                   var funcs = [                       function() { readReviewsForMovie(movie, funcs.shift()); },                       function() { readPostersForMovie(movie, funcs.shift()); },                       function() { readExtraDataForMovie(index + 1); }                   ];                   funcs.shift()();               }           };           readExtraDataForMovie(0);       }   });}function readReviewsForMovie(movie, nextFunc) {   var reviewsTable = tables.getTable('MovieReview');   reviewsTable.where({ movieId: movie.id }).read({       success: function (reviews) {           movie.reviews = reviews;           nextFunc();       }   });}function readPostersForMovie(movie, nextFunc) {   var postersTable = tables.getTable('MoviePosters');   postersTable.where({ movieId: movie.id }).read({       success: function (posters) {           movie.posters = posters;           nextFunc();       }   });}

• Anonymous
  December 26, 2013
  @BeginningWithAzureMobileServices if you didn't use AMS and went with your own or some other methodservice of saving up a collection of objects, you'll still need to implement some form customserialization.  

• Anonymous
  June 19, 2014
  This was so easy to setup -- another good post!

• Anonymous
  August 05, 2014
  I'm curious what you recommend for update and delete

• Anonymous
  August 06, 2014
  @Chris that depends on how you implement your relationship.If you flatten the object (storing everything in a single table):   - Deleting is trivial, as the "children" will just be deleted once the "parent" is deleted   - Updating will depend on what you mean by updating the parent.       - Do the children coming from the client override the ones from the server? If so, the update is also trivial.       - Do you want to somehow "merge" the previous with the new children? You'll need to first read the existing data, apply your merge policy, then perform the actual update.If you store the parent and children in different tables:   - When deleting the parent, first delete the associated children (if delete cascade is your policy) or verify whether there are any associated children and return an error if any (if you can only delete items without relations). The script at blogs.msdn.com/.../supporting-complex-types-in-azure-mobile-services-clients-implementing-1-n-table-relationships.aspx shows the first option   - When updating the parent, you'll also need to determine what it means with respect to the children.       - Children from the parent override the previous ones: first delete the existing children, then add the ones from the client. The same post mentioned above shows how this can be implemented       - Merge: like in the previous case, first read the existing children, then merge them with the ones coming from the client, remove the ones which didn't "survive" the match and add any new ones.

• Anonymous
  November 13, 2014
  Carlos,First of all, great blog! I've learned a lot from your posts regarding Azure Mobile Services.  Then again, I'm stuck with a problem regarding nested lists and I hope you know the answer to this one:Suppose you have a snooker app that contains an object Match which contains a list of Frame objects.  Each Frame object contains a list of Shot objects. So we have nested lists in our object model.As I see it, I should just serialize the Match to JSON, writing a FrameListConverter to serialize the Frames correctly. Then in my Match table insert script, I just add each frame to the Frames table as you did in your example. This works so no problem so far.I thought to do the same for the Frame table insert script, adding the Shots to the Shots table, but curiously enough, when you call framesTable.insert() from within the Match table insert script, the insert script from the frames Table is not triggered and I get the error{"code":400,"error":"Error: The value of property 'shots' is of type 'object' which is not a supported type."}I then tried to add the shots in the match script also, adding the shots for each Frame in the frames loop, using a "var insertNextShot = function()", but that didn't work either. I didn't like this option in the first place as it is not the Match's responsibility to add the shots, but the Frame's and moreover, if I want to save a Frame by itself, the shots should also be saved, so I'll have to implement this part anyway in the Frames table insert script.The real problem seems to be that the "insert" function of the Frames table is not triggered when called from the Match script. Is this normal and if so, how would you solve this? Maybe the error in the Match insert script is firing immediately when you try to insert a Frame that contains a property Shots of type object...Your answer would really help me out because I'm really stuck at the moment and I don't want to leave the "serverside path" to save my matches correctly because of this.Thanks already!

• Anonymous
  November 14, 2014
  Koen,This is currently not supported (table scripts being invoked by calls of tables.getTable(<name>).[insert/read/update/delete] - there's a suggested feature at feedback.azure.com/.../4180025-support-crud-table-scripts-via-the-js-table-object which you can vote up. One possible workaround (not ideal, I know) is to make the code that "flattens" the object into a shared module, and use that module in the scripts for both tables.Hope this helps.

• Anonymous
  November 16, 2014
  Thanks a lot for your response.  I already voted up the idea.  Your solution is exactly what I was going to try out and now I'm sure this is not a stupid idea :-)  Like you said, it's not ideal, but I can't think of a better way to fix this for now.Again, many thanks and keep up the good work!

• Anonymous
  January 27, 2015
  Great post! Saved me hours if not days, and made the transition from Parse backend to Azure much easier.When doing the update, can you avoid deleting and re-adding all the children and just update them if you have all the child ids? I've posted a full question on SO: stackoverflow.com/.../azure-mobile-server-update-script-w-complex-field-type