Browse Tag: web api

Angular 2 server side paging using ng2-pagination

Angular 2 is not quite out of beta yet (Beta 12 at the time of writing) but I’m in the full flow of developing with it for production use. A common feature, for good or bad, is to have lists/tables of data that the user can navigate through page by page, or even filter, to help find something useful.

Angular 2 doesn’t come with any out of the box functionality to support this, so we have to implement it ourselves. And of course what the means today is to use a third party package!

To make this happen, we will utilise n2-pagination, a great plugin, and Web API.

I’ve chosen Web API because that is what I’m using in my production app, but you could easily use ExpressJS or (insert your favourite RESTful framework here).

Checklist

Here is a checklist of what we will do to make this work;

  • Create a new Web API project (you could very easily use an existing project)
  • Enable CORS, as we will use using a seperate development server for the Angular 2 project
  • Download the Angular 2 quick start, ng2-pagination and connect the dots
  • Expose some sample data for testing

I will try to stick with this order.

Web API (for the back end)

Open up Visual Studio (free version here) and create a new Web API project. I prefer to create an Empty project and add Web API.

Add a new controller, called DataController and add the following code;

public class DataModel
{
    public int Id { get; set; }
    public string Text { get; set; }
}

[RoutePrefix("api/data")]
public class DataController : ApiController
{
    private readonly List<DataModel> _data;

    public DataController()
    {
        _data = new List<DataModel>();

        for (var i = 0; i < 10000; i++)
        {
            _data.Add(new DataModel {Id = i + 1, Text = "Data Item " + (i + 1)});
        }
    }

    [HttpGet]
    [Route("{pageIndex:int}/{pageSize:int}")]
    public PagedResponse<DataModel> Get(int pageIndex, int pageSize)
    {
        return new PagedResponse<DataModel>(_data, pageIndex, pageSize);
    }
}

We don’t need to connect to a database to make this work, so we just dummy up 10,000 “items” and page through that instead. If you chose to use Entity Framework, the code is exactly the same, except you initialise a DbContext and query a Set instead.

PagedResponse

Add the following code;

public class PagedResponse<T>
{
    public PagedResponse(IEnumerable<T> data, int pageIndex, int pageSize)
    {
        Data = data.Skip((pageIndex - 1)*pageSize).Take(pageSize).ToList();
        Total = data.Count();
    }

    public int Total { get; set; }
    public ICollection<T> Data { get; set; }
}

PagedResponse exposes two properties. Total and Data. Total is the total number of records in the set. Data is the subset of data itself. We have to include the total number of items in the set so that ng2-pagination knows how many pages there are in total. It will then generate some links/buttons to enable the user to skip forward several pages at once (or as many as required).

Enable CORS (Cross Origin Resource Sharing)

To enable communication between our client and server, we need to enable Cross Origin Resource Sharing (CORS) as they will be (at least during development) running under different servers.

To enable CORS, first install the following package (using NuGet);

Microsoft.AspNet.WebApi.Cors

Now open up WebApiConfig.cs and add the following to the Register method;

var cors = new EnableCorsAttribute("*", "*", "*");
config.EnableCors(cors);
config.MessageHandlers.Add(new PreflightRequestsHandler());

And add a new nested class, as shown;

public class PreflightRequestsHandler : DelegatingHandler
{
    protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        if (request.Headers.Contains("Origin") && request.Method.Method == "OPTIONS")
        {
            var response = new HttpResponseMessage {StatusCode = HttpStatusCode.OK};
            response.Headers.Add("Access-Control-Allow-Origin", "*");
            response.Headers.Add("Access-Control-Allow-Headers", "Origin, Content-Type, Accept, Authorization");
            response.Headers.Add("Access-Control-Allow-Methods", "*");
            var tsc = new TaskCompletionSource<HttpResponseMessage>();
            tsc.SetResult(response);
            return tsc.Task;
        }
        return base.SendAsync(request, cancellationToken);
    }
}

Now when Angular makes a request for data, it will send an OPTIONS header first to check access. This request will be intercepted above and will reply with Access-Control-Allow-Origin header with value any (represented with an asterisk).

Format JSON response

If, like me, you hate Pascal Case JavaScript (ThisIsPascalCase), you will want to add the following code to your Application_Start method;

var formatters = GlobalConfiguration.Configuration.Formatters;
var jsonFormatter = formatters.JsonFormatter;
var settings = jsonFormatter.SerializerSettings;
settings.Formatting = Formatting.Indented;
settings.ContractResolver = new CamelCasePropertyNamesContractResolver();

Now lets set up the front end.

Front-end Angular 2 and ng2-pagination

If you head over the to Angular 2 quickstart, you will see there is a link to download the quick start source code. Go ahead and do that.

I’ll wait here.

Ok you’re done? Lets continue.

Install ng2-pagination and optionally bootstrap and jquery if you want this to look pretty. Skip those two if you don’t mind.

npm install --save-dev ng2-pagination bootstrap jquery

Open up index.html and add the following scripts to the header;

<script src="node_modules/angular2/bundles/http.dev.js"></script>
<script src="node_modules/ng2-pagination/dist/ng2-pagination-bundle.js"></script>

<script src="node_modules/jquery/dist/jquery.js"></script>
<script src="node_modules/bootstrap/dist/js/bootstrap.js"></script>

Also add a link to the bootstrap CSS file, if required.

<link rel="stylesheet" href="node_modules/bootstrap/dist/css/bootstrap.css">

Notice we pulled in Http? We will use that for querying our back-end.

Add a new file to the app folder, called app.component.html. We will use this instead of having all of our markup and TypeScript code in the same file.

ng2-pagination

Open app.component.ts, delete everything, and add the following code instead;

import {Component, OnInit} from 'angular2/core';
import {Http, HTTP_PROVIDERS} from 'angular2/http';
import {Observable} from 'rxjs/Rx';
import 'rxjs/add/operator/map';
import 'rxjs/add/operator/do';
import {PaginatePipe, PaginationService, PaginationControlsCmp, IPaginationInstance} from 'ng2-pagination';

export interface PagedResponse<T> {
    total: number;
    data: T[];
}

export interface DataModel {
    id: number;
    data: string;
}

@Component({
    selector: 'my-app',
    templateUrl: './app/app.component.html',
    providers: [HTTP_PROVIDERS, PaginationService],
    directives: [PaginationControlsCmp],
    pipes: [PaginatePipe]
})
export class AppComponent implements OnInit {
    private _data: Observable<DataModel[]>;
    private _page: number = 1;
    private _total: number;

    constructor(private _http: Http) {

    }
}

A quick walk-through of what I’ve changed;

  • Removed inline HTML and linked to the app.component.html file you created earlier. (This leads to cleaner seperation of concerns).
  • Imported Observable, Map, and Do from RX.js. This will enable us to write cleaner async code without having to rely on promises.
  • Imported a couple of class from angular2/http so that we can use the native Http client, add added HTTP_PROVIDERS as a provider.
  • Imported various objects required by ng2-pagination, and added to providers, directives and pipes so we can access them through our view (which we will create later).
  • Defined two interfaces, one called PagedResponse<T> and DataModel. You may notice these are identical to those we created in our Web API project.
  • Add some variables, we will discuss shortly.

We’ve got the basics in place that we need to call our data service and pass the data over to ng2-pagination. Now lets actually implement that process.

Retrieving data using Angular 2 Http

Eagle eyed readers may have noticed that I’ve pulled in and implemented the OnInit method, but not implemented the ngOnInit method yet.

Add the following method;

ngOnInit() {
    this.getPage(1);
}

When the page loads and is initialised, we want to automatically grab the first page of data. The above method will make that happen.

Note: If you are unfamiliar with ngOnInit, please read this helpful documentation on lifecycle hooks.

Now add the following code;

getPage(page: number) {
this._data = this._http.get("http://localhost:52472/api/data/" + page + "/10")
    .do((res: any) => {
        this._total = res.json().total;
        this._page = page;
    })
    .map((res: any) => res.json().data);
}

The above method does the following;

  • Calls out to our Web API (you may need to change the port number depending on your set up)
  • Passes in two values, the first being the current page number, the second being the number of results to retrieve
  • Stores a reference to the _data variable. Once the request is complete, do is executed.
  • Do is a function (an arrow function in this case) that is executed for each item in the collection received from the server. We’ve set up our Web API method to return a single object, of type PagedResponse, so this method will only be executed once. We take this opportunity to update the current page (which is the same as the page number passed into the method in the first place) and the _total variable, which stores the total number of items in the entire set (not just the paged number).
  • Map is then used to pull the data from the response and convert it to JSON. The way that RX.js works is that an event will be emitted to notify that the collection has changed.

Implement the view

Open app.component.html and add the following code;

<div class="container">
    <table class="table table-striped table-hover">
        <thead>
            <tr>
                <th>Id</th>
                <th>Text</th>
            </tr>
        </thead>
        <tbody>
            <tr *ngFor="#item of _data | async | paginate: { id: 'server', itemsPerPage: 10, currentPage: _page, totalItems: _total }">
                <td>{{item.id}}</td>
                <td>{{item.text}}</td>
            </tr>
        </tbody>
    </table>    
    <pagination-controls (pageChange)="getPage($event)" id="server"></pagination-controls>
</div>

There are a few key points on interest here;

  • On our repeater (*ngFor), we’ve used the async pipe. Under the hood, Angular subscribes to the Observable we pass to it and resolves the value automatically (asynchronously) when it becomes available.
  • We use the paginate pipe, and pass in an object containing the current page and total number of pages so ng2-pagination can render itself properly.
  • Add the pagination-controls directive, which calls back to our getPage function when the user clicks a page number that they are not currently on.

As we know the current page, and the number of items per page, we can efficiently pass this to the Web API to only retrieve data specific data.

So, why bother?

Some benefits;

  • Potentially reduce initial page load time, because less data has to be retrieved from the database, serialized and transferred over.
  • Reduced memory usage on the client. All 10,000 records would have to be held in memory!
  • Reduced processing time, as only the paged data is stored in memory, there are a lot less records to iterate through!

Drawbacks;

  • Lots of small requests for data could reduce server performance (due to chat. Using an effective caching strategy is key here.
  • User experience could be degegrated. If the server is slow to respond, the client may appear to be slow and could frustrate the user.

Summary

Using ng2-pagination, and with help from RX.js, we can easily add pagination to our pages. Doing so has the potential to reduce server load and initial page render time, and thus can result in a better user experience. A good caching strategy and server response times are important considerations when going to production.

Create a RESTful API with authentication using Web API and Jwt

Web API is a feature of the ASP .NET framework that dramatically simplifies building RESTful (REST like) HTTP services that are cross platform and device and browser agnostic. With Web API, you can create endpoints that can be accessed using a combination of descriptive URLs and HTTP verbs. Those endpoints can serve data back to the caller as either JSON or XML that is standards compliant. With JSON Web Tokens (Jwt), which are typically stateless, you can add an authentication and authorization layer enabling you to restrict access to some or all of your API.

The purpose of this tutorial is to develop the beginnings of a Book Store API, using Microsoft Web API with (C#), which authenticates and authorizes each requests, exposes OAuth2 endpoints, and returns data about books and reviews for consumption by the caller. The caller in this case will be Postman, a useful utility for querying API’s.

In a follow up to this post we will write a front end to interact with the API directly.

Set up

Open Visual Studio (I will be using Visual Studio 2015 Community edition, you can use whatever version you like) and create a new Empty project, ensuring you select the Web API option;

Where you save the project is up to you, but I will create my projects under *C:\Source*. For simplicity you might want to do the same.

New Project

Next, packages.

Packages

Open up the packages.config file. Some packages should have already been added to enable Web API itself. Please add the the following additional packages;

install-package EntityFramework
install-package Microsoft.AspNet.Cors
install-package Microsoft.AspNet.Identity.Core
install-package Microsoft.AspNet.Identity.EntityFramework
install-package Microsoft.AspNet.Identity.Owin
install-package Microsoft.AspNet.WebApi.Cors
install-package Microsoft.AspNet.WebApi.Owin
install-package Microsoft.Owin.Cors
install-package Microsoft.Owin.Security.Jwt
install-package Microsoft.Owin.Host.SystemWeb
install-package System.IdentityModel.Tokens.Jwt
install-package Thinktecture.IdentityModel.Core

These are the minimum packages required to provide data persistence, enable CORS (Cross-Origin Resource Sharing), and enable generating and authenticating/authorizing Jwt’s.

Entity Framework

We will use Entity Framework for data persistence, using the Code-First approach. Entity Framework will take care of generating a database, adding tables, stored procedures and so on. As an added benefit, Entity Framework will also upgrade the schema automatically as we make changes. Entity Framework is perfect for rapid prototyping, which is what we are in essence doing here.

Create a new IdentityDbContext called BooksContext, which will give us Users, Roles and Claims in our database. I like to add this under a folder called Core, for organization. We will add our entities to this later.

namespace BooksAPI.Core
{
    using Microsoft.AspNet.Identity.EntityFramework;

    public class BooksContext : IdentityDbContext
    {

    }
}

Claims are used to describe useful information that the user has associated with them. We will use claims to tell the client which roles the user has. The benefit of roles is that we can prevent access to certain methods/controllers to a specific group of users, and permit access to others.

Add a DbMigrationsConfiguration class and allow automatic migrations, but prevent automatic data loss;

namespace BooksAPI.Core
{
    using System.Data.Entity.Migrations;

    public class Configuration : DbMigrationsConfiguration&lt;BooksContext&gt;
    {
        public Configuration()
        {
            AutomaticMigrationsEnabled = true;
            AutomaticMigrationDataLossAllowed = false;
        }
    }
}

Whilst losing data at this stage is not important (we will use a seed method later to populate our database), I like to turn this off now so I do not forget later.

Now tell Entity Framework how to update the database schema using an initializer, as follows;

namespace BooksAPI.Core
{
    using System.Data.Entity;

    public class Initializer : MigrateDatabaseToLatestVersion&lt;BooksContext, Configuration&gt;
    {
    }
}

This tells Entity Framework to go ahead and upgrade the database to the latest version automatically for us.

Finally, tell your application about the initializer by updating the Global.asax.cs file as follows;

namespace BooksAPI
{
    using System.Data.Entity;
    using System.Web;
    using System.Web.Http;
    using Core;

    public class WebApiApplication : HttpApplication
    {
        protected void Application_Start()
        {
            GlobalConfiguration.Configure(WebApiConfig.Register);
            Database.SetInitializer(new Initializer());
        }
    }
}

Data Provider

By default, Entity Framework will configure itself to use LocalDB. If this is not desirable, say you want to use SQL Express instead, you need to make the following adjustments;

Open the Web.config file and delete the following code;

<entityFramework>
    <defaultConnectionFactory type="System.Data.Entity.Infrastructure.LocalDbConnectionFactory, EntityFramework">
        <parameters>
            <parameter value="mssqllocaldb" />
        </parameters>
    </defaultConnectionFactory>
    <providers>
        <provider invariantName="System.Data.SqlClient" type="System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer" />
    </providers>
</entityFramework>

And add the connection string;

<connectionStrings>
    <add name="BooksContext" providerName="System.Data.SqlClient" connectionString="Server=.;Database=Books;Trusted_Connection=True;" />
</connectionStrings>

Now we’re using SQL Server directly (whatever flavour that might be) rather than LocalDB.

JSON

Whilst we’re here, we might as well configure our application to return camel-case JSON (thisIsCamelCase), instead of the default pascal-case (ThisIsPascalCase).

Add the following code to your Application_Start method;

var formatters = GlobalConfiguration.Configuration.Formatters;
var jsonFormatter = formatters.JsonFormatter;
var settings = jsonFormatter.SerializerSettings;
settings.Formatting = Formatting.Indented;
settings.ContractResolver = new CamelCasePropertyNamesContractResolver();

There is nothing worse than pascal-case JavaScript.

CORS (Cross-Origin Resource Sharing)

Cross-Origin Resource Sharing, or CORS for short, is when a client requests access to a resource (an image, or say, data from an endpoint) from an origin (domain) that is different from the domain where the resource itself originates.

This step is completely optional. We are adding in CORS support here because when we come to write our client app in subsequent posts that follow on from this one, we will likely use a separate HTTP server (for testing and debugging purposes). When released to production, these two apps would use the same host (Internet Information Services (IIS)).

To enable CORS, open WebApiConfig.cs and add the following code to the beginning of the Register method;

var cors = new EnableCorsAttribute("*", "*", "*");
config.EnableCors(cors);
config.MessageHandlers.Add(new PreflightRequestsHandler());

And add the following class (in the same file if you prefer for quick reference);

public class PreflightRequestsHandler : DelegatingHandler
{
    protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        if (request.Headers.Contains("Origin") && request.Method.Method == "OPTIONS")
        {
            var response = new HttpResponseMessage {StatusCode = HttpStatusCode.OK};
            response.Headers.Add("Access-Control-Allow-Origin", "*");
            response.Headers.Add("Access-Control-Allow-Headers", "Origin, Content-Type, Accept, Authorization");
            response.Headers.Add("Access-Control-Allow-Methods", "*");
            var tsc = new TaskCompletionSource<HttpResponseMessage>();
            tsc.SetResult(response);
            return tsc.Task;
        }
        return base.SendAsync(request, cancellationToken);
    }
}

In the CORS workflow, before sending a DELETE, PUT or POST request, the client sends an OPTIONS request to check that the domain from which the request originates is the same as the server. If the request domain and server domain are not the same, then the server must include various access headers that describe which domains have access. To enable access to all domains, we just respond with an origin header (Access-Control-Allow-Origin) with an asterisk to enable access for all.

The Access-Control-Allow-Headers header describes which headers the API can accept/is expecting to receive. The Access-Control-Allow-Methods header describes which HTTP verbs are supported/permitted.

See Mozilla Developer Network (MDN) for a more comprehensive write-up on Cross-Origin Resource Sharing (CORS).

Data Model

With Entity Framework configured, lets create our data structure. The API will expose books, and books will have reviews.

Under the Models folder add a new class called Book. Add the following code;

namespace BooksAPI.Models
{
    using System.Collections.Generic;

    public class Book
    {
        public int Id { get; set; }
        public string Title { get; set; }
        public string Description { get; set; }
        public decimal Price { get; set; }
        public string ImageUrl { get; set; }

        public virtual List<Review> Reviews { get; set; }
    }
}

And add Review, as shown;

namespace BooksAPI.Models
{
    public class Review
    {
        public int Id { get; set; }    
        public string Description { get; set; }    
        public int Rating { get; set; }
        public int BookId { get; set; }
    }
}

Add these entities to the IdentityDbContext we created earlier;

public class BooksContext : IdentityDbContext
{
    public DbSet<Book> Books { get; set; }
    public DbSet<Review> Reviews { get; set; }
}

Be sure to add in the necessary using directives.

A couple of helpful abstractions

We need to abstract a couple of classes that we need to make use of, in order to keep our code clean and ensure that it works correctly.

Under the Core folder, add the following classes;

public class BookUserManager : UserManager<IdentityUser>
{
    public BookUserManager() : base(new BookUserStore())
    {
    }
}

We will make heavy use of the UserManager<T> in our project, and we don’t want to have to initialise it with a UserStore<T> every time we want to make use of it. Whilst adding this is not strictly necessary, it does go a long way to helping keep the code clean.

Now add another class for the UserStore, as shown;

public class BookUserStore : UserStore&lt;IdentityUser&gt;
{
    public BookUserStore() : base(new BooksContext())
    {
    }
}

This code is really important. If we fail to tell the UserStore which DbContext to use, it falls back to some default value.

A network-related or instance-specific error occurred while establishing a connection to SQL Server

I’m not sure what the default value is, all I know is it doesn’t seem to correspond to our applications DbContext. This code will help prevent you from tearing your hair out later wondering why you are getting the super-helpful error message shown above.

API Controller

We need to expose some data to our client (when we write it). Lets take advantage of Entity Frameworks Seed method. The Seed method will pre-populate some books and reviews automatically for us.

Instead of dropping the code in directly for this class (it is very long), please refer to the Configuration.cs file on GitHub.

This code gives us a little bit of starting data to play with, instead of having to add a bunch of data manually each time we make changes to our schema that require the database to be re-initialized (not really in our case as we have an extremely simple data model, but in larger applications this is very useful).

Books Endpoint

Next, we want to create the RESTful endpoint that will retrieve all the books data. Create a new Web API controller called BooksController and add the following;

public class BooksController : ApiController
{
    [HttpGet]
    public async Task<IHttpActionResult> Get()
    {
        using (var context = new BooksContext())
        {
            return Ok(await context.Books.Include(x => x.Reviews).ToListAsync());
        }
    }
}

With this code we are fully exploiting recent changes to the .NET framework; the introduction of async and await. Writing asynchronous code in this manner allows the thread to be released whilst data (Books and Reviews) is being retrieved from the database and converted to objects to be consumed by our code. When the asynchronous operation is complete, the code picks up where it was up to and continues executing. (By which, we mean the hydrated data objects are passed to the underlying framework and converted to JSON/XML and returned to the client).

Reviews Endpoint

We’re also going to enable authorized users to post reviews and delete reviews. For this we will need a ReviewsController with the relevant Post and Delete methods. Add the following code;

Create a new Web API controller called ReviewsController and add the following code;

public class ReviewsController : ApiController
{
    [HttpPost]
    public async Task<IHttpActionResult> Post([FromBody] ReviewViewModel review)
    {
        using (var context = new BooksContext())
        {
            var book = await context.Books.FirstOrDefaultAsync(b => b.Id == review.BookId);
            if (book == null)
            {
                return NotFound();
            }

            var newReview = context.Reviews.Add(new Review
            {
                BookId = book.Id,
                Description = review.Description,
                Rating = review.Rating
            });

            await context.SaveChangesAsync();
            return Ok(new ReviewViewModel(newReview));
        }
    }

    [HttpDelete]
    public async Task<IHttpActionResult> Delete(int id)
    {
        using (var context = new BooksContext())
        {
            var review = await context.Reviews.FirstOrDefaultAsync(r => r.Id == id);
            if (review == null)
            {
                return NotFound();
            }

            context.Reviews.Remove(review);
            await context.SaveChangesAsync();
        }
        return Ok();
    }
}

There are a couple of good practices in play here that we need to highlight.

The first method, Post allows the user to add a new review. Notice the parameter for the method;

[FromBody] ReviewViewModel review

The [FromBody] attribute tells Web API to look for the data for the method argument in the body of the HTTP message that we received from the client, and not in the URL. The second parameter is a view model that wraps around the Review entity itself. Add a new folder to your project called ViewModels, add a new class called ReviewViewModel and add the following code;

public class ReviewViewModel
{
    public ReviewViewModel()
    {
    }

    public ReviewViewModel(Review review)
    {
        if (review == null)
        {
            return;
        }

        BookId = review.BookId;
        Rating = review.Rating;
        Description = review.Description;
    }

    public int BookId { get; set; }
    public int Rating { get; set; }
    public string Description { get; set; }

    public Review ToReview()
    {
        return new Review
        {
            BookId = BookId,
            Description = Description,
            Rating = Rating
        };
    }
}

We are just copying all he properties from the Review entity to the ReviewViewModel entity and vice-versa. So why bother? First reason, to help mitigate a well known under/over-posting vulnerability (good write up about it here) inherent in most web services. Also, it helps prevent unwanted information being sent to the client. With this approach we have to explicitly expose data to the client by adding properties to the view model.

For this scenario, this approach is probably a bit overkill, but I highly recommend it keeping your application secure is important, as well as is the need to prevent leaking of potentially sensitive information. A tool I’ve used in the past to simplify this mapping code is AutoMapper. I highly recommend checking out.

Important note: In order to keep our API RESTful, we return the newly created entity (or its view model representation) back to the client for consumption, removing the need to re-fetch the entire data set.

The Delete method is trivial. We accept the Id of the review we want to delete as a parameter, then fetch the entity and finally remove it from the collection. Calling SaveChangesAsync will make the change permanent.

Meaningful response codes

We want to return useful information back to the client as much as possible. Notice that the Post method returns NotFound(), which translates to a 404 HTTP status code, if the corresponding Book for the given review cannot be found. This is useful for client side error handling. Returning Ok() will return 200 (HTTP ‘Ok’ status code), which informs the client that the operation was successful.

Authentication and Authorization Using OAuth and JSON Web Tokens (JWT)

My preferred approach for dealing with authentication and authorization is to use JSON Web Tokens (JWT). We will open up an OAuth endpoint to client credentials and return a token which describes the users claims. For each of the users roles we will add a claim (which could be used to control which views the user has access to on the client side).

We use OWIN to add our OAuth configuration into the pipeline. Add a new class to the project called Startup.cs and add the following code;

using Microsoft.Owin;
using Owin;

[assembly: OwinStartup(typeof (BooksAPI.Startup))]

namespace BooksAPI
{
    public partial class Startup
    {
        public void Configuration(IAppBuilder app)
        {
            ConfigureOAuth(app);
        }
    }
}

Notice that Startup is a partial class. I’ve done that because I want to keep this class as simple as possible, because as the application becomes more complicated and we add more and more middle-ware, this class will grow exponentially. You could use a static helper class here, but the preferred method from the MSDN documentation seems to be leaning towards using partial classes specifically.

Under the App_Start folder add a new class called Startup.OAuth.cs and add the following code;

using System;
using System.Configuration;
using BooksAPI.Core;
using BooksAPI.Identity;
using Microsoft.AspNet.Identity;
using Microsoft.AspNet.Identity.EntityFramework;
using Microsoft.Owin;
using Microsoft.Owin.Security;
using Microsoft.Owin.Security.DataHandler.Encoder;
using Microsoft.Owin.Security.Jwt;
using Microsoft.Owin.Security.OAuth;
using Owin;

namespace BooksAPI
{
    public partial class Startup
    {
        public void ConfigureOAuth(IAppBuilder app)
        {            
        }
    }
}

Note. When I wrote this code originally I encountered a quirk. After spending hours pulling out my hair trying to figure out why something was not working, I eventually discovered that the ordering of the code in this class is very important. If you don’t copy the code in the exact same order, you may encounter unexpected behaviour. Please add the code in the same order as described below.

OAuth secrets

First, add the following code;

var issuer = ConfigurationManager.AppSettings["issuer"];
var secret = TextEncodings.Base64Url.Decode(ConfigurationManager.AppSettings["secret"]);
  • Issuer – a unique identifier for the entity that issued the token (not to be confused with Entity Framework’s entities)
  • Secret – a secret key used to secure the token and prevent tampering

I keep these values in the Web configuration file (Web.config). To be precise, I split these values out into their own configuration file called keys.config and add a reference to that file in the main Web.config. I do this so that I can exclude just the keys from source control by adding a line to my .gitignore file.

To do this, open Web.config and change the <appSettings> section as follows;

<appSettings file="keys.config">
</appSettings>

Now add a new file to your project called keys.config and add the following code;

<appSettings>
  <add key="issuer" value="http://localhost/"/>
  <add key="secret" value="IxrAjDoa2FqElO7IhrSrUJELhUckePEPVpaePlS_Xaw"/>
</appSettings>

Adding objects to the OWIN context

We can make use of OWIN to manage instances of objects for us, on a per request basis. The pattern is comparable to IoC, in that you tell the “container” how to create an instance of a specific type of object, then request the instance using a Get<T> method.

Add the following code;

app.CreatePerOwinContext(() => new BooksContext());
app.CreatePerOwinContext(() => new BookUserManager());

The first time we request an instance of BooksContext for example, the lambda expression will execute and a new BooksContext will be created and returned to us. Subsequent requests will return the same instance.

Important note: The life-cycle of object instance is per-request. As soon as the request is complete, the instance is cleaned up.

Enabling Bearer Authentication/Authorization

To enable bearer authentication, add the following code;

app.UseJwtBearerAuthentication(new JwtBearerAuthenticationOptions
{
    AuthenticationMode = AuthenticationMode.Active,
    AllowedAudiences = new[] { "Any" },
    IssuerSecurityTokenProviders = new IIssuerSecurityTokenProvider[]
    {
        new SymmetricKeyIssuerSecurityTokenProvider(issuer, secret)
    }
});

The key takeaway of this code;

  • State who is the audience (we’re specifying “Any” for the audience, as this is a required field but we’re not fully implementing it).
  • State who is responsible for generating the tokens. Here we’re using SymmetricKeyIssuerSecurityTokenProvider and passing it our secret key to prevent tampering. We could use the X509CertificateSecurityTokenProvider, which uses a X509 certificate to secure the token (but I’ve found these to be overly complex in the past and I prefer a simpler implementation).

This code adds JWT bearer authentication to the OWIN pipeline.

Enabling OAuth

We need to expose an OAuth endpoint so that the client can request a token (by passing a user name and password).

Add the following code;

app.UseOAuthAuthorizationServer(new OAuthAuthorizationServerOptions
{
    AllowInsecureHttp = true,
    TokenEndpointPath = new PathString("/oauth2/token"),
    AccessTokenExpireTimeSpan = TimeSpan.FromMinutes(30),
    Provider = new CustomOAuthProvider(),
    AccessTokenFormat = new CustomJwtFormat(issuer)
});

Some important notes with this code;

  • We’re going to allow insecure HTTP requests whilst we are in development mode. You might want to disable this using a #IF Debug directive so that you don’t allow insecure connections in production.
  • Open an endpoint under /oauth2/token that accepts post requests.
  • When generating a token, make it expire after 30 minutes (1800 seconds).
  • We will use our own provider, CustomOAuthProvider, and formatter, CustomJwtFormat, to take care of authentication and building the actual token itself.

We need to write the provider and formatter next.

Formatting the JWT

Create a new class under the Identity folder called CustomJwtFormat.cs. Add the following code;

namespace BooksAPI.Identity
{
    using System;
    using System.Configuration;
    using System.IdentityModel.Tokens;
    using Microsoft.Owin.Security;
    using Microsoft.Owin.Security.DataHandler.Encoder;
    using Thinktecture.IdentityModel.Tokens;

    public class CustomJwtFormat : ISecureDataFormat<AuthenticationTicket>
    {
        private static readonly byte[] _secret = TextEncodings.Base64Url.Decode(ConfigurationManager.AppSettings["secret"]);
        private readonly string _issuer;

        public CustomJwtFormat(string issuer)
        {
            _issuer = issuer;
        }

        public string Protect(AuthenticationTicket data)
        {
            if (data == null)
            {
                throw new ArgumentNullException(nameof(data));
            }

            var signingKey = new HmacSigningCredentials(_secret);
            var issued = data.Properties.IssuedUtc;
            var expires = data.Properties.ExpiresUtc;

            return new JwtSecurityTokenHandler().WriteToken(new JwtSecurityToken(_issuer, null, data.Identity.Claims, issued.Value.UtcDateTime, expires.Value.UtcDateTime, signingKey));
        }

        public AuthenticationTicket Unprotect(string protectedText)
        {
            throw new NotImplementedException();
        }
    }
}

This is a complicated looking class, but its pretty straightforward. We are just fetching all the information needed to generate the token, including the claims, issued date, expiration date, key and then we’re generating the token and returning it back.

Please note: Some of the code we are writing today was influenced by JSON Web Token in ASP.NET Web API 2 using OWIN by Taiseer Joudeh. I highly recommend checking it out.

The authentication bit

We’re almost there, honest! Now we want to authenticate the user.

using System.Linq;
using System.Security.Claims;
using System.Security.Principal;
using System.Threading;
using System.Threading.Tasks;
using System.Web;
using BooksAPI.Core;
using Microsoft.AspNet.Identity;
using Microsoft.AspNet.Identity.EntityFramework;
using Microsoft.AspNet.Identity.Owin;
using Microsoft.Owin.Security;
using Microsoft.Owin.Security.OAuth;

namespace BooksAPI.Identity
{
    public class CustomOAuthProvider : OAuthAuthorizationServerProvider
    {
        public override Task GrantResourceOwnerCredentials(OAuthGrantResourceOwnerCredentialsContext context)
        {
            context.OwinContext.Response.Headers.Add("Access-Control-Allow-Origin", new[] {"*"});

            var user = context.OwinContext.Get<BooksContext>().Users.FirstOrDefault(u => u.UserName == context.UserName);
            if (!context.OwinContext.Get<BookUserManager>().CheckPassword(user, context.Password))
            {
                context.SetError("invalid_grant", "The user name or password is incorrect");
                context.Rejected();
                return Task.FromResult<object>(null);
            }

            var ticket = new AuthenticationTicket(SetClaimsIdentity(context, user), new AuthenticationProperties());
            context.Validated(ticket);

            return Task.FromResult<object>(null);
        }

        public override Task ValidateClientAuthentication(OAuthValidateClientAuthenticationContext context)
        {
            context.Validated();
            return Task.FromResult<object>(null);
        }

        private static ClaimsIdentity SetClaimsIdentity(OAuthGrantResourceOwnerCredentialsContext context, IdentityUser user)
        {
            var identity = new ClaimsIdentity("JWT");
            identity.AddClaim(new Claim(ClaimTypes.Name, context.UserName));
            identity.AddClaim(new Claim("sub", context.UserName));

            var userRoles = context.OwinContext.Get<BookUserManager>().GetRoles(user.Id);
            foreach (var role in userRoles)
            {
                identity.AddClaim(new Claim(ClaimTypes.Role, role));
            }

            return identity;
        }
    }
}

As we’re not checking the audience, when ValidateClientAuthentication is called we can just validate the request. When the request has a grant_type of password, which all our requests to the OAuth endpoint will have, the above GrantResourceOwnerCredentials method is executed. This method authenticates the user and creates the claims to be added to the JWT.

Testing

There are 2 tools you can use for testing this.

Technique 1 – Using the browser

Open up a web browser, and navigate to the books URL.

Testing with the web browser

You will see the list of books, displayed as XML. This is because Web API can serve up data either as XML or as JSON. Personally, I do not like XML, JSON is my choice these days.

Technique 2 (Preferred) – Using Postman

To make Web API respond in JSON we need to send along a Accept header. The best tool to enable use to do this (for Google Chrome) is Postman. Download it and give it a go if you like.

Drop the same URL into the Enter request URL field, and click Send. Notice the response is in JSON;

Postman response in JSON

This worked because Postman automatically adds the Accept header to each request. You can see this by clicking on the Headers tab. If the header isn’t there and you’re still getting XML back, just add the header as shown in the screenshot and re-send the request.

To test the delete method, change the HTTP verb to Delete and add the ReviewId to the end of the URL. For example; http://localhost:62996/api/reviews/9

Putting it all together

First, we need to restrict access to our endpoints.

Add a new file to the App_Start folder, called FilterConfig.cs and add the following code;

public class FilterConfig
{
    public static void Configure(HttpConfiguration config)
    {
        config.Filters.Add(new AuthorizeAttribute());
    }
}

And call the code from Global.asax.cs as follows;

GlobalConfiguration.Configure(FilterConfig.Configure);

Adding this code will restrict access to all endpoints (except the OAuth endpoint) to requests that have been authenticated (a request that sends along a valid Jwt).

You have much more fine-grain control here, if required. Instead of adding the above code, you could instead add the AuthorizeAttribute to specific controllers or even specific methods. The added benefit here is that you can also restrict access to specific users or specific roles;

Example code;

[Authorize(Roles = "Admin")]

The roles value (“Admin”) can be a comma-separated list. For us, restricting access to all endpoints will suffice.

To test that this code is working correctly, simply make a GET request to the books endpoint;

GET http://localhost:62996/api/books

You should get the following response;

{
  "message": "Authorization has been denied for this request."
}

Great its working. Now let’s fix that problem.

Make a POST request to the OAuth endpoint, and include the following;

  • Headers
    • Accept application/json
    • Accept-Language en-gb
    • Audience Any
  • Body
    • username administrator
    • password administrator123
    • grant_type password

Shown in the below screenshot;

OAuth Request

Make sure you set the message type as x-www-form-urlencoded.

If you are interested, here is the raw message;

POST /oauth2/token HTTP/1.1
Host: localhost:62996
Accept: application/json
Accept-Language: en-gb
Audience: Any
Content-Type: application/x-www-form-urlencoded
Cache-Control: no-cache
Postman-Token: 8bc258b2-a08a-32ea-3cb2-2e7da46ddc09

username=administrator&password=administrator123&grant_type=password

The form data has been URL encoded and placed in the message body.

The web service should authenticate the request, and return a token (Shown in the response section in Postman). You can test that the authentication is working correctly by supplying an invalid username/password. In this case, you should get the following reply;

{
  "error": "invalid_grant"
}

This is deliberately vague to avoid giving any malicious users more information than they need.

Now to get a list of books, we need to call the endpoint passing in the token as a header.

Change the HTTP verb to GET and change the URL to; http://localhost:62996/api/books.

On the Headers tab in Postman, add the following additional headers;

Authorization Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1bmlxdWVfbmFtZSI6ImFkbWluaXN0cmF0b3IiLCJzdWIiOiJhZG1pbmlzdHJhdG9yIiwicm9sZSI6IkFkbWluaXN0cmF0b3IiLCJpc3MiOiJodHRwOi8vand0YXV0aHpzcnYuYXp1cmV3ZWJzaXRlcy5uZXQiLCJhdWQiOiJBbnkiLCJleHAiOjE0NTgwNDI4MjgsIm5iZiI6MTQ1ODA0MTAyOH0.uhrqQW6Ik_us1lvDXWJNKtsyxYlwKkUrCGXs-eQRWZQ

See screenshot below;

Authorization Header

Success! We have data from our secure endpoint.

Summary

In this introduction we looked at creating a project using Web API to issue and authenticate Jwt (JSON Web Tokens). We created a simple endpoint to retrieve a list of books, and also added the ability to get a specific book/review and delete reviews in a RESTful way.

This project is the foundation for subsequent posts that will explore creating a rich client side application, using modern JavaScript frameworks, which will enable authentication and authorization.

ASP .NET 5 (vNext), first thoughts

Microsoft ASP .NET 5 is a major shift from traditional ASP .NET methodologies. Whilst I am not actively developing ASP .NET 5 applications at the minute, .NET has always been my bread and butter technology. When I look at industry trends here in the UK, all I see is .NET .NET .NET, therefore it is important to have one eye on the future. I’ve watched all the introduction videos on the ASP .NET website, but I also wanted to take a look at what ASP .NET 5 means to me.

This is not meant to be a fully formed post. This will come later down the line. Right now, I think ASP .NET 5 is evolving too quickly to be “bloggable” fully.

Version disambiguation and terminology

Lets take a second to disambiguate some terminology. Microsoft’s understanding of versioning has always been different to everybody else. This tweet from Todd Motto really sums it up;

Looks like versioning is not going to get any simpler for the time being

ASP .NET 5 (ASP .NET 4.6 is the current version)

Previously known as ASP .NET vNext, ASP .NET 5 is the successor of ASP .NET 4.6. In the past, versions of ASP .NET have followed the .NET Framework release cycle. It looks like that is coming to an end now. ASP .NET should not be confused with MVC. ASP .NET is a technology, MVC is a framework.

ASP .NET 5 is currently scheduled for release in the first quarter of 2016, as per this tweet from Scott Hansleman; (I suspect this date will slip though)

The ASP .NET team would rather “get it right” and take longer, than rush the product and get it wrong (which would spell long term disaster for the platform)

MVC 6

This is the new version of Microsoft’s Model-View-Controller framework. There is a nice post on StackOverflow that describes the new features of MVC 6. Here are a few of the best;

  • “Cloud optimization” … so better performance.
  • MVC, WebAPI and Web Pages are now unified.
  • Removed dependency on System.Web, which results in more an 10x reduction in request overhead.
  • Built in dependency injection, which is pluggable, so it can be switched out for other DI providers.
  • Roslyn enables dynamic compilation. Save your file, refresh the browser. Works for C# too. No compilation required.
  • Cross platform.

DNX (.NET Execution Environment)

The .NET Execution Environment, DNX, is a cross platform thing that will run your .NET applications. DNX is built around the .NET Core, which is a super lightweight framework for .NET applications, resulting in drastically improved performance thanks to a reduce pipeline. Dependency on the dinosaur assembly System.Web has gone, but in return you are restricted to more of a subset of features. This is a good thing, my friend. System.Web has every featured imagined over the last 13 years, 75% of which you probably don’t even care about.

Interesting new features and changes

  • Use of data annotations for things that would previously have been HTML helpers (Tag helpers)
  • Environment tag on _Layout. Enables a simple means to specify which resources to load depending on the application configuration (Debug mode, release mode etc)
  • Bower support
  • Gulp out of the box (interesting that they chose Gulp over Grunt, I think its Gulps superior speed that has won the day.)
  • .NET Core. Drastically reduced web pipeline, could result in 10x faster response in some cases (remains to be seen!).
  • Noticeably faster starting up.
  • Save to build. With Roslyn, it is now not necessary to build every time you make a change to a CSharp (.cs) code file. Just save and refresh. Compilation is done in memory.
  • Intellisense hints that assembly is not available in .NET Core (nice!)
  • Built in dependency injection, which can be switched out for a third party mechanism.
  • Web API is now no longer a separate component. Web API was originally a separate technology from MVC. The two were always very alike, and it makes sense that the two should be merged together.

Deleted stuff

  • Web.config has finally been removed and exchanged for a simpler JSON formatted file. Parties have been thrown for less.
  • packages.config has gone, seems redundant now that things are in line with how the rest of the web develop, i.e. using package.json

Bad points

  • Still heavy use of the Viewbag in default projects. I’d like to see the ViewBag removed entirely, but I suspect that will never happen.
  • The default project template is still full of “junk”, although it is now a bit simpler to tidy up. Visual Studio automatically managers bower and npm packages, so removing a package is as simple as deleting it from the package.json file.

Summary

I am very keen to get cracking with ASP .NET 5 (vNext), although at the time of writing I feel that it still a little bit too dynamic to start diving in to at a deep level. The introduction of .NET Core, a cross platform, open source subset of the .NET framework is awesome… I can’t wait to see the benefits of using this in the wild (reduced server costs!!! especially when running on a Linux based machine, although it remains to be seen). The ViewBag still exists, but we can’t have it all I suppose.

At this point, we’re at least 5-6 months away from a release, so develop with it at your own risk!

Quick tip: Avoid ‘async void’

When developing a Web API application recently with an AngularJS front end, I made a basic mistake and then lost 2 hours of my life trying to figure out what was causing the problem … async void.

Its pretty common nowadays to use tasks to improve performance/scalability when writing a Web API controller.  Take the following code:

public async Task<Entry[]> Get()
{
    using (var context = new EntriesContext())
    {
        return await context.Entries.ToArrayAsync();
    }
}

At a high level, when ToArrayAsync is executed the call will be moved off onto another thread and the execution of the method will only continue once the operation is complete (when the data is returned from the database in this case).  This is great because it frees up the thread for use by other requests, resulting in better performance/scalability (we could argue about how true this is all day long, so lets not do this here! Smile).

So what about when you still want to harness this functionality, but you don’t need to return anything to the client? async void? Not quite

Take the following Delete method:

public async void Delete(int id)
{
    using (var context = new EntriesContext())
    {
        Entry entity = await context.Entries.FirstOrDefaultAsync(c => c.Id == id);
        if (entity != null)
        {
            context.Entry(entity).State = EntityState.Deleted;
            await context.SaveChangesAsync();
        }
    }
}

The client uses the Id property to do what it needs to do, so it doesn’t care what actually gets returned…as long as the operation (deleting the entity) completes successfully.

To help illustrate the problem, here is the client side code (written in AngularJS, but it really doesn’t matter what the client side framework is);

$scope.delete = function () {

<pre><code>var entry = $scope.entries[0];

$http.delete('/api/Entries/' + entry.Id).then(function () {
    $scope.entries.splice(0, 1);
});
</code></pre>

};

When the delete operation is completed successfully (i.e. a 2xx response code), the then call-back method is raised and the entry is removed from the entries collection.  Only this code never actually runs.  So why?

If you’re lucky, your web browser will give you a error message to let you know that something went wrong…

browser-error

I have however seen this error get swallowed up completely.

To get the actual error message, you will need to use a HTTP proxy tool, such as Fiddler.  With this you can capture the response message returned by the server, which should look something like this (for the sake of clarity I’ve omitted all the HTML code which collectively makes up the yellow screen of death);

An asynchronous module or handler completed while an asynchronous operation was still pending.

Yep, you have a race condition.  The method returned before it finished executing.  Under the hood, the framework didn’t create a Task for the method because the method does not return a Task.  Therefore when calling FirstOrDefaultAsync, the method does not pause execution and the error is encountered.

To resolve the problem, simply change the return type of the method from void to Task.  Don’t worry, you don’t actually have to return anything, and the compiler knows not to generate a build error if there is no return statement.  An easy fix, when you know what the problem is!

Summary

Web API fully supports Tasks, which are helpful for writing more scalable applications.  When writing methods that don’t need to return a value to the client, it may make sense to return void.  However, under the hood .NET requires the method to return Task in order for it to properly support asynchronous  functionality.

Create a telephone directory with Bootstrap, KnockoutJS, jQuery, Web API and Entity Framework (Part 2 of 2)

I find that the best way to learn any new technology, or technology that are unfamiliar with, is to sit down and practice.  The purpose of this very simple tutorial is to learn the basics of creating a single page website that can be used to capture peoples names and telephone numbers. 

By following this tutorial you will learn;

  1. How to use Bootstrap to create a simple UI
  2. How to use KnockoutJS for binding to/displaying of your data
  3. How to use jQuery to asynchronously retrieve data from a web service
  4. How to create a simple Web API using C#/ASP .NET
  5. How to persist data using Entity Framework code first

This is part 2 of 2.  You can read the first part here.

Create the UI

Our user interface has several jobs to do;

  • It must look good (Bootstrap to the rescue here)
  • It must allow the user to enter their name and number
  • It must allow the user to submit their details to the server
  • It must allow the user to reset the data
  • It must allow the user to edit and delete existing entries

What we don’t want is we don’t want to see the page refresh.  This is going to be such a simple page that the act of refreshing the entire page will feel clunky (not to mention the additional overhead of contacting the server to retrieve our entities again).  To aid this, we will use KnockoutJS, which provides JavaScript bindings out of the box.

You have already created the basic structure of your web page, add the following code to the body to create the basic design;

<div class="container-narrow">
    <div class="row">
        <h1>Telephone Directory</h1>
    </div>
    <div class="row shaded padded">
        <div class="col-sm-3">
            <label for="firstName">First Name</label>
            <input id="firstName" name="firstName" type="text" class="form-control" required="required" />
        </div>
        <div class="col-sm-3">
            <label for="lastName">Last Name</label>
            <input id="lastName" name="lastName" type="text" class="form-control" required="required" />
        </div>
        <div class="col-sm-3">
            <label for="phoneNumber">Phone Number</label>
            <input id="phoneNumber" name="phoneNumber" type="text" class="form-control" required="required" />
        </div>
        <div class="col-sm-12">
            <button id="add" name="add" type="submit">Add</button>
            <button id="reset" name="reset" type="reset">Reset</button>
        </div>
    </div>
</div>
<div class="container-narrow">
    <div class="row">
        <table class="table table-striped">
            <thead>
                <tr>
                    <th>First Name</th>
                    <th>Last Name</th>
                    <th>Phone Number</th>
                    <th>&nbsp;</th>
                </tr>
            </thead>
            <tbody>
                <tr>
                    <td><span></span></td>
                    <td><span></span></td>
                    <td><span></span></td>
                    <td>
                        <a href="#">Edit</a>&nbsp;<a href="#">Delete</a>
                    </td>
                </tr>
            </tbody>
        </table>
    </div>
</div>

Also, you should update your site.css file to include a few styles, as follows;

.shaded {
    background-color: #f9f9f9;
    border-radius: 15px;
}

.padded {
    padding: 10px;
}

.container-narrow {
    margin: 25px auto 0 auto;
    max-width: 700px;
}

    .container-narrow > .row > h1 {
        margin-bottom: 20px;
    }

    .container-narrow > .row > .col-sm-12 {
        margin-top: 15px;
        margin-bottom: 5px;
    }

Note that discussion about Bootstrap is generally out of scope of this tutorial, but basically we have used Bootstrap to create a container (a narrow one) with a couple of rows and a few columns to get the layout we desire.  We have also included a little custom CSS to add some shading, and margins.  We also have a couple of empty spans, which are currently working as placeholders, which will be used to display data about each of our entities.

Create a view model using KnockoutJS

With our UI (view) now in place, we now need a view model.  The purpose of the view model will be to send requests to/receive requests from our Web API, and relay the information to the user.

Open index.js and add the following code;

/// <reference path="jquery-1.9.0.intellisense.js" />
/// <reference path="knockout-3.1.0.debug.js" />

function TelephoneEntry(data) {
}

function TelephoneViewModel() {
    var self = this;
   
    self.add = function () {
    };
    self.edit = function (telephoneEntry) {
    };
    self.delete = function (telephoneEntry) {
    };
    self.reset = function () {
    };
    self.load = function () {
    };

    self.post = function (telephoneEntry) {
    };
}
ko.applyBindings(new TelephoneViewModel());

Note that the references at the top of the file are cues to Visual Studio IntelliSense, and aren’t strictly required (although they do make life a lot easier).

We have some functions here for performing certain actions (which are hopefully self explanatory).  The view model gets bound to the view at at the end, and using some funky syntax (which we will see later) we can access the properties/methods on our view model very simply.

Observable properties

An observable property in KnockoutJS is similar to that in WPF.  Basically when the value of an observable property changes, a change notification is raised, which causes the value of the property on the view to be refreshed.  Add the following observable properties to the view model (under var self = this;).  (this defines each property, and sets a default value)

self.id = ko.observable(0);
self.firstName = ko.observable('');
self.lastName = ko.observable('');
self.number = ko.observable('');

And at the very end of the view model, under all the methods we have defined, add the following;

self.telephoneEntries = ko.observableArray([]);
self.load();

An observable array raises change notifications when items are added to it/removed from it.  It does not raise change notifications where the entities themselves are changed (a common misunderstanding).

Add a new telephone entry

We want to capture the details entered by the user on the form, and display them in a list so they can easily be read back (and persist the data back to the database, more on this later).  Update the TelephoneEntry class to store the basic properties of each entity (you may notice this class is identical to that you created for Entity Framework earlier);

function TelephoneEntry(data) {
    var self = this;
    self.id = data.id;
    self.firstName = data.firstName;
    self.lastName = data.lastName;
    self.number = data.number;
}

Now update the add method to add the new entity, as follows;

self.add = function () {
    var entry = new TelephoneEntry({
        id: self.id(),
        firstName: self.firstName(),
        lastName: self.lastName(),
        number: self.number()
    });

    self.telephoneEntries.push(entry);
};

Add bindings to the view

Switch back to the view.  We can now use Knockout bindings to bind the properties we just created on our view model to our text fields, and our placeholder spans that we created earlier.  To specify a binding, we simply use the data-bind attribute on whatever elements we desire.

Locate the firstName input field and add the data-bind attribute, as shown;

<input id="firstName" name="firstName" type="text" class="form-control" data-bind="value: firstName" required="required" />

We have used a value binding here with the property firstName.  This means that we want to bind the value property of the input field to the firstName property on our view model.  Go ahead and update the other input fields, as shown;

<input id="lastName" name="lastName" type="text" class="form-control" data-bind="value: lastName" required="required" />

<input id="phoneNumber" name="phoneNumber" type="text" class="form-control" data-bind="value: number" required="required" />

Next, we want to bind the click event of our Add button to the add method on our view model.  Can you guess how this might work?  See the code below to see if you were correct!

<button id="add" name="add" type="submit" data-bind="click: add">Add</button>

KnockoutJS is very powerful, flexible and pretty easy to use, wouldn’t you agree?

We’re almost there, but first we need to update our table to show a row for each entity in our telephoneEntries observable array that we created earlier.  For this we need a new binding, the foreach binding.  Update the tbody element, as shown below;

 <tbody data-bind="foreach: telephoneEntries">

Basically what is going to happen here is that for each entity in our telephoneEntries observable array, a tr is going to be output.  Each tr will be bound to its corresponding entity in the array.  Use the text binding to see the text property for each span placeholder, as shown below;

<tbody data-bind="foreach: telephoneEntries">
    <tr>
        <td>
            <span data-bind="text: firstName"></span>
        </td>
        <td>
            <span data-bind="text: lastName"></span>
        </td>
        <td>
            <span data-bind="text: number"></span>
        </td>
        <td>
            <a href="#" data-bind="click: $parent.edit">Edit</a>&nbsp;<a href="#" data-bind="click: $parent.delete">Delete</a>
        </td>
    </tr>
</tbody>

I’ve also added a link for editing and deleting entities, which we’ll flesh out later.  To access properties/methods on the view model (rather than whatever is in the current context) we use the $parent object.

If you run the application now, you should be able to enter a first name, last name and number, click the Add button and see a new row added to the table underneath.  Lets now wire up the rest of the functionality.

Add some new properties to you view model, as follows;

self.addText = ko.observable('Add');
self.resetText = ko.observable('Reset');
self.selectedIndex = -1;

These will be used for the add/update methods to determine which is which.  Now update the add method so that it can determine weather it should perform an add, or an update, as shown;

self.add = function () {
        var entry = new TelephoneEntry({
            id: self.id(),
            firstName: self.firstName(),
            lastName: self.lastName(),
            number: self.number()
        });
        if (self.addText() == 'Add') {
            self.telephoneEntries.push(entry);
        }
        else {
            var oldTelephoneEntry = self.telephoneEntries()[self.selectedIndex];
            self.telephoneEntries.replace(oldTelephoneEntry, entry);
        }

        self.post(entry);
        self.reset();
    };

Next, update the edit method to allow editing of each entry.  The method simple reads out the given entity (provided by the context aware aspect of KnockoutJS) and populates the publicly bound properties with those values;

self.edit = function (telephoneEntry) {
    self.id(telephoneEntry.id),
    self.firstName(telephoneEntry.firstName);
    self.lastName(telephoneEntry.lastName);
    self.number(telephoneEntry.number);
    self.addText('Update');
    self.resetText('Cancel');
    self.selectedIndex = self.telephoneEntries.indexOf(telephoneEntry);
};

The delete method is quite simple, all we have to do is remove the entity (again given to us by KnockoutJS from the telephoneEntries observable array);

self.delete = function (telephoneEntry) {
    self.telephoneEntries.destroy(telephoneEntry);
};

Update the reset method, which will allow the user to clear down the form (and we will call this too when performing certain operations);

self.reset = function () {
    self.id(0);
    self.firstName('');
    self.lastName('');
    self.number('');
    self.addText('Add');
    self.resetText('Reset');
    self.selectedIndex = -1;
};

You will need to update the Add/Reset buttons on your view to include the addText and resetText properties, so that we can perform the appropriate actions when editing/adding;

<button id="add" name="add" type="submit" data-bind="click: add, text: addText">Add</button>

<button id="reset" name="reset" type="reset" data-bind="click: reset, text: resetText">Reset</button>

Communication with the Web API using jQuery

The application should now be completely usable (just a reminder, the full source code in on GitHub if you are struggling).  You may notice, however, that if you refresh the page all your data that you have meticulously typed out has been lost.  We need to make use of that Web API that we created earlier to store our data across page loads.  The quickest and easiest way to do this is with jQuery.

To achieve what we require here, we need to make use of the jQuery getJSON, Post and AJAX methods.

Update the load method as follows;

self.load = function () {
    $.getJSON('http://localhost:62129/api/Data/', function (data) {
        $.each(data, function (index, item) {
            self.telephoneEntries.push(new TelephoneEntry({
                id: item.id,
                firstName: item.firstName,
                lastName: item.lastName,
                number: item.number
            }));
        });
    });
};

We call the Web API using the getJSON method.  This passes a ContentType of application/JSON header under the hood to ensure that the server responds in that format.  When the process is complete, it calls our callback function, passing the retrieved data in as the data parameter.  Its then a simple case of iterating over the resulting array and adding each entity to our telephoneEntries observable array.  And because its an observable array, the view (our table) is updated automatically for us.

If you refresh your view, after a second or two (this will take longer the first time) your seed data that you created earlier should appear!

To post a new entity to the Web API, update the post method as follows;

self.post = function (telephoneEntry) {
 $.post('http://localhost:62129/api/Data/', telephoneEntry, function (id) {
        telephoneEntry.id = id;
    });
};

The HTTP POST verb will be sent along with the request.  Note that the callback for this request might not be what you are expecting.  In order to make future edits and deletes, we need the ID value for the new entity we just created.  Note that when you create a new entity on the client, its ID is unknown, so by default the ID is always 0.  When we wrote up the Post method on our API earlier, after the new entity is saved, its new entity ID is returned back to us.  Again thanks to the observable property provided by KnockoutJS, everything just updates magically for us.

The delete operation is slightly different.  Take the following code;

self.delete = function (telephoneEntry) {
    self.telephoneEntries.destroy(telephoneEntry);
    $.ajax({
        url: 'http://localhost:54946/api/Data/' + telephoneEntry.id,
        type: 'DELETE',
        contentType: "application/json;charset=UTF-8",
        data: JSON.stringify({ id: telephoneEntry.id }),
        dataType: "json"
    });};

We have to explicitly send the  DELETE HTTP verb, the Content-Type and a stringified version of the JSON (and the actual data type) along with the request.  The reason why is unclear, but failing to do so results in a 500 Internal Server Error.  Interestingly, however, the operation still completes in its entirety.  If anybody can explain this to me I’m more than happy to listen.

Create a telephone directory with Bootstrap, KnockoutJS, jQuery, Web API and Entity Framework (Part 1 of 2)

I find that the best way to learn any new technology, or technology that are unfamiliar with, is to sit down and practice.  The purpose of this very simple tutorial is to learn the basics of creating a single page website that can be used to capture peoples names and telephone numbers. 

By following this tutorial you will learn;

  1. How to use Bootstrap to create a simple UI
  2. How to use KnockoutJS for binding to/displaying of your data
  3. How to use jQuery to asynchronously retrieve data from a web service
  4. How to create a simple Web API using C#/ASP .NET
  5. How to persist data using Entity Framework code first

This is part 1 of 2.  The second part will be available soon.

A screenshot of the end result;

FinalProduct

Prerequisites/Set Up

We will not be using the standard ASP .NET template, for the sake of keeping the code simple and light.

Regardless of whether you are using Visual Studio 2012 or 2013, start by creating a new ASP .NET Empty Web Application.  Call the project TelephoneDirectory.

NewProject

We’re using this template to avoid a lot of the bulk that comes with the other default templates.  The empty project is truely empty, apart from a web.config file and a couple of references.

Third party dependencies

Next, use the Package Manager Console to add our third party dependencies;

  • Bootstrap (Install-Package bootstrap (this will also bring down jQuery, which is a dependency)) >> Used to give us a super pretty user interface.
  • Entity Framework (Install-Package entityframework) >> Used for data persistence
  • KnockoutJS (install-package knockoutjs (which surprisingly, has no dependencies itself)) >> Used for model binding our form/displaying our data
  • WebAPI (Install-Package Microsoft.AspNet.WebApi) >> Used as the back end data service
  • Newtonsoft.Json (install-package Newtonsoft.Json) >> Used to JSON-ify our data servers responses

Other files

Add the following files in their respective folders;

  • Scriptsindex.js
  • Contentsite.css
  • index.html
  • Global.asax (Add New Item > Global Application Class)

Open up index.html and update the markup as follows; (be sure to substitute the version numbers for the current version number)

  • Add bootstrap.min.css and site.css to the header
  • Add jquery-1.9.0.min.js, bootstrap.min.js, knockout-3.1.0.js and index.js to the body (just above the closing body tag)

Remember that order matters.  You markup should look as follows;

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <title></title>
    <link href="Content/bootstrap.min.css" rel="stylesheet" />
    <link href="Content/site.css" rel="stylesheet" />
</head>
<body>
    <script src="Scripts/jquery-1.9.0.min.js"></script>
    <script src="Scripts/knockout-3.1.0.js"></script>
    <script src="Scripts/index.js"></script>
</body>
</html>

Implement your model

Its always a bit tricky with where to start on these tutorials.  Normally you might spec out a simple UI first and experiment with feeding data to it and evolve the design over a little time, but as this is a guided tutorial, I already know what the end result will be.  So stick with me.

We are using the Entity Framework code first approach for simplicity, so we can create our entities straight in code. 

We will need a property to store the ID of the telephone entry (more on this as we go along), the first name, the last name and the number.  Add a new folder called Models and add a new class called TelephoneEntry, as follows;

public class TelephoneEntry
{
    public string FirstName { get; set; }
    public int Id { get; set; }
    public string LastName { get; set; }
    public string Number { get; set; }
}

Now we need a database context, which we can use to query our database.  Create a DbContext class, called DataContext as follows (you can add this class to the Models folder for simplicity, ideally all your data access code should be split out into its own project, but that is out of the scope of this post);

public class DataContext : DbContext
{
    public DbSet<TelephoneEntry> TelephoneEntries { get; set; }
}

To help aid our testing further down the line, it will be helpful to have some seed data.  Create a custom initializer, called Initializer, which inherits from the  DropCreateDatabaseAlways database initializer.  This will ensure that we start with a fresh database every time our application starts (you would want to change this in a production environment, but its super handy for debugging purposes);

public class Initializer : DropCreateDatabaseAlways<DataContext>
{
    protected override void Seed(DataContext context)
    {
        context.TelephoneEntries.Add(new TelephoneEntry { FirstName = "Jon", LastName = "Preece", Number = "4444" });
    }
}

Override the seed data and add in a new entry (or several if you like), as shown above.

For the final step, we need to initialize our database.  You can do this in Global.asax.cs, inside the Application_Start method;

protected void Application_Start()
{
     Database.SetInitializer(new Initializer());
}

From the database creation perspective, we’re done.  We will query the database a little later on when creating our Web API.

Set up your Web API

Web API is very easy to work with out of the box.  At a high level, it pretty much “just works”, which is fantastic.  One could write a whole book on the ins-and-outs of Web API, but we only need to make use of its basic functionality at this time.

Create a new folder called App_Start and add a new class called WebApiConfig.cs.  This class will contain routing instructions so that our API Controller (coming next) can be accessed as a Restful service.

Add the following static method;

public static void Register(HttpConfiguration config)
{
    config.MapHttpAttributeRoutes();
    config.Routes.MapHttpRoute("DefaultApi", "api/{controller}/{id}", new { id = RouteParameter.Optional });

   JsonMediaTypeFormatter jsonFormatter = config.Formatters.OfType<JsonMediaTypeFormatter>().First();
    jsonFormatter.SerializerSettings.ContractResolver = new CamelCasePropertyNamesContractResolver();
}

You may now be wondering what is the purpose of this JsonMediaTypeFormatter? Well, when a client attempts to access the Web API, it can specify what type of content it accepts in response (by passing an appropriate Content-Type header).

Out of the box, Web API and respond in two formats; XML and JSON.  XML is supported because it is widely used and has been around a long time (although its scarily bulky).  JSON is a very lean duck typed language that can be very efficiently transferred and interpreted.  Unfortunately Web API, when replying with data to the client, responds in Pascal Case (ThisIsPascalCase), whereas JavaScript is typically written in Camel Case (thisIsCamelCase). 

The JsonMediaTypeFormatter ensures that our JSON is sent to the client in camel case, so that it can be properly consumed by the client.

You should call the Register method from the Global.asax.cs Application_Start method, as shown;

protected void Application_Start()
{
    GlobalConfiguration.Configure(WebApiConfig.Register);
    Database.SetInitializer(new Initializer());
}

Design your Web API

Lets take a step back a minute to discuss what Web API is.  Perfectly summed up by Wikipedia;

A server-side web API is a programmatic interface to a defined request-response message system, typically expressed in JSON or XML, which is exposed via the web—most commonly by means of an HTTP-based web server.

Web API is a publically accessible interface for accessing data via HTTP.  Web API uses conventions (or you can actually use routing attributes, but that’s another discussion) to map your methods to the associated HTTP verbs.

Get

So, if you want to respond to a HTTP GET request, you add a Get method to your controller. If you want to respond to a HTTP DELETE request, you add a Delete method to your controller … and so on.

For this tutorial, we will add methods to respond to Get (with overloads), Delete and Post requests.   Create a new folder called Controllers, and add a new class called DataController.  Make sure the class inherits from ApiController (rather than the standard Controller base class) and add the following method;

public class DataController : ApiController
{
     public async Task<IEnumerable<TelephoneEntry>> Get()
     {
        using (DataContext context = new DataContext())
        {
            return await context.TelephoneEntries.ToListAsync();
        }
    }
 }

In the interest of maximum scalability (its important to think ahead) you can make your method asynchronous (by using the async and await keywords) and use the Async versions of methods where possible, although you should know that this is not required (but a good habit to get into).

Interestingly, you can now open up a web browser (by pressing F5) and query this method.  Your browser will send a Get request by default; (note that your port number will vary to mine)

GetData

Well done, your API is now working.

The method you have added will return all the telephone entries by default.  Now, add the following method;

public async Task<TelephoneEntry> Get(int id)
{
    using (DataContext context = new DataContext())
    {
        return await context.TelephoneEntries.FirstOrDefaultAsync(t => t.Id == id);
    }
}

Change the URL to pass in an ID as follows; http://localhost:62129/api/Data/1 or http://localhost:62129/api/Data?id=1.

A little challenge for you, can you add a couple more methods to query the data based on a search query and a first/last name? I’ll wait here whilst you try.

All done? See how yours compares to mine;

public async Task<List<TelephoneEntry>> Get(string query)
{
    using (DataContext context = new DataContext())
    {
        return await context.TelephoneEntries.Where(t => string.Equals(t.FirstName, query) || string.Equals(t.LastName, query)).ToListAsync();
    }
}

public async Task<List<TelephoneEntry>> Get(string firstName, string lastName)
{
    using (DataContext context = new DataContext())
    {
        return await context.TelephoneEntries.Where(t => string.Equals(t.FirstName, firstName) && string.Equals(t.LastName, lastName)).ToListAsync();
    }
}

Delete

If you want the client to be able to delete data (and we do), you will want to add a delete method.  To call the delete method, a DELETE HTTP verb will need to be sent with the request (we will see this in action later).  An identifier for the entity to delete will also need to be sent (in this case, the ID.  Again we will see this later).

Add the following delete method;

public void Delete(int id)
{
    using (DataContext context = new DataContext())
    {
        TelephoneEntry entity = context.TelephoneEntries.FirstOrDefault(t => t.Id == id);
        if (entity != null)
        {
            context.Entry(entity).State = EntityState.Deleted;
        }

        context.SaveChanges();
    }
}

Post

A post request is slightly different to the other methods already discussed.  In this case, we need to pass a more complex object from the client than just a simple ID or a string.  We want to pass the actual TelephoneEntry entity that we want to create.  For completeness, we need to add the FromBody attribute to our parameter, which tells Web API to use the model binder (media type binder to be precise) to extract values from the HTTP message (rather than the URI), and create the object for us without us having to worry about it.

Add your post method as follows;

public async Task<int> Post([FromBody] TelephoneEntry telephoneEntry)
{
    using (DataContext context = new DataContext())
    {
        if (telephoneEntry.Id == 0)
        {
            context.Entry(telephoneEntry).State = EntityState.Added;
        }
        else
        {
            context.Entry(telephoneEntry).State = EntityState.Modified;
        }

        await context.SaveChangesAsync();
        return telephoneEntry.Id;
    }
}

There you go.  You data is now ready to be accessed and consumed by the outside world!

Summary

So far we have explored the basic structure for our project, we have installed all our dependent packages, we have created a database using Entity Framework code first, and we have created a publicly available Web API. In the second part of this post, we will flesh out our view, add KnockoutJS to make our page interactive, and use jQuery to tie everything together.

The entire source code can be found on GitHub