Thursday, 18 November 2010

ASP.NET MVC and LINQ to SQL – LINQPad to the Rescue

Recently I was rewriting  a broken algorithm in some ASP.NET MVC code.  For part of the fix I needed to write some LINQ to SQL. I needed to do a comparison against a nullable field but I was thinking in C# not in SQL. Here is a simplified version of the real problem. Suppose we have a table of customers where LastName is a nullable field. We want to return customers whose last name is NULL.

string lastName = null;

var customers =
from c in Customers
where c.LastName == lastName
select c;


This was unexpectedly returning no values. The correct solution is



var customers =
from c in Customers
where c.LastName == null
select
c;


The reason was due to the “IS NULL vs. = NULL”  issue in the underlying SQL. This was easy to check by launching the excellent LINQPad application that allows you to view the underlying SQL after running a LINQ statement. I’d just assumed that from the C#/LINQ side it would automatically generate the appropriate code.



Comparing directly against null from the C# generates



SELECT [t0].[CustomerID], [t0].[FirstName], [t0].[LastName]

FROM [Customers] AS [t0]

WHERE [t0].[LastName] IS NULL



Comparing against a variable that’s null generates



DECLARE @p0 VarChar(1000) = null

SELECT [t0].[CustomerID], [t0].[FirstName], [t0].[LastName]

FROM [Customers] AS [t0]

WHERE [t0].[LastName] = @p0



In my real problem I was comparing against a nullable ID column and I wanted to return both null and non-null matches in one clause of a compound where. So I wanted to do a more elaborate version of this.



string name = (lastName != null) ? "Fred" : null;

var customers =
from c in Customers
where c.LastName == name
select c;


But I was forced to use an if-else instead.



if (lastName != null)
{
var customers =
from c in Customers
where c.LastName == lastName
select c;
// ...more code
}
else
{
var customers =
from c in Customers
where c.LastName == null
select
c;
// ...more code
}


I couldn’t think of a more concise way around this. Anyway, at least I solved the bigger problem of which this was a part. :)

Wednesday, 25 August 2010

JavaScript Unit Testing with QUnit

Recently, I’ve been trying to learn more about the Reactive Extensions library (currently still in Microsoft DevLabs) which I’ve blogged about previously. Since then Microsoft have pushed out a JavaScript version of the library and published a couple of Hands-on Labs for both the .NET and JavaScript versions. I thought I’d start looking at the JavaScript version as well. However, since the lab makes use of the jQuery JavaScript library it helps to be familiar with it too. So I’ve spent some time with jQuery as well. This in turn led me to realise that I needed to learn more about JavaScript in general.

In following Matthew Podwysocki’s Rx for JavaScript tutorials (not got very far yet) I noticed that he makes use of QUnit, a JavaScript unit testing library that is an offshoot of jQuery. So I made a diversion into this. It turns out that there are a number of unit testing libraries for JavaScript, but I’d not bothered to look into them before. Although QUnit is an offshoot of jQuery, and is what jQuery itself uses to test jQuery, it can be used for generic JavaScript. I have a number of JavaScript utility methods so after an initial play with QUnit I thought I’d try creating a few tests. Lo and behold I discovered a bug in the first method I tried.  Ah, the benefits of unit testing.

There’s a one-page runnable example on the home page and a couple of tutorial links. Following the lead of the second tutorial I created a web site in Visual Studio, added the tester page and then separate “project under test” and unit tests JavaScript files.

<script type="text/javascript" src="Scripts/qunit.js"></script> 
<!-- Library -->
<script type="text/javascript" src="Scripts/utility.js"></script>
<!-- Tests -->
<script type="text/javascript" src="Scripts/tests.js"></script>


I have a method called isInteger(text) that checks whether a text entry is a [positive] integer. Its unit tests are:



/// <reference path="utility.js" />
module('isInteger Tests');

test('isInteger_positiveInteger_true', function () {
var text = '32';
ok(isInteger(text), text + ' is an integer');
})

test('isInteger_negativeInteger_false', function () {
var text = '-32';
ok(!isInteger(text), text + ' is not an integer');
})

test('isInteger_zero_true', function () {
var text = '0';
ok(isInteger(text), text + ' is an integer');
})

test('isInteger_decimal_false', function () {
var text = '32.3';
ok(!isInteger(text), text + ' is not an integer');
})

test('isInteger_alpha_false', function () {
var text = 'abc';
ok(!isInteger(text), text + ' is not an integer');
})

test('isInteger_alphanumeric_false', function () {
var text = 'abc123';
ok(!isInteger(text), text + ' is not an integer');
})

test('isInteger_whitespace_false', function () {
var text = ' ';
ok(!isInteger(text), 'Whitespace text is not an integer');
})

test('isInteger_empty_false', function () {
var text = '';
ok(!isInteger(text), text + 'Empty text is not an integer');
})



QUnit produces out put like this (showing tests 1 to 5 for isInteger with some tests collapsed):



image



This is just the simplest usage of QUnit. It has some more assertions and also an asynchronous testing capability. It is this that Matthew Podwysocki uses in one of his Rx for JavaScript blog posts.

Friday, 11 June 2010

A Useful New String Method in .NET 4.0

In .NET 2.0 the String class introduced a method, IsNullOrEmpty, that allows you to check whether a string is null or the Empty string. However, sometimes you also want to disallow a string that contains just white space. I created a static utility method called IsNullOrEmptyOrBlank that additionally checked for white space.

public static bool IsNullOrEmptyOrBlank(string s)
{
return String.IsNullOrEmpty(s) || s.Trim().Length == 0;
}



In .NET 3.5 I turned this into an extension method.



public static bool IsNullOrEmptyOrBlank(this string s)
{
return String.IsNullOrEmpty(s) || s.Trim().Length == 0;
}



Then to my pleasure I noticed that in .NET 4.0 Microsoft has fixed this oversight and added this functionality to the String class.  The method is IsNullOrWhiteSpace. The description is: “Indicates whether a specified string is null, empty, or consists only of white-space characters.”



Kudos to Microsoft.  :-)

Monday, 7 June 2010

Finding the Missing Number in a Sequence

This is one of those algorithm problems that come up in programmer interview tests from time to time. Suppose you receive a sequence of numbers from 1 to 100 in random order but one is missing.  How do you find the missing number?

The laborious way would be to sort the numbers in ascending order and then loop until the difference in two successive numbers is 2. But this would be very slow if we had to sort 10 million numbers.

A slick solution I encountered was to make use of the formula for the sum of an arithmetic progression which, for a sequence of n consecutive numbers starting from 1, is n(n+1)/2. The full formula is n(2a + (n- 1)d)/2 where a is the first term and d is the common difference. If we substitute a = 1 and d = 1 then it simplifies to the first formula.

To find the missing number we simply add up the numbers from the random sequence and then subtract that sum from the sum given by the arithmetic progression formula. No sorting involved.

I thought I’d try this out using my unique random number generator class. This class takes any range of numbers and randomly shuffles them. So we can take a range of consecutive numbers from 1 to n and generate a random set. Then we can remove one at random and find it using the foregoing technique.

// Generate ordered sequence of numbers
int start = 1;
int count = 10;
var orderedNumbers =
Enumerable.Range(start, count);

// Print ordered numbers
Console.WriteLine("Ordered Numbers");
orderedNumbers
.ToList()
.ForEach(x => Console.Write(x + " "));
Console.WriteLine();

// Randomly shuffle them
var randomNumbers = new HashSet<int>();
var g = new UniqueRandomNumberGenerator(orderedNumbers);

// Print random numbers
Console.WriteLine("Random Numbers");
while (g.RemainingNumbersCount > 0)
{
int number = g.NewRandomNumber();
randomNumbers.Add(number);
Console.Write(number + " ");
}
Console.WriteLine();

// Verify we have same numbers after random shuffle
Debug.Assert(randomNumbers.SetEquals(orderedNumbers));

// Randomly generate a number to remove
var rand = new Random();
int removedNumber =
rand.Next(1, orderedNumbers.Count() + 1);
Console.WriteLine(
"Number to remove is " + removedNumber
);

// Remove it from the random numbers
randomNumbers.Remove(removedNumber);

// Print random numbers with number removed
Console.WriteLine(
"Random Numbers with {0} missing", removedNumber
);
randomNumbers
.ToList()
.ForEach(x => Console.Write(x + " "));
Console.WriteLine();

// Sum random numbers
int randomNumbersSum = 0;
var enumerator =
randomNumbers.GetEnumerator();
while (enumerator.MoveNext())
{
int number = enumerator.Current;
randomNumbersSum += number;
}

// Sum ordered numbers using arithmetic progression formula
int n = orderedNumbers.Count();
int orderedNumbersSum = n * (n + 1) / 2;

// The missing number is the difference between
// the sums of the ordered and random numbers
Console.WriteLine("Searching for missing number...");
int missingNumber = orderedNumbersSum - randomNumbersSum;
Console.WriteLine("Missing number is " + missingNumber);



I could have computed the sums using IEnumerable.Sum() on the two sets of numbers but I wanted to illustrate the algorithms.



Typical output for 10 numbers is:



Ordered Numbers

1 2 3 4 5 6 7 8 9 10


Random Numbers


10 3 5 9 1 7 4 2 8 6


Number to remove is 3


Random Numbers with 3 missing


10 5 9 1 7 4 2 8 6


Searching for missing number...


Missing number is 3

Tuesday, 30 March 2010

Reactive Extensions, Silverlight and WCF

Here I will illustrate how Reactive Extensions (Rx) can be applied to asynchronous calls to a WCF web service in a Silverlight application.

We start with a simple customer database consisting of a single table of first and last names.

ID  First   Last

1   Fred    McFarlane
4   Susan   McFarlane
6   David   Green
10  Joe     Bloggs

Then we set up a WCF service to return a collection of customers. I used LINQ to SQL to generate the CustomersDemo DataContext instance.

[ServiceContract(Namespace = "")]
[AspNetCompatibilityRequirements(
RequirementsMode =
AspNetCompatibilityRequirementsMode.Allowed)
]
public class CustomersService
{
private string connection =
@"[My connection]";

[OperationContract]
public List<Customers> GetCustomers()
{
CustomersDemo db = new CustomersDemo(connection);
var customers = from customer in db.Customers
select customer;
return customers.ToList();
}
}

Asynchronous Access Without Rx


To consume the service in the Silverlight client…


Make the asynchronous call in the Page constructor.

public Page()
{
InitializeComponent();

CustomersServiceClient client =
new CustomersServiceClient();
client.GetCustomersCompleted +=
new EventHandler<GetCustomersCompletedEventArgs>(
client_GetCustomersCompleted
);
client.GetCustomersAsync();
}

Handle the completed event and display customers in a data grid.

void client_GetCustomersCompleted(
object sender,
GetCustomersCompletedEventArgs e
)
{
if (e.Error == null)
{
var customers = e.Result;
dg.ItemsSource = customers;
}
}

Results:


image


Asynchronous Access With Rx


In this case we dispense with the completed handler and centralise the code in the Page constructor. Instead of having a separate handler we create an IObservable from the GetCustomersCompleted event using the Observable.FromEvent method. The appended Take(1) is for returning a single value from the start of the observable sequence. It also implicitly unsubscribes the observer.

public Page()
{
InitializeComponent();

CustomersServiceClient client =
new CustomersServiceClient();
IObservable<IEvent<GetCustomersCompletedEventArgs>> observable =
Observable.FromEvent<GetCustomersCompletedEventArgs>(
client, "GetCustomersCompleted"
).Take(1);

observable.Subscribe(
e =>
{
if (e.EventArgs.Error == null)
{
var customers = e.EventArgs.Result;
dg.ItemsSource = customers;
}
});
client.GetCustomersAsync();
}
This produces the same results:

image

Concurrent File Processing Using Reactive Extensions

This follows on from my first post on Reactive Extensions for .NET back in January. We now consider a simple scenario of performing a number of operations on a file concurrently. This example was inspired by a post from functional programming guru, Matthew Podwysocki. Here I just flesh out his suggestion at the end of that post.

Here is a simple text file – test.txt – consisting of two lines (there is a new line after the first period)

We're proud to announce the availability of Reactive Extensions for JavaScript.
This port brings the power of Reactive programming to JavaScript.

I use a short file so we can easily check the results.

Now let’s perform three concurrent asynchronous operations on it.

  1. Count the words
  2. Count the letters
  3. Count the vowels

This seems quite simple but was in fact a bit trickier than meets the eye. The first thing to do is load the contents of the file into an enumerable collection. We do this so that we can subsequently convert the IEnumerable into an IObservable to which we’ll subscribe three times, once for each operation. There are various ways of reading the file but it turned out to be easiest to read it into a single item collection.

Then create an observable

List<string> text = new List<string>();
text.Add(File.ReadAllText(@"test.txt"));
var observable = text.ToObservable();

Count the words

private static void CountWords(string text)
{
string[] separator =
new string[] { " ", ",", ".", "\r\n"};

string[] words =
text.Split(
separator,
StringSplitOptions.RemoveEmptyEntries
);
int count = words.Count();
Console.WriteLine("Number of words = {0}", count);
}

Count the letters

private static void CountLetters(string text)
{
string[] separator =
new string[] { " ", ",", ".", "\r\n" };

string[] words =
text.Split(
separator,
StringSplitOptions.RemoveEmptyEntries
);
int count = words.Sum(word => word.Length);
Console.WriteLine("Number of letters = {0}", count);
}

Count the vowels

private static void CountVowels(string text)
{
char[] vowels = {'a', 'e', 'i', 'o', 'u'};
char[] chars = text.ToCharArray();
int count =
chars.Count(
c => vowels.Contains(Char.ToLowerInvariant(c))
);
Console.WriteLine("Number of vowels = {0}", count);
}

Putting it all together…

// Read file and store as single item string collection
List<string> text = new List<string>();
text.Add(File.ReadAllText(@"test.txt"));

// Create an observable from our collection
var observable = text.ToObservable();

// Print some file stats concurrently
using (observable.Subscribe(CountWords))
using (observable.Subscribe(CountLetters))
using (observable.Subscribe(CountVowels))

Results:

Number of words = 21
Number of letters = 123
Number of vowels = 47

Thursday, 25 February 2010

Sharing Web Page Links in Discussion Forums

Quite often in discussion forums an answer will refer you to some link(s) that provides the solution to, or sheds some light on, the original question. However, there is nothing more frustrating than finding that a link is broken. So now I've been trying to get into the habit of supplying the page or article title together with the link so that at least the visitor can search on it and stand a chance of finding the relocated link. It's also just nice to see what the title of the article is in any case.

The Firefox browser has a handy add-on, called Multiple Tab Handler, that has a feature that allows you to copy the page title plus uri to the clipboard. This is available on the right-click menu.

Here's a typical result of a right-click:

New Recommendations for Using Strings in .NET 2.0
http://msdn.microsoft.com/en-us/library/ms973919.aspx

This saves me from going to the page, then copying and pasting the link; then returning to the page and copying and pasting the title.

Friday, 29 January 2010

Exploring Reactive Extensions for .NET (Rx)

Microsoft quietly pushed out Reactive Extensions (aka LINQ to Events) for .NET 3.5 SP1 and .NET 4.0 before Christmas. Actually it wasn't done that quietly but there hasn't been that much fanfare about it compared to other features of .NET 4.0. It's a DevLabs project and official documentation is rather scant at the moment. The main sources are videos, blog posts and an MSDN forum.

Reactive Extensions (Rx) used to be called Reactive Framework. An earlier incarnation was quietly slipped out with the Silverlight 3 Toolkit in mid-2009. The purpose of Rx is to make asynchronous programming easier. Conceptually, I'm not sure whether it does but it certainly makes such code expressively more elegant.

If you want to see a reasonably organised set of code snippets that you can run then I recommend the Rx Wiki which hosts 101 Rx Samples. The Rx Wiki also lists the available Channel 9 videos.

Getting Started

Watch the short Getting Started video.

I downloaded the .NET 3.5 SP1 version. For a bit of conceptual background try reading the first few sections of Introducing Rx. It does contain a few typos though.

Let's start as simple as possible. We're going to call the following method from a Console application and execute it synchronously, then asynchronously the old way, then asynchronously the "reactive" way. We'll make some observations as we go along.

Example 1 - Running Code Synchronously

private static void DoSomething()
{
Console.WriteLine("Calculating...");
Thread.Sleep(2000);
Console.WriteLine("Done.");
}
private static void RunCodeSynchronously()
{
DoSomething();
}

Result:
Calculating...
Done.

Example 2 - Running Code Asynchronously the Old Way

We define a delegate for our DoSomething method and use BeginInvoke to fire it asynchronously. We also annotate the main and asynchronous threads for clarity.
private delegate void DoSomethingDelegate();

private static void DoSomething()
{
Console.WriteLine("Async thread {0}.",
Thread.CurrentThread.GetHashCode());
Console.WriteLine("Calculating...");
Thread.Sleep(2000);
Console.WriteLine("Done.");
}
private static void DoSomethingCompleted(IAsyncResult result)
{
}
private static void RunCodeAsynchronouslyOldWay()
{
Console.WriteLine("Main thread {0}.",
Thread.CurrentThread.GetHashCode());
DoSomethingDelegate d =
new DoSomethingDelegate(DoSomething);
d.BeginInvoke(
new AsyncCallback(DoSomethingCompleted),
null
);
Console.ReadKey();
Console.WriteLine("Back in main thread.");
}
For illustration purposes we aren't interested in the AsyncCallback delegate so it's just a no-op. (The Console.ReadKey is introduced to allow the asynchronous thread to complete before the main thread. )

Result:
Main thread 1.
Async thread 3.
Calculating...
Done.
Back in main thread.

Example 3 - Running Code Asynchronously using Rx

private static void RunCodeAsynchronouslyRx()
{
Console.WriteLine("Main thread {0}.",
Thread.CurrentThread.GetHashCode());

// Pass an Action delegate representing our operation
// and invoke asynchronously
var o = Observable.Start(DoSomething);
o.Subscribe();

Console.ReadKey();
Console.WriteLine("Back in main thread.");
}

Result:
Main thread 1.
Async thread 3.
Calculating...
Done.
Back in main thread.


Here the asynchronous code becomes an observable to which we subscribe. It "pushes" its operations to the observer which is our invoking code.

Now let's expand these examples a bit. We'll introduce an error in the asynchronous thread and see how exceptions are handled in each case.

Example 4 - Running Code Asynchronously the Old Way With an Error

RunCodeAsynchronouslyOldWay remains unchanged from above. DoSomething changes to:
private static void DoSomething()
{
try
{
Console.WriteLine(
"Async thread {0}.",
Thread.CurrentThread.GetHashCode()
);
Console.WriteLine("Calculating...");
Thread.Sleep(1000);
// Introduce a divide-by-zero error
int a = 1;
int b = 0;
int x = a / b;
Console.WriteLine("Done.");
}
catch (Exception ex)
{
Console.WriteLine("Error: " + ex.Message);
}
}

Result:
Main thread 1.
Async thread 3.
Calculating...
Error: Attempted to divide by zero.
Back in main thread.

Example 5 - Running Code Asynchronously using Rx With an Error

This time we don't require a try-catch block in DoSomething.
private static void DoSomething()
{
Console.WriteLine("Async thread {0}.",
Thread.CurrentThread.GetHashCode());
Console.WriteLine("Calculating...");
Thread.Sleep(1000);
int a = 1;
int b = 0;
int x = a / b;
Console.WriteLine("Done.");
}
Instead we subscribe to the error from our invoking code.
private static void RunCodeAsynchronouslyRx()
{
Console.WriteLine("Main thread {0}.",
Thread.CurrentThread.GetHashCode());

// Pass an Action delegate representing some operation
// and invoke asynchronously
var o = Observable.Start(DoSomething);

o.Subscribe(
x => {},
ex => Console.WriteLine("Error: " + ex.Message)
);

Console.ReadKey();
Console.WriteLine("Back in main thread.");
}

Result:
Main thread 1.
Async thread 3.
Calculating...
Error: Attempted to divide by zero.
Back in main thread.

The Subscribe method is overloaded. In the overload above we are using a value handler and an exception handler. Think of the value handler as corresponding to the DoSomethingCompleted handler from Example 2. It's code we wish to run after the asynchronous code has completed. The value handler is an Action<T> delegate. We wish to do nothing so I've just supplied a no-op.

One last thing, Subscribe returns an IDisposable so we should call Dispose when we're done or else wrap it in a using statement. I left out those details for simplicity of exposition.

Well, this is just the briefest of introductions. The idea was to use as simple logic as possible so that we can focus on what's new. Often when new concepts are introduced the business logic of the examples is so non-trivial that it obscures the concepts. Reactive Extensions packs in a lot. The Observable class, for example, is huge. I will continue to explore it and hope to post more in due course.