The Code King

Amazon AWSDeploy To Provision Multiple Websites

noreply@blogger.com (TheCodeKing) — Sun, 03 Feb 2013 23:04:22 PST

Introduction

Recently I've been on a project where we are implementing BlueGreen deployments using Amazon CloudFormation, and came across a number of unanswered questions:

How do you deploy multiple applications to a single instance and support auto-scaling?
How do you deploy non-website applications using AWSDeploy?
How can you pass in environmental config during provisioning?

The official answer to the first 2 questions appears to be that you can't without additional scripts/tools. The third (with some caveats) is that you need to bake them into your package prior to your deployment.

This seems frustrating for such common real world use cases, but luckily there is a solution which requires minimal effort.

First Some Background

CloudFormation is Amazon's offering for provisioning a collection of AWS resources on the amazon infrastructure through declarative templates. The collection of resources is known as a stack and can be used for reliable and repeatable infrastructure deployments.

AWSDeploy is the tool which can be used with the Microsoft stack to provision CloudFormation templates and to deploy web applications onto these provisioned resources. This uses Microsoft WebDeploy under the hood, and as such a WebDeploy package must be provided when provisioning a CloudFormation stack using the tool.

The WebDeploy package provided by the user is uploaded to AWS by the tool and stored in an S3 bucket. When the EC2 web server spins up, AWSDeploy installs the package on the web server as part of initialising the stack. By default any failures will result in a stack rollback.

Once a web application is provisioned via AWSDeploy, the stack will manage failover and auto-scaling scenarios automatically by redeploying the associated WebDepoy package stored on S3 to any new instances. In doing so it is able to manage resources hands-free and replicate perfect copies of your application on demand.

With Great Power Comes Great Limitations

As described above, AWSDeploy is incredibly useful for failover and scaling scenarios. Your web application can be replicated and auto-provisioned without manual interaction.

The downside is a number of limitations:

Only 1 WebDeploy package can be associated with a web server instance
No parameters can be passed when AWSDeploy installs the WebDeploy package
As such WebDeploy parameterisation cannot be used for config transforms
Environmental config is limited to a small set of pre-defined appSettings keys, which can be injected during provisioning

The impact is that to work with AWSDeploy you generally have to modify your build artifact for every deployment environment and bake in configuration settings. This is against Continuous Delivery principles and adds complexity to any deployment scripts.

An Idea

WebDeploy in it's own right is an extremely flexible and versatile tool. In a single WebDeploy package it's possible to deploy multiple websites, deploy directories of arbitrary files, execute commands, and change Acl permissions.

So, instead of baking environment configuration into the WebDeploy packages to support AWSDeploy, why not instead create a wrapper WebDeploy package and use an embedded RunCommand to pass environment variables to a child package during provisioning. In this way the original build artifact is unmodified and the full power of WebDeploy can be used to inject parameters during installation.

This approach has additional benefits in that it allows the install of multiple web applications via a single AWSDeploy package, as well as the ability to deploy Windows Services and Console applications during auto-provisioning. Additionally it's then relatively easy to cheery-pick applications for deployment to a particular instance with very little rework of deployment scripts.

The Problem

On creating my first wrapper package I immediately ran into a problem when provisioning. The AWSDeploy logs revealed the issue.

ERROR 10 AWSDeploymentHostManager.Tasks.UpdateAppVersionTask - Deploy failed, not iisApp.

It seems AWSDeploy expects the WebDeploy package to contain an iisApp within it's manifest. So I included a stub website in my WebDeploy package and tried again. Surprisingly I hit the same error!

After much investigation it turns out that when AWSDeploy executes the WebDeploy package, it passes the following parameter to the package. This is to enable customisation of the target IIS Website name during provisioning, and can be set via a AWSDeploy configuration setting.

-setParam:"IIS Web Application Name"="Default Web Site/"

The Solution

Knowing the parameter requirement, the solution is simple. Provided the WebDeploy package contains a parameters file with this parameter defined within, then AWSDeploy will successfully deploy the package regardless of it's contents. It also means that the WebDeploy package does not actually need to include an iisApp instance as the error first suggested.

Putting It All Together

First place any WebDeploy packages to be installed on the server instance in a folder called C:\Packages. Include any set-parameter configuration files for any environmental settings and config transforms.

Create a PowerShell install script called Install.ps1 and add this to the C:\Packages directory. The script will be executed on the server during provisioning. It will be used to deploy the WebDeploy packages in this folder on the server, and can be used to invoke explicit WebDeploy commands during install; such as to leverage environment specific parameterisation.

Set-Alias msdeploy "C:\Program Files\IIS\Microsoft Web Deploy V3\msdpeloy.exe"

msdeploy -verb:sync -source:package=WebsiteA.zip -dest:auto -setParamFile:WebSiteA-Params.xml
msdeploy -verb:sync -source:package=WebsiteB.zip -dest:auto -setParamFile:WebSiteB-Params.xml

Next create the parameters file for inclusion in the WebDeploy wrapper package as follows. Name the file parameters.xml and place it in a folder called C:\Build.

<parameters>
 <parameter name="IIS Web Application Name" defaultValue="Default Web Site/" tags="IisApp" />
</parameters>

Now create a WebDeploy manifest file for the wrapper package as follows. This will provision the contents of C:\Packages to the server instance and use the RunCommand provider to execute the Install.ps1 PowerShell script in the root of this directory. Name the file manifest.xml and place it in the C:\Build folder.

<siteManifest>
  <contentPath path="C:\Packages" />
  <runCommand path="%SystemRoot%\System32\WindowsPowerShell\v1.0\PowerShell.exe
       -inputformat none -command &quot;Set-ExecutionPolicy Unrestricted -force&quot;" 
       waitInterval="10000" />
  <runCommand path="%SystemRoot%\System32\WindowsPowerShell\v1.0\PowerShell.exe 
       -inputformat none -command C:\Packages\Install.ps1" 
       waitInterval="300000" />
</siteManifest>

Finally package up the WebDeploy wrapper package using the command line as follows, which should be executed from the C:\Build folder. The trick is the declareParamFile parameter and parameters.xml file which keeps AWSDeploy happy.

msdeploy -verb:sync -source:manifest=manifest.xml -dest:package=Deploy.zip
         -declareParamFile=parameters.xml

The WebDeploy package produced is now ready to be used with the AWSDeploy tool to provision 1 or more applications on your EC2 server. It is also fully compatible with auto-scaling and failover scenarios.

You can test the install locally by executing the following command in the C:\Build directory.

msdeploy -verb:sync -source:package=Deploy.zip -dest:auto

Further Enhancements

The solution minimises the customisation of deployment scripts for different environments, and results in a solution much less coupled to Amazon AWSDeploy. It also works around the limitations of AWSDeploy and provides a much greater level of flexibility in how instances can be configured to support auto-provisioning.

This is an improvement, but we are still having to build different artifacts per environment which is not ideal. A further enhancement would be to leverage AWS UserData from within the PowerShell script. UserData is configured during provisioning and made available to each server instance via a web service at a fixed address:

http://169.254.169.254/latest/user-data

UserData is persisted during failover & scaling, and could be used to either feed the PowerShell script entire set-parameter files dynamically, or provide flags to indicate to the PowerShell script which pre-deployed set-parameter file to use per environment when installing packages. In this way a single artifact could be used across all environments inline with best practice, as well as a single install and deployment script.

XDMessaging 4.0

noreply@blogger.com (TheCodeKing) — Tue, 12 Mar 2013 01:19:42 PDT

XDMessaging .Net Library 4.0

Project source

Project documentation

Download demo

Introduction

The XDMessaging library provides an easy-to-use, zero configuration solution to inter-process communication for .NET applications. It provides a simple API for broadcasting and receiving messages across application domain, process, and even network boundaries.

The library allows the use of user-defined pseudo channels through which messages may be sent and received. Any application can send a message to any channel, but it must register as a listener with the channel in order to receive. In this way developers can quickly and programmatically devise how their applications will communicate with each other best to work in harmony.

The messages may optionally be propagated to other processes over a network automatically.

Installation

Install the library using Nuget.

PM> Install-Package XDMessaging

Advantages

The XDMessaging library offers some advantages over other IPC technologies like WCF, .Net Remoting, Sockets, NamedPipes and MailSlots. To begin with the library does not require a server-client relationship as there is no physical connection between processes.

With XDMessaging messages can be broadcast by multiple applications and instantly received by multiple listeners in a disconnected fashion. It's also worth noting that most of the existing IPC implementations require the opening of specific ports and somewhat painful configuration of settings to make work. With XDMessaging there is no configuration, the API determines where messages are sent, and which messages are received using pseudo channels.

Network Propagation

Network propagation is a feature of the library that allows messages to leverage HighPerformanceUI or Compatibility modes, whilst additionally distributing messages to a remote server. RemoteNetwork mode is used under the hood to transfer messages to disconnected servers, and messages are rebroadcast by a slave IXDListener instance using the original transport mode. If the slave instance terminates for any reason, then another listener instance will automatically take it’s place.

Using the Library

To use the library, create an instance of a IXDBroadcaster and use this to send a message to a named channel. You can then create an instance of IXDListener to receive messages on a particular channel. The channels are arbitrary strings chosen to represent a channel, and are not case sensitive. Before creating the broadcaster and listener instances you must first decide which transport mode you want to use for the library. There are 3 modes as follows and each has advantages over the other. Modes are abstracted from their specific implementations for extensibility, and loosely coupled. In order to use one of the modes the appropriate assembly must be referenced in the project in addition to XDMessaging.

Transport Modes

Compatibility: The default implementation uses file based IO to broadcast messages via a shared directory. A FileSystemWatcher is used within listener classes to monitor changes and trigger the MessageReceived event containing the broadcast message. This mode can be used in Windows Services, console applications, and Windows Forms based applications. Channels are created as separate directories on the file system for each channel. The temporary directories should be accessible by all processes, and there should be no need for manual configuration. To use this implementation add a reference to XDMessaging.Transport.IOStream in your project.
HighPerformanceUI: The default implementation uses the WM_COPYDATA Windows Message to copy data between applications. The broadcaster implementation sends the Windows Messages directly to a hidden window on the listener instance, which dispatches the MessageReceived event with the copied data. Channels are created by adding/removing Windows properties. This offers the most performant solution for Windows Forms based applications, but does not work for Windows Services, Console apps, or other applications without a message pump. To use this implementation add a reference to XDMessaging.Transport.WindowsMessaging in your project.
RemoteNetwork: By default this uses Amazon Web Services to implement a subscriber/publisher implementation for broadcasting messages over a network and interprocess. There may be associated costs involved in using this mode, and you will need to supply valid Amazon account credentials. This mode is used internally to send messages from other transport modes over the network when using network propagation mode. To use this implementation add a reference to XDMessaging.Transport.Amazon in your project.

Note: Messages broadcast using a particular mode can only be read by applications using a listener in the same mode. For example, Compatibility listeners cannot read messages broadcast in the HighPerformanceUI mode. It is possible to broadcast in several modes at the same time, see documentation for further details. See the included sample applications for more information on using the library in Windows Forms applications or within a Windows Service.

Messaging Demo

Download the source code and demo application from GitHub. Refer to user guide for more information.

Twitter OAuth authentication using .Net

noreply@blogger.com (TheCodeKing) — Mon, 29 Aug 2011 15:52:30 PDT

Source Code

TwitterOAuth.zip

Introduction

In this article I want to demonstrate how to implement OAuth authentication in .Net. I've previously written about my dislike of third party SDKs for social media integration and how we should leverage technology based solutions instead. One of the sticking points in doing this tends to be that implementing OAuth based authentication is relatively difficult compared with actually making the requests themselves. There is documentation available, but there seems to be a lack of .NET example code to go with it.

In keeping with my thoughts in previous articles I would recommend using open source OAuth based libraries to solve this problem, and again avoid resorting to third party Twitter/Facebook implementations which more strongly couple code to specific APIs. This keeps the solution more reusable and builds on specific technologies to better future proof your application.

I've also previously shown how client-side plugins can be used in combination with server-side code to speed development in this area. However sometimes authentication does need to be implemented purely on the server-side.

So how difficult is this?

It turns out implementing OAuth on the server-side in .Net isn't too difficult, the battle is getting the encoding and authentication signature right. With so few examples it can be a little daunting, so here's an example written in pure .NET using the official Twitter OAuth documentation and a bit of trial and error.

Background

The following example shows how to authenticate against the Twitter APIs using a registered Twitter application. Any interaction with the APIs when authenticated in this manner will behave as if coming from the Twitter account under which the application has been registered. It's therefore useful for sending out status updates or sending out notifications from a specific account.

Usually OAuth requires redirecting the user to a login screen to obtain an oAuth token which requires a bit more work. However when authenticating via a Twitter application this step is skipped as your application already has an oAuth token provided (access token). Whether you are using the application oAuth token or a user oAuth token, the following code can be used to authenticate against the twitter APIs.

The Code

The first step is to visit the Twitter developer section and register a new application. On completion you will be provided with a set of public/private keys which you will need the replace in the example below in order to run. The values I have used directly correspond with the documented example here. Make sure you replace them with your own.

var oauth_token           = "819797-Jxq8aYUDRmykzVKrgoLhXSq67TEa5ruc4GJC2rWimw";
var oauth_token_secret    = "J6zix3FfA9LofH0awS24M3HcBYXO5nI1iYe8EfBA";
var oauth_consumer_key    = "GDdmIQH6jhtmLUypg82g";
var oauth_consumer_secret = "MCD8BKwGdgPHvAuvgvz4EQpqDAtx89grbuNMRd7Eh98";

We also need to define some details about the request. This includes a unique oauth_nonce parameter which must be generated per request, and a timestamp.

var oauth_version          = "1.0";
var oauth_signature_method = "HMAC-SHA1";
var oauth_nonce            = Convert.ToBase64String(
                                  new ASCIIEncoding().GetBytes(
                                       DateTime.Now.Ticks.ToString()));
var timeSpan               = DateTime.UtcNow
                                  - new DateTime(1970, 1, 1, 0, 0, 0, 0,
                                       DateTimeKind.Utc);
var oauth_timestamp        = Convert.ToInt64(timeSpan.TotalSeconds).ToString();
var resource_url           = "http://api.twitter.com/1/statuses/update.json";
var status                 = "Updating status via REST API if this works";

The next step is to generate an encrypted oAuth signature which Twitter will use to validate the request. To do this all of the request data is concatenated into a particular format as follows.

var baseFormat = "oauth_consumer_key={0}&oauth_nonce={1}&oauth_signature_method={2}" +
                "&oauth_timestamp={3}&oauth_token={4}&oauth_version={5}&status={6}";

var baseString = string.Format(baseFormat,
                            oauth_consumer_key,
                            oauth_nonce,
                            oauth_signature_method,
                            oauth_timestamp,
                            oauth_token,
                            oauth_version,
                            Uri.EscapeDataString(status)
                            );

baseString = string.Concat("POST&", Uri.EscapeDataString(resource_url), "&", Uri.EscapeDataString(baseString));

Using this base string we then encrypt the data using a composite of the secret keys and the HMAC-SHA1 algorithm.

var compositeKey = string.Concat(Uri.EscapeDataString(oauth_consumer_secret),
                        "&",  Uri.EscapeDataString(oauth_token_secret));

string oauth_signature;
using (HMACSHA1 hasher = new HMACSHA1(ASCIIEncoding.ASCII.GetBytes(compositeKey)))
{
    oauth_signature = Convert.ToBase64String(
        hasher.ComputeHash(ASCIIEncoding.ASCII.GetBytes(baseString)));
}

The oAuth signature is then used to generate the Authentication header. This requires concatenating the public keys and the token generated above into the following format.

var headerFormat = "OAuth oauth_nonce=\"{0}\", oauth_signature_method=\"{1}\", " +
                   "oauth_timestamp=\"{2}\", oauth_consumer_key=\"{3}\", " +
                   "oauth_token=\"{4}\", oauth_signature=\"{5}\", " +
                   "oauth_version=\"{6}\"";

var authHeader = string.Format(headerFormat,
                        Uri.EscapeDataString(oauth_nonce),
                        Uri.EscapeDataString(oauth_signature_method),
                        Uri.EscapeDataString(oauth_timestamp),
                        Uri.EscapeDataString(oauth_consumer_key),
                        Uri.EscapeDataString(oauth_token),
                        Uri.EscapeDataString(oauth_signature),
                        Uri.EscapeDataString(oauth_version)
                );

We are now ready to send the request, which is the easy part. Note we must also disable the Expect: 100-Continue header using the ServicePointManager. Without this code the .Net sends the header by default which is not supported by Twitter.

var postBody = "status=" + Uri.EscapeDataString(status);

ServicePointManager.Expect100Continue = false;

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(resource_url);
request.Headers.Add("Authorization", authHeader);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
using (Stream stream = request.GetRequestStream())
{
    byte[] content = ASCIIEncoding.ASCII.GetBytes(postBody);
    stream.Write(content, 0, content.Length);
}
WebResponse response = request.GetResponse();

Summary

This example hopefully shows how OAuth can be implemented with fairly little effort. In the example provided I've kept everything inline for clarity, however in the real world you would obviously refactor the code into more sensible layers.

In this way it's possible to build some highly testable lightweight classes in order to generate the required message signature, make the requests, and handle the response.

ASP.NET Session End event fires immediately after Session Start - Resolved

noreply@blogger.com (TheCodeKing) — Sun, 28 Aug 2011 12:06:35 PDT

The Question

Whilst browsing StackOverflow I came across an old unanswered question from Feb 10 which caught my eye.

At first I was quick to dismiss it as nonsense, but then I started looking into it further and became interested. It seems a lot of people are reporting an issue whereby Session_End is sometimes called immediately after Session_Start.

This can be reproduced in .NET 2.0/3.5 using the following steps:

Create a new .Net 2.0 WebForms WebApplication
Configure session for InProc, and set the timeout to 1 minute
Add a break point on Session_Start and Session_End events
Debug the application and F5 to skip the first Session_Start break point
Wait until the Session_End fires after 1 minute
Refresh the browser, and notice Session_End is called immediately after Session_Start
Note that adding a session variable to the page OnLoad solves the issue

According to documentation, there are 2 activities that can cause the Session_End event to fire.

When Session.Abandon() is called

Immediately after the Session expires

In this example neither are true. So how can this be?

The Answer

Doing some research on the internet I found a clue in a blog post on session state. It revealed an interesting quirk about how session state is handled.

This is that ASP.NET will only create a new InProc session if the session is actually used by the application. If not then the session is not persisted, and it will not even issue a session cookie (where it doesn't already exist).

So here's what's happening.

When the browser is refreshed after the session timeout, the old session cookie is sent in the request. This causes ASP.NET to reuse the original session ID for a new session. However because at the end of the request the application has not yet used the session, ASP.NET affectively abandons it to avoid persistance and to free resources. This causes an early Session_End event.

Now because ASP.NET does not actually delete the pre-existing session cookie, every subsequent request is essentially restarting the session ID and repeating the sequence of events, such that Session_Start/Session_End are fired repeatedly until either the session is used or the cookie deleted.

The Fix

The issue has already been resolved in .NET 4.0. I suspect the session now remains persisted in the case of a session restart.

If your application is using .NET 2.0/3.5 and has expensive code hooked into the session start/end events, then I recommend you add a dummy session variable to your session start routine. This solves the problem by flagging the session as in use, and helps protect your server resources from excessive session events.

Children Of The Social Revolution

noreply@blogger.com (TheCodeKing) — Sun, 31 Jul 2011 12:56:50 PDT

Introduction

The social revolution is upon us, and we are all a part of it. Within our generation the world has changed. Individuals now form vast community networks which span the globe, and social media is prolific.

In the past decade the balance of power has shifted to the consumer through the uptake in social media. It affects how we search for content, how we consume content, and how we increasingly hold corporations accountable.

Beyond Our Origins

Facebook was the first player to take social media mainstream after it launched in 2004, but it wasn't the first of it's kind. Sites such as SixDegrees were around in the late 90s and many more niche sites briefly gained notoriety around the same time such as FaceParty in the UK; a site at one time more popular than eBay or Amazon with 7 million users^[1].

Before the newer generation of social networking sites, users were constrained to smaller isolated communities built around forums, newsgroups and IRC (Internet Relay Chat). Before these it all started with BBS (bulletin board system); a rudimentary system for exchanging data over dial-up connections in the 70s.

Even before wide availability of computers in the household in the 50s, underground groups came together through the phenomenon of phreaking. Originally involving exploits in the analog phone system, it allowed individuals to route free calls with home made electronic devices such as blue boxes. As well as being used for pranking, notably by the Apple founders Steve Wozniak and Steve Jobs, phreaking became a social outlet not constrained by physical location^[2]. In doing so small communities formed in the darkness via a network of party lines and long distance calls.

Going back even further amateur radio also allowed people to span great distances more legitimately, and formed niche communities from as early as the 20th century. What distinguishes this use of technology from the common telephone is that people were drawn together through common interests, rather than being centered around those already known to them.

It seems the human desire to reach out and form communities is in our DNA. It is the secret to our evolutionary success, yet it’s technology that empowers us beyond our humble origins. It not only gives us a voice, but ensures we are heard.

The Evolution Of Social

What’s interesting to observe looking back is how social media has evolved both in terms of acceptance within our society and in terms of the technology that takes it forward.

First the audience has changed. From the early days in the 70s, 80s, and 90s there was generally a stereotypical stigma attached to those who participated in the practice of online socialising. This stigma existed across all age groups with even young teenagers snubbing the practice. Slowly over generations as technology became increasingly more accessible and more integrated into everyday life the stigma dissipated. Online dating and gaming has helped take acceptance into new age groups and today social media is the norm embracing all generations. Within just the last year my entire family is now on Facebook, and I’m sure most of yours are too.

What this means is there are now almost no social boundaries left within the industry, at least within the western world. This makes it a huge market to tap into and puts the onus on the big players to gain our trust and keep us onside. In return they glean revenue from billions of page views via carefully positioned advertising.

The other interesting observation is that it’s not the older more established sites that are the most successful today. Instead it’s the youngest sites which have used smarter technologies and learnt from the failings of others that thrive, toppling the once dominating leaders in their wake. At the time of writing GooglePlus has just launched and already seeing unprecedented widespread adoption that could fast become a real threat to Facebook. It has already gained 20 million users in the first 3 weeks to become the fastest growing network of it’s kind^[3].

Social Power

It’s truly a fierce and highly competitive market which has changed the way social media engages with us. As a result it empowers consumers to heavily influence functional design based on our evolving expectations over privacy and security concerns. Through this process we have already achieved much greater transparency over how our data is shared. On platforms such as Facebook for instance, we no longer get rogue applications spamming our feeds with notifications at every conceivable opportunity. Instead third party apps are now relegated to the edges where they can only solicit our data.

Besides evolving with us, social media also empowers us beyond the platform. By giving us a voice and allowing us to form these vast social networks, it is changing the internet as we know it. Today the digital consumer is king. Any bad experiences can be instantly shared with thousands, retweeted, liked, and +1 to the masses. Suddenly our opinions matter. In the same way that advertising spreads it’s idealistic branding that companies spend millions to promote, social chatter is beginning to spread a more realistic viewpoint based on real experiences. Increasingly every customer interaction is key; both online and in the physical world.

Likewise whereas search engines were once the most important avenue for driving traffic and promoting brands, online social communities are now establishing their own independent content streams through content sharing. This is becoming a process of natural selection where popular content is not necessarily based on search placement.

Social media is the future of digital. It evolves with us, and it empowers us. We are truly children of the social revolution, and together through these online hubs we shape the future.

References

Facebook Integration via Hammock

noreply@blogger.com (TheCodeKing) — Tue, 30 Aug 2011 05:00:31 PDT

Source Code

FacebookDemo.zip

Introduction

Recently, I wrote about Social Media and strategies for integration. I intentionally didn't go into detail as I wanted to take a more philosophical view on our development processes. I made the distinction between frameworks where community driven libraries thrive, and evolving APIs where they often become the kiss of death for projects.

I believe we need to change the way we work with Social Media APIs and stop trying to create SDKs for these APIs in the technology of our choice. The technologies evolve too quickly and instead we need to embrace the core technologies, whilst keeping the integration layers lightweight.

In this article, I want to give an example of one approach by demonstrating some integration with Facebook. The key point here is that rather than using one of the many .NET libraries which wrap Facebook APIs, I'm instead using a generic library that is entirely decoupled from the Facebook APIs themselves.

Hammock - REST, easy

Hammock is an open source project built in .NET which facilitates the consumption of RESTFul services in a generic way. It offers support for asynchronous operation, query caching, mocking, and rate limiting among other features. There are of course other libraries which could equally be used.

The Code

The following shows an example of retrieving posts from a user's wall in ASP.NET. We could achieve the same functionality through JavaScript alone, but let's assume we want to do something with the data on the server. This requires that the user has granted the read_stream permission to the Facebook application and assumes the user is authenticated. In the source code provided, I've shown how to achieve both of these tasks using Social Plugins and JavaScript with minimal effort.

The first thing we need to do is create an instance of Hammock's RestClient class. This contains the base URI of the service EndPoint and can also contain other default properties such as a UserAgent string.

IRestClient client = new RestClient
{
 Authority = "https://graph.facebook.com/",
 UserAgent = "MyFacebookApp"
};

Next we create an instance of Hammock's RestRequest class. This allows us to make RESTFul calls to a particular path relative to the base EndPoint URI defined above. Using Hammock's API it's then simple to add request parameters to the call. In this case, the user's access_token is appended to the querystring to provide the required authorization.

// call https://graph.facebook.com/[userid]/feed
var request = new RestRequest
{
 Path = string.Concat(UserId, "/feed"),
 Method = WebMethod.Get
};
request.AddParameter("access_token", AccessToken);

Note the UserId and AccessToken values shown in the sample above are obtained from the currently authenticated Facebook user. In the source code provided, I'm passing these values from an ajax call triggered by a Social Plugin login event.

OK so far it's easy right? Any changes to the Facebook API means we can quickly add or remove parameters that are sent to the API.

The next step is to make the request and parse the response. This is done using the RestClient we created earlier.

var response = client.Request(request);

// check for errors and do something
if (response.StatusCode != HttpStatusCode.OK)
{
 throw new CommunicationException(
  string.Format("The EndPoint {0}{1} returned an unexpected StatusCode {2}",
  client.Authority, client.Path, response.StatusCode));
}

// this is the raw JSON response
var content = response.Content;

OK, so now the interesting bit, and this is where I come back to my point about keeping this lightweight and loosely coupled. We could use Hammock's deserializer to type the response directly to our own object, but that's too rigid for my liking. Instead I want to use the dynamic type. This will allow us to access the JSON properties in a loose fashion which can more easily be changed if the API is updated.

Now I couldn't figure out how to get Hammock's out-of-box deserialization to work with dynamic objects. It may be possible, and if anyone knows, please let me know. Instead I created my own JavaScriptConverter extension so I can use .NETs in-built JavaScriptSerializer class. You could also register your own custom deserialization classes via Hammock's interfaces but I've kept it simple.

// set up deserialization classes
JavaScriptSerializer serializer = new JavaScriptSerializer();
serializer.RegisterConverters(new[] { new JsonConverter() });

// deserialize the raw JSON
dynamic result = serializer.Deserialize<ExpandoObject>(response.Content);

// parse the results 
foreach (dynamic obj in result.data)
{
 Response.Write(obj.from.name + " wrote " + obj.message);
}

Static vs Dynamic Typing

The nice thing about ExpandoObject is that not only does it provide dynamic access to the response object to keep it loosely coupled, but it also implements IDictionary<string, object>. This means it can easily be converted to a typed object using JavaScriptSerializer. This is useful if we decide to move the code into a service layer and want to provide a strong typed model to our business layer. Only properties that are defined on both source and target will be mapped across in the conversion, other properties are simply ignored.

// define a class for post content
public class Post
{
 public string Id { get; set; }
 public string Message { get; set; }
}

foreach (var obj in result.data)
{
 // convert ExpandoObject to the Post class
 var post = serializer.ConvertToType(obj);

 // we now have strong typed access to content
 var message = post.Message;
}

Running The Demo

To run the demo code included, you should replace the Facebook AppId in value FBHelper class with your own Facebook application key. See Facebook documentation for more details on how to obtain this.

You will notice in the source code provided, that I've created a custom expando class called JsonExpandoObject. This is because ExpandoObject throws exceptions if you try to access a property that doesn't exist. My version returns null instead and therefore works much better with JSON.

Summary

So to summarize, generic third party libraries such as Hammock can be used to speed integration with Social Media APIs. This has the advantage over third party Social Media SDKs in that it keeps the codebase more loosely coupled. In doing so, it allows developers to build applications around long established core technologies and produces a codebase more adaptive to change.

By implementing late binding via dynamic types, you can further decouple hard dependencies on APIs. This is particularly useful for lightweight manipulation of content within the data layer. For passing objects to the business layer, I've shown how JavaScriptSerializer can be used to quickly convert dynamic types into static types as required.

Static types are of course checked at compile time and offer many advantages over dynamic types. The trick here is that we are able to leverage static types in our business layer, but map/convert from objects which are late-bound to API code. This mapping/conversion code becomes the single touch point for maintenance when APIs change.

The example included also shows how we can leverage the emerging Social Plugins and JavaScript APIs which are more resilient to change. These can be used for core functionality and should always be favoured where possible.

Twitter The Undiscovered Truth

noreply@blogger.com (TheCodeKing) — Thu, 14 Jul 2011 13:05:13 PDT

Suspect Activity Detected

Here's a thought. I recently became active on Twitter. Having never really used it before I thought I should embrace the technology in order to help me better understand what is quickly becoming my industry.

Within 3 days I have about 40 followers. Fair enough I posted my Twitter name on various sites and I expect to get some followers who might be interested in my articles. But here's the thing, I'm getting an usual amount of young ladies following me and I have to wonder why.

Yes I know what you're thinking, but even with my good looks I'm still a geek writing about geeky things. So why on earth would an 18 year old babe from California be interested in anything I have to say?

A Pessimist's Theory

So heres my pessimistic theory. If I were a shrewd entrepreneur who saw an opportunity to provide Twitter accounts pre-populated with followers in the magnitude of 10s of thousands of users, then here's what I'd do.

I'd write a bot to harvest Twitter user accounts by auto following users just often enough to stay under the radar. I'd set up that follower account with pictures of hot girls with names like Thumper and Sandy and give them cute profiles like "I love lollipops" and "looking to make friends".

For every 5 accounts I follow I'd probably get about 3 follow me back. Over time my accounts would ripen and be ready for market. A quick change in username, picture and profile, and I'd be ready to cash in the account to some high flying exec for $$$. A CEO profile to boast status, power, and influence among their peers, or a powerful marketing tool providing an instant voice to 1000s.

The Million Dollar Question

Now it's just a theory, and it could well be I'm just too damn good looking for my own good, who's to say! However a quick search for Twitter accounts for sale and the evidence speaks for itself.

Now here's the million dollar question. Do I publish this article and inform others of my findings in a selfless stride to make the world a better place? Or do I get straight to work building a bigger & better bot of my own!

Screw it I'll do both ;)

The Great Social Change

noreply@blogger.com (TheCodeKing) — Sun, 10 Jul 2011 17:25:30 PDT

Social Change?

Wow what a title. No you haven't stumbled upon a piece of broad-sheet journalism reflecting on the evolution of our society in modern time. This article is about something far more important, Social Media! Or rather, Social Media and strategies for integration.

I will be referring to Social Media in the broad sense as I think it's more relevant than focusing on Facebook or Twitter in particular. Of course a lot of what I'm referring to is based on my direct experiences with the two big players.

Full Disclosure

I must admit, although in my work I build a lot of Social Media enabled sites, this is something that usually gets implemented by front-end web developers. It's not something I have had to deal with too often in terms of the software engineering side of things, but times are (rightly) changing. As we increasingly push boundaries with Social Media it can no longer be considered a value-add or an after-thought. Social Media is forever ingrained in our lives and is becoming the hub for our online experiences. More than ever it's crucial to understand these technologies and how to harness them. As such I've been researching Social Media to get a better idea of what's out there and how we can leverage them in our projects without introducing significant risk.

The Colour Of Chaos

So what's out there? Well the bad news is it's chaos! The good news is its getting better.

So what do I mean by chaos? Well Social Media has evolved rapidly, not only the technologies themselves but also our understanding and acceptance of them. The result is a sway of privacy and policy changes influencing functional change and several generations of progressive APIs leaving a trail of misinformation.

Now change is a great thing, it's the key the to success and acceptance of Social Media in our life's, but it gives us developers a terrible time. Just search for samples and tutorials for working with Social Media and there's a vast sea of misinformation out there referring to out-dated APIs, functionality long since removed, and links to abandoned libraries. What's more is that the applications we build today, don't necessarily work tomorrow.

OK so we have the official documentation right? The problem with those Social Media giants is that they are busy people. They spend a lot of time working to enrich our lives, inventing new features, refining APIs, and re-organising their sites. All this takes its toll on documentation which can be quite slim and sparse of specific examples.

The Kiss Of Death

I don't need to tell you, we're a friendly bunch us developers. We love to share our ideas and contribute to open source communities. Usually this is a great thing, it brings a competitive edge to the industry; it broadens our experiences; makes us proud; and allows us to achieve so much more of our potential.

Now the thing about Social Media is that they certainly aren't in the business of creating frameworks. And where those lovely community libraries can lead to faster development times and reduce costs when working with frameworks, they can be the kiss of death when working with third party APIs. It's an unnecessary layer between you and a moving target, and if you can't adapt fast enough, you're dead in the water.

The internet is littered with these good-willed projects that once sort to enrich our lives by bringing Social Media to our particular brand of technology. They now lay abandoned where the effort in upkeep overtook good-will or where functional changes have crippled the once sparkly widget that we all craved. Whereas Apple's move to block third party libraries on their platform was unexpected, it's clear that this was some clever thinking on their part to keep the community and platform nibble. The truth is Apple are more focused on producing APIs than developing frameworks, and it makes perfect sense.

Now there are some good libraries out there, notably the Facebook C# SDK and Twitterizer, but I question the need for these and personally I believe it's only time before these go the way of their predecessors. Social Media has grown up and their APIs are built on simple technologies, but unlike frameworks they will continue to adapt and be redefined as the community demands.

Write Code Not Libraries

So how do we deal with change whilst still enabling cost effective development? Well as it evolves Social Media is becoming much smarter and a more robust than ever before. Facebook is moving forward with Social Plugins, a technology that removes previous coupling between developer code and the Facebook hub. This approach now allows us to take Social Media completely off-platform where it can self-update and become much more resilient to change. We can utilize the Facebook JavaScript API directly without compiling a single line of code, and share consistant experiences both within and outside of Facebook from a single codebase.

The other emerging feature of Social Media APIs are the streaming content services which move away from more traditional RESTFul integration techniques. This allows much more efficient B2B communication, and the handling of much higher volumes of data than ever before.

A New Hope

To really leverage Social Media and take advantage of the advancements being made we need to create more lightweight adaptive solutions that build on the core technologies, and not the APIs.

OK so what do I mean by that? Well both Twitter and Facebook are built on RESTFul technologies. Rather than looking for C# libraires (for example) that provide an easy hook into the platform and are tightly coupled to a particular version of the API, I suggest effort is better spent investigating, developing, and implementing RESTFul libraries written in C# (for example), that make it easier to call the Social API. This promotes code reuse without coupling the application to one approach or one technology. Any changes to the API can then easily be rectified by tweaks to the high level codebase without functional changes.

The Capitalists

Lastly I want to talk about another emerging solution to integration woes. These are service based companies that are springing up. Many of these are in their infancy with Alpha or Beta products but their offerings are attractive and come at a premium. These companies take on the integration effort across numerous Social Media Networks so you don't have to. Unlike community based projects this is becoming big business and they are forming tight partnerships with the big players. Whereas they may not be useful for helping you develop your Facebook application or run your Twitter campaign they can really help you access the results and get out the raw data. Common features include the aggregation and normalization of data across multiple Social Networks, and the provision of that data which you may not otherwise have access to.

For example access to 100% of the unfiltered Twitter firehose (i.e. all the data) is very difficult to achieve due to the way the APIs are request limited and the way public streams are filtered to reduce noise. This data is massively valuable so of course companies are keen to offer solutions at a premium. Prices vary and usually charged on a per message basis. These services could prove themselves very valuable, although I suspect we may start to see these offers being made more directly in the future.

Key To Success

The Social APIs are constantly in flux and we shouldn't try to pin them down with middleware. The nature of Social Media is that it's driven by the will of the community and natural evolution. It requires a different thought and development process than traditional frameworks. Embrace the core technologies and build on those rather than the APIs directly. Keep the actual integration light and adaptive, watch out for depreciated APIs. Embrace the new integration mechanisms on offer such as Social Plugins and streaming services. Keep an eye out for data mining services to help aggregate and normalize your data.

In Closing

The one thing Social Media has taught us is that it's no longer acceptable to force our messaging on the user, or jump on every opportunity to spam them with content. This is reflected in the emerging APIs which curb our dark desires. We need to educate our clients on this and find new ways to reach out. The message is clear, it's about quality, relevance and user consent.

Build it well and they will come!

WinRM in mixed domain environments

noreply@blogger.com (TheCodeKing) — Fri, 08 Jul 2011 11:01:30 PDT

Introduction

Windows Remoting works perfectly for same domain situations, and the set-up is relatively straight-forward. It's extremely powerful when it works, and offers a highly flexible way to securely execute commands remotely.

Problems arise however when trying to use WinRM in mixed domain environments, or where only one machine is on a domain. As I discovered this requires some additional configuration steps outlined below.

Enabling Remoting

Enabling WinRM between computers on the same domain is straight forward by running the following command in an elevated Powershell console on the remote server. This enables WinRM and configures the firewall so that it can accept incoming commands.

Enable-PSRemoting

Once this is done commands can then be executed against the machine remotely as follows.

Invoke-Command -ComputerName RM-SERVER -ScriptBlock {Get-Process}

This example will list processes on the remote machine named RM-SERVER using the Get-Process cmdlet.

Mixed domain

Mixed domain environments require some additional configuration to get working. To do this the following command needs to be executed on the remote server. This will allow the client server to authenticate from a different domain, or with an account local to the remote server.

New-Itemproperty -name LocalAccountTokenFilterPolicy -path HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System -propertyType DWord -value 1

With this configuration it's now possible to authenticate and execute a command remotely with explicit credentials.

$credential = Get-Credential
Invoke-Command -ComputerName RM-SERVER -ScriptBlock {Get-Process} -Credential $credential

Or create the credential explicitly in script as shown below.

$securePassword = ConvertTo-SecureString "Password" -AsPlainText -force
$credential = New-Object System.Management.Automation.PsCredential("domain\username",$securePassword)
Invoke-Command -ComputerName RM-SERVER -ScriptBlock {Get-Process} -Credential $credential

Troubleshooting

WinRM cannot process the request

Beware, I spent several hours on this. No matter what I did I kept getting the following error. It turns out this was caused by anti-virus software running on the remote server which had to be disabled.

Connecting to remote server failed with the following error message : WinRM cannot process the request. The following error occured while using Kerberos authentication: There are currently no logon servers available to service the logon request.
Possible causes are:
  -The user name or password specified are invalid.
  -Kerberos is used when no authentication method and no user name are specified.
  -Kerberos accepts domain user names, but not local user names.
  -The Service Principal Name (SPN) for the remote computer name and port does not exist.
  -The client and remote computers are in different domains and there is no trust between the two domains.
After checking for the above issues, try the following:
  -Check the Event Viewer for events related to authentication.
  -Change the authentication method; add the destination computer to the WinRM TrustedHosts configuration setting or use HTTPS transport.
Note that computers in the TrustedHosts list might not be authenticated.
   -For more information about WinRM configuration, run the following command: winrm help config. For more information, see the about_Remote_Troubleshooting Help topic.
    + CategoryInfo          : OpenError: (:) [], PSRemotingTransportException
    + FullyQualifiedErrorId : PSSessionStateBroken

Trusted hosts

The remote server can be configured to allow commands from only specific hosts. To set this run the following in an elevated Powershell console on the remote server. By default this is set to allow any host (*).

Set-item wsman:localhost\client\trustedhosts -value RM-Client1,RM-Client2

The line above configures the remote server to allow only commands from machines RM-Client1 and RM-Client2.

Firewall configuration

WinRm requires port 5985 for http, or port 5986 for https. The Enable-PSRemoting cmdlet will auto-configure the Windows software firewall, but do ensure these ports are accessible across your network infrastructure.

Blogger Hacks Part 2: Creating a navigation structure

noreply@blogger.com (TheCodeKing) — Wed, 09 Feb 2011 14:49:42 PST

Introduction

In Part 1 I showed how to create a static homepage. In this article I'll explain a way to gain more control over the navigation of your site. This is for users like myself who may want to get more out of Blogger than purely the journal functionality.

Update

This article has been updated to reflect the techniques I'm currently using on this site.

The Principle

OK so here's the basic principle, relax it's not rocket science.

With a static homepage in place users entering the site will be faced with a single post (rather than usual list of latest posts). If we then remove the Blog Archive gadget from the template, then we can remove the default blog based navigation. This leaves us with a clean slate to build our own navigation structure.

To remove the default navigation, login to the admin console and navigate to Design -> Page Elements. Locate the Archive gadget, click edit and then remove.

Simple Navigation

This is the technique I'm currently using to create the menus on the left-sidebar of my site. It's done by simply adding a Links gadget to the Blogger template, and then using the gadget to create links to the various posts as desired. You can add multiple gadgets to the template in order to create multiple menus.

With this technique it's a good idea to also disable the auto-archive function of Blogger. Do this by logging into the admin console and navigating to Settings -> Archiving. Set the Archive Frequency setting to "No Archive".

A further enhancement you may wish to implement is to disable the footer navigation via some custom CSS. This further restricts navigation in the site, and avoids the ugly list view.

Static Pages

A quick & easy way to create 2-tier navigation is to create a post (or static page) that simply contains links to other posts in your site. This can then be added to a menu as described above to create a second level navigation.

Personally I prefer not to use this technique as think the experience is a little clunky.

Label Navigation

This technique creates a similar experience to the Static Pages technique described above as is no longer used on my site. I've included it for reference, but don't recommend it.

To implement label based navigation use the Blogger designer to add a Labels gadget to your template. When creating posts you can then label them as desired to form the navigation structure.

Now by default the gadget will display each label with a post count, and provide a link to all posts with the corresponding label. To avoid this unwanted list view of posts you can implement a hack to create an interstitial navigation page. This is a more advanced set-up so you should only proceed if you are comfortable editing the Blogger template and understand CSS.

The following steps implement conditional CSS based on the view type, and is used to hide the post body content in the list view to form a list of formatted links.

To implement the behaviour login to the console and navigate to the template editor Design -> Edit HTML.

Backup your existing template before making changes.

Using the editor scroll down to the <head> tag, and insert the following code directly beneath this line. You may need to change the CSS to work with your particular template.

<style>
   <b:if cond='data:blog.pageType == "item"'>
  .post-footer, .post-body 
  { 
   display: inline;
  }
 <b:else />
  .post-footer, .post-body 
  { 
   display: none;
  }  
 </b:if>
</style>

Posts without labels do not appear in the categorized navigation menu, and therefore we can use these hidden pages as static pages which we link to from other posts.

Summary

The techniques described above vary greatly in complexity and can be used together or individually to customise the Blogger navigation. This can help change the look & feel from a standard blog site into a more conventional website structure. Hopefully the sample code included also gives you some ideas for how you can further customise the Blogger templates using the markup. See here for more info: http://code.blogger.com.

Blogger Hacks Part 1: Creating a static homepage

noreply@blogger.com (TheCodeKing) — Fri, 11 Feb 2011 02:01:27 PST

Introduction

After searching around for somewhere to host my site I finally settled with Blogger. Why? Because I decided I preferred the choice of templates and in particular the drag-drop gadgets and page elements.

What I didn't like is that it is purely a blogging site. This is quite restrictive in terms of content as I really wanted the ability to create more of a website than a blog.

Well after a bit of trial and error, it turns out the new Blogger templates are fairly easy to 'hack', and as such I've managed to get the best of both worlds. I have the cool WYSIWYG gadgets and templates of Blogger, but have also fashioned a static home page and navigation structure.

Update

This article has been updated to reflect the simplified technique that I'm currently using for this website. There are also several alternative techniques which involve editing the Blogger template directly, or by making use of the new static pages. I've briefly described these solutions below for reference.

Simplified Technique

This technique is simply a Blogger setting which allows a user to limit the number of blogs displayed on the default page. By changing this setting to 1, a static homepage is created which will always display the most recent post. By manually configuring the publish dates of the blogs you can then control which content will will displayed on the homepage.

To set the post limit login to the Blogger console and navigate to Settings -> Formatting. Change the "show at most" setting to "1 post".

Next override the publish date of the post you wish to display on the homepage. This can be found in the Post Options menu at the bottom of the blog editor.

Template Redirect Hack

An alternative technique I've previously written about involves editing the Blogger template, and adding code to redirect to a URL of choice. This has the advantage that you don't have to manually edit the post dates every time you create a new post, but does break the Blogger designer view and is not SEO friendly.

To implement the redirect on the homepage login to the console and navigate to the template editor Design -> Edit HTML.

Backup your existing template before making changes.

Using the editor scroll down to the <head> tag, and insert the following code directly beneath this line. Replace http://mysite.com/post.html with the URL you wish to use as your homepage (this should be the URL of a specific post, or you could use a static page now supported by Blogger).

<b:if cond='data:blog.url == data:blog.homepageUrl'>
       <meta content='0;url=http://mysite.com/post.html' http-equiv='refresh'/>
</b:if>

The code above causes a client-side redirect as soon as the page loads. If you really want to go all-out and improve the implementation further, you can also include the following edits. This will reduce the time taken for the initial page-load prior to the redirect and further improve the user experience. It also provides a manual link for users with JavaScript disabled. I'd recommend only making these changes if you are comfortable editing HTML.

Using the editor scroll down and locate the <body> tag in the template. You now need to wrap the entire contents of the <body> tag with the following code. This is everything inside the <body> tag and the </body> tag.

<body>
    <b:if cond='data:blog.url != data:blog.homepageUrl'>
          [existing template code]
     <b:else/> 
           <p> If the page doesn't automatically refresh click <a href='http://mysite.com/post.html'>here</a>.</p>
     </b:if>
</body>

Remember that with this technique in place you will not be able to use the Blogger designer to manage your gadgets. To do this you will need to temporarily disable the hack.

Summary

That's all there is to it. In the next article I'll explain how to set up a navigation structure within Blogger.

AutoObjectBuilder.Net

noreply@blogger.com (TheCodeKing) — Sat, 09 Feb 2013 16:58:47 PST

AutoObjectBuilder.Net Beta

Introduction

AutoObjectBuilder.Net is my latest open source project designed to assist with Unit testing by automatically building simple and complex object graphs with predictable test data.

Now available on NuGet.

PM> Install-Package AutoObjectBuilder

Features

Recursive instance creation - injecting constructor dependencies as required
Able to create internal, protected and constructor-less instances
Auto generation of interface/abstract class proxies
Provides instances recursively populated with predicable test data
Provides a fluent interface for overriding default construction behaviour to cover different edge cases
Auto populates ICollection, IList, IDictionary, and Array types with a configurable number of items
Enables use of custom configuration via extension methods.

The library can be used to automatically create and build objects recursively, populating them with predicable test data as they are constructed. These instances can then be used to compliment other Mocking frameworks, by allowing them to be injected as Mocked return values. This avoids bloating unit tests with extensive set-up code for complex objects.

The following simplistic example shows how the library might be used with the Moq framework to return an auto-generated instance pre-populated with test values. The test takes advantage of the fact that the Username/Password members of the generated instance are predicable, and therefore is able to validate the application logic within the UserService Login method.

     IPerson person = Auto.Make<IPerson>().Object;

     var moq = new Mock<IUserRespository>();
     var moq.SetUp(o => o.FindUser(It.IsAny<int>())).Return(person);

     UserService service = new UserService(moq.Object);
     var loggedInUser = service.Login("Username", "Password");

     Assert.That(loggedInUser, Is.EqualTo(person));
     Assert.That(loggedInUser.Username, Is.EqualTo("Username"));
     Assert.That(loggedInUser.Password, Is.EqualTo("Password"));

Making Log4Net work with the .Net Client Profile

noreply@blogger.com (TheCodeKing) — Sun, 12 Sep 2010 03:02:37 PDT

Log4net for Client Profile

log4net.zip

Background

I've recently been working on a desktop application targeted at the Microsoft .Net Framework 4 Client Profile and ran into a problem using log4net. The issue is that the .Net Client Profile does not include any web related assemblies which are required by the log4net assembly. In particular for the AspNetTraceAppender class.

The result is that the C# compiler cannot resolve the log4net references and this results in a "reference not found" compiler error. When using Visual Studio it appears as if it cannot find the log4net dll when the project is built, but it's actually caused by missing dependencies. The error displayed is as follows:

Error 1 The type or namespace name 'log4net' could not be found (are you missing a using directive or an assembly reference?)

There are some open tickets (LOG4NET-174, LOG4NET-233) raised against this issue, but there is no indication of when (or if) this will be resolved.

The Solution

Luckily log4net is open source and the solution is relatively straight forward, although perhaps not entirely obvious. The following steps will allow you to build a custom version of log4net that will work with an application targeted at the 3.5 or 4 Client Profile.

Download the log4net source
Open & upgrade the solution using Visual Studio 2010
Remove the System.Web project reference
Exclude Appender\AspNetTraceAppender.cs class from the project
Add a reference to System.Configuration
Navigate to Project -> log4net properties, and select the application tab
Change the target framework to .NET Framework 3.5 Client Profile
Select the Build tab, and change the configuration to Debug
Under Conditional compilation symbols change this to NET;NET_1_0;NET_2_0;
Change the configuration to Release
Under Conditional compilation symbols change this to STRONG;NET;NET_1_0;NET_2_0;
Edit the AssemblyInfo.cs class and update the AssemblyKeyFile attribute with a valid strong key
Compile the project in Release mode and distribute the new assembly

Note log4net must be compiled for 3.5 Client Profile and not 4 Client Profile. This allows the assembly to work correctly with both versions of the framework. For some reason compiling for 4 results in a runtime error deep with the framework which I didn't have time to resolve.

For convenience I've provided a version of the log4net dll compiled for client profile via this page, but be aware that the strong name does not match that of the official release.

Creating a localized Windows Installer & bootstrapper: Part 1

noreply@blogger.com (TheCodeKing) — Sun, 22 Aug 2010 11:40:11 PDT

Part 1: Introduction & Setup

Part 2: Defining Conditions

Part 3: Localization

Part 4: Provision Prerequisites

Download Source Code

Introduction

It's been a while still my last article so I thought I'd write about my recent experiences with Wix for creating Windows Installer packages. I last used Wix 1.0 about 4 years ago with great results and had forgotten how satisfying it is to use. Unfortunately at the time of writing Wix 3.5 isn't out, but it sounds like there are some great additions in the pipeline including the much anticipated Burn utility (a bootstrap compiler), as well as Votive support for Visual Studio 2010.

In this series of articles I want to provide a complete end-to-end solution for deploying a .NET application based on my recent experiences. The solution provided is based around the following real-world requirements:

Provide a MSI installer which deploys a .NET application
Use NGen to pre-cache .NET assemblies for faster start-up times
Only allow the installer to run on a specific OS
Only allow the installer to run based on the results of an WMI lookup
Only allow the installer to run if prerequisites are installed (.NET 4.0)
Inject version info and other specifics into the installer from a build script
Provide localized versions within a single MSI installer and auto detect the user language
Create a localized bootstrapper as a single file that installs the necessary prerequisites (.NET 4.0)

Note because in my example I'm targeting a Visual Studio 2010 project which isn't supported by the Wix 3.0's Votive project, I'm instead using a NAnt script to automate the build process.

Technology Stack

Note about GUIDs

It's important to generate your own GUIDs when working with Wix to avoid conflicts with other products. As it's common to copy & paste examples, I've deliberately masked my GUIDs to prevent any duplicates ending up in production code. To work with any of the examples in the solution, be sure to replace all ids of the format NEWGUID-XXXX-XXXX-XXXX-XXXXXXXXXXXX with your own unique GUIDS. You'll find these have been factored into Config.Wsi for convenience.

Solution Overview

The MSI installer and EXE bootstrapper are produced from a single NAnt script. The script performs the following steps which are described in more detail throughout this series of articles. To build the project execute the included build.bat file.

Use HEAT to produce a Wix component fragment
Transform HEAT output to inject NGen elements
Use CANDLE to compile the Wix project, injecting build variables
Use LIGHT linker to generate an English version of the MSI
Use LIGHT linker to generate a language variation of the MSI
Use TORCH to create a transform (delta) file by comparing the output of steps 4 & 5
Use WISUBSTG.VBS to embed the MST into the original MSI file
Repeat steps 5-7 for each language supported
Pre-process the dotNetInstaller config file to inject build parameters
Compile the bootstrapper

Using HEAT

The HEAT utility is used to generate Wix code fragments. The solution uses HEAT to create a Wix component fragment that describes the target files that will be deployed by the installer.

As we want the installer to deploy these files to the user's chosen installation directory, we need to nest the generated component element within a specific directory node that relates to this location. Luckily HEAT provides an argument for this, and we just need to pass the correct directory id into the tool using the -dr parameter. As we will be using the WixUI library the installation directory id must be set to WIXUI_INSTALLDIR.

heat dir MyProject\bin\Release
     -o Component.wsx
     -dr WIXUI_INSTALLDIR
     -cg AppFiles 
     -var var.InputSourceFolder
     -g1 -gg -srd -sfrag

We also pass the -cg argument so that HEAT writes out a generated ComponentGroup with a specific id. This allows us to reference it from the main Wix project and include it in a Wix feature. The var argument tells HEAT to swap the absolute paths to the target files with a custom variable var.InputSourceFolder. This allows us to define the location of the target files at compile time.

Provision NGen

The Native Image Generator (Ngen.exe) is a .NET tool that improves the performance of managed applications start-up time by pre-caching a compiled image. Normally with .NET applications the JIT compiler compiles the image on first run and therefore causes an initial delay. By running NGen against the application during installation the compiled .NET application image is pre-cached and will therefore execute faster on first launch.

Wix comes with the WixNetFxExtension.dll extension that can be used to provision the NGen tool. To use the extension we just add the netfx namespace to the Main.wxs project file, and add the NativeImage element to any file nodes that we want to pre-cache. Lastly we must ensure that we pass the extension to CANDLE during compilation.

The following shows how NGen is provisioned as a child of a parent file node within a component definition.

<Component Id="..." Guid="....">
   <File Id="..." KeyPath="yes" Source="$(var.InputSourceFolder)\MyAppLibrary.dll">
      <netfx:NativeImage Priority="1" Id="MyAppLibrary.dll" AppBaseDirectory="WIXUI_INSTALLDIR" />
   </File>
</Component>

However there's a problem, as HEAT is used to generate the component fragment dynamically, how can we then inject our NativeImage nodes?

Well luckily Wix comes to the rescue in the form of an additional parameter that can be passed to the HEAT tool. This parameter describes an optional XSLT file that is applied to the fragment during output. This is used in the solution to inject the NativeImage nodes into the component fragment for all files with an extension of DLL or EXE.

Our HEAT command now looks something like this:

heat.exe dir MyProject\bin\Release
     -o Component.wsx
     -dr WIXUI_INSTALLDIR
     -cg AppFiles 
     -g1 -gg -srd -sfrag
     -t:Components.xslt

WixUI Library

In the example solution provided I'm using the stock WixUI_MONDO interface provided by the WixUI library. This is simply included in Main.wsx with the following reference. In doing this we must include the WixUIExtension.dll extension when executing LIGHT.

   <UI>
      <UIRef Id="WixUI_Mondo" />
      <UIRef Id="WixUI_ErrorProgressText" />
   </UI>

In our solution we also swap out the stock images & agreement files used by the library dialogs by overriding some WixUI variables.

    <WixVariable Id="WixUIBannerBmp" Value="binary\banner.bmp" />
    <WixVariable Id="WixUIDialogBmp" Value="binary\dialog.bmp" />
    <WixVariable Id="WixUILicenseRtf" Value="binary\EULA_en-gb.rtf" />

When using WixUI_Mondo the target installation directory defined in the source code must use an id of WIXUI_INSTALLDIR. This relates back to the reference in the auto-generated Component.wsx file, and allows a user to override the location via the WixUI interface. If the user does not customize the installation location then files are deployed to the folder within Program Files as defined by the fragment below. This is using custom variables injected at compile time to determine the default folder names.

<Directory Id='TARGETDIR' Name='SourceDir'>
    <Directory Id='ProgramFilesFolder' Name='PFiles'>
      <Directory Id='ManufacturerDir' Name='$(var.CompanyName)'>
        <Directory Id='WIXUI_INSTALLDIR' Name='$(var.FriendlyName) v$(var.ProductVersion)' />
      </Directory>
    </Directory>
      ... 
</Directory>

Using CANDLE

The Wix CANDLE utility compiles the Wix source files into a intermediate file format with a WixOBJ extension. Build parameters can also be injected at this point as Wix custom define variables. We can use these to pass in the product name and version number, as well as the location of the target files to be included by the installer.

The CANDLE command looks something like this:

candle.exe -o bin\Release\
     -ext WixNetFxExtension.dll
     -dInputSourceFolder=MyProject\bin\Release\
     -dFriendlyName="My App"
     -dVersion=1.0.0.0
     Main.wxs 
     Component.wxs

In the solution provided you'll note I'm using a NAnt task provided by Wix for the CANDLE command. The build script is able to pass in the custom variables from build properties which could originate from a build server such as CruiseControl.

Using LIGHT

LIGHT is the linker used to generate the MSI file from the intermediate WixObj libraries to produce a MSI file. The command line to do this in our solution would be as follows, but again in the solution we are actually using Wix NAnt task instead.

light.exe -o bin\Release\Setup.msi
     -ext WixNetFxExtension.dll
     -ext WixUIExtension.dll
     Main.wixobj 
     Component.wixobj

Summary

In this part we have created a fully functional Windows Installer file for deploying the .NET application. In Part 2 I'll discuss adding condition checks to enhance the installer further. Part 2.

Creating a localized Windows Installer & bootstrapper: Part 4

noreply@blogger.com (TheCodeKing) — Sun, 22 Aug 2010 10:08:00 PDT

Part 1: Introduction & Setup

Part 2: Defining Conditions

Part 3: Localization

Part 4: Provision Prerequisites

Download Source Code

Introduction

In Part 1,2 and 3 we created a localizable MSI installer using the Wix tools. We also created conditions to ensure that the application can only be installed on Dell hardware running Windows 7 and the Microsoft .NET Framework 4 Client Profile. In our original requirements we stated that this must be deployable as a single file, and should install any prerequisites specifically .NET 4. To achieve this we need to create a bootstrapper that will preform the prerequisites installations from unmanaged code. The bootstrapper will be deployable as a single self-contained EXE with the original MSI file embedded inside.

I investigated many utilities for creating a bootstrapper, including the Wix SETUPBLD tool, and the MSBuild GenerateBootStrapper task. In the end I settled on a powerful free utility called dotNetInstaller. This allows the creation of highly configurable and localizable bootstrappers which can be compiled from the command line.

DotNetInstaller

DoNetInstaller includes several utilities for authoring and compiling the bootstrapper. The bootstrapper is compiled from a single XML config file by the InstallerLinker.exe tool. To author the configuration file an editor is provided in the form of InstallerEditor.exe.

Using the editor language variations can quickly be set up through the UI and prerequisites defined by adding child nodes. Any prerequisite installers can either be embedded directly into the bootstrapper or a URL provided from which they will be downloaded and installed. In our case we embed the Web Installer for the Microsoft .NET Framework 4 Client Profile. This will launch the installation process and download the necessary components on demand to keep the size of the bootstrapper down.

Once the prerequisites are installed dotNetInstaller allows us to define a custom command to launch our own embedded MSI file using the complete_command property. We can set up different commands for different languages and therefore explicitly pass a transform to the MSI file after the prerequisite checks. If all prerequisites are found we can hide the bootstrapper interface altogether and simply launch the MSI file.

DotNetinstaller includes it's own auto-language detection based on either user or system settings. To leverage this we can refer to a #LCID token in the config file that will evaluate to the detected LCID at runtime. This allows us to pass the LCID explicitly to our MSI in order to launch a particular language variation.

Through the command line we can still inject properties into our MSI for overriding the hardware check for example. The syntax for this would be:

setup.exe /CompleteCommandArgs "SKIPCHECK=1"

The features of dotNetInstaller are too many to cover in this article but there is plenty of documentation and it's very feature rich.

Build Automation

To complete our bootstrapper we really want to integrate this into our build script and inject parameters such as product name and version. To achieve this we can use a NAnt script that performs token replacement on the dotNetInstaller config file prior to compilation. I could have equally used XML manipulation, but NAnt makes the token replacement somewhat easy.

This is implemented using a filterchain with the NAnt copy task as shown below. The original config file is copied to a new location where it is modified and then parsed to the installer linker.

<copy tofile="${installer.dir}\obj\${build.configuration}\Configuration.xml"
          file="${installer.dir}\bootstrapper\Configuration.xml" overwrite="true"
          inputencoding="utf-8"
          outputencoding="utf-8"
          >
      <filterchain>
        <replacetokens>
          <token key="APPLICATION_MANUFACTURER" value="${project.companyname}" />
          <token key="APPLICATION_NAME" value="${app.name}" />
          <token key="APPLICATION_VERSION" value="v${version}" />
          <token key="INSTALLER_VERSION" value="${installer.version}" />
          <token key="INSTALLER_COPYRIGHT" value="Copyright © ${datetime::get-year(datetime::now())} ${project.companyname}" />
          <token key="MSI_PATH" value="${installer.dir}\bin\${build.configuration}\Setup.msi" />
        </replacetokens>
        <tabstospaces />
      </filterchain>
    </copy>

Summary

In this final part we have fulfilled our original requirements and provided a build script for dynamically generating a localizable installer & self-contained bootstrapper. All of the source code and scripts are provided as a download on this page, and I hope it's useful for demonstrating the technologies in a real-world scenario. When building the demo solution don't forget to replace the GUIDs in the source Config.wsi file with your own unique ids.

Creating a localized Windows Installer & bootstrapper: Part 3

noreply@blogger.com (TheCodeKing) — Sun, 22 Aug 2010 10:07:48 PDT

Part 1: Introduction & Setup

Part 2: Defining Conditions

Part 3: Localization

Part 4: Provision Prerequisites

Download Source Code

Introduction

In Part 1 & 2 we created a Windows Installer using Wix in the form of an MSI file. In the real-world installers often require localization and in this part we'll look at a solution for localizing the installer into a second language; in this example Italian.

Strictly speaking Windows Installer doesn't support multiple languages and can only store one set of strings at any point in time. However it does provide a mechanism which allows for localization via an implementation of transforms. A transform is a delta MST file which can sit inside or outside of the main MSI package. By supplying the transform to Windows Installer the MSI content is replaced by matching content in the MST file at runtime before installation begins, which is ideal for localization.

Our localization solution involves the following steps:

Make the Wix source localizable
Generate an Italian version of the MSI file
Create a MST transform file
Embed the transform file into the original MSI

Localizing Wix

Wix contains all of the tools required for localization, and the WixUI library we are using already comes with stock translations for the UI dialogs provided. The localization authoring features allow us to move localizable strings into separate Wix translation files, which can then be swapped out by LIGHT when generating an MSI. This will ultimately make it easier to generate transforms later on in the process.

In the solution provided all of the custom strings have been moved into external localization files and named with the relevant culture code. These are then referenced in the main source files using the syntax !(loc.VariableName). The WixLocalization element in the localization file must contain correct values for Culture and Codepage attributes for the localized version to function correctly with different character sets. We also add the language LCID as a translation string so that we can reference the language code as a variable in the main project and override the product language attribute.

The final localization file looks something like this:

<WixLocalization Culture="en-GB" xmlns="http://schemas.microsoft.com/wix/2006/localization" Codepage='1252'>
  <String Id='LANG' Overridable="yes">1033</String>
  <String Id="Windows7Required" Overridable="yes">This application only runs on Windows 7.</String>
   ...  
</WixLocalization>

The main source code references these localization strings as follows:

<Condition Message='!(loc.Windows7Required)'>
   <![CDATA[VersionNT = 601 AND WindowsBuild > 7100]]>
</Condition>

Once the strings are re-factored into the localization files, we can go ahead and get these translated into multiple languages. A set of localization files are created for each new language and the culture specific attributes set up accordingly.

Create Localized MSI

LIGHT provides some handy features that allow us to create a localized version of the MSI. Firstly the utility takes multiple translation files that can contain a mixture of cultures. The output file however will only contain a single set of translation strings. So how does this work?

The trick is the cultures argument that takes a list of cultures in order of precedence. From left to right LIGHT will look through the supplied translations looking for the first matching culture for each string. This allows us to build fall-backs into our translation set.

Please note that WixVariables cannot reference localized translation variables, and therefore to swap out the path to the T&Cs agreement we must change our solution so that this is passed through on the command line. We can then pass in references to different agreement files for each localized installer.

So implementing this technique we can create a Italian version of the Installer as follows:

light.exe -o bin\Release\Setup.msi
     -ext WixNetFxExtension.dll
     -ext WixUIExtension.dll
     -cultures:"it-IT,en-GB"
     -dWixUILicenseRtf=Binary\EULA_it-IT.rtf
     Main.wixobj 
     Component.wixobj
     -loc it-IT.wxl
     -loc en-GB.wxl

The example above will look for Italian translations, and for any strings not found it will fall back to English. Note, the naming of the WXL files is irrelevant to the linker.

Creating a Transform

Now that we have an English and Italian version of the MSI we can generate a transform file. The transform will contain a set of differences between the two files. In this case it will contain just the Italian translations and a new agreement file.

The Wix TORCH tool is used to create the transform MST file as follows:

torch.exe setup.msi it-IT\setup.msi -o it-IT.mst

The resultant it-IT.mst file is smaller than the original and contains the transforms which can be applied to the original MSI file in order to localize the installer. It could be deployed alongside the MSI file, but in our case we want to embed the transform and create a single MSI for ease of deployment.

This is where we can leverage an undocumented and unsupported feature of Windows Installer which allows us to auto-detect the language of the installer from the user's region settings. By naming the transform with the corresponding LCID Windows Installer will automatically apply the transform at runtime.

The tool used to embed the MST with a specific name is a script taken from the Microsoft Windows Software Development Kit. This is used as follows, where 1040 is the Italian LCID:

cscript.exe WiSubStg.vbs setup.msi it-IT.mst 1040

Once embedded the transform can be applied explicitly through the command line to launch the Italian version of the installer.

msiexec /i setup.msi TRANSFORMS=":1040"

Note the TRANSFORMS=":1040" syntax (with colon) applies embedded transforms, whereas TRANSFORMS="it-IT.mst" (without colon) will apply an external transform file.

To test the auto-language detection functionality change the regional settings by opening control panel and navigating to regional settings. On the first dialog which defines date and time Formats, select Italy from the language selector. Apply the settings and launch the MSI with no parameters. You should then see the Italian version of the installer.

Summary

In this part of the series we talked through localizing the installer by leveraging some of the Wix tools and by taking advantage of some undocumented features of Windows Installer. Our installer is almost complete. In part 4 I'll show how to create bootstrapper for the installer that will detect and install any prerequisites. Part 4.

Creating a localized Windows Installer & bootstrapper: Part 2

noreply@blogger.com (TheCodeKing) — Sun, 22 Aug 2010 10:07:27 PDT

Part 1: Introduction & Setup

Part 2: Defining Conditions

Part 3: Localization

Part 4: Provision Prerequisites

Download Source Code

Introduction

In Part 1 we looked at the steps required to create a Windows Installer using Wix that dynamically packages the binaries from a target .NET application ready for deployment. In this part we'll look at adding some condition checks into the installer that prevents a user from installing the software unless some conditions are met.

In our solution we will create condition checks for the following:

Install only on Windows 7
Install only if the Microsoft .NET Framework 4 Client Profile is installed
Install only on Dell hardware

Wix provides a variety of ways to add condition checks using registry or file system searches for example. When a condition element is added as a child node of the Project element it will only allow installation when the condition is met. If not the installation will be terminated with the given error message.

Windows 7 Condition

To add a condition that prevents users from installing the application on versions of Windows other than Windows 7 we can leverage some existing properties populated by Windows Installer. These are the VersionNT and WindowsBuild properties. If we look at the documentation for Operating System Property Values we find that the required values for detecting Windows 7 are VersionNT 601 and WindowsBuild greater than 7100.

To implement the condition we add the following element into the Main.wxs file within the Project node.

<Condition Message='[ProductName] can only be installed on Windows 7' >
  VersionNT = 601 AND WindowsBuild > 7100
</Condition>

The result is that the installer will now only install the application on devices running Windows 7, otherwise an error is thrown and the installation is terminated.

Microsoft .NET Framework 4 Client Profile Condition

The WixNetFxExtension.dll extension used in Part 1 provides a set of useful utilities related to the .NET framework and this includes properties that can be used for detecting the version of the .NET framework that is currently installed.

Unfortunately version 3.0 of the framework doesn't include a property for detecting .NET 4.0 and this will only be available in Wix 3.5. Luckily detecting if a particular version of the framework is installed isn't too difficult, and we can use a simple registry search as follows:

<Property Id="NETFRAMEWORK40CLIENT">
   <RegistrySearch Id="NetFramework40Client" 
      Root="HKLM" 
      Key="SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Client" 
      Name="Install"
      Type="raw" />
</Property>

This populates a property named NETFRAMEWORK40CLIENT with a value of 1 if the registry key is found. Using this we can then implement the condition check. We skip the condition check on uninstall in case the framework is subsequently removed.

<Condition Message="[ProductName] requires the Microsoft .NET Framework 4 Client Profile">
    Installed Or NETFRAMEWORK40CLIENT
</Condition>

Dell Hardware Condition

To implement a hardware check we need to perform a WMI lookup to determine the hardware manufacturer. To get the WMI value into Windows Installer so that we can implement a condition we need to use a custom action.

There are a variety of ways to implement a custom action. For simplicity in this case we will use an embedded VBScript called WMIQuery.vbs. The script will perform the WMI lookup and poke this into the installer as a property named WMI_Manufacturer.

To begin we add a reference to the WMIQuery.vbs file using a Binary element. We then define a custom action that will invoke a function within the script and schedule this to run after AppSearch in the InstallExecuteSequence and InstallUISequence sequences. This makes the property available prior to running the condition checks during install.

The new code looks something like this:

<Condition Message='This application cannot be installed on this hardware.'>
  <![CDATA[
    SKIPCHECK 
    OR Installed 
    OR WMI_Manufacturer >< "dell"
  ]]>
</Condition>

<CustomAction Id="WMIQuery" Return="check" BinaryKey="WMIQuery" VBScriptCall="Main" />
<Binary Id="WMIQuery" SourceFile="Binary\WMIQuery.vbs" />

<InstallExecuteSequence>
  <Custom Action="WMIQuery" After="AppSearch" />
</InstallExecuteSequence>

<InstallUISequence>
  <Custom Action="WMIQuery" After="AppSearch" />
</InstallUISequence>

Note the use of the contains string operator >< for the condition check. The operator characters must be escaped within CDATA blocks to prevent validation issues with the XML. We have also added an override property called SKIPCHECK that will allow us to override the hardware condition when run from the command line for testing. To do this we execute the MSI as follows:

msiexec /i setup.msi SKIPCHECK=1

Summary

So far in this series we have looked at creating a Windows Installer using Wix which dynamically packages a .NET application and applies NGen against the assemblies during installation. We have now also added some conditions to prevent the application being installed in specific scenarios using a range of techniques. In Part 3 we will look at localizing the installer and providing auto-language detection using the user's region settings. Part 3.

Error: Only Content controls are allowed directly in a content page

noreply@blogger.com (TheCodeKing) — Thu, 25 Feb 2010 01:25:17 PST

I recently came across the following error which happened during a full content deployment, and despite the fact that the template in question was ghosted:

An error occurred during the processing of . Only Content controls are allowed directly in a content page that contains Content controls.

It turns out (rather randomly) that this can be caused when a lowercase is used for the asp:content user controls in the template. The fix is to modify these to use an uppercase as follows asp:Content.

XDMessaging 3.0 : Downloads

noreply@blogger.com (TheCodeKing) — Sun, 27 Mar 2011 12:52:25 PDT

XDMessaging .Net Library 3.0

Project home

Project downloads

Project documentation

Latest Release 3.0.0.0

XDMessaging Library 3.0.0.0

What's new in this release

Enhancements to IOStream mode for performance.
Refactoring code to split DataGram into various implementations.
This version adds additional error handling around mixed elevation scenarios.
Obsolete APIs are now removed.
Codebase migrated to VS2010.

Note: This library is free for personal and educational use. Please ask permission before using in commercial applications and consider making a small donation.

Archive Versions

Version 2.1.0.3

XDMessaging Library 2.1.0.3

Enhancements to IOStream mode for performance.
Refactoring code to split DataGram into various implementations.

Version 2.1.0.0

XDMessaging Library 2.1.0.0

Adding error handling around mixed elevation scenarios.
Obsolete APIs are now removed.
Codebase migrated to VS2010.

Version 2.0.3.0

XDMessaging Library 2.0.3.0

Adding filter to remove duplicate MailSlot messages sent once per protocol.

Version 2.0.2.8

XDMessaging Library 2.0.2.8

Fixes to DataGram dispose bug, causing crashing on 64bit.

Version 2.0.2.7 (beta)

XDMessaging Library 2.0.2.7

Fixing IOStream Mutex

Version 2.0.2.6 (beta)

XDMessaging Library 2.0.2.6

Fixes to ensure MailSlot read handles are closed correctly.
Removing Mutex on MailSlot writes.

Version 2.0.2.5 (beta)

XDMessaging Library 2.0.2.5

Correcting MailSlot implementation to block rather than poll.

Version 2.0.2.4 (beta)

XDMessaging Library 2.0.2.4

Workaround for MailSlot throughput limitations to minimize dropping of messages under load

Version 2.0.2.3 (beta)

XDMessaging Library 2.0.2.3

Improving MailSlot listener to ensure MailSlot can be opened

Version 2.0.2.2 (beta)

XDMessaging Library 2.0.2.2

Improving exception handling, adding exception log to demo applications

Version 2.0.2.1 (beta)

XDMessaging Library 2.0.2.1

Increasing robustness of MailSlot implementation through retry on IO fail

Version 2.0.2.0 (beta)

XDMessaging Library 2.0.2.0

Implementation of MailSlots for IPC broadcasting as well as over a local Domain or Workgroup
Option to broadcast messages in any mode over the local network
Added Multi-broadcast API for broadcasting in multiple modes

Version 2.0.1.0

XDMessaging Library 2.0.1.0

Performance enhancements
Implementation of IDisposable on listener classes
Mutex process synchronization on clean up of File IO
Changed WindowsMessaging to only iterate and use top-level Windows
Changed WindowsMessaging to use async message processing

Version 2.0.0.0

Added support Windows Services, Console applications
Added concept of transport modes, and refactored API to allow additional implementations
Implemented IOStream for IO file based communication

Version 1.0.0.0

XDMessaging Library 1.0.0.0

Windows Services, Console applications are not supported

XDMessaging 3.0 : Documentation

noreply@blogger.com (TheCodeKing) — Sun, 27 Mar 2011 13:45:11 PDT

XDMessaging .Net Library 3.0

Project home

Project downloads

Project documentation

Getting started

The following API is all you need to start broadcasting and receiving messages across application, process and local network boundaries in your .Net application.

XDTransportMode

This defines the implementation to use for broadcasting and receiving messages. There are currently three modes as follows, each with advantages over the other.

IOStream: This uses file based IO to broadcast messages via a shared directory. A FileSystemWatcher is used within listener classes to monitor changes and trigger the MessageReceived event containing the broadcast message. This mode can be used in Windows Services, console applications, and Windows Forms based applications. Channels are created as separate directories on the file system for each channel. The temporary directories should be accessible by all processes, and there should be no need for manual configuration.
WindowsMessaging: This uses the WM_COPYDATA Windows message to copy data between applications. The broadcast implementation sends the Windows messages directly to a hidden window on the listener instance, which dispatches the MessageReceived event with the copied data. Channels are created by adding/removing Windows properties. This offers the most performant solution for Windows Forms based applications, but does not work for Windows Services, console apps, or other applications without a message pump.
MailSlot: This uses a MailSlot to broadcast messages over the local network and interprocess. A limitation of the MailSlot is that only one process may read the message within the scope of a single machine. The library uses a Mutex to synchronize access to the MailSlot across the system, and ensures that when one process terminates another listener can pick up further messages. This mode is used internally to send messages from other transport modes over the local network. Machines must be a member of the same Domain or Workgroup in order to send and receive MailSlot messages. MailSlot are not robust when used over the network and there may be a high drop rate using this implementation.

IXDBroadcast

Use these implementations to broadcast string messages on named channel to other processes. Messages will be received by listeners in the same mode which have registered with the same named channel.

Example: Creating an instance.

// Create an instance of IXDBroadcast using IOStream mode
IXDBroadcast broadcast = XDBroadcast.CreateBroadcast(XDTransportMode.IOStream);

// or create an instance of IXDBroadcast using WindowsMessaging mode
IXDBroadcast broadcast = XDBroadcast.CreateBroadcast(XDTransportMode.WindowsMessaging);

// or create an instance of IXDBroadcast using MailSlot mode
IXDBroadcast broadcast = XDBroadcast.CreateBroadcast(XDTransportMode.MailSlot);

Example: Sending a message.

// Send shutdown message to a channel named commands
broadcast.SendToChannel("commands", "shutdown");

// Send serializable object to a channel named binary1
broadcast.SendToChannel("binary1", new CustomData("Some text"));

Example: Network propagation

Set the optional propagateNetwork parameter to true in order to send messages to listeners on other machines. Network messages are received by all listener instances which are in the same mode and channel as the broadcast instance. For network communication the machines must be on the same Domain or Workgroup.


// create an instance of IXDBroadcast and allow network propagation
IXDBroadcast broadcast = XDBroadcast.CreateBroadcast(XDTransportMode.WindowsMessaging, true);

// Send a message on the commands channel to all listeners on the local network
// which are in WindowsMessaging mode
broadcast.SendToChannel("commands", "doWork");

Example: Multi-broadcast

This example shows how to send messages using multiple modes at the same time.

// create an instance of IXDBroadcast which uses multiple modes
IXDBroadcast broadcast = XDBroadcast.CreateBroadcast(
                               XDTransportMode.WindowsMessaging,
                               XDTransportMode.IOStream);

// Dispatch to all listeners in either modes
broadcast.SendToChannel("commands", "doWork");

IXDListener

Use these implementations to receive events when messages are broadcast from other processes. Events will only occur if the listener is in the same mode as the broadcasting instance, and if it's listening on the same channel.

By default each listener is capable of acting as a system-wide listener for network messages propagated over the network in order that they may be re-distributed to all processes on the same machine. By setting the optional excludeNetworkMessages flag to true the listener will not take part in listening for network messages. With this setting however the instance may still receive network messages that have been re-broadcast by other listeners in the same mode.

Example: Creating an instance.

// Create our listener instance using IOStream mode
IXDListener listener = XDListener.CreateListener(XDTransportMode.IOStream);

// or create our listener using WindowsMessaging mode
IXDListener listener = XDListener.CreateListener(XDTransportMode.WindowsMessaging);

Example: Registering an unregistering channels.

// Register channels to listen on
listener.RegisterChannel("events");
listener.RegisterChannel("status");
listener.RegisterChannel("commands");

// Stop listening on a specific channel
listener.UnRegisterChannel("status");

Example: Handling the MessageReceived event.

// Attach an event handler to a listener instance
listener.MessageReceived+=XDMessageHandler(this.listener_MessageReceived);

// process the message
private void listener_MessageReceived(object sender, XDMessageEventArgs e)
{
    // e.DataGram.Message is the message
    // e.DataGram.Channel is the channel name
    if (e.DataGram.Channel == "commands")
    {
       switch(e.DataGram.Message)
       {
           case "shutdown":
             this.Close();
             break;
       }
    }
}

Example: Handling a serialized messages:

// Attach an event handler to a listener instance
listener.MessageReceived+=XDMessageHandler(this.listener_MessageReceived);
 
// process the message
private void listener_MessageReceived(object sender, XDMessageEventArgs e)
{
 switch(e.DataGram.Channel)
 {
  case "binary1":
      TypedDataGram<CustomData> typedMsg = e.DataGram;
      UserMessage customData = typedMsg.Message;
      // do something with customData
      break;
 }
}

Example: Dispose IXDListener to release resources.

// Create our listener instance using IOStream mode
IXDListener listener = XDListener.CreateListener(XDTransportMode.IOStream);

// when no longer needed ensure it's disposed
// without this the IOStream listener will continue using resources to listen for messages
listener.Dispose();

// create a new instance
listener = XDListener.CreateListener(XDTransportMode.WindowsMessaging);

Ghosting the Unghostable

noreply@blogger.com (TheCodeKing) — Sat, 05 Dec 2009 18:26:22 PST

Download the sample code.

CodeKing.Reghost.zip

Introduction

This article shows a way to remap a ghosted file to a new location, and also shows how to ghost a non-ghostable file, such as one originally created via SharePoint designer. This assumes a basic knowledge of SharePoint and the concept of ghosted and unghosted files.

This is actually a trick which overwrites the existing file with a new ghosted version. To actually modify a non-ghostable file so that it becomes ghostable is near on impossible without committing the cardinal sin of modifying the content database directly. An alternative is to delete the original and deploy the ghosted version in its place; however, problems arise with layouts (for example) that may still be referenced by existing content. I found the following technique overcomes these issues and allows files to be swapped out with a ghosted version seamlessly.

Note: if your desire is simply to reghost an existing file, then I recommend one of the many existing tools available, such as gl-reghostfile.

The Solution

In this example, we will remap the ArticleLeft layout from the SharePoint publishing template to an updated version deployed via our sample feature. This changes the location of the ghosted file to that of our feature, and allows us to deploy subsequent changes to the ArticleLeft layout by upgrading the new solution. The sample adds the word MODIFIED to the page title of the template.

The first step is to save out the existing file from the SharePoint database using the SharePoint designer. This is then added to a feature in the usual way as if you were provisioning it for the first time. In the case of layouts, be sure to also provision the properties from the original file; otherwise, the new layout will not behave correctly.

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <Module Url="_catalogs/masterpage" Path="Layouts">
    <File Url="ArticleLeft_Ghosted.aspx" 
         Type="GhostableInLibrary" IgnoreIfAlreadyExists="true">
      <Property Name="Title" 
         Value="Modified Article page with image on left" />
      <Property Name="MasterPageDescription" 
         Value="The modified article page with image on left contains 
                an image field and a rich text field." />
      <Property Name="ContentType" 
         Value="Page Layout" />
      <Property Name="PublishingPreviewImage" 
         Value="/_catalogs/masterpage/en-US/Preview Images/ArticleLeft.png, 
                /_catalogs/masterpage/en-US/Preview Images/ArticleLeft.png" />
      <Property Name="PublishingAssociatedContentType" 
         Value=";#Article Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007
                948130EC3DB064584E219954237AF3900242457EFB8B24247815D688C526CD44D;#" />
    </File>
  </Module>
</Elements>

The next step is to rename the file so that it can initially be deployed alongside the the existing layout. In the example included, I have named the file ArticleLeft_Ghosted.aspx.

Lastly, we add a feature receiver with the following code to overwrite the target file with the new ghosted version. Note: this code has been simplified for brevity; see the sample for full code.

public override void FeatureActivated(SPFeatureReceiverProperties properties)
{
    SPSite site = properties.Feature.Parent as SPSite;
 
    PublishingWeb pubweb = PublishingWeb.GetPublishingWeb(site.RootWeb);
 
    // get reference to newly provisioned file
    SPFile file = site.RootWeb.GetFile("/_catalogs/masterpage/ArticleLeft_Ghosted.aspx");
    if (file != null)
    {
        // overwrite the old layout with the ghosted version
        file.MoveTo("/_catalogs/masterpage/ArticleLeft.aspx", true);
        file.Update();
    }
}

Once overwritten, the new version of the file remains ghosted. In the case of layouts and master pages, all existing content will begin to use the ghosted version. The key to the technique is the MoveTo command which allows any referenced layout to be overwritten without breaking references to existing content.

Intermittent access denied

noreply@blogger.com (TheCodeKing) — Fri, 11 Dec 2009 15:04:55 PST

The following is an interesting error I recently came across in SharePoint on an application inherited from another company. For some reason (probably accidently) they had enabled the batch compilation setting in the web.config as follows.

<compilation batch="true" debug="false" />

This resulted in a particular page on the website throwing a 401 error intermittently, and prompting the user to authenticate. After repeated requests the error would disappear and the page would display as normal. We finally worked out that this was happening on application re-start. SharePoint logs recorded an unhelpful Access Denied entry with no further details.

It turns out that the batch compilation setting is not supported in SharePoint as discussed in this article.

The fix was to turn off batch compilation, and the page then behaved as normal.

<compilation batch="false" debug="false" />

Exception from HRESULT: 0x80040E14

noreply@blogger.com (TheCodeKing) — Fri, 11 Dec 2009 15:05:12 PST

During restore of a SharePoint application with stsadm -o restore command I recently came across the following error:

Exception from HRESULT: 0x80040E14

It turns out this happens when the database does not have enough disk space to complete the task. Freeing up some space on the db server resolved the issue.

Using SharePoint and log4net in harmony

noreply@blogger.com (TheCodeKing) — Sat, 05 Dec 2009 18:28:19 PST

Introduction

With traditional .NET applications, configuring log4net is relatively straightforward by simply changing the web.config and using xcopy deployment. In SharePoint, however, things become a little more complicated, and getting log4net working effectively takes a little more effort.

Firstly, it's not recommended that you modify the web.config manually or use xcopy deployments when working with SharePoint. This breaks the whole SharePoint model, and is likely to cause issues later on (for instance, when joining a new FTE server to the farm or extending an application). I'm a firm believer in repeatable builds and deploying everything via solutions and features, so this is not an option.

Secondly, not all code executes within the web application context. Take feature receivers and jobs, for example; depending on how they are invoked, these could run within the stsadm process or within OWSTimer. Setting up your log4net settings in the web.config can therefore be somewhat futile for debugging these components.

Solution

The solution I recommend is to deploy log4net into the GAC, and log4net.config to the 12 hive (using WSPBuilder). It's then possible to pass the location of the config file to log4net on initialize by simply adding the following line to your AssemblyInfo.cs file.

[assembly: log4net.Config.XmlConfigurator(ConfigFile = 
  @"C:\Program Files\Common Files\Microsoft Shared\" + 
  @"Web Server Extensions\12\CONFIG\log4net.config", Watch = true)]

Note that although this requires hard-coding the location of your previously deployed config file, I feel that it's acceptable as the path never changes between SharePoint installations. I would deploy the log4net assembly and config within a globally scoped solution rather than packaging with your main application codebase. The reasons for this are discussed briefly in a previous article here, and will reduce risk with managing shared resources in the farm.

Often, you may wish to have different log4net configurations per environment. If this is the case, I recommend using a different technique for configuring the location of the log4net config file within your codebase. This allows you to inject some additional application logic to determine the location of your config file.

public static void InitializeLog4Net(string environment)
{ 
   string configFile = string.Concat("log4net.",environment,".config");
   FileInfo configFileInfo = new FileInfo(@"C:\Program Files\Common Files\" + 
     @"Microsoft Shared\Web Server Extensions\12\CONFIG\log4net\"+ configFile);
   log4net.Config.XmlConfigurator.ConfigureAndWatch(configFileInfo);
}

Summary

The beauty of this solution is that log4net settings may be managed across the entire farm centrally. It also doesn't matter how your code is executed or what process it's running in, all log messages will be processed according to your settings. Use filters to customize log messages for different applications from the one config file.

For debugging feature receivers and jobs, I highly recommend configuring the log4net OutputDebugStringAppender and using the Sysinternals DebugView. This allows you to see your debug code in real-time, and can easily be switched on in various environments without the need for a debug build.

Working with SPWebConfigModification

noreply@blogger.com (TheCodeKing) — Sat, 05 Dec 2009 18:28:58 PST

Introduction

The following tips relate to working with the SharePoint SPWebConfigModification API in order to manage changes to the web.config. Note it's not recommended that you modify the web.config directly as this may have side effects with the running of your SharePoint instance, and changes may be lost when installing updates or service packs.

Avoiding Duplicates

When working with the SPWebConfigModification API, the name property should be set to an xPath expression that identifies the configuration change being made. This should be an expression relative to the Path property. This prevents duplicate changes being made in the web.config. If an arbitrary name is used instead, then the system will not throw an error but may lead to duplicate entries.

SPWebService service = SPWebService.ContentService;

SPWebApplication webApp = properties.Feature.Parent as SPWebApplication;

myModification.Path = "configuration/system.web/httpModules";

myModification.Name = "add[@name='myModule']";

myModification.Sequence = 0;
myModification.Owner = this.GetType().FullName;
myModification.Type = SPWebConfigModification.SPWebConfigModificationType.EnsureChildNode;
myModification.Value = "<add name='myModule' type='myModule, myAssembly' />";

service.WebConfigModifications.Add(myModification);
service.Update();
service.ApplyWebConfigModifications();

Removing Entries

To ensure your entries can be removed via code (such on feature deactivate), it's a good idea to set the owner of the SPWebConfigModification instance as in the example above. It's then possible to loop through the config settings and remove the entries that were previously set by the same codebase

SPWebApplication webApp = properties.Feature.Parent as SPWebApplication;
Collection modsCollection = webApp.WebConfigModifications;
int count = modsCollection.Count;
for (int i = count-1; c >= 0; i--)
{
 SPWebConfigModification mod = modsCollection[i];
 if (mod.Owner == this.GetType().FullName)
 {
  modsCollection.Remove(mod);
 }
}
webApp.Update();
webApp.Farm.Services.GetValue().ApplyWebConfigModifications();