Design

logging

Logging is an essential component of developing applications. It helps in many ways, but still many developers being overconfident about their code never add proper logging to their applications.

It is not a difficult task, I think it is one of the most easiest thing that developers can do to make their code more reliable and easy to debug. There are many frameworks that can be used, most of which are extremely simple. Sometimes developers add these framework to their projects, but they seldom use them properly.

For example, what will you do if you see a log message that says:

We can not submit your request.

Apart from knowing something failed, what help can anyone get from this? Nothing at all. This message is logged as a custom message and it could have been a little more verbose.

Another form of message is this,

Object reference not set to an instance of object.

Now you know exactly which error happened, but again, can you do anything about this? Little stack trace info would have been awesome. This message is of no use without information about error source location.

Many times we validate our data transfer objects for data validity. You submit something from client to service and request fails due to invalid data. Most of the time you will see a message like this:

Request failed due to validation issue. ID – 1. Followed by stack trace and error location.

Now this message is definitely more helpful, but still we can have more information here like which validation failed and what was the data value for which validation failed.

Most of the applications follow layered architecture which means they will have a data layer, business layer and UI. Business layer most of the time is implemented via services. In this architecture, you will usually have 2 log files. One for UI and other one for service. Now, suppose error happened and you go to UI log file and see this:

UI – Log. Service failed to process this request. Please contact administrator.

Now, you open service log and see more than one error messages. Huh…how will you now relate UI error message with service error message. If your app is used by few people, only few times a day, you can use log message time stamps to relate these errors. Most of the time this will not work and you will have multiple log message with almost similar time stamp. One way to easily solve this is by generating a correlation Id (Guid), logging that with error message in service log and then pass that same correlation Id as part of service fault. Log the UI error message with same correlation ID and you are done. So you will see something like this in UI log file;

(ID: 1234-23322-34322….) Service failed to process this request. Please contact administrator.

and service log will have error logged like this;

(ID: 1234-23322-34322….) Invalid argument [User Name – Brad John]. Data not found in data store. Followed by stack trace.

Now you can use this pattern to show generic error message with correlation ID on user interface and then ask user to pass that correlation ID to your admin/ support team. They can easily get to the real problem.

At a high level, you should do this for logging:

  • don’t shy away from logging verbose errors
  • don’t just log error message (ex.Message)
  • don’t forget to add stack trace information
  • use throw in place of throw ex. If throwing new exception, then add old exception as inner exception. This will preserve stack trace information.
  • provide extensive information that can be used for debugging
  • log exactly due to which data value a business validation failed
  • log informational messages that can be easily turned off via configuration
  • use correlation pattern to relate error message logs in different app layers

These are few things which I’ve on top of my head. Over the years I’ve used multiple logging frameworks with each one having their own pros and cons. In next few posts I’ll discuss these frameworks and thier usage.

Advertisements

Fluent API – Improvements

Please read my earlier article on Fluent API before reading this post. I wrote my old article to get you started with Fluent API design. There are few essential improvement that we should do in that implementation.

In our Fluent interface all methods were returning similar type of object, IFluentPolicyEngine

public interface IFluentPolicyEngine
{
IFluentPolicyEngine CreatePolicy(string name, string description);

IFluentPolicyEngine PerformsOperationOnData(string operation);

IFluentPolicyEngine HavingDataFilter(string type);

IFluentPolicyEngine Where(string filter);

IFluentPolicyEngine Validate();

void Save();
}

This makes it very easy to chain methods but in real world applications this will not work well. There should be a predefined way of chaining methods. Above interface let’s you chain methods in any order. There will be some business rules that should be used while creating these chains. Designing your fluent API based on business rules helps in:

  • Business rule driven data validation
    Suppose our policy has a business rule, a policy can have filter statements added via Where(…) only if data filter is added to the policy via HavingDataFilter(…). It is not possible to restrict chaining using our above interface. Only possible solution is to do a check in API method and throw exception. Nahhh :(, we should strive for compile time safety rather than runtime.
  • Improving code readability
    If API is designed carefully based on business rules then rather than sprinkling validation code in all methods we can have far better data validation routines

Let’s modify our fluent API to support following business rules:

  • Client can only add policy filter after adding business operation
  • Client can only add filter statements via Where(…) after data filter is added via AddDataFilter(…)
  • Client can only validate policy when an operation and at least one filter statement is added to the policy
  • Validation can only be performed after adding operation and filter to the policy
  • Save can only be called on a validated policy

New interface design looks like this:

public interface IFluentPolicyEngine
{
IFluentPolicyEngine CreatePolicy(string name, string description);

IFluentPolicyWithOperation PerformOperationOnData(string operation);
}

public interface IFluentPolicyWithOperation
{
IFluentPolicyWithFilter HavingDataFilter(string type);
}
public interface IFluentPolicyWithFilter
{
IFluentPolicyWithFilter Where(string filter);

IValidatedPolicy Validate();
}

public interface IValidatedPolicy
{
void Save();
}

Modify the class where old single interface was implemented by implementing all new interfaces and changing method return types. Client code remains the same.

Now you can only write client code with proper business rule driven method chaining. Ad hoc chaining will result in compile time issues. Download the code and try changing things.

Download Code : http://1drv.ms/1o3lnyC

Starting with Fluent API

Last week, while trying out a new logging mechanism, Serilog, I stumbled upon a new style of API design. It is known as Fluent API and looked like an interesting topic to understand.

What is Fluent API?

This term was coined by object oriented programming guru Martin Fowler in 2005. Not so new 🙂 His idea was to develop client friendly code that is easy to understand (read) and easy to maintain.

Business processes are defined by developing mix of objects and then chaining them together via some internal domain specific language. LINQ is a perfect place to start with. The way multiple methods are chained together to create a SQL like business operation is an example of Fluent API. We keep on adding Where(), Select(), Take() etc. methods to our code statement and each operation returns IEnumerable object. This is commonly known as method chaining. See following example to understand it better:

var underAgeCustomerCount = Customers                
.Where(c => c.Age <= 18)
.OrderBy(c => c.FirstName)
.Count();

How it is done?

I’ll take one of my real life project as a sample and show how we can add a very basic Fluent interface to that.

Business Goal of the project

This project contains a policy engine where users can define policies based on which operations are performed on data. Users can create multiple policies and save them. When data passes through this policy engine, appropriate policy is kicked off and a specific operation is performed.

Each policy consists of an operation and set of filters. Consider filter as a SQL WHERE clause. It can have multiple filter statements and these statements are added together by either AND or OR. So a filter can look like this:

WHERE Name = 'Gaurav Sharma' AND Age > 18 AND City = 'HYD'

Above is a filter, having 3 statements and uses AND as separator.

Operation is a method that is executed if data matches with defined filter.

Current Implementation

There are POCO classes defined for Policy, Operation, Filter and FilterStatement. Then there is a PolicyManager class that provided functionality to Validate and Save policy. Code looks like this:

public class Policy
{
public Guid ID { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public Filter Filter { get; set; }
public Operation Operation { get; set; }
}


public class Filter
{
public string Type { get; set; }
public List<FilterStatement> FilterStatements { get; set; }
}
public class FilterStatement
{
public string Statement { get; set; }
}
public class Operation
{
public string Name { get; set; }
}
 
public class PolicyManager
{
public void Validate(Policy policy)
{
// check for invalid properties in policy object
// throw exception in case of validation failures
}

public Policy Save(Policy policy)
{
// create policy and save to store

// create a new ID and update the object
policy.ID = Guid.NewGuid();

return policy;
}
}
 
This is very basic representation of what we are actually doing in our project but for this blog I think it is sufficient. Now if you have to write client code for this then it would look similar to:
 
var policy = new Policy();

policy.Name = "Sample policy";
policy.Description = "This is a sample policy";

var operation = new Operation();
operation.Name = "Create Order";

var statement1 = new FilterStatement();
statement1.Statement = "City = New Delhi";

var statement2 = new FilterStatement();
statement1.Statement = "CustomerName = Gaurav Sharma";

var statement3 = new FilterStatement();
statement1.Statement = "TotalAmount > 100000";

var filterStatements = new List<FilterStatement>();
filterStatements.Add(statement1);
filterStatements.Add(statement2);
filterStatements.Add(statement3);

var filter = new Filter();
filter.FilterStatements = filterStatements;

policy.Operation = operation;
policy.Filter = filter;

var manager = new PolicyManager();
manager.Validate(policy);
policy = manager.Save(policy);

Console.WriteLine("Successfully saved policy with ID {0}", policy.ID);

Remember good old ADO.NET days when we use to write this type of code for data operations. Remember DataConnection, DataReader, DataSet etc. It was painful.

We can clean this a little more by using different constructors and object initializes. Let’s skip that for now. Now we will add very simple Fluent interface implementation to our project and see how it improves client code.

public interface IFluentPolicyEngine
{
IFluentPolicyEngine CreatePolicy(string name, string description);

IFluentPolicyEngine PerformsOperationOnData(string operation);

IFluentPolicyEngine HavingDataFilter(string type);

IFluentPolicyEngine Where(string filter);

IFluentPolicyEngine Validate();

void Save();
}
 
See the return type defined for our new fluent interface methods. This helps us in chaining these methods together. Let’s implement this interface.
 
public class FluentPolicyEngine : IFluentPolicyEngine
{
public Policy Policy { get; set; }

public FluentPolicyEngine()
{
Policy = new Policy();
}
public IFluentPolicyEngine CreatePolicy(string name,
string description)
{
Policy.Name = name;
Policy.Description = description;
return this;
}

public IFluentPolicyEngine PerformsOperationOnData(string operation)
{
var op = new Operation
{
Name = operation
};
Policy.Operation = op;
return this;
}

public IFluentPolicyEngine HavingDataFilter(string type)
{
var filter = new Filter
{
FilterStatements = new List<FilterStatement>(),
Type = type
};
Policy.Filter = filter;
return this;
}

public IFluentPolicyEngine Where(string filter)
{
var filterStatement = new FilterStatement
{
Statement = filter
};
Policy.Filter.FilterStatements.Add(filterStatement);

return this;
}

public IFluentPolicyEngine Validate()
{
if (Policy.Operation == null)
throw new InvalidDataException("Operation is not provided.");

if (!(Policy.Filter != null
&& Policy.Filter.FilterStatements != null
&& Policy.Filter.FilterStatements.Count > 0))
throw new InvalidDataException("Filter is missing.");

return this;
}

public void Save()
{
Policy.ID = Guid.NewGuid();
}
}
 
This is again a very simple implementation of our interface but sufficient for this write up. Methods are simple and self explanatory. Now our new client code looks like:
 
var policyEngine = new FluentPolicyEngine();
policyEngine
.CreatePolicy("Fluent Policy", "Fluent policy sample")
.PerformsOperationOnData("Create Order")
.HavingDataFilter("All")
.Where("City = New Delhi")
.Where("CustomerName = Gaurav Sharma")
.Where("TotalAmount > 100000")
.Validate()
.Save();

Console.WriteLine("Successfully saved policy with ID {0}",
policyEngine.Policy.ID);
 
See how readable this code is now.
Some important points
  • Unless client developer understand Fluent API business domain, it is very tough to write this type of code. Think about a developer knowing nothing about SQL and then trying his hands on LINQ.
  • Debugging is little difficult in this type of client code. You can not put breakpoint on a specific method call. 
  • There are multiple ways of doing what I just showed you. For example, in C# we can use extension methods to achieve similar results.
  • Lot of configuration APIs provide fluent interfaces. Entity framework is one of them.
  • Although it simplifies client code, your API code can become complex. Keep a watch on that.

It is not easy to create efficient Fluent API. You have to think a lot. So think.then.code.

Download Code: http://1drv.ms/1khBvvD

Library Design – Documenting Exceptions

I’m sure most of you might already know but I thought of sharing this. If we are coding for a class library then we should always try to explicitly provide exception type that our code (method) will throw. If we provide exception values in XML Comments then those exceptions will appear in intellisense while we are using that library method. This will be handy for developers using our library. I remember this is also mentioned somewhere in framework design guidelines.

  • Method definition: Explicit exception details provided

  • Method Usage: Intellisense contains list of exceptions


Cheers!
Currently Playing : PROTOTYPE

Singleton – One of a Kind Objects (Part II)

How will we tame the multi threading beast?
I’m a .Net developer having CLR as my run time. I never worked on Java, but C# and JAVA are almost similar. C# is advance form of JAVA and CLR is much more refine than JVM. Things that comes later are generally more advanced than things that came earlier. I was first introduced to this multi threading problem while going throgh Head First Design Patterns book. According to the book it was damn simple to fix this problem by making access to MySingleton.GetMySingletonInstance() synchronized. Code for this is simple as Singleton class itself,

using System;
using System.Runtime.CompilerServies;
internal sealed class MySingleton
{
//static variable to hold instance reference
private static MySingleton onlyInstance;
private MySingleton();
//Global i.e. public and static point of access
//No two treads may enter the method at the same time

[MethodImpl(MethodImplOptions.Synchronized)]
public static MySingleton GetMySingletonInstance()
{
if (onlyInstance==null)
{
onlyInstance=new MySingleton();
}
return onlyInstance;
}
}

This is as simple as it can get.We just decorated our method with MethodImpl attribute. This is available in System.Runtime.CompilerServices namespace. Now, no two threads can enter our GetMySingletonInstance() method at the same time. This solves the problem but leaves us with another headache, performance. We all know that this synchronization is only needed when onlyInstance is null i.e. we haven’t yet created the MySingleton object. So adding synchronization lock to our method doesn’t makes sense. Its just like solving one problem and creating another.

Now What?
There is a well known technique used to solve this problem. This is called Double Check Locking Technique. It’s used when you want to have lazy instantiation i.e. you want your object to be constructed when your application first requests it. This is not a well known technique because its particularly interesting or useful. It’s well known because a lot is written about it and a lot of Java developers use it. This was quite heavily used in Java but lately its discovered that this technique doesn’t guarantee that it would work everywhere. If you want to know why this fails then just read through http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html. Just to show you how this double checking is implemented in C# I’ll add some code here,

internal sealed class MySingleton
{
private static volatile MySingleton onlyInstance= null;
protected MySingleton() {}
public static MySingleton GetMySingletonInstance()
{
if (onlyInstance == null)
{
lock (typeof(MySingleton))
{
if (onlyInstance == null)
{
onlyInstance = new MySingleton();
}
}
}
return onlyInstance; }
}
What is the best implementation?
Use Type Initializer (static constructor) to create a simple and best implementation of singleton.
internal sealed class MySingleton
{
//Type constructor will be called when any member of type is accessed
//This initialization code will be added to Type Constructor by JIT compiler
private static MySingleton onlyInstance= new MySingleton();
private MySingleton() {}
public static MySingleton OnlyInstance
{
get { return onlyInstance;}
}
}
CLR automatically calls type’s class constructor when the code attempts the access a member of the class. CLR ensures that calls to a type constructor are thread safe. Double check locking is less efficient than this technique as you need to create your own lock object and write all additional locking code yourself. Double check locking is interesting only if class has lots of members and you want to have your singleton constructed only when one of the member is called.

I hope! I answered my friend Pramod’s question related to minimize overhead associated with double check locking technique.