False Positives in Database Integration Tests

You may sometimes find that all your tests pass, but in production your entities cannot be hydrated from the database. Take this example:

@PersistenceContext(unitName="shipmentOrder")
EntityManager entityManager;

@Test
@Transactional
public void maps_correctly_to_and_from_database() throws Exception {

	ShipmentOrder state = new ShipmentOrder();
	state.setCustomerName("CALLERNAME");
	state.getDeliveryAddress().setCountry("NIRVANA");

	entityManager.persist(state);
	entityManager.flush(); // enforce writing to the database

	ShipmentOrder foundState = entityManger.find(ShipmentOrder.class, state.getId());

	assertEquals("CALLERNAME", foundState.getCustomerName());
	assertEquals("NIRVANA", foundState.getDeliveryAddress().getCountry());
}

This test will happily pass, but the same call fails in production. Why?

Below are the entity classes - note, that JPA by default does not allow @Embeddable's to inherit (see e.g. the Hibernate Annotations Reference, section "3.2.4.4. Inherit properties from superclasses" for more details). Even worse, Hibernate does not complain, but silently ignores inherited fields.

@Entity
class ShipmentOrder {
	String customerName;
	@Embedded
	DeliveryAddress deliveryAddress;

	String getCustomerName() { return customerName; }
	DeliveryAddress getDeliveryAddress() { return deliveryAddress; }
}

@Embeddable
class DeliveryAddress extends Address{
	String deliveryLocation;
	String getDeliveryLocation();
}

class Address {
	String country;
	String getCountry();
}

So, why does the test pass then?

The problem is, that while within a transaction, Hibernate reuses objects from the 1st Level cache. This means the our call to "flush()" causes Hibernate to write changes to the database, but the entity instance is still kept in memory. If we execute the find() method, Hibernate simply returns the entity instance from the cache and will never try to hydrate the instance from the database.

Solution

EntityManager.clear() allows to manually purge the 1st level cache. Changing the test to

repository.store(state);
entityManager.flush(); // enforce writing to the database

entityManager.clear(); // purge 1st level cache

state = repository.findById(state.getId());

will successfully catch the mapping error.

Here is a method we use in integration tests to automatically flush+purge all persistence contexts defined in Spring:

protected void flushAndClearPersistenceContexts() {
    Map<String, EntityManagerFactory> entityManagers = this.getApplicationContext().getBeansOfType(EntityManagerFactory.class);
    for(EntityManagerFactory emf:entityManagers.values()) {
        EntityManagerHolder emHolder = (EntityManagerHolder) TransactionSynchronizationManager.getResource(emf);
        if (emHolder != null) {
            final EntityManager entityManager = emHolder.getEntityManager();
            try {
                entityManager.flush();
            } catch (TransactionRequiredException e) {
                // ignore - couldn't find a way to properly check if we are within a transaction
            }
            entityManager.clear();
        }
    }
}

NOTE: never use flush() and clear() in your production code as this seriously kills the performance of hibernate

You can find the sample code reproducing this issue on my git repository.

A little Christmas present - for .NET

Just read Eberhard Wolff's newest blog post and simply couldn't resist to borrow the idea and show you, how the same things can be done using Spring.NET. First, here's the .NET version of Eberhard's example:

public class MyRepository
{ }

public class MyService
{
    public MyRepository MyRepository { get; set; }
}

Here it is, the smallest possible Spring.NET object. Notice, due to Java's lack of properties the .NET version is even smaller. Still, no attributes, no Spring dependencies - pure code.

How do you wire them? Spring.NET's XML configuration unfortunately lacks Spring.Java's component-scanning as demonstrated in Eberhard's post. But using the new code-based configuration I presented recently, you easily can wire your objects by writing

appContext.Configure(cfg => cfg
     .Scan(scan => scan
        .TheCallingAssembly()
        .Include(t => t.FullName.EndsWith("Service") || t.FullName.EndsWith("Repository"))
        .With<AutowiringConvention>()
));

The example code can be downloaded at SmallestSpringObject.zip

Merry Christmas!

NetMX & Spring.NET - configure your Apps at runtime

Ever wished you could change some settings of your application during runtime? Easily retrieve runtime information from your components? You tried WMI and Performance Counters?

Well I did and to put it polite: I don't like WMI. Performance Counters are a nice way to retrieve peformance information about an application, but when it comes to changing an application's settings or behavior at runtime, you need another way. WMI is still COM-based, although a WMI.NET binding is available via the System.Management components. Alas this binding is not complete and functions are spread across different namespaces for historical reasons. To put a long story short: It is nothing for the fainthearted.

Brief Introduction to Java Management Extensions (JMX)

In the Java universe for this purpose there is JMX. Basically it allows an application to expose management objects via a JMX server. A monitoring client can then connect to this server and dynamically obtain runtime information about the exposed management objects. In contrast to WMI there is no schema involved, everything is discovered dynamically. For instance the server application could expose the following management bean:

public interface SampleMBean
{
    public String getHello();
    public void setHello(String helloMessage);
    public String sayHello();
}

public class Sample implements SampleMBean {

    private String hello = "Hello World(s)!";

    public Sample() {
    }

    public String getHello() {
        return hello;
    }

    public void setHello(String helloMessage) {
      hello = helloMessage;
    }

    public String sayHello() {
      return hello;
    }
}

Using the JConsole tool, one can easily connect to this application's virtual machine and remotely retrieve/change the properties as well as invoke the exposed method(s):

 

The server application may expose its management objects over a variety of different connectors, as it is shown in this image (copied from the Wikipedia article about JMX)

And Now: .NET Management Extensions

Thanks to the amazing work of Szymon Pobiega, the power of JMX is available for the .NET platform as well. The NetMX implementation is available from CodePlex. The really nice part is, that it also comes with a working implementation of the JSR262 JMX connector, so that it is possible to connect to your .NET application using JConsole (yes, that's true and I'm going to proof that ;-)). Of course NetMX also comes with a .NET version of the console, please check out the project for more. I grabbed the sources and compiled them. After playing around a while, I decided to implement a simple usage scenario.

The Sample Scenario

Every now and then I wanted to increase decrease the logging level of my applications (among other things ...). Using NetMX it should be easy. And - well it was! Using Spring.NET, it was easy to weave a logging advice around my components, using NetMX it was easy to expose a management object for this logging advice, so that the log settings can be changed at runtime. The picture below illustrates the scenario setup I aimed for:

The Logging Advice intercepts every call to the service object and logs the method calls to the log system. By default, logging is turned off, using JConsole I want to change those settings during runtime.

My sample application again is the MovieFinder application. To make the effect visible, the client code retrieves the movielist every second and prints the timestamp + the moviename on the console. Also the logging system is configured to write to the console, but disabled by default.

The Solution

The sources for my experiment can be fetched from my subversion repository. What you also need is the JSR262 enhanced version of JConsole, which is included in the samples of the JSR262 Connector download. Unpack the connector download and checkout $/jsr262-ri-ea4/samples/index.html for more.

So launching the application results in a screen similar to this:

Now launch JConsole and connect to our .NET application using the url "service:jmx:ws://127.0.0.1:9998/jmxws":

Expanding the tree shown in the left pane, you should see this:

  Now let's change some settings. Double-Click into the "Value" column and enter the following:

é voila! Our console has significantly changed now:

We just reconfigured the logging advice on the fly to also log all method calls to our service and also dumps the arguments passed in!

Here's the configuration snipped required to make this happen (using Spring.NET CodeConfig from my last post):

var appContext = new GenericApplicationContext(false);
appContext.Configure()
   .FromConfiguration<MovieFinderConfiguration>()
    .EnableLogging(log =>
    {
       log.TypeFilter = type => type.IsDefined(typeof(ServiceAttribute), true);
       log.Logger.LogLevel = LogLevel.Off;
    })
   .EnableNetMX(netmx =>
   {
     netmx.AddConnector<Jsr262ConnectorServerProvider>(new Uri("http://0.0.0.0:9998/jmxws"));
     netmx.ExportMBean<SimpleLoggingManager>();
   });

Enjoy and again thanks to Szymon for his great work!

Merry XMLless! CodeConfig for Spring.NET

Straight to the point: XML is not meant for humans. Fullstop. The only way for humans to deal with XML is when it is hidden behind proper tooling support. Without a tool hiding XAML you wouldn't write XAML code by hand, would you? Currently being on a greenfield project with my team collegues not familar with Spring.NET it quickly turned out that XML configuration can be quite a hurdle, burying the gain of power in the pain of configuration. Using the simple MovieFinder example from the Spring.NET quickstart examples, I would like to introduce you to a new way to do familar things.

Introducing MovieFinder

In the following I will use the following - very simple - example to show you different ways of wiring the components. A MovieLister can be used to obtain a list of movies directed by a particular director. To access persistent storage, it uses the MovieFinder repository component:

Here's how the client code might look like:

IMovieLister lister = ...
var movies = lister.MoviesDirectedBy("Roberto Benigni")

The traditional way of wiring

The traditional way of configuring the Spring.NET container looks like this:

<objects xmlns='http://www.springframework.net'>

  <object id='movieLister' type='MovieFinder.Core.MovieLister, MovieFinder.Core'>
    <constructor-arg index='0' ref='movieFinder' />
  </object>
  
  <object id='movieFinder' type ='MovieFinder.Data.SimpleMovieFinder, MovieFinder.Data' />    
  
</objects>

Code Example: Traditional Configuration

There are a couple of problems with that approach, here are some:

• not pleasant to read
• much 'noise', xml syntax hiding the actual wiring structure
• lack of refactoring support
• unmanageable for large object numbers

You can imagine that in large applications without any tool support this can be become very painful. Already less known is the fact that Spring is able to autowire components. Here's the example to autowire components based on type-matching contructor arguments (see the reference documentation for other values of 'default-autowire'):

<objects xmlns='http://www.springframework.net' default-autowire='constructor'>

  <object id='movieLister' type='MovieFinder.Core.MovieLister, MovieFinder.Core' />  
  <object id='movieFinder' type ='MovieFinder.Data.SimpleMovieFinder, MovieFinder.Data' />    
  
</objects>

Code Example: Using constructor autwiring

This saves you from having to manually specifiying all dependencies. Of course, renaming your classes still may break this configuration.

Mark Pollack blogged an extensive post about configuring the Spring.NET container, including the even less known container API for configuring the container.

Going fluent

A lot of frameworks out there have already introduced a configuration style known as fluent API. A nice example for the Windsor container can be found here. There's a lot more, Fluent NHibernate being another very populare example.

Recently, Tom Farnbauer released such a fluent configuration API for Spring.NET, Recoil for Spring.NET. Using Recoil, our MovieFinder example could look like this:

public class MyWiringContainer : WiringContainer
{
    public override void SetupContainer()
    {
        Define( () => new MovieFinder() )
             .As<IMovieFinder>;

        Define( () => new MovieLister( Wire<IMovieFinder>() ) )
             .As<IMovieLister>;
    }
 
}

var myContext = new GenericApplicationContext();
myContext.Configure()
  .With<MyWiringContainer>();
myContext.Refresh();

Code Example: Wiring using Recoil for Spring.NET

Unfortunately I quickly found while introducing Recoil into my current project, that the API has some flaws and - after all - when looking at fluent apis, they usually add as much codenoise to your configuration as xml already does. Also fluent APIs tend to be less extensible than other approaches. Imho Fluent NHibernate suffers this fate a lot (probably others too, but this is the latest example crossing my way).

Of course we already achieved at least 1 important goal: We are safe against refactoring. Any class moves or renames will automatically be reflected in our configuration - because it is code.

Back to the whiteboard

I didn't find any of those approaches really satisfying. After all, all I want to do is

var movieFinder = new MovieFinder();
var movieLister = new MovieLister( movieFinder );

So lets start with the least minimal configuration container I can think of. A simple class, who's member methods return the objects you are asking for:

public class MovieFinderConfiguration 
{
   public IMovieFinder MovieFinder() 
  {
     return new MovieFinder();
  }

  public IMovieLister MovieLister()
  {
    return new MovieLister( MovieFinder() )
  }
}

Code Example: Wiring using ... plain C#

This already does a lot of what we want:

• it is readable
• it doesn't cause more codenoise than your standard code would have
• we have full refactoring support
• we automatically get the whole object graph once we call MovieLister():

var container = new MovieFinderConfiguration();
var movieLister = container.MovieLister();
var movies = movieLister.MoviesDirectedBy("Roberto Bergnini");

Code Example: Obtaining the object graph from our C# "container"

Unfortunately our minimal container is lacking a couple of things:

• no support for managing the object lifecycle - everytime we call MovieLister() it will return a new graph
• no support for applying aspects to care for crosscutting concerns
• no support for all the other features you get from the containers out there

New on stage: CodeConfig for Spring.NET

Rooting in a great idea of Rod Johnson, Chris Beams started working on bringing this idea to the Java world, the result is JavaConfig (and since yesterday's release of our mother-project integral part of Spring 3.0, congrats at this point to the Java team!). Mark Pollack implemented the first POC for Spring.NET, due to the needs in my current project, I decided to take that POC and continue develop it.

Yet again, Spring.NET surprised me. Due to being incredibly flexible, it was almost a breeze to merge the above mentioned concept with the container infrastructure. How does our MovieFinder look like using CodeConfig? Simple:

[Configuration]
public class MovieFinderConfiguration 
{
   public virtual IMovieFinder MovieFinder() 
  {
     return new MovieFinder();
  }

  public virtual IMovieLister MovieLister()
  {
    return new MovieLister( MovieFinder() )
  }
}

Code Example: The CodeConfig configuration - not much difference!

Note the additional [Configuration] attribute and the "virtual" keyword added to the methods. Now feed this configuration into the Spring.NET application context and retrieve an instance of the IMovieLister:

var appContext = new GenericApplicationContext();
appContext.Configure()
    .FromConfiguration<MovieFinderConfiguration>();
ApplicationContext.Refresh();

var movieLister = appContext.GetObject<IMovieLister>();
var movies = movieLister.MoviesDirectedBy("Roberto Bergnini");

Code Example: retrieving an IMovieFinder from an CodeConfig-configured application context

Neat, isn't it? This means you can write your configuration in the simplest possible way and still can leverage the full power of the IoC container. Enabling logging on your services? No problem:

appContext.Configure() 
    .FromConfiguration<MovieFinderConfiguration>() 
    .EnableLogging(cfg =>
    {
        cfg.TypeFilter = type => type.IsDefined(typeof(ServiceAttribute), true);
        cfg.Logger.LogExecutionTime = true;
        cfg.Logger.LogMethodArguments = true;
        cfg.Logger.LogReturnValue = true;
        cfg.Logger.LogLevel = LogLevel.Trace;
    });

Using Conventions for Configuration

You don't want to configure each object manually in a large application? Use component-scanning, a feature that I shamelessly stole from StructureMap:

appContext.Configure() 
    .Scan(s => s                
        .AssemblyOfType<MovieLister>()
        .AssemblyOfType<SimpleMovieFinder>()
        .Include(t => t.IsDefined(typeof (ComponentAttribute), true))
        .With<TypeNamingConvention>()
        .With<AutowiringConvention>()
    );

Sources, Documentation and Next Steps

At this moment, you can find the sources in my public SVN repository at XP-Dev. Note, that the code will move to SpringSource's CodeConfig for Spring.NET repository here at a later stage.

For a quick overview of what is already possible, check out the various configuration examples for the MovieFinder example there. Documentation is missing, but reading the JavaConfig reference will give you a good overview of the features. For the component-scanning feature read Jeremy Miller's introduction on assembly scanning.

Beware that this is still a moving target, consider it not being more than a preview yet. But I'd love you to grab the sources, play around and let me know what you think - keen on hearing your feedback!

The first milestone of CodeConfig is scheduled for end of January - and don't forget to check, later this day, Spring.NET 1.3.0 GA will be released.

Merry XMLless!

Sniff HTTP traffic with ASP.NET Development Webserver

During developing webapplications or -services, every now and then you will face a strong desire to be able to see the HTTP traffic that is sent back and forth between a client and the webserver.

Tracing local HTTP traffic is not easy

You will quickly find out, that this task sounds easier than it actually is when your client and your server reside on the same machine. There are tools like Fiddler (and tons of others, but this is my favorite), but they all suffer the same problem: Requests to localhost/127.0.0.1 cannot be captured because they are optimized by the windows network stack and bypass the usual hooks used by capturing tools. Thus you either need to use your network card's IP address for submitting requests or - if you don't have a local ipaddress when e.g. you use DHCP and are disconnected from the network - you need to install the MS Loopback Adapter and use this adapter's IP address (see installing MS Loopback Adapter)

ASP.NET Development Webserver is locked down on "localhost"

If you are like me and like the ASP.NET Development Webserver (aka "Cassini" or "WebDev.Webserver") that comes with Visual Studio (and recently also with the Windows 7 SDK), you can't use it to capture traffic. Probably due to legal issues, the ASP.NET Development WebServer (aka Cassini) contains code that binds the TCP socket to the 127.0.0.1 address only and also contains a check, that the requesting client resides on localhost. In this case you have 2 choices:

a) setup a Webapplication in IIS this is possible but nasty when you want to do it in a build script

b) follow the instructions below to patch the WebDev.WebHost.dll on your system

Steps to patch WebDev.WebServer

Note: All steps below assume you have .NET 3.5 Service Pack 1 and the Windows 7 SDK installed. But with basic familarity of IL code you shouldn't have much problems applying those steps to e.g. the version of WebDev.WebServer that comes with VS 2005 (note, that this version resides under the installroot of VS!)

Step 1

Create a new directory "C:\patchcassini" that we will use to work and change to this directory on the commandline. You can create any directory you want, but I will refer to it as C:\patchcassini below.

Step 2

Copy the WebDev.WebHost.dll from the GAC to your working directory and disassemble into an IL script. The following batch script shows how this is done:

set SDKHOME=C:\Program Files\Microsoft SDKs\Windows\v7.0




cd C:\patchcassini

rem refresh copy from GAC
copy /Y C:\Windows\assembly\GAC_32\WebDev.WebHost\9.0.0.0__b03f5f7f11d50a3a\WebDev.WebHost.dll .

REM create backup
copy WebDev.WebHost.dll WebDev.WebHost.original.dll

REM disassemble
"%SDKHOME%\bin\ildasm.exe" WebDev.WebHost.original.dll /out=WebDev.WebHost.il

This will create 2 new files in your folder: WebDev.WebHost.il and WebDev.WebHost.res. Note that the script also creates a backup of the original assembly

Step 3

Patch the generated IL script. There are 2 things you have to do:

1) Open the script in any editor and find & replace all occurrences of [System]System.Net.IPAddress::Loopback with [System]System.Net.IPAddress::Any

2) Find the method body of Connection::get_IsLocal() (just search for this string), you will see something like

.method assembly hidebysig specialname
        instance bool get_IsLocal() cil managed
{
 ....
} // end of method Connection::get_IsLocal

Replace the whole method body with the code as shown below:

.method assembly hidebysig specialname
        instance bool  get_IsLocal() cil managed
{
.maxstack  2
IL_0014:  ldc.i4.1
IL_0015:  ret
} // end of method Connection::get_IsLocal

Step 4

You need to recompile the IL script and reinstall the patched dll into the GAC. Since we also modified a signed dll, we need to turn off signature validation for this dll. The batch script below shows how this is done:

set FRAMEWORKHOME=C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727
set SDKHOME=C:\Program Files\Microsoft SDKs\Windows\v7.0

REM recompile the patched IL script into WebDev.WebHost.dll
%FRAMEWORKHOME%\ilasm.exe /output=WebDev.WebHost.dll /quiet /resource=WebDev.WebHost.res /debug /dll WebDev.WebHost.il

rem Remove validation
"%SDKHOME%\bin\sn.exe" -Vr WebDev.WebHost.dll

rem Install into GAC, forcing overriding any existing assembly
"%SDKHOME%\bin\gacutil.exe" /i WebDev.WebHost.dll /f

Enjoy!

After patching and reinstalling the WebDev.WebHost.dll, hitting F5 in VS to launch your webapplication allows you to access the Webserver using any local IP address and thus allows tools like Fiddler to capture the traffic.

Hint: You can also make your life easier by registering the Webserver in the context menu of any folder. Just use the registry script below:

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Directory\shell\webdev.webserver.9]
@="ASP.NET Webserver 2008"

[HKEY_CLASSES_ROOT\Directory\shell\webdev.webserver.9\command]
@="\"C:\\Program Files\\Common Files\\Microsoft Shared\\DevServer\\9.0\\WebDev.WebServer.EXE\"  /port:81 /vpath:\"/\" /path:\"%1\""

This allows you to launch the Webserver using a rightclick of your mouse on any arbitrary folder:

Hope this helps!