Creating Expressions dynamically to overcome compiler restrictions

April 15th, 2011

Using lamdas to identify a property in a type safe manner has become a popular idiom in the latest microsoft APIs. So for example in EF 4.1 to ignore a property you can enter

public class Address {
   public Person Property {  protected getset; }     
}
...
modelBuilder.ComplexType<Address>().Ignore(a => a.Property);

However the compiler will only allow you to do this for properties with
an accessible getter. So if a property is protected this would be illegal.
Assuming that there is no alternative API and the method will accept a
protected property a workaround is to dynamically create the expression. 

For example 

var prop = (typeof(Address)).GetProperty("Property");
ParameterExpression p = Expression.Parameter((typeof(Address)), "p");
MemberExpression body = Expression.Property(p, prop);
LambdaExpression expr = Expression.Lambda(body, p);

modelBuilder.ComplexType<Address>().
                   Ignore((Expression<Func<AddressPerson>>)expr);

Naked Objects will recognise IQueryables from the next release

December 9th, 2010

For the next release (we don’t yet have a specific date, but it should be within January) we have some fairly big changes coming. These open up some very powerful new capabilities, but there are also some small changes to existing functionality, so we thought it would be helpful to forewarn you.

One of the biggest changes is the introduction of paging. We have this working very nicely now, in house. Return any large collection and Naked Objects will now display a singlepage, and with standard navigation buttons for paging forwards and backwards. The default page size is 20 objects, but you can annotate any given method with a [PageSize(x)] attribute if you wish.

Perhaps more significant is the fact that Naked Objects will be able to recognise IEnumerable<T> and IQueryable<T> (where T is a domain entity type) – not just ICollections. If your method returns an ICollection<T> or an IEnumerable<T> then the entire collection will be retrieved (eacg page refresh) and only the current page viewed. However, if your method returns specifically an IQueryable<T> then the paging mechanism will only retrieve from the database the objects displayed on the current page – which is obviously much more efficient. For this reason, [i]from the next release, we will be strongly recommending that you make all query methods return IQueryable<T>.[/i] This is much more useful anyway as it is easier to write query methods that filter other queries – you can always ToList() them if you need programmatic access to the objects. However, this is optional.

The important change is to the feature that we just released last month: actions contributed to collections. From the next release, such actions will [i]only [/i]be contributed to IQueryable<T> results – [i] they will not be contributed to ICollections[/i]. This to is help reduce the risk of side effects, caused by the ’stateless’ nature of Naked Objects MVC.

For example, let’s say, in the present model, that you had an action contributed to a collection of Payment objects, defined as Print(ICollection<Payment> payments). This is fine if you are invoking it on a collection of payments that you had just retrieved via a query. Now, say, you invoked a method on a Supplier called GeneratePaymentsForAllOutstandingInvoices, returning a collection of Payments. That returned collection would currently offer ‘Print’ as a contributed action. The problem is that, at present, due to the stateless design, Print would first re-invoke GeneratePaymentsForAllOutstandingInvoices, thereby potentially duplicating the payments. (Obviously you can and should guard against duplicated payments in the logic of that method – but it serves as a simple illustration). We don’t want to facilitate accidental side effects like this.

(It has previously been suggested that we might provide an attribute to indicate whether a particular method was [i]idempotent [/i]or not – and this would solve the issue. But many programmers are rightly wary of the risks with that approach.

We have therefore decided to adopt and recommend the following convention:

If a method that returns a collection of objects is a query or other idempotent method then return an IQueryable<T>. As previously stated, for query methods this has big performance advantages. Naked Objects then infers that this method can be safely re-invoked and will contribute appropriate actions to the IQueryable. If your method is not a query or otherwise idempotent, then return an IEnumerable<T> or an ICollection<T> – which is probably more convenient anyway since the collection won’t have arisen directly from a query itself. Then Naked Objects will assume that it is not safe to re-invoke the method and so will not contribute actions. We will be making this distinction clearer at the user interface.

A second benefit from this change, however, is that we can now support ‘covariance’. At present, if you have a Contributed Action that takes a Customer as a parameter then it will also be contributed to sub-classes of Customer (e.g. Store and Individual in AdventureWorks), but if you have a contributed action that takes ICollection<Customer> as a parameter, then this will not be contributed to an ICollection<Store> because ICollection<Store> is not a sub-type of ICollection<Customer>. However in the new model (next release) a contributed action that takes IQueryable<Customer> will also be contributed to an IQueryable<Store> because we have been able to make this covariant.

Using the WatiN test framework with Naked Objects MVC

November 26th, 2010

I have recently been experimenting with the open source WatiN framework (see http://watin.sourceforge.net/) to write end-to-end tests for applications written in Naked Objects MVC. The following example code is lifted directly from their website, and shows how you can write a test by launching a browser (you can even parameterise your tests so that you repeat them all with several different browsers):

[TestMethod]
public void SearchForWatiNOnGoogle() {
using (var browser = new IE("http://www.google.com")) {
browser.TextField(Find.ByName("q")).TypeText("WatiN");
browser.Button(Find.ByName("btnG")).Click();
Assert.IsTrue(browser.ContainsText("WatiN"));
}
}

(I just changed their [Test] attribute to [TestMethod] because I use it with the Microsoft test framework rather than NUnit.)

Tests of a realistic application can get rather long-winded. But here I’ve found that Naked Objects MVC offers a big advantage, because the (default) user interface is completely generic, and involves a small fixed set of HTML patterns. It was thefore a straightforward matter to write a set of helper methods that translate standard user actions into the WatiN idioms. The net result is tests that are easy to write and read. Here’s a small fragment of such a test, running against our AdventureWorks demo:

[TestMethod]
public void RetrieveOrdersForACustomer() {
browser.ClickAction("CustomerRepository-FindCustomerByAccountNumber");
browser.GetField("CustomerRepository-FindCustomerByAccountNumber-AccountNumber").InputText("AW00000546");
browser.ClickOK();
browser.ClickAction(”Store-SearchForOrders”);
browser.GetField(”OrderContributedActions-SearchForOrders-Customer”).AssertObjectHasTitle(”Field Trip Store, AW00000546″);
browser.GetField(”OrderContributedActions-SearchForOrders-FromDate”).InputText(”01/01/2003″);
browser.GetField(”OrderContributedActions-SearchForOrders-ToDate”).InputText(”31/12/2003″);
browser.ClickOK();
br.AssertPageTitleEquals(”4 Sales Orders”);
// etc
}

We’ve also got some simple DatabaseUtils methods that make it easy to back-up and/or restore the database between tests:

private const string server = @"MyServer\SqlExpress";
private const string database = "AdventureWorks";
private const string backup = "AdventureWorksDBAsShipped";

[ClassInitialize]
public static void InitialiseClass(TestContext context)
{
DatabaseUtils.RestoreDatabase(database, backup, server);
}

I tend to do the restore at class level, and ensure that all tests in a class are working on non-overlapping data sets; but you could equally well restore at method level (using [TestInitialize]) for even greate safety at the cost of slower test performance.

I’m planning to include these helper methods in the Naked Objects framework at some future point. In the meantime, if anyone is interested to start using WatiN with Naked Objects MVC, just get in touch with me offline and I’ll happily send you an early version of them.

Using the Entity Designer Database Generation Power Pack

September 2nd, 2010

I have several times commented that the thing I would most like to see added to Entity Framework would be automatic management of database schema changes while preserving data.  Somehow I managed to miss the  Entity Designer Database Generation Power Pack, which does just that. I have used it with Naked Objects MVC and it allowed me to evolve my model, preserving the data that I had entered.

The power pack is clearly experimental, and suffers from a number of limitations. For example, it copes well with additions of new entities , new properties on existing entities, and changes to the attributes (e.g. string length) on existing properties.  However, it does not cope with renaming, or deletion, of existing entities or properties.  For that you will need to revert to SQL queries.

But it is to be hoped that Microsoft will address these limitations in the next release  -  and that the power pack will eventually be included within the Visual Studio release.

Also, at present this capability is designed to work with Entity Framework in ‘Model first’ mode, but there is no reason (that I can see) why this shouldn’t in future be compatible with Code first.  In the interim, finding this power pack has shifted my own focus back from Code first to Model first.

I also got to thinking that the combination of Naked Objects MVC with this Entity Designer Database Generation Power Pack gives us a real competitor to Rails as a tool to support iterativer development of data-oriented web applications.  To be sure the Rails capabilities for data migration go further than this, but then Naked Objects MVC has many advantages over Rails in other ways  -  not least because it uses generic views rather than code-generated scaffold views  – so you end up with far less code to maintain.

Richard

Running NakedObjects MVC on Microsoft Azure

August 13th, 2010

We’ve experimented and discovered that deploying a NakedObjects MVC application on Microsoft Azure is very straightforward.  These are the high level steps we went through -

Prerequisites

- A working NakedObjects MVC application with database
- Microsoft Azure developer account
- Microsoft Azure SDK installed on your PC

First create a new SQL Azure server and database. Connect to it via Sql Server Management Studio or VS2010. We populated the schema from our local database by using the .edmx.sql file you get when when you ‘Generate Database from Model’ on the .edmx file, then we populated the data with some generate scripts from our local database.  (To connect see here)

Next stage was to create a new ‘Cloud’ project in VS2010 with a single MVC2 Role, using the ‘Add new project’ templates. Then you can either copy across all your controllers, views etc from your existing MVC project to the new MVC web role project or alternatively copy across the Web.Role file and merge the config into your existing project. In the second case update the Web Service to point to your existing project and delete the unused Web Role template.

Now update the connection string in the web.config file to point to the SQL Azure database (see here) and you should be able to run the application locally in the Azure simulation environment against the Sql Azure database.

Once that works ‘Publish’ the service and the on the Azure site create a new service and deploy. Once the service is ‘Ready’ you should be able to navigate to the URL and see your application running in the cloud.

‘Code First’ development using Entity Framework

August 4th, 2010
Working ‘Code First’
A couple of weeks ago, Microsoft  released it’s fourth version of the Entity Framework Feature CTP (http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=4e094902-aeff-4ee2-a12d-5881d4b0dd3e), which enables ‘Code First’ development.

A couple of weeks ago, Microsoft  released it’s fourth version of the Entity Framework Feature CTP, and we have just released a new version of Naked Objects MVC to work with this ‘CTP4′.

This series of Feature CTPs is concerned with what Microsoft calls ‘Code First’ development (using Entity Framework).  What they mean by that is that you should be able to write your domain objects as pure POCOs, and have them persisted on a relational database, without having to write any additional code or even configuration.  This is an impressive idea, but the earlier versions of Code First (CTPs 1,2,3)  had a number of severe limitations that meant that Code First was not really suitable for prime time development. With CTP4 those limitations have been addressed and it appears to us that Code First is now a viable idea for mainstream development.

You can read about the various specific improvements made to Code First on the ADO.NET team blog and elsewhere.

Philosophically, Code First and Naked Objects follow the same philosophy:  use ‘conventions’ first, annotation (attributes) second, and explicit coding 0r configuration third.  But its more than just a philosophical match.

The combination of the latest incarnation of Code First and Naked Objects MVC  is strongly synergistic.  Code First basically looks after everything underneath the domain objects and Naked Objects looks after everything on top of them.  The net result is that you can build a complete web application for a complex domain model, solely by writing domain objects as pure POCOs.

Richard

Concurrency checking, ASP.Net MVC, Entity Framework and Naked Objects

July 22nd, 2010

The Entity Framework (EF) approach to concurrency is to mark one or more fields as ‘ConcurrencyMode=Fixed’. This can be done either directly in the edmx file or via the Entity Data Model Designer. The EF will then use the original value of the field as a condition on any update. If the value has changed the update will fail and a concurrency exception will be thrown.

This works well for a long running context and so we been happy to use it for thick-client implementations of Naked Objects. However the MVC implementation uses a context that only survives for the duration of the web transaction. The EF concurrency check will then only check for changes during that transaction – this is of some value but it doesn’t catch the situation where an object is opened for edit (in one transaction) changes are made and then submitted (in another transaction).

As a slight aside here we need to be aware of how Naked Objects MVC handles the persistence of an object from one transaction to the next. Naked Objects passes an identity token (in the URL) which is then used to repopulate objects from the data store. These are then used to make any changes or run any actions and then save changes to the store. This has the advantages that it is completely stateless – everything is in the Http Request – and cleanly decoupled from the object store implementation. It does though have the disadvantage of an additional object store query.

With EF an alternative would be to serialize all or some of the object state, recreate the entity and attach to the context. This is an option for the future but there’s quite a few hoops to jump through to get this working, especially in an object store independent way and within the constraints of the existing Naked Objects architecture. So for the moment simplicity and object store independence win the day.

So back to concurrency, it seems to be possible to set the token in the object and tell EF to accept it. Then when other changes are applied and the object saved the standard EF concurrency checking should kick in. This is obviously a good solution in that it’s a common concurrency mechanism across thick-client and MVC Naked Objects implementations. This would be easiest if we had adopted the reattaching EF mechanism but should be possible with our current object retrieval mechanism. However it once again means coupling ourselves to EF or building a lot of EF specific stuff into our generic object store API.

Finally then the simple approach we adopted is to save the concurrency token as a hidden field on object edits and views. For edit sessions the original value survives the length of the edit (which may involve intermediate posts). Then when the edited object is saved or an action is invoked we check against the current database value (which we’ve populated in the object anyway) and fail concurrency if it’s changed. This is simple, obvious what’s going on if you want to customize the application and fits in cleanly with our existing architecture. The only disadvantage is that we need a way of identifying the concurrency token fortunately Microsoft have provided a new attribute – ConcurrencyCheckAttribute – which we use. As this is also used in EF Code First to mark those properties that are “ConcurrencyMode=Fixed”  this is all that is necessary for Code First development. For Model and Database first the same fields marked “ConcurrencyMode=Fixed”  should also be annotated ‘ConcurrencyCheckRequired’.


Naked Objects MVC: a milestone for Naked Objects – and for MVC

July 9th, 2010
Earlier today we released the Evaluation Version of our newest product – Naked Objects MVC – which combines the power of Naked Objects for .NET with ASP.NET MVC. (If you haven’t seen this, visit our website, watch the demo videos and download the free Evaluation Version).
 
 This is a milestone for Naked Objects, for two reasons: it means that we can generate a pure HTML user interface from a domain object model, and it means that we have an interface that is fully customisable.
 
I also think that this is a milestone for MVC, and here I don’t just mean ASP.NET MVC, I mean the MVC pattern in general. Let me explain why.The ‘naked objects pattern’ was the product of my PhD research, which I undertook at Trinity College, Dublin. In my thesis I was quite critical of the impact that the MVC pattern had made on object-oriented design. Although this was never the intent of the MVC pattern, in practice it had led to people putting too much domain logic in the Controllers, and even the Views – resulting in ‘anaemic model objects’.

The original idea contained in the thesis was that it ought to be possible to create a user interface that was a direct reflection of the Model, without requiring any UI coding at all. A framework that implemented this ‘naked objects pattern’ would probably use Views and Controllers internally, but they would be invisible to the domain model programmer.

When I completed my thesis in 2003 my supervisor faced the usual challenge of trying to find an external examiner who could do it justice. You can imagine my feelings when he decided to invite Trygve Reenskaug – who first proposed the MVC pattern back in 1978. But Tyrgve was very gracious, and in fact I was awarded my PhD before any cross-examination by the panel of examiners (which is quite unusual). At the viva (defense) Trygve also revealed something that did not appear in any of the literature. He said that it was always the intention of his team that Models should have default Views/Controllers, but that they had never realised that idea.

Ever since then, I have had the dream of a framework that would allow you to span both worlds: to have a default View/Controller that would give you a complete user-interface for free (and not just CRUD operations – full object behaviour as well), but then allow you to write your own Views and/or Controllers as you needed. And, preferably, I wanted to realise that dream using someone else’s MVC framework, rather than invent a new one from scratch.

Microsoft’s ASP.NET MVC framework (specifically the second release ‘MVC 2′) proved up to the job. If you look at the Naked Objects MVC implementation you’ll see that we manage with just a single GenericController and only half a dozen generic views. An experienced ASP.NET MVC programmer could probably re-create this from scratch in a few weeks. But, like the proverbial swan (elegant on the surface, but lots of paddling beneath it) this only works because of the phenominal capabilities of the existing Naked Objects for .NET framework underneath.

Richard

 

 

Entity Framework: helping to bridge the cultural gap between DBAs and Programmers

September 23rd, 2009

Microsoft’s Entity Framework acts as a technical bridge between the domain of objects and the domain of the database – as does any Object-Relational Mapper. Over the last year or so I have worked quite extensively with Entity Framework, in real business situations, and I have made the following observation: Entity Framework (combined with LINQ) not only bridges the technical gap, but can also help to bridge the cultural gap between DBAs and application programmers. Let me explain.

For many years I have been a strong advocate of object-oriented design and development. The Naked Objects pattern is really just the logical outcome of a commitment to very pure OO design.

I have long argued that the people who ought to embrace this concept most enthusiastically are DBAs. DBAs are used to working with entity-relationship models; an object model is really just an entity-relationship model, but with behaviour encapsulated on those entities. Normalisation – eliminating duplicate data – is a fundamental discipline for DBAs; good object modelling requires the same discipline, but with the scope expanded to eliminate duplicate functionality. DBAs are just as capable of writing behaviour as programmers – it’s just that they typically write it in SQL rather than a object-oriented language.

Yet in my experience, DBAs have often been amongst the most reluctant of parties to buy into the object-oriented way. Some of this reluctance might be attributed to traditional shibboleths such as fear of change, or protection of turf. But we all exhibit these traits to some extent, and I don’t think that is a sufficient explanation.

A deeper issue is that many DBAs distrust application programmers: they can’t be relied on to consistently enforce business rules; or to ensure data integrity; or to design systems that will perform under load. This is not without historical justification, but one can’t help feeling that this excuse is being over-used. The distrust is reciprocated: programmers (at least those who are committed to agile development) argue that DBAs are not remotely interested in agility – they will think nothing of duplicating business logic that already exists in the application code, or creating hundreds of hard-to-maintain stored procedures with much duplication between them.

Whatever the historical justification for these opinions, they are perpetuated by an over-the-wall methodology: when the application developers have completed their analysis they throw their design over the wall to the DBAs. What comes back is a database schema and/or a set of stored procedures for accessing it. As long as both sides confirm to this interface, both parties are happy to keep the communication to the minimum. And what’s wrong with that, you ask? Isn’t the essence of good design all about maintaining clean interfaces? Not if both sides are independently creating a separate internal representation of the same business requirements.

Entity Framework can help to eliminate this over-the-wall style of operation; coupled with Naked Entities, the effect is even more striking. Using the Code Only mechanism (of Entity Framework v4): as soon as the application designers have modelled any part of the business requirements as a domain objects, they can immediately create a first cut at a database schema – which the DBAs can immediately validate or criticise. In my experience, involving a DBA right from the beginning like this can bring benefits – a DBA focuses on issues that application programmers tend to overlook. ‘OK, so you’ve got a String property there, but does it need to be a VarChar(MAX) – what’s the maximum expected length? Is it Nullable or not? Can it be constrained to one of a set of values? Which properties constitute the key? What constitutes a duplicate instance?’ At a broader level, early translation of an object model into a database schema can show up flaws and/or potential improvements to the object model itself.

By asking these questions early on, we can refine the database schema in parallel. Moreover the answer to these questions could be translated back into code constraints, using annotations or other methods and, most importantly, automated tests can be written to ensure that they are enforced. In this climate there is much more scope for constructive negotiation as to whether a rule should or shouldn’t be duplicated between application and database.

Conversely, if a new application is to be built on top of an existing database then EF allows you to create a first iteration of the object model automatically. To be sure this object model will need refinement, because it is unlikely that an existing database schema will have been optimised for an object-oriented design anyway. And DBAs need to be open to the idea that the act of translating a database schema into an object model can show up some flaws and/or potential improvements to that database schema. (I certainly found this when I created our demonstration application from the existing AdventureWorks database). But object modellers would do well to recognise that a database schema may well form a very good first pass at the object model.

The challenge of modelling associations as interfaces (2)

September 15th, 2009

In several previous postings I have whined about the fact that Entity Framework does not support the modelling of associations that are defined by an Interface rather than a Class, and explained why this capability is important. In my last posting on this subject (http://blog.nakedobjects.net/?p=60 ) I posited a workaround.

With the release of Naked Entities, which runs against Entity Framework v4, I’m pleased to say that we have implemented a much more elegant workaround. Still not perfect, and still a workaround, but much better. It draws on two capabilities:

- Complex Types. This allows the ‘two part foreign key’ (one column defining the Type of the associated object, and the other defining the ID of the instance) to be treated as a single object.

- A new, templated, class in our App Lib called InterfaceAssociation

This, combined with a new code snippet (shortuct: ‘intass’) means that you can implement one of these associations in a jiffy.

I’ve recorded the folliwing short video tutorial, to shows how this works:  http://www.nakedobjects.net/video/interface_associations.shtml