Serialization in C#, shortcomings, advantages and evolution (.NET 1.0 to 3.5)

One topic i like to get into about when it comes to C# is serialization. I am a strong believer in the power this has brought to solving programming problems in a very trivial manner. Today i am going to discuss about serialization issues in C#. Particularly multi dimensional arrays and some disadvantages/advantages of using various serialization classes. First of all i will give a run down on serialization in .NET using C#.

Serialization is a concept in programming (particularly .NET in this discussion anyway) that can be used to construct an already instantiated object/data structure into an output (in the form of a stream of bytes which could be XML or binary). This output is said to be serialized and can be used to reconstruct that object into an identical object again with the same state etc. Reconstructing this output (serialized) data into an actual object instance again is called deserializing. This is particularly useful for web services and local applications that are network based. We can easily save data structures to disk or share them over a network and allow network applications to communicate with one another easily.

There is 3 main classes used for serialization in C#. They are SoapFormatter, BinaryFormatter and DataContractSerializer (.NET 3.5). There is also XmlSerializer but that is beyond the scope of this article and i am deliberatelly avoiding it because it is unuseful in the fact it cannot serialize private data members of a class. So anyway, the three classes have fundamental differences and advantages.

SoapFormatter is a serializer based on the Simple Object Access Protocol. This protocol was invented as a specification for storing and exchanging structured information in XML over networks via e.g. HTTP. It is common in web services today and since it uses XML and SOAP it is an open standard with active parsers in a lot of languages. This means cross platform support for it is good.

A brief simple example of serialization of a string using SoapFormatter:

string strMyName="Ronan", strNameDeserialized="";

MemoryStream nameStream = new MemoryStream();
SoapFormatter formatterName = new SoapFormatter();

// serialize our data structure into a stream of SOAP XML bytes
formatterName.Serialize(nameStream, strMyName);

nameStrm.Position = 0;

// deserialize the serialized output of bytes into a string again
strNameDeserialized=(string)formatterName.Deserialize(nameStream);

BinaryFormatter on the other hand is s serializer that is less open. The constructed output is in binary, which makes it less suitable for deploying on a network. This is because binary uses NULL bytes in it which would break anything null terminated that is string parsed. It is proprietary by Microsoft which means you will not be able to serialize/deserialize the object unless you are using a .NET language. However, it is still useful in the sense that the serialized object output is much more compact than SoapFormatter, even though the output is not as readable.

An example of serialization of a string using BinaryFormatter:

string strMyName="Ronan", strNameDeserialized="";

MemoryStream nameStream = new MemoryStream();
BinaryFormatter formatterName = new BinaryFormatter();

// serialize our data structure into a stream of binary bytes
formatterName.Serialize(nameStream, strMyName);

nameStrm.Position = 0;

// deserialize the serialized output of bytes into a string again
strNameDeserialized=(string)formatterName.Deserialize(nameStream);

To serialize instances of our own custom classes the class definition has to be prefixed with [Serializable] attribute for it to be serializable. Most inbuilt classes in .NET library have this set by default, so this is why we do not do it in our examples. [NoneSerialized()] can also be used on class members not to be serialized. Also sometimes this needs used on data members in a class that is being serialized because sometimes certain data member' s types are not serializable, other times we just dont want a certain data member' s state to be remembered so we dont serialize it). Information about all this can be found at the msdn site.

SoapFormatter generic support limitation?

All is good and well except for one main problem. Since .NET 2.0 generics were introduced into the language. This was a great feature, however the new language feature did not bring with it the needed updated SoapFormatter support to facilitate the serializiation of classes that were either generic or had generic members ! Microsoft to this day discontinued support for SoapFormatter. To see a demonstration of what i mean, either run the below code or place a breakpoint on the catch statement and run it. It should yield figure a.

List<string> strItemsList = new List<string>();

strItemsList.Add("FirstItem");
strItemsList.Add("SecondItem");

using (MemoryStream memStrm = new MemoryStream())
{
SoapFormatter formatterName = new SoapFormatter();

try
{
formatterName.Serialize(memStrm, strItemsList);
memStrm.Position = 0;
List strItems2List = (List<string>)formatterName.Deserialize(memStrm);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}

Figure A.



I encountered this downfall to a painful disadvantage in one of my projects. Particularly the battleships application i wrote for college one year. In Visual studio 2005 there was no fix to this except to force yourself to not use generics for anything that would be serialized. Which could mean refactoring your code if you already had generic classes in place. This takes away the benefits of generics altogether !

New hope with DataContractSerializer (support for generic serialization)

However, there is light at the end of the tunnel. Since Visual Studio 2008 was released, Microsoft released .NET 3.5. Since work on SoapFormatter was discontinued (although still bundled with the software), there was a new replacement class included by the name of DataContractSerializer. This class, by comparison offered serialization of generics at last ! Although this took 3 years to come about, i thought i would go about and refactor an old college project i had done to see if it would fix the serialization of generics problem that had plagued my application. Upon doing so, i discovered that infact this class also had its flaws too.

No support for multi dimensional arrays ?

Central to the functioning of my battleships project design was multi dimensional arrays, also known as rectangular arrays in C#. It turns out that with my playing around i found out to my dismay that DataContractSerializer did not support serialization of rectangular arrays. Below is sample code that declares a rectangular array to invoke such an outcome to demonstrate it:

int[,] myArray = { { 1, 2 }, { 3, 4 }, { 5, 6 }, { 7, 8 } };

using (MemoryStream memStrm = new MemoryStream())
{
DataContractSerializer ser = new DataContractSerializer(typeof(int[,]));

try
{
// serialize
ser.WriteObject(memStrm, myArray);
memStrm.Position = 0;
//deserialize
int[,] myArray2 = (int[,])ser.ReadObject(memStrm);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}

The actual exception being caught can be seen in figure b:

Figure B.



How to overcome this ?

I found this odd because SoapFormatter had previously serialized multi dimensional arrays no problem. It would appear that with one step forward gained in one area with DataContractSerializer, one step backward came in another area. Which is a shame really since i really needed multi dimensional arrays. At the time, .NET 3.5 was only new. DataContractSerializer was only new and no one knew much about it, nevermind it' s shortcomings ! For a year or two after, when the .NET 3.5 framework was more widely adopted and known about, i was randomly browsing around the net to see if anyone else had run into this shortcoming. There was a thing on a Microsoft Msdn forum that discusses the problem on August 2009, it can be found here. In it a Microsoft designer of the .NET framework acknowledges the shortcomings and discusses some workarounds for the limitation. It turns out that DataContractSerializer includes support for Jagged arrays and not Rectangular arrays. So a bit of code refactoring and changing to jagged arrays should fix this.

With a few minor differences and the fact we have to explicitly define the size of each second dimension of the jagged array seperately because jagged arrays are essentially arrays of arrays and each index of the first dimension can have a separatelly defined index count for the second dimension.

For example, for the above piece of sample code to actually work with dataContractSerializer it would be changed to:

int[][] myArray = new int[][] { new int[] { 2, 3, 4 }, new int[] { 5, 6, 7, 8, 9 } };

using (MemoryStream memStrm = new MemoryStream())
{
DataContractSerializer ser = new DataContractSerializer(typeof(int[][]));

try
{
// serialize
ser.WriteObject(memStrm, myArray);
memStrm.Position = 0;
//deserialize
int[][] myArray2 = (int[][])ser.ReadObject(memStrm);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}

Alternatively, SoapFormatter will serialise rectangular arrays no problem at all as long as generics are not being used. Although in my oppinion generics are essential and it is too much of a price to drop them just to be able to serialise multi dimensional arrays. It is also too much of a price (assuming the project is large) to refactor, drop all multi dimensional arrays and change them to their jagged array equivalent. The link i included above about the forum includes code by a Microsoft engineer that shows how to convert multidimensional array into a jagged array on the fly ! This will evidently not be as fast as a solution purely based on jagged arrays as it incurs some overhead but it temporarily solves the problem i suppose and is well worth a read. For anyone implementing something that is going to be serialized using DataContractSerializer i would recommend making sure you are using jagged arrays and not multi dimensional arrays. It really will save a lot of hassle having to refactor it later.

BinaryFormatter unaffected

Also, do bare in mind that BinaryFormatter does also support generics and multi dimensional arrays. Although due to the limitation above and the fact it is not suited for network application or none Microsoft OS/ Languages, it really has limited application. Although it is still perfectly suitable for for example storing application configuration data to disk (so state or field data can be reloaded at a later time) on a windows machine. Although all serializers can be used for this and are not just restricted to network use ! A common trick i use it for is to store a file with the serialized data so i can pause my application, close the application and open it again at a later date to continue off from where i was.

Visual Studio 2010 is almost here and with it, it brings .NET framework 4.0. I have not had time to try it out yet or get a copy of it. I am looking forward to getting my hands on a copy though, getting comfortable with the new features and improvements (if any) that are made on serialization. When i do i may write another blog about it.

- Ronan