Bad Code: Too Many Object Conversions Between Application Layers And How to Avoid Them

Jakub HolyMay 30th, 2013Last Updated: May 30th, 2013

0 54 7 minutes read

Have you ever worked with an application where you had to copy data from one object to another and another and so on before you actually could do something with it? Have you ever written code to convert data from XML to a DTO to a Business Object to a JDBC Statement? Again and again for each of the different data types being processed? Then you have encountered an all too common antipattern of many “enterprise” (read “overdesigned”) applications, which we could call The Endless Mapping Death March. Let’s look at an application suffering from this antipattern and how to rewrite it in a much nicer, leaner and easier to maintain form.

Note: This is more a design discussion than bad vs. good code but I still think it fits this blog. The code for the “good” part is available at GitHub.

The application, The World of Thrilling Fashion (or WTF for short) collects and stores information about newly designed dresses and makes it available via a REST API. Every poor dress has to go through the following conversions before reaching a devoted fashion fan:

Parsing from XML into a XML-specific XDress object
Processing and conversion to an application-specific Dress object
Conversion to a MongoDB’s DBObject so that it can be stored in the DB (as JSON)
Conversion from the DBObject back to the Dress object
Conversion from Dress to a JSON string

Uff, that’s lot of work! Each of the conversions is coded manually and if we want to extend WTF to provide information also about trendy shoes, we will need to code all of them again. (Plus couple of methods in our MongoDAO, such as getAllShoes and storeShoes.) But we can do much better than that!

Eliminating the Manual Conversions

It’s time-consuming, error-prone and annoying to code all the conversions while you actually want to use your time to build business logic and not some boilerplate code. We can eliminate the manual work in two ways:

Generalize the conversions so that they only need to be written once (likely leveraging existing conversion libraries)
Eliminate them, e.g. use the same data format through the complete processing chain

To be fair, I have to admit that the manual approach also has some advantages: you have full (fool? ) power over the form of the objects and can fit them perfectly for the processing required, you don’t need to introduce huge and buggy libraries, and you earn more money, if paid by LOC.

However the disadvantages named above nearly always overweight the advantages, especially if you reuse suitable, mature, high-quality libraries that allow you to customize the processing to any detail on an on-need basis.

One question remains: how do we represent the data? We have two possibilities:

With generic data structures, i.e. maps. This is common in dynamic functional languages such as Clojure and it is extremely easy and comfortable.
- Pros: Less work, very flexible, generic operations can be applied easily (map, filter, etc.)
With objects specific for each data type, i.e. POJOs such as DressVariant, Shoes
- Pros: Type safety, the compiler helps to ensure that your code is correct, it might be easier to understand
- Cons: You have to write and maintain a class for each possible data element being processed

Sidenote: The Business Domain

You might skip this section and only come back later if you want to understand the reasoning behind the design.

WTF has to do some processing of the dress elements that it retrieves, mainly because multiple elements may represent the same dress only with slight variations such as color. WTF thus stores such a group of related elements as a list of DressVariant items inside a parent Dress object, generates a unique ID for the Dress and stores the IDs of the input elements in an attributed named “externalIds”. Therefore N input elements becomes M Dress elements with 1+ DressVariants, M <= N.

WTF also has to do some other processing on its WTF XML input such as detecting which images are real and which are just fake placeholders but we won’t discuss that.

Implementing the Static-Typed Generic Processing

I’ve decided to keep having a class per data type not to diverge too much from the current implementation. How do we now make the manual conversions generic and reusable?

Let’s first see how I would like to construct the processing pipeline:

fetchFrom("http://wtf.example.com/atom/dresses.xml")
   .parseNodesAt("/feed/dress")
   .transform(DressVariant.class, new DressDeduplicatingTransformer()); // Transformer
   .transform(new PojoToDboTransformer()); // Transformer
   .store(new MongoDAO());
// + we'll use DBObject.toMap() + PojoToJson mapper when serving the data via REST

So we fetch XML from a URL, send it to a parser to extract some nodes that are automatically converted to DressVariant objects, next we use a transformer that merges multiple DressVariants into a single, unified Dress object, and finally we convert the resulting POJO into a Mongo DBObject before storing it into the DB. What do we use for the conversions?

XML -> DressVariant: Use JAXB to convert Nodes to our POJO annotated with @XmlRootElement. Notice that you can customize the conversion that JAXB performs very much, if the need be. Thus you only need to create a simple POJO and add one annotation.
DressVariant -> Dress: We will check the MongoDB and either send further an existing Dress or a new Dress object with this DressVariant added (this will result in multiple updates if the dress really has multiple formats in the input feed, but that isn’t a problem for us). This conversion is type-specific, i.e. for each data type we have to code its own transformation. That is good because for example Shoes don’t need any such deduplicating processing/converting.
Dress -> DBObject: We will use the Jackson Mongo Mapper, and extension of the first-class JSON mapping library, that adds support for Mongo DB. It will also performs some special data sanitization required by Mongo, such as replacing ‘.’ in map keys with ‘-’.
DBObject -> MongoDB: We will have one generic method, storeDocument(String collectionName, DBObject doc), where the collection name is derived from the original object (e.g. DressVariant -> “dressVariants”). The doc’s attribute id is expected to be its unique identificator (and thus we will either update or insert based on its [missing] value).
MongoDB -> DBObject: Again a generic method, list(String collectionName)
DBObject -> Map: The DBObject does that itself
Map -> JSON: We will use the PojoMapping feature of the Jersey REST library to automatically convert the Map produced by our methods to JSON when sending it to the clients.
JSON -> clients: We will have one GenericCollectionResource with a list method mapped to the URL /list/{collectionName}”. It will load the collection from Mongo as described and return a List, automatically converted to JSON by Jersey.

Result: Aside of custom data-type-specific transformations, instead of 1 POJO, 4 hand-coded converters, and 2+2 methods for each data type we now need only 1 POJO per data type plus one generic converter, 4 generic methods and one or two libraries. Less coding, less code, less defects, more productivity, more fun.

Notice that thanks to our choice of libraries, if the default conversion schemas turn out not to be sufficient for us, we can tweak them as much as we want – though we most certainly don’t want to go that way. It’s better to sacrifice some flexibility and more fit data formats than doing too many tweaks, struggling with the mapping libraries instead of leveraging them. A wise man chooses his battles.

Sample Code

Sample code demonstrating automatic, generic mappings XML -> Java -> Mongo -> REST with JSON is available at GitHub – generic-pojo-mappers.

Summary and Conclusion

Many applications force developers to convert data between a number of objects, which is very unproductive and error-prone. A better approach is to avoid the conversions and use the same object throughout the whole processing as much as possible, doing conversions only when really necessary. These conversions are better written in a generic and reusable way than hand-coded for each data type and it often pays off to use an existing, mature mapping library for that (though you must make sure your intended use is aligned with its philosophy and design).

Using the same object throughout the processing causes it to be less fitted for the individual processing stages but it makes them much easier and faster to write and maintain. We lose some performance due to using reflection but that is negligible with respect to the I/O (retrieving a file over HTTP, sending data to a DB) and XML parsing.

In the example of the World of Thrilling Fashion, we have cut the amount of manual coding and methods considerably and the result is a smaller, cleaner, and more flexible code (w.r.t. adding new data types).

Criticism

“But I really need to use objects fine-tuned for each layer of processing!”
Your choice, if you really need it, do it – but be aware of how much you pay for it.

“Libraries are evil!”
Well, yes. Sometimes it’s better to hand-code things but not always. Make sure that you don’t use a library in a way different than intended because then you might lose more time fighting it than being productive.

“You are an idiot!”
Yes, many people think so. Thank you for reading.

“Do you say that I’m an idiot if I wrote code like that?”
Not at all, you might have good reasons to do so. Or you might not know the alternatives. Or you just haven’t such a strong dislike of writing mindless code as I do. That’s OK.

The rich Persistent Domain Object + slim Gateway patterns by Adam Bien also make it possible to use the same object (a JPA entity) throughout all the application (web UI – DB) in the name of increased productivity.