I recently have had a chance to work with MongoDB (as in humongoous), which is a document-oriented database written in C++. It is ideal for storing documents which may vary in structure, and it uses a format similar to JSON, which means it supports similar data types and structures as JSON. It provides a rich yet simple query language and still allows us to index key fields for fast retrieval. Documents are stored in collections which effectively limit the scope of a query, but there is really no limitation on the types of heterogeneous data that you can store in a collection. The MongoDB site has decent docs if you need to learn the basics of MongoDB.
MongoDB in Java
The Mongo Java driver basically exposes all documents as key-value pairs exposed as map, and lists of values. This means that if we have to store or retrieve documents in Java, we will have to do some mapping of our POJOs to that map interface. Below is an example of the type of code we would normally have to write to save a document to MongoDB from Java:
BasicDBObject doc = new BasicDBObject();
doc.put("user", "carfey");
BasicDBObject post1 = new BasicDBObject();
post1.put("subject", "spam & eggs");
post1.put("message", "first!");
BasicDBObject post2 = new BasicDBObject();
post2.put("subject", "sorry about the spam");
doc.put("posts", Arrays.asList(post1, post2));
coll.insert(doc);
This is fine for some use cases, but for others, it would be better to have a library to do the grunt work for us.
Enter Morphia
Morphia is a Java library which acts sort of like an ORM for MongoDB – it allows us to seamlessly map Java objects to the MongoDB datastore. It uses annotations to indicate which collection a class is stored in, and even supports polymorphic collections. One of the nicest features is that it can be used to automatically index your collections based on your collection- or property-level annotations. This greatly simplifies deployment and rolling out changes.
I mentioned polymorphic storage of multiple types in the same collection. This can help us map varying document structures and acts somewhat like a discriminator in something like Hibernate.
Here’s an example of how to define entities which will support polymorphic storage and querying. The Return class is a child of Order and references the same collection-name. Morphia will automatically handle the polymorphism when querying or storing data. You would pretty much do the same thing for annotating collections that aren’t polymorphic, but you wouldn’t have multiple classes using the same collection name.
Note: This isn’t really an example of the type of data I would recommend storing in MongoDB since it is more suited to a traditional RDBMS, but it demonstrates the principles nicely.
@Entity("orders") // store in the orders collection
@Indexes({ @Index("-createdDate, cancelled") }) // multi-column index
public class Order {
@Id private ObjectId id; // always required
@Indexed
private String orderId;
@Embedded // let's us embed a complex object
private Person person;
@Embedded
private List<Item> items;
private Date createdDate;
private boolean cancelled;
// .. getters and setters aren't strictly required
// for mapping, but they would be here
}
@Entity("orders") // note the same collection name
public class Return extends Order {
// maintain legacy name but name it nicely in mongodb
@Indexed
@Property("rmaNumber") private String rma;
private Date approvedDate;
private Date returnDate;
}
Now, below I will demonstrate how to query those polymorphic instances. Note that we don’t have to do anything special when storing the data. MongoDB stores a className attribute along with the document so it can support polymorphic fetches and queries. Following the example above, I can query for all order types by doing the following:
// ds is a Datastore instance
Query<Order> q = ds.createQuery(Order.class).filter("createdDate >=", date);
List<Order> ordersAndReturns = q.asList();
// and returns only
Query<Return> rq = ds.createQuery(Return.class).filter("createdDate >=", date);
List<Return> returnsOnly = rq.asList();
If I only want to query plain orders, I would have to use a className filter as follows. This allows us to effectively disable the polymorphic behaviour and limit results to a single target type.
Query<Order> q = ds.createQuery(Order.class)
.filter("createdDate >=", cutOffDate)
.filter("className", Order.class.getName());
List<Order> ordersOnly = q.asList();
Morphia currently uses the className attribute to filter results, but at some point in the future is likely to use a discriminator column, in which case you may have to filter on that value instead.
Note: At some point during startup of your application, you need to register your mapped classes so they can be used by Morphia. See here for full details. A quick example is below.
Morphia m = ... Datastore ds = ... m.map(MyEntity.class); ds.ensureIndexes(); //creates all defined with @Indexed ds.ensureCaps(); //creates all collections for @Entity(cap=@CappedAt(...))
Problems with Varying Structure in Documents
One of the nice features of document-oriented storage in MongoDB is that it allows you to store documents with different structure in the same collection, but still perform structured queries and index values to get good performance.
Morphia unfortunately doesn’t really like this as it is meant to map all stored attributes to known POJO fields. There are currently two ways I’ve found that let us deal with this.
The first is disabling validation on queries. This will mean that values which exist in the datastore but can’t be mapped to our POJOs will be dropped rather than blowing up:
// drop unmapped fields quietly Query<Order> q = ds.createQuery(Order.class).disableValidation();
The other option is to store all unstructured content under a single bucket element using a Map. This could contain any basic types supported by the MongoDB driver including Lists and Maps, but no complex objects unless you have registered converters with Morphia (e.g. morphia.getMapper().getConverters().addConverter(new MyCustomTypeConverter()) .
@Entity("orders")
public class Order {
// .. our base attributes here
private Map<String, Object> attributes; // bucket for everything else (
}
Note that Morphia may complain on startup that it can’t validate the field (since the generics declaration is not strict), but as of the current release version (0.99), it will work with no problem and store any attributes normally and retrieve them as maps and lists of values.
Note: When it populates a loosely-typed map from a retrieved document, it will use the basic MongoDB Java driver types BasicDBObject and BasicDBList. These implement Map and List respectively, so they will work pretty much as you expect, except that they will not be equals() to any input maps or lists you may have stored, even if the structure and content appear to be equal. If you want to avoid this, you can use the @PostLoad annotation to annotate a method which can perform normalization to JDK maps and lists after the document is loaded. I personally did this to ensure we always see a consistent view of MongoDB documents whether they are pulled from a collection or not yet persisted.
Reference: Using MongoDB with Morphia from our JCG partners at the Carfey Software blog.
Related Articles :
