We are getting closer to the first Release Candidate for JavaParser 3.0. One of the last features we added was support for observing changes to all nodes of the Abstract Syntax Tree. While I wrote the code for this feature I received precious feedback from Danny van Bruggen (a.k.a. Matozoid) and Cruz Maximilien. So I use “we” to refer to the JavaParser team.
What observers on AST nodes could be used for?
I think this is a very important feature for the ecosystem of JavaParser because it makes easier to integrate with JavaParser by reacting to the changes made on the AST. Possible changes that can be observed are setting a new name for a class or add a new field. Different tools could react to those changes in different ways. For example:
- an editor could update its list of symbols, which could be used for things like auto-completion
- some frameworks could regenerate source code to reflect the changes
- validation could be performed to verify if the new change lead to an invalid AST
- libraries like JavaSymbolSolver could recalculate the types for expressions
These are just a few ideas that come to mind but I think that most scenarios in which JavaParser is used could benefit from the possibility to react to changes.
The AstObserver
The JavaParser 3.0 AST is based on Nodes and NodeLists. A Node, like a TypeDeclaration for instance, can have different groups of children. When these groups can contain more than one node we use NodeLists. For example a TypeDeclarations can have multiple members (fields, methods, inner classes). So each TypeDeclaration has a NodeList to contain fields, one to contain methods, etc. Other children, like the name of a TypeDeclaration, are instead directly contain in a node.
We introduced a new interface named AstObserver. An AstObserver receive changes on the Nodes and NodeLists.
/**
* An Observer for an AST element (either a Node or a NodeList).
*/
public interface AstObserver {
/**
* Type of change occurring on a List
*/
public enum ListChangeType {
ADDITION,
REMOVAL
}
/**
* The value of a property is changed
*
* @param observedNode owner of the property
* @param property property changed
* @param oldValue value of the property before the change
* @param newValue value of the property after the change
*/
void propertyChange(Node observedNode, ObservableProperty property, Object oldValue, Object newValue);
/**
* The parent of a node is changed
*
* @param observedNode node of which the parent is changed
* @param previousParent previous parent
* @param newParent new parent
*/
void parentChange(Node observedNode, Node previousParent, Node newParent);
/**
* A list is changed
*
* @param observedNode list changed
* @param type type of change
* @param index position at which the changed occurred
* @param nodeAddedOrRemoved element added or removed
*/
void listChange(NodeList observedNode, ListChangeType type, int index, Node nodeAddedOrRemoved);
}What to observe
Now we have an AstObserver and we need to decide which changes it should received. We thought of three possible scenarios:
- Observing just one node, for example a ClassDeclaration. The observer would receive notifications for changes on that node (e.g., if the class change name) but not for any of its descendants. For example if a field of the class change name the observer would not be notified
- For a node and all its descendants at the moment of registration of the observer. In this case if I register an observer for the ClassDeclaration I would be notified for changes to the class and all its fields and methods. If a new field is added and later modified I would not receive notifications for those changes
- For a node and all its descendants, both the ones existing at the moment of registration of the observer and the ones added later.
So a Node has now this method:
/**
* Register a new observer for the given node. Depending on the mode specified also descendants, existing
* and new, could be observed. For more details see ObserverRegistrationMode.
*/
public void register(AstObserver observer, ObserverRegistrationMode mode) {
if (mode == null) {
throw new IllegalArgumentException("Mode should be not null");
}
switch (mode) {
case JUST_THIS_NODE:
register(observer);
break;
case THIS_NODE_AND_EXISTING_DESCENDANTS:
registerForSubtree(observer);
break;
case SELF_PROPAGATING:
registerForSubtree(PropagatingAstObserver.transformInPropagatingObserver(observer));
break;
default:
throw new UnsupportedOperationException("This mode is not supported: " + mode);
}
}To distinguish these three cases we simply use an enum (ObserverRegistrationMode). Later you can see how we implemented the PropagatingAstObserver.
Implementing support for observers
If JavaParser was based on some meta-modeling framework like EMF this would be extremely simple to do. Given this is not the case I needed to add a notification call in all the setters of the AST classes (there are around 90 of those).
So when a setter is invoke on a certain node it notifies all the observers. Simple. Take for example setNamein TypeDeclaration<T>:
@Override
public T setName(SimpleName name) {
notifyPropertyChange(ObservableProperty.NAME, this.name, name);
this.name = assertNotNull(name);
setAsParentNodeOf(name);
return (T) this;
}Given we do not have a proper metamodel we have no definitions for properties. Therefore we added a list of properties in an enum, named ObservableProperty. In this way an Observer can check which property was changed and decide how to react.
Internal hierarchy of observers
For performance reasons each node has its own list of observers. When we want to observe all descendants of a node we simply add the same observer to all nodes and nodelists in that subtree.
However this is not enough, because in some cases you may want to observe also all nodes which are added to the subtree after you have placed your observers. We do that by using a PropagatingAstObserver. It is an AstObserver that when see a new node been attached to a node it is observing start to observe the new node as well. Simple, eh?
/**
* This AstObserver attach itself to all new nodes added to the nodes already observed.
*/
public abstract class PropagatingAstObserver implements AstObserver {
/**
* Wrap a given observer to make it self-propagating. If the given observer is an instance of PropagatingAstObserver
* the observer is returned without changes.
*/
public static PropagatingAstObserver transformInPropagatingObserver(final AstObserver observer) {
if (observer instanceof PropagatingAstObserver) {
return (PropagatingAstObserver)observer;
}
return new PropagatingAstObserver() {
@Override
public void concretePropertyChange(Node observedNode, ObservableProperty property, Object oldValue, Object newValue) {
observer.propertyChange(observedNode, property, oldValue, newValue);
}
@Override
public void concreteListChange(NodeList observedNode, ListChangeType type, int index, Node nodeAddedOrRemoved) {
observer.listChange(observedNode, type, index, nodeAddedOrRemoved);
}
@Override
public void parentChange(Node observedNode, Node previousParent, Node newParent) {
observer.parentChange(observedNode, previousParent, newParent);
}
};
}
@Override
public final void propertyChange(Node observedNode, ObservableProperty property, Object oldValue, Object newValue) {
considerRemoving(oldValue);
considerAdding(newValue);
concretePropertyChange(observedNode, property, oldValue, newValue);
}
@Override
public final void listChange(NodeList observedNode, ListChangeType type, int index, Node nodeAddedOrRemoved) {
if (type == ListChangeType.REMOVAL) {
considerRemoving(nodeAddedOrRemoved);
} else if (type == ListChangeType.ADDITION) {
considerAdding(nodeAddedOrRemoved);
}
concreteListChange(observedNode, type, index, nodeAddedOrRemoved);
}
public void concretePropertyChange(Node observedNode, ObservableProperty property, Object oldValue, Object newValue) {
// do nothing
}
public void concreteListChange(NodeList observedNode, ListChangeType type, int index, Node nodeAddedOrRemoved) {
// do nothing
}
@Override
public void parentChange(Node observedNode, Node previousParent, Node newParent) {
// do nothing
}
private void considerRemoving(Object element) {
if (element instanceof Observable) {
if (((Observable) element).isRegistered(this)) {
((Observable) element).unregister(this);
}
}
}
private void considerAdding(Object element) {
if (element instanceof Node) {
((Node) element).registerForSubtree(this);
} else if (element instanceof Observable) {
((Observable) element).register(this);
}
}
}Observers in action
Let’s see how this works in practice:
// write some code and parse it
String code = "class A { int f; void foo(int p) { return 'z'; }}";
CompilationUnit cu = JavaParser.parse(code);
// set up our observer
List changes = new ArrayList<>();
AstObserver observer = new AstObserverAdapter() {
@Override
public void propertyChange(Node observedNode, ObservableProperty property, Object oldValue, Object newValue) {
changes.add(String.format("%s.%s changed from %s to %s", observedNode.getClass().getSimpleName(), property.name().toLowerCase(), oldValue, newValue));
}
};
cu.getClassByName("A").register(observer, /* Here we could use different modes */);
// Doing some changes
cu.getClassByName("A").setName("MyCoolClass");
cu.getClassByName("MyCoolClass").getFieldByName("f").setElementType(new PrimitiveType(PrimitiveType.Primitive.Boolean));
cu.getClassByName("MyCoolClass").getMethodsByName("foo").get(0).getParamByName("p").setName("myParam");
// Here we are adding a new field and immediately changing it
cu.getClassByName("MyCoolClass").addField("int", "bar").getVariables().get(0).setInit("0");
// If we registered our observer with mode JUST_THIS_NODE
assertEquals(Arrays.asList("ClassOrInterfaceDeclaration.name changed from A to MyCoolClass"), changes);
// If we registered our observer with mode THIS_NODE_AND_EXISTING_DESCENDANTS
assertEquals(Arrays.asList("ClassOrInterfaceDeclaration.name changed from A to MyCoolClass",
"FieldDeclaration.element_type changed from int to boolean",
"VariableDeclaratorId.name changed from p to myParam"), changes);
// If we registered our observer with mode SELF_PROPAGATING
assertEquals(Arrays.asList("ClassOrInterfaceDeclaration.name changed from A to MyCoolClass",
"FieldDeclaration.element_type changed from int to boolean",
"VariableDeclaratorId.name changed from p to myParam",
"FieldDeclaration.modifiers changed from [] to []",
"FieldDeclaration.element_type changed from empty to int",
"VariableDeclaratorId.array_bracket_pairs_after_id changed from com.github.javaparser.ast.NodeList@1 to com.github.javaparser.ast.NodeList@1",
"VariableDeclarator.init changed from null to 0"), changes);Conclusions
I am quite excited about this new feature because I think it enables more cool stuff to be done with JavaParser. I think our work as committers is to enable other people to do things we are not foreseeing right now. We should just act as enablers and then get out of the way.
I am really curious to see what people will build. By the way, do you know any project using JavaParser that you want to make known to us? Leave a comment or open an issue on GitHub, we are looking forward to hearing from you!
| Reference: | Observers for AST nodes in JavaParser from our JCG partner Federico Tomassetti at the Federico Tomassetti blog. |
