Why I created the java-symbol-solver?
A few years ago I started using JavaParser and then I started contributing. After a while I realized that many operations we want to do on Java code cannot be done just by using the Abstract Syntax Tree produced by a parser, we need also to resolve types, symbols and method calls. For this reason I have created the JavaSymbolSolver. It is now been used to produce static analysis tools by Coati.
One thing that is missing is documentation: people open issues on JavaParser asking how to answer a certain question and the answer is often “for this you need to use JavaSymbolSolver”. Starting from these issues I will show a few examples.
Inspired by this issue I will show how to produce a list of all calls to a specific method.
How can we resolve method calls in Java using the java-symbol-solver?
It can be done in two steps:
- You use JavaParser on the source code to build your ASTs
- You call JavaSymbolSolver on the nodes of the ASTs representing method calls and get the answer
We are going to write a short example. At the end we will get an application that given a source file will produce this:
* L55 setId(id) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setId(com.github.javaparser.ast.body.VariableDeclaratorId) * L59 setId(new VariableDeclaratorId(variableName)) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setId(com.github.javaparser.ast.body.VariableDeclaratorId) * L71 setId(id) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setId(com.github.javaparser.ast.body.VariableDeclaratorId) * L72 setInit(init) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setInit(com.github.javaparser.ast.expr.Expression) * L76 setId(new VariableDeclaratorId(variableName)) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setId(com.github.javaparser.ast.body.VariableDeclaratorId) * L77 setInit(init) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setInit(com.github.javaparser.ast.expr.Expression) * L82 setId(id) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setId(com.github.javaparser.ast.body.VariableDeclaratorId) * L83 setInit(init) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setInit(com.github.javaparser.ast.expr.Expression) * L88 v.visit(this, arg) -&qt; com.github.javaparser.ast.visitor.GenericVisitor.visit(com.github.javaparser.ast.body.VariableDeclarator, A) * L93 v.visit(this, arg) -&qt; com.github.javaparser.ast.visitor.VoidVisitor.visit(com.github.javaparser.ast.body.VariableDeclarator, A) * L106 setAsParentNodeOf(this.id) -&qt; com.github.javaparser.ast.Node.setAsParentNodeOf(com.github.javaparser.ast.Node) * L112 setAsParentNodeOf(this.init) -&qt; com.github.javaparser.ast.Node.setAsParentNodeOf(com.github.javaparser.ast.Node) * L121 setAsParentNodeOf(this.init) -&qt; com.github.javaparser.ast.Node.setAsParentNodeOf(com.github.javaparser.ast.Node) * L128 getParentNodeOfType(NodeWithElementType.class) -&qt; com.github.javaparser.ast.Node.getParentNodeOfType(java.lang.Class<T&qt;) * L130 wrapInArrayTypes(elementType.getElementType(), elementType.getArrayBracketPairsAfterElementType(), getId().getArrayBracketPairsAfterId()) -&qt; com.github.javaparser.ast.type.ArrayType.wrapInArrayTypes(com.github.javaparser.ast.type.Type, java.util.List<com.github.javaparser.ast.ArrayBracketPair&qt;...) * L130 elementType.getElementType() -&qt; com.github.javaparser.ast.nodeTypes.NodeWithElementType.getElementType() * L131 elementType.getArrayBracketPairsAfterElementType() -&qt; com.github.javaparser.ast.nodeTypes.NodeWithElementType.getArrayBracketPairsAfterElementType() * L132 getId().getArrayBracketPairsAfterId() -&qt; com.github.javaparser.ast.body.VariableDeclaratorId.getArrayBracketPairsAfterId() * L132 getId() -&qt; com.github.javaparser.ast.body.VariableDeclarator.getId() * L137 ArrayType.unwrapArrayTypes(type) -&qt; com.github.javaparser.ast.type.ArrayType.unwrapArrayTypes(com.github.javaparser.ast.type.Type) * L138 getParentNodeOfType(NodeWithElementType.class) -&qt; com.github.javaparser.ast.Node.getParentNodeOfType(java.lang.Class<T&qt;) * L142 nodeWithElementType.setElementType(unwrapped.a) -&qt; com.github.javaparser.ast.nodeTypes.NodeWithElementType.setElementType(com.github.javaparser.ast.type.Type<?&qt;) * L143 nodeWithElementType.setArrayBracketPairsAfterElementType(null) -&qt; com.github.javaparser.ast.nodeTypes.NodeWithElementType.setArrayBracketPairsAfterElementType(java.util.List<com.github.javaparser.ast.ArrayBracketPair&qt;) * L144 getId().setArrayBracketPairsAfterId(unwrapped.b) -&qt; com.github.javaparser.ast.body.VariableDeclaratorId.setArrayBracketPairsAfterId(java.util.List<com.github.javaparser.ast.ArrayBracketPair&qt;) * L144 getId() -&qt; com.github.javaparser.ast.body.VariableDeclarator.getId()
when executed on this source file:
/*
* Copyright (C) 2007-2010 J?lio Vilmar Gesser.
* Copyright (C) 2011, 2013-2016 The JavaParser Team.
*
* This file is part of JavaParser.
*
* JavaParser can be used either under the terms of
* a) the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* b) the terms of the Apache License
*
* You should have received a copy of both licenses in LICENCE.LGPL and
* LICENCE.APACHE. Please refer to those files for details.
*
* JavaParser is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*/
package com.github.javaparser.ast.body;
import com.github.javaparser.Range;
import com.github.javaparser.ast.ArrayBracketPair;
import com.github.javaparser.ast.Node;
import com.github.javaparser.ast.expr.Expression;
import com.github.javaparser.ast.expr.NameExpr;
import com.github.javaparser.ast.nodeTypes.NodeWithElementType;
import com.github.javaparser.ast.nodeTypes.NodeWithType;
import com.github.javaparser.ast.type.ArrayType;
import com.github.javaparser.ast.type.Type;
import com.github.javaparser.ast.visitor.GenericVisitor;
import com.github.javaparser.ast.visitor.VoidVisitor;
import com.github.javaparser.utils.Pair;
import java.util.List;
import static com.github.javaparser.ast.type.ArrayType.wrapInArrayTypes;
/**
* @author Julio Vilmar Gesser
*/
public final class VariableDeclarator extends Node implements
NodeWithType<VariableDeclarator&qt; {
private VariableDeclaratorId id;
private Expression init;
public VariableDeclarator() {
}
public VariableDeclarator(VariableDeclaratorId id) {
setId(id);
}
public VariableDeclarator(String variableName) {
setId(new VariableDeclaratorId(variableName));
}
/**
* Defines the declaration of a variable.
*
* @param id The identifier for this variable. IE. The variables name.
* @param init What this variable should be initialized to.
* An {@link com.github.javaparser.ast.expr.AssignExpr} is unnecessary as the <code&qt;=</code&qt; operator is
* already added.
*/
public VariableDeclarator(VariableDeclaratorId id, Expression init) {
setId(id);
setInit(init);
}
public VariableDeclarator(String variableName, Expression init) {
setId(new VariableDeclaratorId(variableName));
setInit(init);
}
public VariableDeclarator(Range range, VariableDeclaratorId id, Expression init) {
super(range);
setId(id);
setInit(init);
}
@Override
public <R, A&qt; R accept(GenericVisitor<R, A&qt; v, A arg) {
return v.visit(this, arg);
}
@Override
public <A&qt; void accept(VoidVisitor<A&qt; v, A arg) {
v.visit(this, arg);
}
public VariableDeclaratorId getId() {
return id;
}
public Expression getInit() {
return init;
}
public VariableDeclarator setId(VariableDeclaratorId id) {
this.id = id;
setAsParentNodeOf(this.id);
return this;
}
public VariableDeclarator setInit(Expression init) {
this.init = init;
setAsParentNodeOf(this.init);
return this;
}
/**
* Will create a {@link NameExpr} with the init param
*/
public VariableDeclarator setInit(String init) {
this.init = new NameExpr(init);
setAsParentNodeOf(this.init);
return this;
}
@Override
public Type getType() {
NodeWithElementType<?&qt; elementType = getParentNodeOfType(NodeWithElementType.class);
return wrapInArrayTypes(elementType.getElementType(),
elementType.getArrayBracketPairsAfterElementType(),
getId().getArrayBracketPairsAfterId());
}
@Override
public VariableDeclarator setType(Type type) {
Pair<Type, List<ArrayBracketPair&qt;&qt; unwrapped = ArrayType.unwrapArrayTypes(type);
NodeWithElementType<?&qt; nodeWithElementType = getParentNodeOfType(NodeWithElementType.class);
if (nodeWithElementType == null) {
throw new IllegalStateException("Cannot set type without a parent");
}
nodeWithElementType.setElementType(unwrapped.a);
nodeWithElementType.setArrayBracketPairsAfterElementType(null);
getId().setArrayBracketPairsAfterId(unwrapped.b);
return this;
}
}Setting up the project
We are going to use Kotlin and Gradle. Our build file looks like this:
buildscript {
ext.kotlin_version = '1.0.4'
repositories {
mavenCentral()
maven {
name 'JFrog OSS snapshot repo'
url 'https://oss.jfrog.org/oss-snapshot-local/'
}
jcenter()
}
dependencies {
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
}
}
apply plugin: 'kotlin'
apply plugin: 'application'
apply plugin: 'java'
apply plugin: 'idea'
apply plugin: 'antlr'
repositories {
mavenLocal()
mavenCentral()
jcenter()
}
dependencies {
compile "me.tomassetti:java-symbol-solver-core:0.3.1"
compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
compile "org.jetbrains.kotlin:kotlin-reflect:$kotlin_version"
testCompile "org.jetbrains.kotlin:kotlin-test:$kotlin_version"
testCompile "org.jetbrains.kotlin:kotlin-test-junit:$kotlin_version"
testCompile "junit:junit:latest.release"
}
idea {
module {
excludeDirs += file('src/main/resources')
}
}
* L55 setId(id) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setId(com.github.javaparser.ast.body.VariableDeclaratorId)
* L59 setId(new VariableDeclaratorId(variableName)) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setId(com.github.javaparser.ast.body.VariableDeclaratorId)
* L71 setId(id) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setId(com.github.javaparser.ast.body.VariableDeclaratorId)
* L72 setInit(init) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setInit(com.github.javaparser.ast.expr.Expression)
* L76 setId(new VariableDeclaratorId(variableName)) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setId(com.github.javaparser.ast.body.VariableDeclaratorId)
* L77 setInit(init) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setInit(com.github.javaparser.ast.expr.Expression)
* L82 setId(id) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setId(com.github.javaparser.ast.body.VariableDeclaratorId)
* L83 setInit(init) -&qt; com.github.javaparser.ast.body.VariableDeclarator.setInit(com.github.javaparser.ast.expr.Expression)
* L88 v.visit(this, arg) -&qt; com.github.javaparser.ast.visitor.GenericVisitor.visit(com.github.javaparser.ast.body.VariableDeclarator, A)
* L93 v.visit(this, arg) -&qt; com.github.javaparser.ast.visitor.VoidVisitor.visit(com.github.javaparser.ast.body.VariableDeclarator, A)
* L106 setAsParentNodeOf(this.id) -&qt; com.github.javaparser.ast.Node.setAsParentNodeOf(com.github.javaparser.ast.Node)
* L112 setAsParentNodeOf(this.init) -&qt; com.github.javaparser.ast.Node.setAsParentNodeOf(com.github.javaparser.ast.Node)
* L121 setAsParentNodeOf(this.init) -&qt; com.github.javaparser.ast.Node.setAsParentNodeOf(com.github.javaparser.ast.Node)
* L128 getParentNodeOfType(NodeWithElementType.class) -&qt; com.github.javaparser.ast.Node.getParentNodeOfType(java.lang.Class<T&qt;)
* L130 wrapInArrayTypes(elementType.getElementType(), elementType.getArrayBracketPairsAfterElementType(), getId().getArrayBracketPairsAfterId()) -&qt; com.github.javaparser.ast.type.ArrayType.wrapInArrayTypes(com.github.javaparser.ast.type.Type, java.util.List<com.github.javaparser.ast.ArrayBracketPair&qt;...)
* L130 elementType.getElementType() -&qt; com.github.javaparser.ast.nodeTypes.NodeWithElementType.getElementType()
* L131 elementType.getArrayBracketPairsAfterElementType() -&qt; com.github.javaparser.ast.nodeTypes.NodeWithElementType.getArrayBracketPairsAfterElementType()
* L132 getId().getArrayBracketPairsAfterId() -&qt; com.github.javaparser.ast.body.VariableDeclaratorId.getArrayBracketPairsAfterId()
* L132 getId() -&qt; com.github.javaparser.ast.body.VariableDeclarator.getId()
* L137 ArrayType.unwrapArrayTypes(type) -&qt; com.github.javaparser.ast.type.ArrayType.unwrapArrayTypes(com.github.javaparser.ast.type.Type)
* L138 getParentNodeOfType(NodeWithElementType.class) -&qt; com.github.javaparser.ast.Node.getParentNodeOfType(java.lang.Class<T&qt;)
* L142 nodeWithElementType.setElementType(unwrapped.a) -&qt; com.github.javaparser.ast.nodeTypes.NodeWithElementType.setElementType(com.github.javaparser.ast.type.Type<?&qt;)
* L143 nodeWithElementType.setArrayBracketPairsAfterElementType(null) -&qt; com.github.javaparser.ast.nodeTypes.NodeWithElementType.setArrayBracketPairsAfterElementType(java.util.List<com.github.javaparser.ast.ArrayBracketPair&qt;)
* L144 getId().setArrayBracketPairsAfterId(unwrapped.b) -&qt; com.github.javaparser.ast.body.VariableDeclaratorId.setArrayBracketPairsAfterId(java.util.List<com.github.javaparser.ast.ArrayBracketPair&qt;)
* L144 getId() -&qt; com.github.javaparser.ast.body.VariableDeclarator.getId()Building the AST
Building an AST is quite easy, you simply call this method:
JavaParser.parse(file)
I have also used a few methods to navigate the AST and get specific nodes. In particular I will use this to take only the method calls. If you are interested they look like this:
class SpecificNodeIterator<T&qt;(private val type: Class<T&qt;, private val nodeHandler: SpecificNodeIterator.NodeHandler<T&qt;) {
interface NodeHandler<T&qt; {
fun handle(node: T): Boolean
}
fun explore(node: Node) {
if (type.isInstance(node)) {
if (!nodeHandler.handle(type.cast(node))) {
return
}
}
for (child in node.childrenNodes) {
explore(child)
}
}
}
// this is a method extension: we had this method to the existing class "Node"
fun <T&qt; Node.descendantsOfType(type: Class<T&qt;) : List<T&qt; {
val descendants = LinkedList<T&qt;()
SpecificNodeIterator(type, object : SpecificNodeIterator.NodeHandler<T&qt; {
override fun handle(node: T): Boolean {
descendants.add(node)
return true
}
}).explore(this)
return descendants
}Specify the Type Solver
What the hell is a Type Solver? It is the object which knows where to look for classes. When processing source code you will typically have references to code that is not yet compiled, but it is just present in other source files. You could also use classes contained in JARs or classes from the Java standard libraries. You have just to tell to your TypeSolver where to look for classes and it will figure it out.
In our example we will parse the source code from the JavaParser project (how meta?!). This project has source code in two different directories, for proper source code and code generated by JavaCC (you can ignore what JavaCC is, it is not relevant to you). We of course use also classes from the java standard libraries. This is how our TypeSolver looks like:
fun typeSolver() : TypeSolver {
val combinedTypeSolver = CombinedTypeSolver()
combinedTypeSolver.add(JreTypeSolver())
combinedTypeSolver.add(JavaParserTypeSolver(File("src/main/resources/javaparser-core")))
combinedTypeSolver.add(JavaParserTypeSolver(File("src/main/resources/javaparser-generated-sources")))
return combinedTypeSolver
}Our application
This is where we invoke JavaParserFacade, one of the classes provided by JavaSymbolSolver. We just take a method call at the time and we pass it to the method solve of the JavaParserFacade. We get a MethodUsage (which is basically a method declaration + the value of the parameter types for that specific invocation). From it we get the MethodDeclaration and we print the qualified signature, i.e., the qualified name of the class followed by the signature of the method. This is how we get the final output:
var solved = 0
var unsolved = 0
var errors = 0
fun processJavaFile(file: File, javaParserFacade: JavaParserFacade) {
println(file)
JavaParser.parse(file).descendantsOfType(MethodCallExpr::class.java).forEach {
print(" * L${it.begin.line} $it ")
try {
val methodRef = javaParserFacade.solve(it)
if (methodRef.isSolved) {
solved++
val methodDecl = methodRef.correspondingDeclaration
println(" -> ${methodDecl.qualifiedSignature}")
} else {
unsolved++
println(" ???")
}
} catch (e: Exception) {
println(" ERR ${e.message}")
errors++
} catch (t: Throwable) {
t.printStackTrace()
}
}
}Conclusions
There is so plumbing to do but basically JavaSymbolSolver does all the heavy work behind the scene. Once you have a node of the AST you can throw it at the class JavaParserFacade and it will give you back all the information you may need: it will find corresponding types, fields, methods, etc.
The problem is… we need more documentation and feedback from users. I hope some of you will start using JavaSymbolSolver and tell us how we can improve it.
Also, last week the JavaSymbolSolver was moved under the JavaParser organization. This means that in the future we will work more closely with the JavaParser project.
The code is available on GitHub: java-symbol-solver-examples
| Reference: | Resolve method calls in Java code using the JavaSymbolSolver from our JCG partner Federico Tomassetti at the Federico Tomassetti blog. |
