Project Student: Webservice Server with Jersey
This is part of Project Student. Other posts are Webservice Client With Jersey, Business Layer and Persistence with Spring Data.
The second layer of the RESTful webapp onion is the webservice server. It should be a thin layer that wraps calls to the business layer but does not do significant processing of its own. This post has a lot of code but it’s mostly test classes.
Design Decisions
Jersey – I use Jersey for the REST server. I considered alternatives – Spring MVC, Netty, etc., but decided to go with Jersey for the same reason as the client. It’s lightweight and doesn’t constrain the developer.
Dependency Injection – I need dependency injection and that means I need to decide on a framework: Spring, EJB3, Guice, etc. I already know that I’ll be using Spring Data for the persistence layer so it’s a no-brainer to use the spring framework. I will still be careful to minimize any dependencies (ha!) on that framework for maximum flexibility.
Limitations
Jersey – I don’t know how well Jersey will handle high loads. This is a key reason why the REST server must be a thin wrapper around the business layer – it will be relatively painless to change libraries if it becomes necessary.
User Permissions – there is no attempt to restrict access to certain methods to specific users or hosts. This should be handled by the business layer with security exceptions translated to FORBIDDEN status codes by the REST server.
Jersey REST Server
One of our early design documents is the REST API. For the server this means that we implement the layer from the REST server down instead of from the business layer API up. In fact the REST server defines the necessary methods in the business layer API.
There is one small deviation from the standard REST CRUD API: objects are created with a POST instead of a PUT since the semantics of the latter is that the object is created exactly as provided. We can’t do that – for security reasons we never expose our internal ID and must never accept a user-defined UUID. That means we’ll violate the REST API contract so we use a POST instead.
There is also one small cheat: the CRUD contract only requires the ability to create or update objects. This means that we can figure out the required action given just the path – we don’t need to add a specific ‘action’ field. This may change as we extend the implementation to include more than just CRUD actions.
Onward to the code…
@Service @Path("/course") public class CourseResource extends AbstractResource { private static final Logger log = Logger.getLogger(CourseResource.class); private static final Course[] EMPTY_COURSE_ARRAY = new Course[0]; @Context UriInfo uriInfo; @Context Request request; @Resource private CourseService service; /** * Default constructor. */ public CourseResource() { } /** * Unit test constructor. * * @param service */ CourseResource(CourseService service) { this.service = service; } /** * Get all Courses. * * @return */ @GET @Produces({ MediaType.APPLICATION_JSON, MediaType.TEXT_XML }) public Response findAllCourses() { log.debug("CourseResource: findAllCourses()"); Response response = null; try { List<Course> courses = service.findAllCourses(); List<Course> results = new ArrayList<Course>(courses.size()); for (Course course : courses) { results.add(scrubCourse(course)); } response = Response.ok(results.toArray(EMPTY_COURSE_ARRAY)).build(); } catch (Exception e) { if (!(e instanceof UnitTestException)) { log.info("unhandled exception", e); } response = Response.status(Status.INTERNAL_SERVER_ERROR).build(); } return response; } /** * Create a Course. * * @param req * @return */ @POST @Consumes({ MediaType.APPLICATION_JSON, MediaType.TEXT_XML }) @Produces({ MediaType.APPLICATION_JSON, MediaType.TEXT_XML }) public Response createCourse(Name req) { log.debug("CourseResource: createCourse()"); final String name = req.getName(); if ((name == null) || name.isEmpty()) { return Response.status(Status.BAD_REQUEST).entity("'name' is required").build(); } Response response = null; try { Course course = service.createCourse(name); if (course == null) { response = Response.status(Status.INTERNAL_SERVER_ERROR).build(); } else { response = Response.created(URI.create(course.getUuid())).entity(scrubCourse(course)).build(); } } catch (Exception e) { if (!(e instanceof UnitTestException)) { log.info("unhandled exception", e); } response = Response.status(Status.INTERNAL_SERVER_ERROR).build(); } return response; } /** * Get a specific Course. * * @param uuid * @return */ @Path("/{courseId}") @GET @Produces({ MediaType.APPLICATION_JSON, MediaType.TEXT_XML }) public Response getCourse(@PathParam("courseId") String id) { log.debug("CourseResource: getCourse()"); Response response = null; try { Course course = service.findCourseByUuid(id); response = Response.ok(scrubCourse(course)).build(); } catch (ObjectNotFoundException e) { response = Response.status(Status.NOT_FOUND).build(); } catch (Exception e) { if (!e instanceof UnitTestException)) { log.info("unhandled exception", e); } response = Response.status(Status.INTERNAL_SERVER_ERROR).build(); } return response; } /** * Update a Course. * * FIXME: what about uniqueness violations? * * @param id * @param req * @return */ @Path("/{courseId}") @POST @Consumes({ MediaType.APPLICATION_JSON, MediaType.TEXT_XML }) @Produces({ MediaType.APPLICATION_JSON, MediaType.TEXT_XML }) public Response updateCourse(@PathParam("courseId") String id, Name req) { log.debug("CourseResource: updateCourse()"); final String name = req.getName(); if ((name == null) || name.isEmpty()) { return Response.status(Status.BAD_REQUEST).entity("'name' is required").build(); } Response response = null; try { final Course course = service.findCourseByUuid(id); final Course updatedCourse = service.updateCourse(course, name); response = Response.ok(scrubCourse(updatedCourse)).build(); } catch (ObjectNotFoundException exception) { response = Response.status(Status.NOT_FOUND).build(); } catch (Exception e) { if (!(e instanceof UnitTestException)) { log.info("unhandled exception", e); } response = Response.status(Status.INTERNAL_SERVER_ERROR).build(); } return response; } /** * Delete a Course. * * @param id * @return */ @Path("/{courseId}") @DELETE public Response deleteCourse(@PathParam("courseId") String id) { log.debug("CourseResource: deleteCourse()"); Response response = null; try { service.deleteCourse(id); response = Response.noContent().build(); } catch (ObjectNotFoundException exception) { response = Response.noContent().build(); } catch (Exception e) { if (!(e instanceof UnitTestException)) { log.info("unhandled exception", e); } response = Response.status(Status.INTERNAL_SERVER_ERROR).build(); } return response; } }
The implementation tells us that we need three things:
- a service API (CourseService)
- request parameter classes (Name)
- a scrubber (scrubCourse)
I did not show full logging. Request parameters must be scrubbed to avoid log contamination.. As a simple example consider using a logger that writes to a SQL database for the ease of analysis. A naive implementation of this logger – one that does not use positional parameters – would permit SQL injection via carefully crafted request parameters!
OWASP ESAPI contains methods that can be used for log scrubbing. I haven’t included here since it’s a bit of a pain to set up. (It should be in the checked-in code soon.)
Why would you log to a database? One good practice is to log all unhandled exceptions that reach the server layer – you never want to rely on the user to report problems and errors written to the log files are easily overlooked. In contrast reports written to a database are easy to check with simple tools.
Advanced developers can even create new bug reports when unhandled exceptions occur. In this case it’s critical to maintain a separate exception database to avoid submitting duplicate entries and overwhelming developers. (The database can contain details for each exception but the bug reporting system should only have one bug report per exception class + stack trace.)
Service API
The service API for CRUD operations is straightforward.
public interface CourseService { List<Course> findAllCourses(); Course findCourseById(Integer id); Course findCourseByUuid(String uuid); Course createCourse(String name); Course updateCourse(Course course, String name); void deleteCourse(String uuid); }
The API also includes an ObjectNotFoundException. (This should be expanded to include the type of the object that could not be found.)
public class ObjectNotFoundException extends RuntimeException { private static final long serialVersionUID = 1L; private final String uuid; public ObjectNotFoundException(String uuid) { super("object not found: [" + uuid + "]"); this.uuid = uuid; } public String getUuid() { return uuid; } }
As mentioned above we’ll also want an UnauthorizedOperationException eventually.
Request Parameters
The request parameters are simple POJOs that encapsulate POST payloads.
@XmlRootElement public class Name { private String name; public String getName() { return name; } public void setName(String name) { this.name = name; } }
Students and instructors also require email addresses.
@XmlRootElement public class NameAndEmailAddress { private String name; private String emailAddress; public String getName() { return name; } public void setName(String name) { this.name = name; } public String getEmailAddress() { return emailAddress; } public void setEmailAddress(String emailAddress) { this.emailAddress = emailAddress; } }
The final application will have a large number of request parameter classes.
Scrubber
A scrubber serves three purposes. First, it removes sensitive information that should not be provided to the client, e.g., internal database identifiers.
Second, it prevents an massive database dump due to pulling in collections. For instance a student should include a list of current sections, but each section has a list of enrolled students and instructors. Each of those students and instructors has his own list of current sections. Lather, rinse, repeat, and you end up dumping the entire database in response to a single query.
The solution is to only include shallow information about every object that can be independently queried. For instance a student will have a list of current sections but those sections will only contain UUID and name. A very good rule of thumb is that scrubbed collections should contain exactly the information that will be used in pulldown lists and presentation tables, nothing more. Presentation lists can include links (or AJAX actions) to pull in additional information as required.
Finally, it’s a good place to perform HTML encoding and scrubbing. The returned values should be scrubbed to prevent cross-site scripting (CSS) attacks.
public abstract class AbstractResource { /** * Scrub 'course' object. * * FIXME add HTML scrubbing and encoding for string values! */ public Course scrubCourse(final Course dirty) { final Course clean = new Course(); clean.setUuid(dirty.getUuid()); clean.setName(dirty.getName()); // clean.setSelf("resource/" + dirty.getUuid()); return clean; } }
Configuration Classes
We have two configuration classes. The first is always used by the server, the second is only used by the server during integration testing. The latter configuration (and referenced classes) are located in the integration-test source tree.
I prefer to use configuration classes (introduced in Spring 3.0) since they provide the most flexibility – e.g., I could conditionally define beans according to the user running the application or environmental variables – and allow me to still include standard configuration files.
@Configuration @ComponentScan(basePackages = { "com.invariantproperties.sandbox.student.webservice.server.rest" }) @ImportResource({ "classpath:applicationContext-rest.xml" }) // @PropertySource("classpath:application.properties") public class RestApplicationContext { @Resource private Environment environment; }
Spring 3.1 introduced configuration profiles. They work – but the spring-aware jersey servlet I am using appears to be unable to properly set the active profiles.
@Configuration //@Profile("test") public class RestApplicationContextTest { @Bean StudentService studentService() { return new DummyStudentService(); } }
web.xml
We now have enough to implement our web server. The servlet used is a spring-enabled Jersey servlet that uses the configuration classes given in the contextClass parameter. (It is also possible to use configuration files, but not a combination of configuration classes and files.)
The servlet also contains a definition of spring.profiles.active. The intent is to conditionally include the definitions inside of RestApplicationContextTest via the spring 3.1 @Profile annotation but I haven’t been able to get it to work. I’ve left it in for future reference.
<?xml version="1.0" encoding="UTF-8"?> <web-app version="2.5" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"> <display-name>Project Student Webservice</display-name> <context-param> <param-name>contextClass</param-name> <param-value>org.springframework.web.context.support.AnnotationConfigWebApplicationContext</param-value> </context-param> <context-param> <param-name>contextConfigLocation</param-name> <param-value> com.invariantproperties.sandbox.student.webservice.server.config.RestApplicationContext com.invariantproperties.sandbox.student.webservice.server.config.RestApplicationContextTest </param-value> </context-param> <listener> <listener-class> org.springframework.web.context.ContextLoaderListener </listener-class> </listener> <servlet> <servlet-name>REST dispatcher</servlet-name> <servlet-class>com.sun.jersey.spi.spring.container.servlet.SpringServlet</servlet-class> <init-param> <param-name>spring.profiles.active</param-name> <param-value>test</param-value> </init-param> </servlet> <servlet-mapping> <servlet-name>REST dispatcher</servlet-name> <url-pattern>/rest/*</url-pattern> </servlet-mapping> </web-app>
Unit Testing
The unit tests are straightforward.
public class CourseResourceTest { private Course physics = new Course(); private Course mechanics = new Course(); @Before public void init() { physics.setId(1); physics.setName("physics"); physics.setUuid(UUID.randomUUID().toString()); mechanics.setId(1); mechanics.setName("mechanics"); mechanics.setUuid(UUID.randomUUID().toString()); } @Test public void testFindAllCourses() { final List<Course> expected = Arrays.asList(physics); final CourseService service = Mockito.mock(CourseService.class); when(service.findAllCourses()).thenReturn(expected); final CourseResource resource = new CourseResource(service); final Response response = resource.findAllCourses(); assertEquals(200, response.getStatus()); final Course[] actual = (Course[]) response.getEntity(); assertEquals(expected.size(), actual.length); assertNull(actual[0].getId()); assertEquals(expected.get(0).getName(), actual[0].getName()); assertEquals(expected.get(0).getUuid(), actual[0].getUuid()); } @Test public void testFindAllCoursesEmpty() { final List<Course> expected = new ArrayList<>(); final CourseService service = Mockito.mock(CourseService.class); when(service.findAllCourses()).thenReturn(expected); final CourseResource resource = new CourseResource(service); final Response response = resource.findAllCourses(); assertEquals(200, response.getStatus()); final Course[] actual = (Course[]) response.getEntity(); assertEquals(0, actual.length); } @Test public void testFindAllCoursesFailure() { final CourseService service = Mockito.mock(CourseService.class); when(service.findAllCourses()).thenThrow( new UnitTestException(); final CourseResource resource = new CourseResource(service); final Response response = resource.findAllCourses(); assertEquals(500, response.getStatus()); } @Test public void testGetCourse() { final Course expected = physics; final CourseService service = Mockito.mock(CourseService.class); when(service.findCourseByUuid(expected.getUuid())).thenReturn(expected); final CourseResource resource = new CourseResource(service); final Response response = resource.getCourse(expected.getUuid()); assertEquals(200, response.getStatus()); final Course actual = (Course) response.getEntity(); assertNull(actual.getId()); assertEquals(expected.getName(), actual.getName()); assertEquals(expected.getUuid(), actual.getUuid()); } @Test public void testGetCourseMissing() { final CourseService service = Mockito.mock(CourseService.class); when(service.findCourseByUuid(physics.getUuid())).thenThrow( new ObjectNotFoundException(physics.getUuid())); final CourseResource resource = new CourseResource(service); final Response response = resource.getCourse(physics.getUuid()); assertEquals(404, response.getStatus()); } @Test public void testGetCourseFailure() { final CourseService service = Mockito.mock(CourseService.class); when(service.findCourseByUuid(physics.getUuid())).thenThrow( new UnitTestException(); final CourseResource resource = new CourseResource(service); final Response response = resource.getCourse(physics.getUuid()); assertEquals(500, response.getStatus()); } @Test public void testCreateCourse() { final Course expected = physics; final Name name = new Name(); name.setName(expected.getName()); final CourseService service = Mockito.mock(CourseService.class); when(service.createCourse(name.getName())).thenReturn(expected); final CourseResource resource = new CourseResource(service); final Response response = resource.createCourse(name); assertEquals(201, response.getStatus()); final Course actual = (Course) response.getEntity(); assertNull(actual.getId()); assertEquals(expected.getName(), actual.getName()); } @Test public void testCreateCourseBlankName() { final Course expected = physics; final Name name = new Name(); final CourseService service = Mockito.mock(CourseService.class); final CourseResource resource = new CourseResource(service); final Response response = resource.createCourse(name); assertEquals(400, response.getStatus()); } /** * Test handling when the course can't be created for some reason. For now * the service layer just returns a null value - it should throw an * appropriate exception. */ @Test public void testCreateCourseProblem() { final Course expected = physics; final Name name = new Name(); name.setName(expected.getName()); final CourseService service = Mockito.mock(CourseService.class); when(service.createCourse(name.getName())).thenReturn(null); final CourseResource resource = new CourseResource(service); final Response response = resource.createCourse(name); assertEquals(500, response.getStatus()); } @Test public void testCreateCourseFailure() { final Course expected = physics; final Name name = new Name(); name.setName(expected.getName()); final CourseService service = Mockito.mock(CourseService.class); when(service.createCourse(name.getName())).thenThrow( new UnitTestException(); final CourseResource resource = new CourseResource(service); final Response response = resource.createCourse(name); assertEquals(500, response.getStatus()); } @Test public void testUpdateCourse() { final Course expected = physics; final Name name = new Name(); name.setName(mechanics.getName()); final Course updated = new Course(); updated.setId(expected.getId()); updated.setName(mechanics.getName()); updated.setUuid(expected.getUuid()); final CourseService service = Mockito.mock(CourseService.class); when(service.findCourseByUuid(expected.getUuid())).thenReturn(expected); when(service.updateCourse(expected, name.getName())) .thenReturn(updated); final CourseResource resource = new CourseResource(service); final Response response = resource.updateCourse(expected.getUuid(), name); assertEquals(200, response.getStatus()); final Course actual = (Course) response.getEntity(); assertNull(actual.getId()); assertEquals(mechanics.getName(), actual.getName()); assertEquals(expected.getUuid(), actual.getUuid()); } /** * Test handling when the course can't be updated for some reason. For now * the service layer just returns a null value - it should throw an * appropriate exception. */ @Test public void testUpdateCourseProblem() { final Course expected = physics; final Name name = new Name(); name.setName(expected.getName()); final CourseService service = Mockito.mock(CourseService.class); when(service.updateCourse(expected, name.getName())).thenReturn(null); final CourseResource resource = new CourseResource(service); final Response response = resource.createCourse(name); assertEquals(500, response.getStatus()); } @Test public void testUpdateCourseFailure() { final Course expected = physics; final Name name = new Name(); name.setName(expected.getName()); final CourseService service = Mockito.mock(CourseService.class); when(service.updateCourse(expected, name.getName())).thenThrow( new UnitTestException(); final CourseResource resource = new CourseResource(service); final Response response = resource.createCourse(name); assertEquals(500, response.getStatus()); } @Test public void testDeleteCourse() { final Course expected = physics; final CourseService service = Mockito.mock(CourseService.class); doNothing().when(service).deleteCourse(expected.getUuid()); final CourseResource resource = new CourseResource(service); final Response response = resource.deleteCourse(expected.getUuid()); assertEquals(204, response.getStatus()); } @Test public void testDeleteCourseMissing() { final Course expected = physics; final Name name = new Name(); name.setName(expected.getName()); final CourseService service = Mockito.mock(CourseService.class); doThrow(new ObjectNotFoundException(expected.getUuid())).when(service) .deleteCourse(expected.getUuid()); final CourseResource resource = new CourseResource(service); final Response response = resource.deleteCourse(expected.getUuid()); assertEquals(204, response.getStatus()); } @Test public void testDeleteCourseFailure() { final Course expected = physics; final CourseService service = Mockito.mock(CourseService.class); doThrow(new UnitTestException()).when(service) .deleteCourse(expected.getUuid()); final CourseResource resource = new CourseResource(service); final Response response = resource.deleteCourse(expected.getUuid()); assertEquals(500, response.getStatus()); } }
Integration Testing
Question: should REST server integration tests use a live database?
Answer: it’s a trick question. We need both.
The overall architecture has three maven modules. We covered student-ws-client earlier and we’re covering student-ws-server today. Each creates a .jar file. There’s a third module – student-ws-webapp – that creates the actual .war file. The integration tests for the student-ws-server module should use a dummied service layer while the integration tests for the student-ws-webapp module uses the full stack.
We start with the integration tests that mirror the unit tests in the client module.
public class CourseRestServerIntegrationTest { CourseRestClient client = new CourseRestClientImpl( "http://localhost:8080/rest/course/"); @Test public void testGetAll() throws IOException { Course[] courses = client.getAllCourses(); assertNotNull(courses); } @Test(expected = ObjectNotFoundException.class) public void testUnknownCourse() throws IOException { client.getCourse("missing"); } @Test public void testLifecycle() throws IOException { final String physicsName = "Physics 201"; final Course expected = client.createCourse(physicsName); assertEquals(physicsName, expected.getName()); final Course actual1 = client.getCourse(expected.getUuid()); assertEquals(physicsName, actual1.getName()); final Course[] courses = client.getAllCourses(); assertTrue(courses.length > 0); final String mechanicsName = "Newtonian Mechanics 201"; final Course actual2 = client.updateCourse(actual1.getUuid(), mechanicsName); assertEquals(mechanicsName, actual2.getName()); client.deleteCourse(actual1.getUuid()); try { client.getCourse(expected.getUuid()); fail("should have thrown exception"); } catch (ObjectNotFoundException e) { // do nothing } } }
We also need a dummy service class that implements just enough functionality to support our integration tests.
public class DummyCourseService implements CourseService { private Map cache = Collections.synchronizedMap(new HashMap<String, Course>()); public List<Course> findAllCourses() { return new ArrayList(cache.values()); } public Course findCourseById(Integer id) { throw new ObjectNotFoundException(null); } public Course findCourseByUuid(String uuid) { if (!cache.containsKey(uuid)) { throw new ObjectNotFoundException(uuid); } return cache.get(uuid); } public Course createCourse(String name) { Course course = new Course(); course.setUuid(UUID.randomUUID().toString()); course.setName(name); cache.put(course.getUuid(), course); return course; } public Course updateCourse(Course oldCourse, String name) { if (!cache.containsKey(oldCourse.getUuid())) { throw new ObjectNotFoundException(oldCourse.getUuid()); } Course course = cache.get(oldCourse.getUuid()); course.setUuid(UUID.randomUUID().toString()); course.setName(name); return course; } public void deleteCourse(String uuid) { if (cache.containsKey(uuid)) { cache.remove(uuid); } } }
pom.xml
The pom.xml file should include a plugin to run an embedded jetty or tomcat server. Advanced users can spin up and tear down the embedded server as part of the integration test – see the update.
<build> <plugins> <!-- Run the application using "mvn jetty:run" --> <plugin> <groupId>org.mortbay.jetty</groupId> <artifactId>maven-jetty-plugin</artifactId> <version>6.1.16</version> <!-- ancient! --> <configuration> <!-- Log to the console. --> <requestLog implementation="org.mortbay.jetty.NCSARequestLog"> <!-- This doesn't do anything for Jetty, but is a workaround for a Maven bug that prevents the requestLog from being set. --> <append>true</append> </requestLog> <webAppConfig> <contextPath>/</contextPath> <extraClasspath>${basedir}/target/test-classes/</extraClasspath> </webAppConfig> </configuration> </plugin> </plugins> </build>
Update
After a bit more research I have the configuration to setup and teardown a jetty server during integration tests. This configuration uses non-standard ports so we can run it without having to shut down another jetty or tomcat instance running at the same time.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <build> <plugins> <!-- Run the application using "mvn jetty:run" --> <plugin> <groupId>org.eclipse.jetty</groupId> <artifactId>jetty-maven-plugin</artifactId> <version>9.1.0.v20131115</version> <configuration> <webApp> <extraClasspath>${basedir}/target/test-classes/</extraClasspath> </webApp> <scanIntervalSeconds>10</scanIntervalSeconds> <stopPort>18005</stopPort> <stopKey>STOP</stopKey> <systemProperties> <systemProperty> <name>jetty.port</name> <value>18080</value> </systemProperty> </systemProperties> </configuration> <executions> <execution> <id>start-jetty</id> <phase>pre-integration-test</phase> <goals> <goal>run</goal> </goals> <configuration> <scanIntervalSeconds>0</scanIntervalSeconds> <daemon>true</daemon> </configuration> </execution> <execution> <id>stop-jetty</id> <phase>post-integration-test</phase> <goals> <goal>stop</goal> </goals> </execution> </executions> </plugin> </plugins> </build> </project>
Source Code
- The source code is available at http://code.google.com/p/invariant-properties-blog/source/browse/student/student-webservices/student-ws-server.
Hi, How would i compile and do this if i just use a text editor?