Friday, January 25, 2019

Spring Custom Serializers with @JsonIdentityInfo

Intro

Serialization/Deserialization from/to JSON in Spring is widely used in modern Spring-based applications. It is based on Jackson. Jackson can serialize any POJO into JSON and vice versa with ease. This code is well written. I never encountered any issues. It gets more difficult when custom serializers are involved. This post shows how to use custom serializers in Spring with autowired fields.

Defining a Custom Serializer

Usually a custom serializer for a class is inherited from com.fasterxml.jackson.databind.ser.std.StdSerializer. This class defines some constructors but the framework only need a no-argument constructor that should call the superclass, something like this:

public CustomSerializer() {
    this(null);
}

public CustomSerializer(Class<ObjectToSerializet) {
    super(t);
}

Then there is the main method that must be implemented to actually write the JSON:

@Override
public void serialize(ObjectToSerialize value, JsonGenerator gen, SerializerProvider providerthrows IOException {
    gen.writeStartObject();
    ...
    provider.defaultSerializeField("some field"value.getField(), gen);
    ...
    gen.writeEndObject();
}

When the serializer class is created it must be registered as the serializer for ObjectToSerialize. This can be done with the @JsonSerialize annotation on the class:

@JsonSerialize(using = CustomSerializer.class)
public class ObjectToSerialize {

Now Jackson will be using this custom serializer for all instances of this class. If necessary a custom deserializer can be written by subclassing com.fasterxml.jackson.databind.deser.std.StdDeserializer<T>

Circular References and @JsonIdentityInfo

For most commercial applications with Spring and Hibernate the issue of circular references manifests itself sooner or later. Here is a simple example. 
We have 2 classes:

public class Building {

    @Id
    @GeneratedValue(<parameters>)
    private Long id;

    private Set<Apartment> apartments;
}

public class Apartment {

    @Id
    @GeneratedValue(<parameters>)
    private Long id;

    private Building building;
}

If we try to serialize one building that has at least one apartment we get a StackOverflowException.

Jackson has a solution to this problem - @JsonIdentityInfo.

If the annotation @JsonIdentityInfo is added to the class like this:

@JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator.class, property = "id")
public class ObjectToSerialize {

then any ObjectMapper will break the cycle by replacing every occurrence of the object except the first with its id. Like this:

{
    "id": 1,
    "apartments": [
        {
            "id": 2,
            "building": 1 - the object is replaced with its ID
        },
        {
            "id": 3,
            "building": 1 - the object is replaced with its ID
        }
    ]
}

These are the tools Jackson provides to customize serialization and deal with circular references.

JSON Structure Problem


Problem

@JsonIdentityInfo works well for simple applications. But as the application grows in the default form it could affect the structure of the JSON. For example if some method returns the buildings and the districts in one response here is what may occur:
{
    "buildings": [
        {
            "id": 1,
            "apartments": [
                {
                    "id": 2,
                    "building": 1 - the object is replaced with its ID
                },
                {
                    "id": 3,
                    "building": 1 - the object is replaced with its ID
                }
            ]
        }
    ],
    "districts": [
         {
             "buildings": [
                 {
                     "id": 5,
                     ...
                 },
                 1, - the object is replaced with its ID
                 {
                     "id": 6,
                     ...
                 }
             ]
         }
    ]
}

This replacement could be quite unpredictable from the parser's point of view. Within an array it could encounter objects and IDs. And this could happen for any field and any object. Any object where the class is annotated with @JsonIdentityInfo is replaced with its ID if the serialization provider finds it more than once. Every second, third, fourth etc. instance with the same ID found by the serialization provider is replaced with its ID. 

Solution


The solution here is to use a separate ObjectMapper to write parts of the JSON. The lists of already seen IDs are stored in the serialization provider which is created by ObjectMapper. By creating a separate ObjectMapper (with a probably different configuration) the lists are reset. 
For a "composite" JSON result which returns different objects types a custom serializer can be written. In this custom serializer the "header" is written manually with JsonGenerator methods and when the correct level in the JSON is reached we create a new ObjectMapper and write a much better looking JSON.

{
    "buildings": [ - create a new ObjectMapper
        {
            "id": 1,
            "apartments": [
                {
                    "id": 2,
                    "building": 1 - the object is replaced with its ID
                },
                {
                    "id": 3,
                    "building": 1 - the object is replaced with its ID
                }
            ]
        }
    ],
    "districts": [ - create a new ObjectMapper
         {
             "buildings": [
                 {
                     "id": 5,
                     ...
                 },
                 { - the object is written as a JSON Object not an ID
                     "id": 1,
                     ...
                 },
                 {
                     "id": 6,
                     ...
                 }
             ]
         }
    ]
}

To write the JSON to the original generator we can use ObjectMapper.writeValueAsString and JsonGenerator.writeRawValue(String)

P.S. it is also possible to create a new serialization provider by means of DefaultSerializerProvider.createInstance(SerializationConfig, SerializerFactory) but it is potentially more complicated. 

Custom Serializer Autowire Problem


Problem

We'd like to be able to use @Autowire in our custom serializers. It is one of Spring's best features! Actually it works if the default ObjectMapper is used. But if we use the solution to the JSON structure problem it doesn't work for custom serializers instantiated by our own object mappers. 

Solution

Our own object mappers must be configured with a special HandlerInstantiator:

// try to use the default configuration as much as possible
ObjectMapper om = Jackson2ObjectMapperBuilder.json().build();
// This instantiator will handle autowiring properties into the custom serializers
om.setHandlerInstantiator(
new SpringHandlerInstantiator(this.applicationContext.getAutowireCapableBeanFactory()));

If the custom object mappers are created inside another custom serializer which is created by the default ObjectMapper then it can autowire the ApplicationContext.

Tuesday, October 9, 2018

GWT with Spring Boot

Intro

I've been investigating the options for writing UI code in Java recently. In my previous post I investigated Eclipse RAP and found that it can be integrated with Spring Boot in one executable jar. This time I wanted to do the same trick with GWT. 
Everyone likes Spring Boot. It makes a lot of things so much cleaner and easier. But historically the frameworks for creating the UI in the web browser had their own methods to do some of the things Spring Boot does. Unfortunately in many cases their methods look old and outdated. So the goal was to use Spring Boot as much as possible and use GWT only for UI. 
I have to warn the readers this post is actually a classic example of TL;DR :-)). 

GWT Approach

  GWT uses a special compiler to generate Javascript code from Java code. The method is to create a module description file .gwt.xml, use it to import some other modules and write the code in Java with GWT widgets. Then their compiler will generate a lot of javascript code that needs to be included on the html page. They have a tutorial at www.gwtproject.org that explains the basics. 
  They use GWT RPC to call methods on the backend. This approach requires an interface that is shared between the client and the server. The client uses the interface to call the RPC method. The server-side implementation of the method is registered in web.xml as a Servlet with the appropriate URL pattern. 
  One major issue in my opinion is debugging. GWT in latest versions adopted a radical approach of source maps. This means Java code debugging occurs in the browser with source maps enabled and not in Eclipse (or maybe I couldn't make it work in Eclipse). I tried this in Chrome and it actually works but it looks like a bit of a kludge. GWT does not even generate source maps by default. In order to use them one must start the code server and load a different javascript in the html page from this code server. Most people in this case add an option to the compiler.
  I really mean no offence to the GWT team and the supporters of this technology but it looks a little outdated in general. They do not spend too much time developing new features. Even the build plugins are maintained by enthusiasts.

Goals

Here is what I wanted to achieve in my investigation:
  1. Use GWT only for generating the Javascript code that is to be archived together with everything else into the executable jar.
  2. Use Spring Boot for REST endpoints and avoid GWT RPC completely
  3. Use Spring Boot's executable jar to start the application and service the GWT html files with the embedded Tomcat. This also means all the other great Spring Boot features can be used. 

Build Tool

In order to achieve goal #1 we need a good build tool. I've created the sample project from the tutorial with the Maven plugin. Here is the complete configuration that worked for me:
<plugin>
<groupId>net.ltgt.gwt.maven</groupId>
<artifactId>gwt-maven-plugin</artifactId>
<version>1.0-rc-6</version>
<executions>
<execution>
<goals>
<goal>import-sources</goal>
<goal>compile</goal>
<goal>import-test-sources</goal>
<goal>test</goal>
</goals>
</execution>
</executions>
<configuration>
<moduleName>org.example.gwt.StockWatcher</moduleName>
<moduleShortName>StockWatcher</moduleShortName>
<failOnError>true</failOnError>
<!-- GWT compiler 2.8 requires 1.8, hence define sourceLevel here if you use
a different source language for java compilation -->
<sourceLevel>1.8</sourceLevel>
<!-- Compiler configuration -->
<compilerArgs>
<!-- Ask GWT to create the Story of Your Compile (SOYC) (gwt:compile) -->
<arg>-compileReport</arg>
<arg>-XcompilerMetrics</arg>
</compilerArgs>
<!-- DevMode configuration -->
<warDir>${project.build.directory}/${project.build.finalName}</warDir>
<classpathScope>compile+runtime</classpathScope>
<!-- URL(s) that should be opened by DevMode (gwt:devmode). -->
<startupUrls>
<startupUrl>StockWatcher.html</startupUrl>
</startupUrls>
</configuration>
</plugin>

With the GWT Eclipse plugin I made it work and even debugging was working in Chrome because the GWT plugin for Eclipse starts the code server automatically and somehow updates the html file to load the javascript from the code server. 

The bottom line is: the GWT Maven plugin works :-)). But integrating Spring Boot and GWT will be a complex task. I'll need to run the GWT compilation first, then add the resulting javascript to the executable Jar. Maybe it is possible to do this with Maven but for this task I decided to use Gradle

Gradle is a rapidly developing build tool. The DSL and API are not stable yet but it offers substantial flexibility. While Maven has a fairly straight line of build phases Gradle can execute tasks in any order. This flexibility is what I need.
After some digging I found one working Gradle plugin for GWT: de.esoco.gwt. It is a fork of the Putnami plugin. The documentation is good enough to make this plugin work. I didn't notice any major issues. The configuration in build.gradle is inside the gwt block:

gwt {
gwtVersion = gwtVersion

module("org.example.gwt.StockWatcher2", "de.richsource.gradle.plugins.gwt.example.Example")
        // other configuration options
}

This plugin adds some tasks to the gradle build. The most important of them is gwtCompile. This task actually generates the javascript code and puts it in ${buildDir}/gwt/outThese values (both gwt and out) are hardcoded in the Gradle GWT plugin. 
It is important to remember that the code that is compiled into javascript is specified in the GWT module file like this:
<source path='client'/>
<source path='shared'/>

REST and Resty

The next goal is to use Spring Boot's REST endpoints. I found RestyGWT that helped me do just that. They have a simple how-to on the front page. 
I added the required dependencies to build.gradle:

implementation("javax.ws.rs:javax.ws.rs-api:2.0.1")
compileOnly group: "org.fusesource.restygwt", name: "restygwt", version: "2.2.0"
implementation group: "com.fasterxml.jackson.jaxrs", name: "jackson-jaxrs-json-provider", version: "2.8.9"

The JAX-RS dependencies are necessary because RestyGWT uses the annotation from JAX-RS to declare the endpoints. As far as I understood Jackson is also necessary to parse the JSON. 
I added the dependency in the GWT module as well:

<inherits name="org.fusesource.restygwt.RestyGWT"/>

Here is the service I created with RestyGWT:

public interface TestService extends RestService {

    @GET
    @Path("test"void test1(@QueryParam("input") String inp,    
        MethodCallback<TestResultcallback);
}

I call this service in a ClickHandler (I mostly used the code from the original GWT tutorial):

private final TestService testSrv = GWT.create(TestService.class);
btnCallServer.addClickHandler(clkEvent -> {
    testSrv.test1("TestString"new MethodCallback<TestResult>() {

        @Override
        public void onSuccess(Method method, TestResult response) {
            testLabel.setText("Srv success " + response.getStr1());
        }

        @Override
        public void onFailure(Method method, Throwable exception) {
            testLabel.setText("Srv failure " + exception.getMessage());
        }
    });
});

This service calls this simple method in a Spring Boot controller:

@GetMapping("/test")
public TestResult test1(@RequestParam(name="input", required=false) String inp) {
return new TestResult(inp + " qqq");
}

The good thing is all this code is part of one executable jar.

Executable Jar

The third goal is to actually bundle all that into one executable fat jar. In this section I can finally make use of Gradle's flexibility. 
First I put the html files to /src/main/resources/static
I created a task to copy the generated javascript into the static folder in ${buildDir} during the build:

task copyGWTCode(dependsOn: ["gwtCompile"], type: Copy) {
    from file("${buildDir}/gwt/out")
    into file("${buildDir}/resources/main/static")
}

Next I made the bootJar task dependent on this task and copied the jar to the more traditional target directory:

bootJar {
    dependsOn copyGWTCode
    doLast {
        mkdir "${buildDir}/target"
        setDestinationDir(file("${buildDir}/target"))
        copy()
    }
}

Debugging in GWT

One extra chapter on GWT debugging.
I found a fairly simple way to debug GWT UI in Chrome (Chrome can handle it better than Firefox). Here are the steps to make it work. I used the project from the GWT tutorial but renamed it to "stockwatcher2".
1. Add a new html file for debugging to src/main/resources/static. If the original file was for example  StockWatcher2.html the new file should be  StockWatcher2debug.html. In this new file replace the line

<script type="text/javascript" src="stockwatcher2/stockwatcher2.nocache.js"></script>

with this line (javascript from the code server):

<script src="http://localhost:9876/recompile-requester/stockwatcher2"></script>


2. Execute the task bootJar and run it. 
3. Start the code server from the projects folder with "gradle gwtCodeServer".
4. Open http://<host>:<port>/<somepath>/StockWatcher2debug.html in Chrome
5. Now you can find the source maps in Developer Tools -> Sources under 127.0.0.1:9876. The breakpoint can be set and hit in Chrome directly.

The idea with a separate file is to exclude it from production builds but keep it in developer builds. It is easy with Gradle. There is only one issue with this approach and that is the REST endpoints that are called from the debug sources are different from the endpoints that are called from the "normal" sources. Adding one more mapping solves the problem.

Conclusion

I congratulate the heroic persons who have reached this conclusion! You are true programmers and those who gave up are miserable cowards! 
But the bottom line is that working with GWT is quite tough. The build tools are quite unwieldy and lack important features. There is practically no integration (for example with Spring Boot). Debugging is unnecessary complicated. 
If someone were to choose between GWT and Eclipse RAP I would recommend Eclipse RAP. 
No happy ending :-(.