Retrofit is an open source library by Square that turns annotated java interfaces into REST clients. The library is currently in its second beta and changed quite a bit from its first version to better support the Ok stack. In this post we will go through its code, explaining its main techniques and inner workings. This post assumes some knowledge of the library as a user of it.
Retrofit is a simple and small library. In principle you don’t need it, you could do all the work directly in okhttp: build requests manually with `okhttp3.Builder`, then use some custom converter to turn `ResponseBody`s to your own pojos, and perhaps move callbacks to specific threads with `Executor`. Yet Retrofit does all this with a simple, easy to understand, annotation-based API.
The first thing to happen when a method is called is its parsing into an internal model, its `MethodHandler`, that is then cached to avoid future overhead. Afterwards, and on every method call, the method arguments are passed to its handler which has the responsability to create a `Call` object, a description of a request that can be triggered later either synchronously or asynchronously. Before going into the detail of how requests are built, it is interesting to study a bit how Retrofit links interfaces and method handlers.
Dynamic proxies
A proxy is an object that receives calls and delegate them to another object, perhaps adding custom behavior like security checks or instrumentation. This is a classic software design pattern that can be implemented without any special API. The difference here is the “dynamic” part, that is, the possibility to create on-the-fly implementations out of class tokens that represent service interfaces (class tokens are plain `Class<?>` objects).
Think about how Retrofit could implement all its service interfaces: it cannot know them in advance, so the regular way to use `implements` in code is out of question. Nevertheless, logically speaking, to implement an interface you only need to know about the interface itself and the behavior you want to associate with it. Dynamic proxies offer a simple API for it: you pass in an interface by a `Class<?>` object, you pass in the behavior as an `InvocationHandler`, and the API creates the proxy instance for you.
Let’s see how this is implemented in code. We use the code from Square’s repo at commit `90729eb2`:
public T create(final Class service) {
Utils.validateServiceInterface(service);
// ... more validation here ...
return (T) Proxy.newProxyInstance(service.getClassLoader(),
new Class<?>[] { service },
new InvocationHandler() {
private final Platform platform = Platform.get();
@Override public Object invoke(Object proxy, Method method,
Object... args) throws Throwable {
// If the method is a method from Object then defer to normal
// invocation.
if (method.getDeclaringClass() == Object.class) {
return method.invoke(this, args);
}
if (platform.isDefaultMethod(method)) {
return platform.invokeDefaultMethod(method, service, proxy, args);
}
return loadMethodHandler(method).invoke(args);
}
});
}
The key point is the call to `Proxy.newProxyInstance()`, the one that creates the dynamic proxy. Keep also in mind that a class token does not have to represent an interface but interfaces are required for dynamic proxies, hence this is checked beforehand by the utility method `Util.validateServiceInterface()`. Take a look at heterogeneous containers in Effective Java, Item 29 to see some other usages of class tokens.
The real logic is in the `InvocationHandler`, implemented as an anonymous inner class here, which is responsible to load a `MethodHandler` (and cache it) and then to delegate to its `invoke()` method, passing over the arguments.
Some additional details: using the interface’s class loader makes the proxy and the interface be loaded by the same class loader. This is actually required, because if the class loader cannot see the interface, the JVM will throw an `IllegalArgumentException`: this is trivial if there is only one interface like here, but could be a problem if the proxy has to implement multiple interfaces loaded by different class loaders.
Finally, the platform object defines whether Retrofit is running on Java 8, Android, or something else. In Java 8 we can have default methods on interfaces, so there is a check for that. One further check is to let the dynamic proxy handle methods from `Object`, for example `equals()`, `hashCode()`, etc.
The main idea: annotations parsing
In this section, we discuss how Retrofit parses annotations to create a `RequestFactory` that is then used inside `OkHttpCall` to create `Request`s. After this, there will be still some plumbing to do, which is the topic of the next section. Let’s take a look how to create a `MethodHandler`, where the parsing starts:
static MethodHandler create(Retrofit retrofit, Method method) {
CallAdapter<?> callAdapter = createCallAdapter(method, retrofit);
Type responseType = callAdapter.responseType();
Converter<ResponseBody, ?> responseConverter =
createResponseConverter(method, retrofit, responseType);
RequestFactory requestFactory = RequestFactoryParser.parse(method,
responseType, retrofit);
return new MethodHandler(retrofit.callFactory(), requestFactory, callAdapter,
responseConverter);
}
A `MethodHandler` needs four dependencies:
1. A request factory
2. A raw call factory for `okhttp3.Call`s
3. A call adapter
4. A response converter
`RequestFactory` is a simple class that is responsible to create a new `Request` each time its single method `create()` is invoked. (Although this is not necessary as a new instance of `OkHttpCall` is created each time, we will see this later.) `Call`s, call adapters and response converters will be discussed in the next section.
When `create()` is invoked, it does three things:
1. Create a `RequestBuilder`: this is the final builder of a `Request`, it
contains various lower level okhttp builders.
2. Apply an array of `RequestAction`s to the it : those are pending method calls on the builder, created during parsing following the command pattern.
3. Call `build()` on the builder and return the `Request`.
The builder itself is not very complicated, it is responsible for:
- merging the `baseUrl` with `relativeUrl` (using `okhttp3.UrlBuilder`)
- adding headers
- adding and encoding path parameters, for example `user` in
@GET("/users/{user}/repos")
Call<List> listRepos(@Path("user") String user)
- adding and encoding query paramters
- adding form fields to the body of `@FormUrlEncoded` requests (`application/x-www-form-urlencoded`)
- adding part to `@Multipart` requests
- finally building the request using the url builder and the body builders.
The only thing left to see is how `RequestAction`s are created during parsing. Continuing from the code above we call `RequestFactoryParse.parse()`:
static RequestFactory parse(Method method, Type responseType, Retrofit retrofit) {
RequestFactoryParser parser = new RequestFactoryParser(method);
parser.parseMethodAnnotations(responseType);
parser.parseParameters(retrofit);
return parser.toRequestFactory(retrofit.baseUrl());
}
The constructor and the conversion methods do nothing special, so let’s focus on the two parsing methods. The former, `parseMethodAnnotations()`, is responsible to scan the annotations of a method, similar to what we have seen here, and to delegate to `parseHttpMethodAndPath()` the setting of the method type (e.g. get, put, post, delete, etc) and the parsing of the path parameters (e.g. “user” in the url as above).
private void parseMethodAnnotations(Type responseType) {
for (Annotation annotation : method.getAnnotations()) {
if (annotation instanceof DELETE) {
parseHttpMethodAndPath("DELETE", ((DELETE) annotation).value(), false);
} else if (annotation instanceof GET) {
parseHttpMethodAndPath("GET", ((GET) annotation).value(), false);
} // ...
}
// some checks here
}
The latter method, `parseParameter()` is really long (300+ lines) but not so complicated. It parses *parameters* annotations, not method’s, and creates the corresponing `RequestAction`s triggered in the `RequestFactory`.
Only a couple of points to note here: firstly, `@Body`, `@Part` and `@PartMap` are the only actions that use a `Converter` to `RequestBody` instead of to `String`, see the second type parameter. Secondly, Retrofit supports array/varargs and iterable notations to pass headers, query parameters, form fields and body parts, see the docs for the corresponding annotations. This is implemented by the methods `RequestAction.array()` and `RequestAction.iterable()`: here the action will wrap itself in another action which will loop through all the values and perform the original action on each value.
Plumbing: Calls, call adapters and converters
In this section, we talk about the plumbing logic of `Call`s, `CallAdapter`s and `Converter`s. A `Call` object encapsulates a pending request that can be sent either synchronously, with `execute()`, or asynchronously, with `enqueue(Callback)`. In the former case, a `Response` is returned in the return value (with `T` parametrized with the type of the wanted resource) or an `Exception` is raised in case of failure. In the latter case, a `Callback` object is passed in as argument (with `T` still parametrized with the type of the wanted resource) and we get a callback either `onSuccess(Response)` or `onFailure(Throwable)`.
`Call` objects are new to Retrofit 2 and provide a handy interface for manipulating requests, for example to clone them or to stash them away for later use. Furthermore, the returned `Response` objects are useful to obtain both the body of the response, usually as some user defined entity converted by some Json library, and additional information such as the response status code or errors.
How are `Call` objects created? This is done every time `MethodHandler` is invoked:
Object invoke(Object... args) {
return callAdapter.adapt(
new OkHttpCall<>(callFactory, requestFactory, args, responseConverter));
}
The `OkHttpCall` class is a special implementation that adapts a okhttp `Call` to a Retrofit `Call`. It has three main responsabilities:
1. Create raw `okhttp3.Call`s, using an `OkHttpClient` (the call factory).
2. Parse back the response with a `Converter` in `parseResponse()`.
3. Adapt the `okhttp3.Call` API to the `retrofit2.Call` API (e.g. forwarding cancellation and translating callbacks)
Now, you might have noticed that the `OkHttpCall` is not returned directly but “adapted” through a call adapter. This might seem superflous but it is handy to have a flexible API to get responses back as custom wrappers. For example, RxJava fans could get back responses as `Observable` instead of `Call`; see also `CustomCallAdapter` in the samples.
Call adapters have another usage though, namely threading: they are also used to invoke callbacks on a specific thread. This is fundamental on Android where success/failure callbacks run on the main thread. To see this, let’s dive into the `ExecutorCallAdapterFactory.ExecutorCallbackCall` code, in the `enqueue()` method:
// ...
final Call delegate;
// ...
@Override public void enqueue(final Callback callback) {
delegate.enqueue(new Callback() {
@Override public void onResponse(final Response response) {
callbackExecutor.execute(new Runnable() {
@Override public void run() {
if (delegate.isCanceled()) {
// Emulate OkHttp's behavior of throwing/delivering an
// IOException on cancelation
callback.onFailure(new IOException("Canceled"));
} else {
callback.onResponse(response);
}
}
});
}
@Override public void onFailure(final Throwable t) {
callbackExecutor.execute(new Runnable() {
@Override public void run() {
callback.onFailure(t);
}
});
}
});
}
The `delegate` here creates new callbacks to `execute` the real callbacks on the given executor. This looks like the callbacks were jumping to the requested thread: for example, on Android the executor will be `MainThreadExecutor` which will `post()` the real callbacks on the main thread.
Call adapters can be set on the Retrofit buider calling `Retrofit.Builder.addCallAdapterFactory()`: when it’s time to locate one, i.e. when the `MethodHandler` is first created, all the factories will be scanned in order, and the first that matches the given return type (e.g. `Call`) will be used. See the methods
`Retrofit.callAdapter()` and `Retrofit.nextCallAdapter()` for more detail.
Finally, let’s take a look to `Converter`s. They come in three flavors:
1. Response converters, i.e. `Converter<ResponseBody, T>`: used by `OkHttpCall` to convert a `okhttp3.ResponseBody` to `T`.
2. Request converters, i.e. `Converter<T, RequestBody>`: used in `RequestAction.Body`, `RequestAction.Part` and `RequestAction.PartMap` when converting a user defined pojo to a `RequestBody` to be passed in the `RequestBuilder`.
3. String converter, i.e. `Converter<T, String>`: used in all other `RequestAction`s, for example to convert a `@Path` annotated argument to a string.
It must be mentioned that the default string converter is `ToStringConverter` that just call `toString()` on the given argument; if the argument were a primitive the dynamic proxy would wrap it. All three types are added through the Retrofit builder’s method `addConverterFactory()`.