Versioned API in Play Framework 2 – One fit all Controller

Developing RESTful APIs becomes more and more popular but how to develop an API as abstract as possible? In this article I’ll describe how we do header based versioning with one controller for all Versions.

The Problem

For developing a RESTful API there are different approaches. You could separate different versions simply by a url like /api/v1/something. This works pretty well, it is fast to develop and a working solution. When you have a look at the APIs and how they work and what changes you might have in a different version it’s mainly not the routing / controller action – it’s more the data you deliver. The question we were asking our selves was: “Do we really need different controllers?”. The answer was something like: “A new version should not change the access point (routing) it should change the implementation”.

We were also developing APIs with Zend Frameworks Apigility and with this tool you still have different controllers/services but the main entry point and the routing is configured just once – except you have a new action.

For our Scala projects, we work also with the Request/Response principle which was introduced by Robert C. Martin in his book and talks about clean architecture. This means, that our controller receives or creates a request object, passing it to a business logic class and retrieves a response object back. For Api’s this principle is perfect – you get a request and return a response – the main principle of HTTP.

Let’s jump into the implementation

For our implementation we needed a versioning system, in which only the api-version knows it’s implementation. To specify a version we introduced a class called ApiVersion which gets the version number, a namespace, and a response format (class ApiFormat). Also we searched the web and had a look at Apigility, to see, how others do the versioning. A standard solution is to have a Version header like application/vnd.users.v1+json. This is also described in RFC4288. That means we have a vendor user in version 1 and the response is in json format.

package de.securing.api
abstract class ApiFormat(val name: String)
case object JSON extends ApiFormat("json")
case class ApiVersion(version: Int, namespace: String, format: ApiFormat = JSON) {
    override def toString: String = s"application/vnd.$namespace.v$version+${format.name}"
}

Now we are able to specify our versions in a very simple way. When your API should be able to return other formats as well, then ApiVersion should get a list of formats instead and change the get method in ApiVersions which will be shown later – but for now we’re okay with json only. For providing multiple versions we need another class which tells us which version uses which class as the implementation. To reach that, we need also a general abstract class for the Api itself. Let’s define a simple user API with get, patch, delete methods, and a implementation for that abstract class for version 1

package de.securing.api
import play.api.mvc.Request
import scala.util.Try
import org.json4s.native.Serialization

abstract class GenericApi // we will have a look later on that
trait ApiRequest
trait ApiResponse {
    /** json format (json4s package) */
    implicit val formats = Serialization.formats(NoTypeHints)
    def toJson: String
}
trait NoContentResponse extends ApiResponse {
    def toJson: String = ""
}

abstract class UserApi extends GenericApi {

    def get(implicit httpRequest: Request[String]): Try[ApiResponse]
    def patch(id: Int)(implicit httpRequest: Request[String]): Try[ApiResponse]
    def delete(id: Int)(implicit httpRequest: Request[String]): Try[ApiResponse]
}

package de.securing.user.api.v1
import play.api.mvc.Request
import scala.util.Try
class UserV1Api extends UserApi {

    def get(implicit httpRequest: Request[String]): Try[ApiResponse] = ???
    def patch(id: Int)(implicit httpRequest: Request[String]): Try[ApiResponse] = ???
    def delete(id: Int)(implicit httpRequest: Request[String]): Try[ApiResponse] = ???
}

Back to our versioning, we can now map the version to the implementation and register it in the versioning system. The ApiVersions trait has a list of version mapping ApiVersion -> GenericApi and a default version. There is also aget method to return the right API based on the HTTP version header. Using ApiVersions we can create ourApiVersion for our user API and map the implementation.

package de.securing.api
trait ApiVersions[A <: GenericApi] {

    /** mapping from version to api implementation */
    def versions[B <: A]: Map[ApiVersion, A]

    /** the default version if there is no header defined */
    val defaultVersion: ApiVersion

    /**
     * method, to get the api implementation by version header
     *
     * @param headerString the current header string
     * @return
     */
    private[api] def get[B <: A](headerString: Option[String]) = {

        require(versions.contains(defaultVersion), "The defined default version needs to be implemented in the versions map")

        headerString match {
            case Some(version) => versions.find(_._1.toString == version).map(_._2)
            case None => Some(versions(defaultVersion))
        }
    }
}

package de.securing.user.api
import javax.inject.Inject
class UserVersions @Inject()(v1Api: UserV1Api) extends ApiVersions[UserApi] {

    val v1 = ApiVersion(1, "user")
    def versions[B <: UserApi]: Map[ApiVersion, UserApi] = Map(v1 -> v1Api)
    val defaultVersion: ApiVersion = v1
}

The abstract generic API class

Versioning and a mapping for a specific version to its implementation and business logic is done. Before we can create the controller, we need to implement a bit of logic to out GenericApi class. We need to extract the request object from the HTTP-request and provide a method to call the business logic from the controller.

package de.securing.api
import org.json4s.native.Serialization
import org.json4s.native.Serialization.read
abstract class GenericApi {

    /** json format (json4s package) */
    implicit val formats = Serialization.formats(NoTypeHints) ++ additionalFormats

    private def parseHttpContent[R <: ApiRequest](implicit httpRequest: Request[String], m: Manifest[R]): Try[R] = Try {

        Option(httpRequest.body) map read[R] getOrElse {

            throw new ApiNoContentException("no content provided")
        }
    }

    def execute[R <: ApiRequest](requestObjectModify: R => R)(call: R => Try[ApiResponse])(implicit request: Request[String], m: Manifest[R]): Try[ApiResponse] = {
        parseHttpContent[R] flatMap (request => call(requestObjectModify(request)))
    }
}

First we import the json4s serialization format to be able to convert a given json to a case class. Json4s is a small scala library to easily covert data between scala classes and json. The converting is done in parseHttpContentmethod. If there is no content there is an exception thrown to return an error to the user. The interesting part is theexecute method. The first parameter is a callback to be able to modify the request object before we pass it. This is mainly important for get requests or some ids which shouldn’t be in the json coming from a user. If you would trust the users input, the user would be able to change other records – but it really depends on your API and your business logic. The second parameter is a method which takes the Request object and returns a Try of API Response. The next parameter is the play request object and the manifest from our generic request class A for json4s. In the method we create the request object by calling parseHttpContent and then we execute the callback with the modified request object from the other callback.

The abstract Controller

Let’s create an abstract controller to call out API methods from UserV1Api. This controller needs to know about our versioning and our interface/trait for the API. The controller should also be able to call the business logic and transform/convert the response to the ApiFormat(json).

package de.securing.api
import play.api.mvc._
import scala.util.{Failure, Success, Try}
trait ApiController[A <: GenericApi, V <: ApiVersions[A]] extends Results {

    /** requires the Versions class to get the API implementation */
    val apiVersions: V

    private[api] def getApi(request: Request[String]): Try[A] = Try {

        val version = request.headers.get("accept").exists(_.toString.matches("""application\/vnd\.[^.]+\.v[0-9]+\+[a-z]+""")) match {
            case true => request.headers.get("accept").map(_.toString)
            case false => None
        }

        apiVersions.get(version) match {

            case Some(api) => api
            case None => throw new ApiVersionNotFoundException(s"The Api Version $version does not exist.")
        }
    }

    private[api] def toResult[R <: ApiResponse](response: Try[R]): Result = {

        response match {

            case Success(res: NoContentResponse) => NoContent
            case Success(res: ApiResponse) => Ok(res.toJson).as("application/json; charset=utf8")
            case Failure(exception) => ApiProblem.fromException(exception).httpResponse
        }
    }

    def callApi(execute: Request[String] => A => Try[ApiResponse]) = Action(BodyParsers.parse.tolerantText) { implicit request =>

        val response = getApi(request) flatMap (api => execute(request)(api))
        toResult(response)
    }
}

Our abstract controller get’s the two generic classes for an implementation the GenericApi and ApiVersions. There is also a value which needs to be implemented to have access to the versioning class. The getApi method will have a look at the HTTP-request header and search for an accept header. If there is one, we check if the header is based on RFC4288 and if so, we try to retrieve the API class by calling the get method we saw earlier. The toResultmethod is a very simple mapping from Try[ApiResponse] to a valid HTTP response in play or in case of an exception an ApiProblem which I will explain a later in the section Error messages. It’s just an error format which always has the same structure. In callApi we have a callback for Request[String] => GenericApi => Try[ApiResponse]. This means we have a function taking the Request[String]as its input and retuning a function which then takes A (the Api) as it’s input and return a Try of ApiResponse. The implementation is then Play conform – we have our action with a String parser for it’s HTTP body and the implicit request object. From there we get the api and call the execute method we saw earlier and then we return the play response by using the toResult method.

Our own API controller

Because of our abstract classes the controller is now easy going. We just need to define some settings implement our actions and call the abstract controller. To remember, we have get patch and delete methods in our API, so we also need three actions.

package controllers
class User @Inject()(
    val apiVersions: UserVersions
) extends ApiController[UserApi, UserVersions] {

    def get() = callApi { implicit request => api => api.get}
    def patch(id: Int) = callApi { implicit request => api => api.patch(id)}
    def delete(id: Int) = callApi { implicit request => api => api.delete(id)}
}

That’s it. Looks pretty simple and straight forward. Now we need the routing for our new controller.

GET    /api/user                controllers.User.get()
PATCH  /api/user/$id<[\d]+>     controllers.User.patch(id: Int)
DELETE /api/user/$id<[\d]+>     controllers.User.delete(id: Int)

Let’s get back to our UserV1Api to implement the business logic. The get method should return a list of users, the patch method should be able to modify the user data and return the new data, and the delete method should be able to delete a specific user and return no content. For that, we need also request and response object and a storage system. I’ll skip the storage system completely because it’s not really relevant here. Le’ts just consider we have a storage for users with the following fields: id: Int, email: String, name: String to keep it simple. For request objects we need just a PatchRequest object – for get and delete we do not really need one. At secu-ring we do this anyway – therefore we have the requestModifyCallback function we saw in GenericApi I’ll demonstrate that in the following patch and delete method.

package de.securing.user.api.v1
import org.json4s.native.Serialization.write

case class UserEntity(id: Int, email: String, name: String)

case class PatchRequest(id: Int = 0, email: Option[String] = None, name: Option[String] = None) extends ApiRequest
case class DeleteRequest(id: Int = 0) extends ApiRequest
case class UserListResponse(users: List[UserResponse]) extends ApiResponse {
    override def toJson: String = write(this) 
}
case class UserResponse(id: Int, email: String, name: String) extends ApiResponse {
    override def toJson: String = write(this)
}
class DeleteResponse extends ApiResponse with NoContentResponse {
    override def toJson: String = ""
}

class UserV1Api extends UserApi {

    def get(implicit httpRequest: Request[String]): Try[ApiResponse] = Try {
        /** here you need to load users from your data storage */
        val users: List[UserEntity] = dataStorage.getUsers
        val userResponse: List[UserResponse] = users.map{ user => 
            UserResponse(user.id, user.email, user.name)
        }
        UserListResponse(userResponse)
    }
    def patch(id: Int)(implicit httpRequest: Request[String]): Try[ApiResponse] = {
        execute[PatchRequest](_.copy(id = id))(patch)
    }
    def delete(id: Int)(implicit httpRequest: Request[String]): Try[ApiResponse] = {
        execute[DeleteRequest](_.copy(id = id))(delete)
    }

    private def patch(request: PatchRequest): Try[UserResponse] = Try {
        val userEntity = dataStorage.loadUser(request.id)
        val modified = userEntity.copy(
            email = request.email.getOrElse(userEntity.email),
            name = request.name.getOrElse(userEntity.name)
        )

        dataStorage.save(userEntity)
        UserResponse(modified.id, modified.email, modified.name)
    }

    private def delete(request: DeleteRequest): Try[DeleteResponse] = Try {
        val userEntity = dataStorage.loadUser(request.id)
        dataStorage.delete(userEntity)

        new DeleteResponse
    }
}

For the get method, we don’t have a request object so the implementation starts right away. First we load a list of users from our storage system and convert them to UserResponse objects. In that simple way it might look confusing because the objects look exactly the same. If you want, you could also use your entity as a response object but then you face problems in raising your versions. Let’s consider in version 2 you want to split the name of the user in 2 separate fields which requires an entity change. Now Version 1 will break. With the response object, you can still deliver the same data in Version 1. After mapping the entity, we put it to the UserListResponse which should resolve in following json:

{
    "users": [
        {"id": 1, "email": "foo@bar.de", "Foo Bar"},
        {"id": 2, "email": "baz@bar.de", "Baz Bar"}
    ]
}

For patch the things working a bit different, because here we really need the json data provided by the patch request which will be automatically converted into the given request object. After that we use the request modify callback to set the id to the request object to have all parameters in one class. Then we call the method to execute the real business logic. You could also remove the requestObject callback in the GenericApi class and rewrite your methods to something like that:

def patch(id: Int)(implicit httpRequest: Request[String]): Try[ApiResponse] = {
    execute[PatchRequest](patch(id))
}
def patch(id: Int)(request: PatchRequest): Try[UserResponse] = ???

 

API Error messages – Exceptions

As we saw a bit earlier there is this ApiProblem class. This is a standardized format for reporting errors in json format. This is also implemented in Apigility and I really like this. The reason is, that every exception will be reported as an ApiProblem to the user of the API. Therefore we need to translate different errors to different HTTP status codes and mak exceptions to different error messages.

package de.securing.api
import play.api.http.{HeaderNames, Status}
import org.json4s.native.Serialization.write

case class ApiVersionNotFoundException(e: String) extends scala.Exception(e) with scala.Product with scala.Serializable
case class ApiNoContentException(e: String) extends scala.Exception(e) with scala.Product with scala.Serializable
case class ApiMethodNotImplementedException(e: String) extends scala.Exception(e) with scala.Product with scala.Serializable
case class ApiValidationException(e: String) extends scala.Exception(e) with scala.Product with scala.Serializable
case class EntityNotFoundException(e: String) extends scala.Exception(e) with scala.Product with scala.Serializable

case class ApiProblem(
`type`: String, detail: String, status: Int, title: String
) extends Results with HeaderNames with Status {

    /** json format (json4s package) */
    implicit val formats = Serialization.formats(NoTypeHints)
    def toJson = write(this)

    /**Return a valid HTTP Response for play to render an ApiProblem nicely
     */
    def httpResponse = {

        val result = status match {
            case NOT_FOUND => NotFound(toJson)
            case UNPROCESSABLE_ENTITY => UnprocessableEntity(toJson)
            case NOT_ACCEPTABLE => NotAcceptable(toJson)
            case UNAUTHORIZED => Unauthorized(toJson)
            case NOT_IMPLEMENTED => NotImplemented(toJson)
            case _ => BadRequest(toJson)
        }

        result.as("application/problem+json; charset=utf8")
    }
}
object ApiProblem extends play.api.http.Status {

    def fromException(exception: Throwable): ApiProblem = {

        exception match {

            case e: EntityNotFoundException => ApiProblem("Entity Missing", "The requested entity was not found on the server",  NOT_FOUND, "Entity not found")
            case e: ApiMethodNotImplementedException => ApiProblem("Implementation missing", exception.getMessage, NOT_IMPLEMENTED, "Error on processing Request")
            case e: ApiVersionNotFoundException => ApiProblem("Not allowed", exception.getMessage, NOT_IMPLEMENTED, "Error on processing Request")
            case e: ApiValidationException => ApiProblem("Validation error", exception.getMessage, NOT_ACCEPTABLE, "Validation error")
            case e: ApiNoContentException => ApiProblem("No Content", exception.getMessage, UNPROCESSABLE_ENTITY, "Thhere is no data to process to request")
            case _ => ApiProblem("Internal Server error", "Your request cannot be process during an internal server error: %s" format exception.toString,  INTERNAL_SERVER_ERROR, "Server Error")
        }
    }
}

Let’s create version 2

To create a new version of the API we start with our implementation. We will now separate the name in the user entity to really see the benefit of the request / response model. We will now have different request and response classes. First, we modify our user entity and the storage system to apply the changes. The version 1 will now break, so wee need to make sure that the db changes does not have an impact on the first version. The new version could look like this:

case class UserEntity(id: Int, email: String, firstName: String, lastName: String)
class UserV1Api extends UserApi {

    def get(implicit httpRequest: Request[String]): Try[ApiResponse] = Try {
        /** here you need to load users from your data storage */
        val users: List[UserEntity] = dataStorage.getUsers
        val userResponse: List[UserResponse] = users.map{ user =>
            UserResponse(user.id, user.email, s"${user.firstName} ${user.lastName}")
        }
        UserListResponse(userResponse)
    }
    def patch(id: Int)(implicit httpRequest: Request[String]): Try[ApiResponse] = {
        execute[PatchRequest](_.copy(id = id))(patch)
    }
    def delete(id: Int)(implicit httpRequest: Request[String]): Try[ApiResponse] = {
        execute[DeleteRequest](_.copy(id = id))(delete)
    }

    private def patch(request: PatchRequest): Try[UserResponse] = Try {
        val userEntity = dataStorage.loadUser(request.id)
        val modified = userEntity.copy(
            email = request.email.getOrElse(userEntity.email),
            firstName = request.name.flatMap(_.split(" ").headOption).getOrElse(userEntity.firstName),
            lastName = request.name.flatMap(name => Option(name.split(" ")(1))).getOrElse(userEntity.lastName)
        )

        val user = dataStorage.save(userEntity)
        UserResponse(modified.id, modified.email, s"${user.firstName} ${user.lastName}")
    }

    private def delete(request: DeleteRequest): Try[DeleteResponse] = Try {
        val userEntity = dataStorage.loadUser(request.id)
        dataStorage.delete(userEntity)

        new DeleteResponse
    }
}

Now, version 1 runs again and stays the same as before, at least for the endpoint and users of the API. Now we can copy over the version to version 2 and apply the changes regarding the user entity.

package de.securing.user.api.v2
import org.json4s.native.Serialization.write

case class PatchRequest(id: Int = 0, email: Option[String] = None, firstName: Option[String] = None, lastName: Option[String]) extends ApiRequest
case class DeleteRequest(id: Int = 0) extends ApiRequest
case class UserListResponse(users: List[UserResponse]) extends ApiResponse {
    override def toJson: String = write(this) 
}
case class UserResponse(id: Int, email: String, firstName: String, lastName: String) extends ApiResponse {
    override def toJson: String = write(this)
}
class DeleteResponse extends ApiResponse with NoContentResponse {
    override def toJson: String = ""
}

class UserV2Api extends UserApi {

    def get(implicit httpRequest: Request[String]): Try[ApiResponse] = Try {
        /** here you need to load users from your data storage */
        val users: List[UserEntity] = dataStorage.getUsers
        val userResponse: List[UserResponse] = users.map{ user => 
            UserResponse(user.id, user.email, user.firstName, user.lastName)
        }
        UserListResponse(userResponse)
    }
    def patch(id: Int)(implicit httpRequest: Request[String]): Try[ApiResponse] = {
        execute[PatchRequest](_.copy(id = id))(patch)
    }
    def delete(id: Int)(implicit httpRequest: Request[String]): Try[ApiResponse] = {
        execute[DeleteRequest](_.copy(id = id))(delete)
    }

    private def patch(request: PatchRequest): Try[UserResponse] = Try {
        val userEntity = dataStorage.loadUser(request.id)
        val modified = userEntity.copy(
            email = request.email.getOrElse(userEntity.email),
            firstName = request.firstName.getOrElse(userEntity.firstName),
            lastName = request.lastName.getOrElse(userEntity.lastName)
        )

        val user = dataStorage.save(userEntity)
        UserResponse(modified.id, modified.email, modified.firstName, user.lastName)
    }

    private def delete(request: DeleteRequest): Try[DeleteResponse] = Try {
        val userEntity = dataStorage.loadUser(request.id)
        dataStorage.delete(userEntity)

        new DeleteResponse
    }
}

After our implementation is done, we need to map the second version to the second implementation and everything should work.

class UserVersions @Inject()(v1Api: UserV1Api, v2Api: UserV2Api) extends ApiVersions[UserApi] {

    val v1 = ApiVersion(1, "user")
    val v2 = ApiVersion(2, "user")
    def versions[B <: UserApi]: Map[ApiVersion, UserApi] = Map(
        v1 -> v1Api,
        v2 -> v2Api
    )
    val defaultVersion: ApiVersion = v2
}

Done! If we did everything right, the version 1 api should work the same way as before and the new version also. There is now a problem with duplicated source code which can be solved by extending. Version 1 should extend version 2 and overwrite only the methods which are different in 1. Therefore you need to change the methods fromprivate to protected. Maybe you’re wondering why version 1 should extend version 2 and not the other way around. My thought is, that when you have at some point version 6 and don’t want to support version 1 anymore, you can easily delete the version 1 namespace. In the other way, you need to copy code over to version 2 before you can delete version 1.

Conclusion

In my opinion this is a nice and abstract solution for the problem to have just one controller for all versions and having the flexibility of easy version changes. This system is not yet ready to cover all possibilities. For our current internal use it fits the requirements and works pretty well. We’re also able to easily test the business logic of the API. You should see this article as an inspiration how it can work.

I believe that this system can save some time in developing new features or provide new versions and API changes. I’m curious about your opinions and experiences with writing APIs. So don’t hesitate to write a comment.

Happy Coding.

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.