In the previous blog post GraphQL with Intershop (Part 1): Improved Client Performance we gave a brief introduction to GraphQL as well as to our GraphQL-Gateway, which forms a bridge between an existing “classical” REST-API and a GraphQL API. But how does it work in detail? In this post, we are going to take a closer look.
Static and Dynamic Side of Things
As shown in the last blog post, the first step in implementing GraphQL is to create the schema that defines both the possible queries as well as the format of returned data. REST does not have a required equivalent of this, although OpenAPI/Swagger tries to approximate this. So, in the end, our gateway actually consists of two components:
- The “static” component which takes an OpenAPI schema file and generates a GraphQL schema,
- The “dynamic” component which takes GraphQL queries (that match the schema), converts them into corresponding REST requests, reads the REST response and passes it back as the GraphQL result.
The static part can be done once during startup while the dynamic part is done continuously. Now, once we started implementing it, it turned out that the static part is far more involved than the dynamic one. This is because once you have a clear schema defined, the way of converting GraphQL queries into REST requests (and similarly REST to GraphQL responses) comes naturally. Perhaps this is once again an indicator as to why static type systems are so valuable.
Let’s take the the simple Product API from the last post, consisting of a single endpoint /products/{productKey} which returns an object that conforms to the ProductRO-OpenAPI-schema defined like this:
Starting Out Simple
ProductRO:
type: object
properties:
sku:
type: string
productName:
type: string
shortDescription:
type: string
longDescription:
type: string
availability:
type: boolean
listPrice:
$ref: '#/components/schemas/ProductPriceRO'
salePrice:
$ref: '#/components/schemas/ProductPriceRO'
minListPrice:
$ref: '#/components/schemas/ProductPriceRO'It makes sense to first consider how to map this to a GraphQL type. The answer seems fairly obvious: Make it a GraphQL type with the respective properties and repeat recursively for the nested types such as ProductPriceRO. While this general approach works fine in many cases, there are some intrinsic differences between GraphQL and OpenAPI that might pose some problems:
- What if the nested schemas are defined inline, instead of being referenced with via
$ref? In this case the inner schema does not have a name so we would have to guess one. - What do we do with
additionalProperties-schemas? Those behave like maps and have no fixed set of properties, but GraphQL, being inherently static, does not support this. - How to handle custom primitives? GraphQL supports custom scalars (think for example of a custom scalar named
Currency) while OpenAPI doesn’t do this directly. - How to handle input types? GraphQL strictly differentiates between so-called “input types” and “output types”. Input types are the ones which can be passed to a mutation. Output types are returned by queries or mutations. However, in OpenAPI, a schema defined in the components section can be used both as a parameter and as a response schema. This means that the
ProductROfrom above is not mapped to just one GraphQL type but to two: one input and one output type. - How to correctly handle
null-values? Throughout the history of programming languages, the handling ofnullhas been as diverse as it has been controversial. This is no exception: OpenAPI has thenullableproperty while GraphQL has aNonNullType modifier. And have I mentioned that the next version of OpenAPI will go back to null-types instead ofnullable? And how does HTTP 204 No Content fit into all this? As you can see, things are messy and we have to find the best possible compromise in mapping between these two.
Aside from these (still important) details, converting a schema is quite straightforward. How do we represent the actual endpoint then? Well, we convert parameters to field attributes and turn the whole operation into a field of the root type Query:
type Query {
Product(productKey: String!): Product!
}Handling Intershop-specific Concepts
Sometimes, applying this simple conversion scheme does not yield satisfying results. What if we want to show a list of products? The corresponding REST endpoint /products will return a list of links to actual product objects. When we look at the OpenAPI schema, we can see that the endpoint has the following response schema (the version shown here is slightly simplified):
ResourceCollectionROLinkRO:
title: Link List
type: object
properties:
pageable:
type: string
description: the pageable ID
total:
type: integer
description: the pageable amount total
format: int32
offset:
type: integer
description: the pageable offset
format: int32
amount:
type: integer
description: the pageable amount
format: int32
elements:
type: array
description: the list of elements
xml:
wrapped: true
items:
$ref: '#/components/schemas/LinkRO'
LinkRO:
title: Link
type: object
properties:
url:
type: stringAs you can see, LinkRO is just a link to the REST endpoint which will return the actual product. The problem that we have here is that this type of response hides the actual type that is returned, namely the products. Instead, GraphQL only sees a list of links, yielding a result like this:
type Query {
products : ResourceCollectionROLinkRO
}
type ResourceCollectionROLinkRO {
pageable: String!
total: int!
offset: Int!
amount: Int!
elements: [LinkRO!]!
}
type LinkRO {
url: String!
}This, of course, defies the whole purpose of the gateway: The user would still have to perform REST requests manually to resolve the links. However, there is only one piece of information left for the gateway to be able to fully reconstruct the scenario: The actual type of the elements in the link list. Luckily, we can provide just that using OpenAPI extension attributes:
responses:
200:
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/ResourceCollectionROLinkRO'
x-element-type: ProductROIt turned out that this is sufficient for the gateway to achieve reasonable results:
type Query {
products : [Product!]!
}Of course, we have completely ignored pagination here, but the gist remains that we have to use extension attributes to provide the Gateway with hints so it can produce better results. If you are interested in the concrete format of the pagination, you can take a look at the previous post in which we briefly described it.
From Endpoints to Type Hierarchies: Linking Things Together
Following the approach outlined above, you already get a very usable GraphQL-API. However, one problem remains. Let’s consider the following two endpoints:
- /products/{productKey} returning a
ProductRO - /products/{productKey}/reviews returning a list of
ReviewROs. This would translate into the following GraphQL schema:
type Query {
Product(productKey: String!) : Product!
reviews(productKey: String!) : [Review!]!
}What we have here is essentially a shallow hierarchy, one that mimics REST. As discussed in the last blog post, this comes with the usual problems of underfetching and also goes against the design of GraphQL. Thus, we need a way to “link” different endpoints together. Once again, OpenAPI has the mechanics already built-in: OpenAPI Links.
responses:
200:
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/ProductRO'
links:
reviewsLink:
operationId: getReview
parameters:
productKey: $response.body#/sku
description: Link to the reviewsThis link says that the sku of the response of /products/{productKey} can be used as a parameter to /products/{productKey}/reviews (we assume that this operation has the ID getReview). From this link, the gateway can figure out how to attach the getReview endpoint to the product:
type Product {
sku : String!
reviews: [Review!]!
...
}The reviews can then be queried in an intuitive manner:
query {
Product(productKey : "12345") {
name,
reviews {
content
}
}
...
}What is even more important: The client only has to run one single GraphQL query! The Gateway then does the following:
- It calls /products/12345 and extracts the
nameand theskuof the result. - It uses the returned
skuto call /products/12345/reviews which returns a list of links. - It resolves each link to a review and extracts its content.
The advantage of GraphQL here is evident, only one query is required to get all of that information. One might ask now what we have won, since it’s now the Gateway’s job to handle these REST calls – There are still many REST calls made, just by the Gateway instead of the client. However, the Gateway is usually located in the same data center (maybe even on the same machine) as the actual server, so those requests can be handled much faster.
We have added a fallback mechanism in case adding OpenAPI links is not feasible. In that case, the Gateway will try to match the endpoints by their URL prefixes. The example above will still work, because /products/{productKey} is recognized to be a prefix of /products/{productKey}/reviews. As in REST this structure usually indicates subobjects, it is reasonable to assume a connection between the two.
Extending from Outside: Reverse Links
It is common for companies to extend the Intershop API by providing custom data for the data models. This is no problem for our Gateway: It can be fed as many OpenAPI models as required. It will then process them all into one single GraphQL schema. Let’s imagine that a company stores reference manuals for products on their servers and provides a custom REST API that is compatible with the Intershop API:
GET /manuals/{productKey}/
=> "https://example.com/manuals/demo-manual-1.pdf"A corresponding OpenAPI schema might look like this:
/manuals/{productKey}:
get:
operationId: getManuals
parameters:
- name: productKey
in: path
schema:
type: string
required: true
responses:
200:
description: OK
content:
text/plain:
schema:
type: string
204:
description: No ContentThe company now wants to integrate the manuals API with the main Intershop API via the GraphQL Gateway. However, as we’ve just seen, that requires an OpenAPI link from the product resource to the manuals resource. However, the company usually does not have access to Intershop’s OpenAPI model, so it is not possible to add a link to the manual’s resource from the outside. How can we still attach the manual resource to the product?
One option would be to reformat the resource to be on the /products/{productKey}/manuals endpoint. In this case the Gateway will automatically recognize the connection, as mentioned above. However, this seems to be a somewhat tedious solution. It is not always easily possible to just change the endpoint as it can be connected to a lot of backend infrastructure.
This is why we have implemented a custom concept to extend regular OpenAPI links, which we call reverse links. Reverse links are just like regular links, but they are not placed at the link source, but at the target. We can then specify the source using an extension attribute:
/manuals/{productKey}:
get:
operationId: getManuals
parameters:
- name: productKey
in: path
schema:
type: string
required: true
responses:
200:
description: OK
content:
text/plain:
schema:
type: string
links:
manualFromProduct:
operationId: getManuals
parameters:
productKey: '$response.body#/sku'
x-source-operation: getProductSingle # points to /products/{productKey}
204:
description: No Content
x-graphql-field: manualsThe x-source-operation marks the link as a reverse link and specifies the source of the link as the products resource. The target is still specified in the operationId and points to ourselves, the manuals resource. This way, we have specified a connection between the two resources without modifying the Intershop OpenAPI schema or changing our URLs. The result is just as we expect it:
type Product {
...
manuals : String
}You might have noticed the x-graphql-field attribute. It can be used to specify a better name for the field (normally the link name, in our case manualFromProduct, would be used).
Summary and Outlook
We’ve seen how the GraphQL Gateway is both a static bridge between OpenAPI and the GraphQL schema as well as a dynamic bridge between GraphQL queries and REST requests.
Different linking techniques can be used to turn a shallow REST API into a connected hierarchy of GraphQL types that solves the problems of over- and underfetching described in the previous post.
We’ve also seen how the Gateway provides an elegant approach to extensibility: Companies can provide their own REST APIs that can then be fully integrated into the existing Intershop ones, resulting in a single, concise GraphQL schema, allowing the client to focus on retrieving data instead of routing data back and forth.
In the future, we might write a post describing how the Gateway can be applied to the Intershop PWA, providing a real-life use case. This would also be an example of a smooth migration from REST to GraphQL: The Gateway could be seen as a nice transition that one can use before switching fully to GraphQL.