Filtering, Sorting, and Pagination
Filtering
―LHS Brackets
GET /items?price[gte]=10&price[lte]=100―RHS Colon
GET /items?price=gte:10&price=lte:100―Search Query Param
GET /items?q=title:red chair AND price:[10 TO 100]Pagination
―Offset Pagination
GET /items?limit=20&offset=100, returns the 20 rows starting with the 100th row―Keyset Pagination
GET /items?limit=20&created:lte:2018-01-20T00:00:00It uses the filter values of the last page to fetch the next set of items
―Seek Pagination
GET /items?limit=20&after_id=20Extension of Keyset paging. By adding an after_id or start_id URL parameter
Sorting
―Multi-Column Sort
GET /users?sort_by=desc(last_modified),asc(email) GET /users?sort_by=-last_modified,+emailThe good API design is a critical component for your Developer Experience (DX). API specifications can outlast many underlying server implementations which require thinking about future use cases for your API.
https://www.moesif.com/blog/technical/api-design/REST-API-Design-Filtering-Sorting-and-Pagination/#
Getting, creating, updating or deleting multiple resources in one API call
―Working with a single resource
Creating a resource
POST /resources HTTP/1.1 { "some": "some data", "other": "some other data" }
Getting a resource
GET /resources/ID
Updating a resource
PATCH /resources HTTP/1.1 { "other": "modified data" }
Replacing or creating a resource
PUT /resources/ID HTTP/1.1 { "some": "some new data", "other": "some other new data" }
Deleting a resource
DELETE /resources/ID
―But why explaining all that? I want to work with multiple resources!
―Same action on resources of the same type
A request containing multiple resources
GET /resources POST /resources PATCH /resources PUT /resources DELETE /resources
Create multiple resources
POST /resources [ { "id": "CREATION1", "body": {"resource's": "data"} }, { "id": "CREATION2", "body": {"resource's": "data"} } ] POST /resources { "CREATION1" : {"resource's": "data"}, "CREATION2" : {"resource's": "data"} }
Update or replace multiple resources
PATCH /resources { "ID1" : {"resource's": "data"}, "ID2" : {"resource's": "data"} }
Get or delete multiple resources
GET /resources?ids=ID1,ID2 DELETE /resources?ids=ID1,ID2
―A response containing responses
The 207 (Multi-Status) status code provides status for multiple independent operations:
[ { "id": "ID1", "status": "201", "headers": [ {"header's name": "header's value"} ], "body": { "the": "response's body"} }, { "id": "ID2", "status": "400", "headers": [ {"header's name": "header's value"} ], "body": { "the": "response's body"} } ] { "ID1": { "status": "201", "headers": [ {"header's name": "header's value"} ], "body": { "the": "response's body"} }, "ID2": { "status": "400", "headers": [ {"header's name": "header's value"} ], "body": { "the": "response's body"} } }
―Two levels of error
In that case, we must be aware that there are two types of errors, the one concerning one or more of the resources and the one concerning the multiple request itself.
―Single and multiple creations with the same endpoint
Use a list/map for both case
{ "CREATION1" : {"resource's": "data"} }
Accept both a list/map and single object
{"resource's": "data"} //and { "CREATION1" : {"resource's": "data"} "CREATION2" : {"resource's": "data"} }
―Different actions on resources of the same type
POST /resources or POST /resource-modifications [ { "id": "ID1", "method": "DELETE" }, { "id": "ID2", "method": "PATCH", "body": {"resource's": "data"} } ]
―Different actions on resources of different types
POST /bulk or POST /batch [ { "id": "ACTION1", "uri": "/resources/ID1", "method": "DELETE" }, { "id": "ACTION2", "uri": "/another-resources/ID2", "method": "PATCH", "body": {"resource's": "data"} } , { "id": "ACTION3", "uri": "/resources", "method": "POST", "body": {"resource's": "data"} } ]
Best Practices for Designing a Pragmatic RESTful API
―An API is a user interface for a developer - make it pleasant
―Use RESTful URLs and actions
GET /tickets - Retrieves a list of tickets GET /tickets/12 - Retrieves a specific ticket POST /tickets - Creates a new ticket PUT /tickets/12 - Updates ticket #12 PATCH /tickets/12 - Partially updates ticket #12 DELETE /tickets/12 - Deletes ticket #12 //relations: GET /tickets/12/messages - Retrieves list of messages for ticket #12 GET /tickets/12/messages/5 - Retrieves message #5 for ticket #12 POST /tickets/12/messages - Creates a new message in ticket #12 PUT /tickets/12/messages/5 - Updates message #5 for ticket #12 PATCH /tickets/12/messages/5 - Partially updates message #5 for ticket #12 DELETE /tickets/12/messages/5 - Deletes message #5 for ticket #12
Use plural
―Use SSL everywhere, no exceptions
―An API is only as good as its documentation - so have great documentation
―Version via the URL, not via headers
―Use query parameters for advanced filtering, sorting & searching
//filtering GET /tickets?state=open //sorting GET /tickets?sort=-priority GET /tickets?sort=-priority,created_at //searching --Retrieve recently updated tickets GET /tickets?sort=-updated_at --Retrieve recently closed tickets GET /tickets?state=closed&sort=-updated_at --Retrieve the highest priority open tickets mentioning the word 'return' GET /tickets?q=return&state=open&sort=-priority,created_at //Aliases for common queries GET /tickets/recently_closed
―Provide a way to limit which fields are returned from the API
GET /tickets?fields=id,subject,customer_name,updated_at&state=open&sort=-updated_at
―Return something useful from POST, PATCH & PUT requests
―HATEOAS isn't practical just yet
'Hypermedia As The Engine of Application State' (say return the data and links to other relevant endpoints related to the current endpoint)
―Use JSON where possible
―You should use camelCase with JSON, but snake_case is 20% easier to read
―Pretty print by default & ensure gzip is supported
Since the cost of pretty-printing is relatively small, it's best to pretty print by default
―Don't use response envelopes by default
//AVOID { "data" : { "id" : 123, "name" : "John" } } //PREFER { "id" : 123, "name" : "John" }
―Consider using JSON for POST, PUT and PATCH request bodies
―Paginate using Link headers
Link: <https://api.github.com/user/repos?page=3&per_page=100>; rel="next", <https://api.github.com/user/repos?page=50&per_page=100>; rel="last"―Provide useful response headers for rate limiting
//The number of allowed requests in the current period X-Rate-Limit-Limit //The number of remaining requests in the current period X-Rate-Limit-Remaining //The number of seconds left in the current period X-Rate-Limit-Reset
―Auto loading related resource representations
GET /tickets/12?embed=customer.name,assigned_user { "id" : 12, "subject" : "I have a question!", "summary" : "Hi, ....", "customer" : { "name" : "Bob" }, assigned_user: { "id" : 42, "name" : "Jim", } }
―Use token-based authentication, transported over OAuth2 where delegation is needed
―Include response headers that facilitate caching
If-None-Match Last-Modified Modified-Since
―Define a consumable error payload
//AVOID { "code" : 1234, "message" : "Something bad happened :(", "description" : "More details about the error here" } //PREFER { "code" : 1024, "message" : "Validation Failed", "errors" : [ { "code" : 5432, "field" : "first_name", "message" : "First name cannot have fancy characters" }, { "code" : 5622, "field" : "password", "message" : "Password cannot be blank" } ] }
―Effectively use HTTP Status codes
//Response to a successful GET, PUT, PATCH or DELETE. Can also be used for a POST that doesn't result in a creation. 200 OK //Response to a POST that results in a creation. Should be combined with a Location header pointing to the location of the new resource 201 Created //Response to a successful request that won't be returning a body (like a DELETE request) 204 No Content //Used when HTTP caching headers are in play 304 Not Modified //The request is malformed, such as if the body does not parse 400 Bad Request //When no or invalid authentication details are provided. Also useful to trigger an auth popup if the API is used from a browser 401 Unauthorized //When authentication succeeded but authenticated user doesn't have access to the resource 403 Forbidden //When a non-existent resource is requested 404 Not Found //When an HTTP method is being requested that isn't allowed for the authenticated user 405 Method Not Allowed //Indicates that the resource at this end point is no longer available. Useful as a blanket response for old API versions 410 Gone //If incorrect content type was provided as part of the request 415 Unsupported Media Type //Used for validation errors 422 Unprocessable Entity 429 Too Many Requests
Common disagreements over REST API design
―“If the engine of application state (and hence the API) is not being driven by hypertext, then it cannot be RESTful and cannot be a REST API.”
―PUT vs POST: Which should you use for creating new resources?
If you know the location of the resource then use PUT, otherwise, POST is probably more appropriate.
―Doing PATCH “properly”
PATCH /my/data HTTP/1.1 Host: example.org Content-Length: 326 Content-Type: application/json-patch+json If-Match: "abc123" [ { "op": "test", "path": "/a/b/c", "value": "foo" }, { "op": "remove", "path": "/a/b/c" }, { "op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ] }, { "op": "replace", "path": "/a/b/c", "value": 42 }, { "op": "move", "from": "/a/b/c", "path": "/a/b/d" }, { "op": "copy", "from": "/a/b/d", "path": "/a/b/e" } ]
―The correct way of doing Authentication: stateless
―“A REST API must not define fixed resource names or hierarchies. Servers must have the freedom to control their own namespace. Instead, allow servers to instruct clients on how to construct appropriate URIs.”
―REST and CRUD are not the same things.
―“A REST API should not be dependent on any single communication protocol”
―Doing versioning 'right': putting it on the URI, using a custom request header or adding it to the HTTP Accept header.
REST cookbook
The RESTful cookbook website is inspired by the puppet cookbook from Dean Wilson, who created cookbook recipes on how to deal with some of the issues people are facing over and over again when using the Puppet Configuration management tool. This site, however, is about dealing with issues and questions people are facing over and over again when trying to create RESTful APIs.
―Basics
- How do I let users log into my RESTful API?
- Caching your REST API
- How do I version my REST API?
- What is HATEOAS and why is it important?
- What is the code-on-demand constraint?
―HTTP headers
―Mediatypes
HTTP API Design Guide
This guide describes a set of HTTP+JSON API design practices, originally extracted from work on the Heroku Platform API.
This guide informs additions to that API and also guides new internal APIs at Heroku.
It’s also of interest to API designers outside of Heroku.
The goals here are consistency and focusing on business logic while avoiding design bikeshedding.
Looking for a good, consistent, well-documented way to design APIs, not necessarily the only/ideal way.
It is assumed you’re familiar with the basics of HTTP+JSON APIs
This guide will not cover all of the fundamentals.
It is available for online reading and in multiple formats at gitbook.
―Foundations
- Separate Concerns
- Require Secure Connections
- Require Versioning in the Accepts Header
- Support ETags for Caching
- Provide Request-Ids for Introspection
- Divide Large Responses Across Requests with Ranges
―Requests
- Accept serialized JSON in request bodies
- Resource names
- Actions
- Use consistent path formats
- Downcase paths and attributes
- Support non-id dereferencing for convenience
- Minimize path nesting