本文从网上搜集了很多资料，看完应该会对restful有一个全面的认识。

• Restful API Design Guide
  • What is REST
    • Guiding Principles of REST
      • Client–server separating
      • Stateless
      • Cacheable
      • Uniform interface
      • Layered system
      • Code on demand (optional)
    • Resource
    • Resource Methods
    • Http
  • REST Resource Naming Best Practices
    • Use nouns to represent resources
    • Resource Archetypes
    • Use hyphens (-) to improve the readability of URIs
    • Use lowercase letters in URIs
    • Do not use file extenstions
    • Never use CRUD function names in URIs
    • Use query component to filter URI collection
    • Do not use trailing forward slash (/) in URIs
    • Use forward slash (/) to indicate a hierarchical relationships
  • REST API Versioning
    • When to Version
    • How to Version
  • REST API Security Essentials
    • REST Security Design Principles
    • Best Practices to Secure REST APIs
      • 1. Keep it Simple, Basic Authentication
      • 2. Always Use HTTPS
      • 3. Use Password Hash
      • 4. Never expose information on URLs
      • 5. Consider OAuth
      • 6. Consider Adding Timestamp in Request
      • 7. Input Parameter Validation
      • 8. API Key
      • 9. Validating SSL Certificates
      • 10. Whitelist IP addresses
      • 11. Two-factor authentication
      • 12. SSO, SAML, OpenID Connect
      • 13. Asymmetric encryption
      • 14. Rate limit
  • Caching REST API Response
    • Caching in REST APIs
      • Expires
      • Cache-Control
      • ETag
      • Last-Modified
  • REST Resource Representation Compression
    • Accept-Encoding
    • Content-Encoding
  • REST – Content Negotiation
  • HATEOAS Driven REST APIs
  • Idempotent REST APIs
    • HTTP POST
    • HTTP GET, HEAD, OPTIONS and TRACE
    • HTTP PUT
    • HTTP DELETE
    • POST vs PUT
  • Statelessness
  • REST API – N+1 Problem
  • Error Code And Response
  • Richardson Maturity Model
  • Framework And Example
    • Java - JAX-RS 2.0 Specification
    • Node.js Framework
    • Relationship and difference between HAL and HATEOAS
    • Example from Github
      • Cross origin resource sharing
      • Rate limiting
      • Conditional requests
      • Hypermedia
  • Feel Less Guilty About Being Non-RESTful
  • Reference

Restful API Design Guide

REST is a matured architectural style to make us become one of world-class performers.

GraphQL is yet another web service API design style, which offers significantly more flexibility for integrators. OntPass will support GraphQL in the future.

What is REST

REST is acronym for REpresentational State Transfer, was first presented by Roy Fielding in 2000 in his famous dissertation.

It is an architecture style for designing loosely coupled applications, that is often used in the development of web services. REST does not enforce any rule regarding how it should be implemented at lower level, it just put high level design guidelines and leave you to think of your own implementation.

Guiding Principles of REST

Like any other architectural style, REST also does have it’s own 6 guiding constraints which must be satisfied if an interface needs to be referred as RESTful. These principles are listed below.

Client–server separating

By separating the user interface concerns from the server concerns, we improve the portability of the user interface across multiple platforms and improve scalability by simplifying the server components.

Servers and clients may also be replaced and developed independently, as long as the interface between them is not altered.

Stateless

Make all client-server interaction stateless. Server will not store anything about latest HTTP request client made. It will treat each and every request as new. No session, no history.

If client application needs to be a stateful application for the end user, where user logs in once and do other authorized operations thereafter, then each request from the client should contain all the information necessary to service the request – including authentication and authorization details.

No client context shall be stored on the server between requests. The client is responsible for managing the state of the application.

Cacheable

Cache constraints require that the data within a response to a request be implicitly or explicitly labeled as cacheable or non-cacheable. If a response is cacheable, then a client cache is given the right to reuse that response data for later, equivalent requests.

In today’s world, caching of data and responses is of utmost important wherever they are applicable/possible. The webpage you are reading here is also a cached version of the HTML page. Caching brings performance improvement for client side, and better scope for scalability for a server because the load has reduced.

In REST, caching shall be applied to resources when applicable and then these resources MUST declare themselves cacheable. Caching can be implemented on the server or client side.

Well-managed caching partially or completely eliminates some client-server interactions, further improving scalability and performance.

Uniform interface

By applying the software engineering principle of generality to the component interface, the overall system architecture is simplified and the visibility of interactions is improved. In order to obtain a uniform interface, multiple architectural constraints are needed to guide the behavior of components. REST is defined by four interface constraints: identification of resources; manipulation of resources through representations; self-descriptive messages; and, hypermedia as the engine of application state.

As the constraint name itself applies, you MUST decide APIs interface for resources inside the system which are exposed to API consumers and follow religiously. A resource in the system should have only one logical URI and that should provide a way to fetch related or additional data.

Any single resource should not be too large and contain each and everything in its representation. Whenever relevant, a resource should contain links (HATEOAS) pointing to relative URIs to fetch related information.

Also, the resource representations across system should follow certain guidelines such as naming conventions, link formats or data format (xml or/and json).

All resources should be accessible through a common approach such as HTTP GET and similarly modified using a consistent approach.

Once a developer becomes familiar with one of your API, he should be able to follow the similar approach for other APIs.

Layered system

The layered system style allows an architecture to be composed of hierarchical layers by constraining component behavior such that each component cannot "see" beyond the immediate layer with which they are interacting.

REST allows you to use a layered system architecture where you deploy the APIs on server A, and store data on server B and authenticate requests in Server C, for example. A client cannot ordinarily tell whether it is connected directly to the end server, or to an intermediary along the way.

Code on demand (optional)

REST allows client functionality to be extended by downloading and executing code in the form of applets or scripts. This simplifies clients by reducing the number of features required to be pre-implemented.

Most of the time you will be sending the static representations of resources in form of XML or JSON. But when you need to, you are free to return executable code to support a part of your application e.g. clients may call your API to get a UI widget rendering code. It is permitted.

All above constraints help you build a truly RESTful API and you should follow them. Still, at times you may find yourself violating one or two constraints. Do not worry, you are still making a RESTful API – but not "truly RESTful".

Resource

The key abstraction of information in REST is a resource. Any information that can be named can be a resource: a document or image, a temporal service, a collection of other resources, a non-virtual object (e.g. a person), and so on. REST uses a resource identifier to identify the particular resource involved in an interaction between components.

The state of resource at any particular timestamp is known as resource representation. A representation consists of data, metadata describing the data and hypermedia links which can help the clients in transition to next desired state.

The data format of a representation is known as a media type. The media type identifies a specification that defines how a representation is to be processed. A truly RESTful API looks like hypertext. Every addressable unit of information carries an address, either explicitly (e.g., link and id attributes) or implicitly (e.g., derived from the media type definition and representation structure).

According to Roy Fielding:

Hypertext (or hypermedia) mean the simultaneous presentation of information and controls such that the information becomes the affordance through which the user (or automaton) obtains choices and selects actions. Remember that hypertext does not need to be HTML (or XML or JSON) on a browser. Machines can follow links when they understand the data format and relationship types.

Further, resource representations shall be self-descriptive: the client does not need to know if a resource is employee or device. It should act on basis of media-type associated with resource. So in practice, you will end up creating lots of custom media-types – normally one media-type associated with one resource.

Every media type defines a default processing model. For example, HTML defines a rendering process for hypertext and the browser behavior around each element. It has no relation to the resource methods GET/PUT/POST/DELETE/… other than the fact that some media type elements will define a process model that goes like "anchor elements with an href attribute create a hypertext link that, when selected, invokes a retrieval request (GET) on the URI corresponding to the CDATA-encoded href attribute."

Resource Methods

Other important thing associated with REST is resource methods to be used to perform the desired transition. A large number of people wrongly relate resource methods to HTTP GET/PUT/POST/DELETE methods.

Roy Fielding has never mentioned any recommendation around which method to be used in which condition. All he emphasizes is that it should be uniform interface. If you decide HTTP POST will be used for updating a resource – rather than most people recommend HTTP PUT – it’s alright and application interface will be RESTful.

Ideally, everything that is needed to change the resource state shall be part of API response for that resource – including methods and in what state they will leave the representation.

A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand).
[Failure here implies that out-of-band information is driving interaction instead of hypertext.]

Another thing which will help you while building RESTful APIs is that query based API results should be represented by a list of links with summary information, not by arrays of original resource representations because query is not a substitute for identification of resources.

Http

A lot of people prefer to compare HTTP with REST. REST and HTTP are not same.

In simplest words, in the REST architectural style, data and functionality are considered resources and are accessed using Uniform Resource Identifiers (URIs). The resources are acted upon by using a set of simple, well-defined operations. The clients and servers exchange representations of resources by using a standardized interface and protocol – typically HTTP.

Resources are decoupled from their representation so that their content can be accessed in a variety of formats, such as HTML, XML, plain text, PDF, JPEG, JSON, and others. Metadata about the resource is available and used, for example, to control caching, detect transmission errors, negotiate the appropriate representation format, and perform authentication or access control.And most importantly, every interaction with a resource is stateless.

All these principles help RESTful applications to be simple, lightweight, and fast.

REST Resource Naming Best Practices

In REST, primary data representation is called Resource. Having a strong and consistent REST resource naming strategy – will definitely prove your one of the best design decisions in long term.

REST APIs use Uniform Resource Identifiers (URIs) to address resources. REST API designers should create URIs that convey a REST API’s resource model to its potential client developers. When resources are named well, an API is intuitive and easy to use. If done poorly, that same API can feel difficult to use and understand.

The constraint of uniform interface is partially addressed by the combination of URIs and HTTP verbs and using them in line with the standards and conventions.

Below are a few tips to get you going when creating the resource URIs for your new API.

Use nouns to represent resources

RESTful URI should refer to a resource that is a thing (noun) instead of referring to an action (verb) because nouns have properties which verbs do not have – similar to resources have attributes. Some examples of a resource are:

1. Users of the system
2. User Accounts
3. Network Devices etc.

and their resource URIs can be designed as below:

http://api.example.com/device-management/managed-devices 
http://api.example.com/device-management/managed-devices/{device-id} 
http://api.example.com/user-management/users/
http://api.example.com/user-management/users/{id}

Resource Archetypes

For more clarity, let’s divide the resource archetypes into four categories (document, collection and controller) and then you should always target to put a resource into one archetype and then use it’s naming convention consistently. For uniformity’s sake, resist the temptation to design resources that are hybrids of more than one archetype.

1. document

A document resource is a singular concept that is akin to an object instance or database record. In REST, you can view it as a single resource inside resource collection. A document’s state representation typically includes both fields with values and links to other related resources.

Use "singular" name to denote document resource archetype.

http://api.example.com/device-management/managed-devices/{device-id}
http://api.example.com/user-management/users/{id}
http://api.example.com/user-management/users/admin

2. collection

A collection resource is a server-managed directory of resources. Clients may propose new resources to be added to a collection. However, it is up to the collection to choose to create a new resource, or not. A collection resource chooses what it wants to contain and also decides the URIs of each contained resource.

Use "plural" name to denote collection resource archetype.

http://api.example.com/device-management/managed-devices
http://api.example.com/user-management/users
http://api.example.com/user-management/users/{id}/accounts

3. controller
   A controller resource models a procedural concept. Controller resources are like executable functions, with parameters and return values; inputs and outputs.

Use "verb" to denote controller archetype.

http://api.example.com/cart-management/users/{id}/cart/checkout
http://api.example.com/song-management/users/{id}/playlist/play

Use hyphens (-) to improve the readability of URIs

To make your URIs easy for people to scan and interpret, use the hyphen (-) character to improve the readability of names in long path segments.

1. https://api.github.com/repos/api-playground/projects-test/issues/3
2. https://stackoverflow.com/questions/5262224/how-are-reddit-and-hacker-news-ranking-algorithms-used

underscores ( _ ) ： You should press shift to input _, and it’s possible that the underscore (_) character can either get partially obscured or completely hidden in some browsers or screens, in addition, _ might be hidden by cursor.

Use lowercase letters in URIs

When convenient, lowercase letters should be consistently preferred in URI paths.

RFC 3986 defines URIs as case-sensitive except for the scheme and host components. e.g.

http://api.example.org/my-folder/my-doc  //1
HTTP://API.EXAMPLE.ORG/my-folder/my-doc  //2
http://api.example.org/My-Folder/my-doc  //3

In above examples, 1 and 2 are same but 3 is not as it uses My-Folder in capital letters.

Do not use file extenstions

File extensions look bad and do not add any advantage. Removing them decrease the length of URIs as well. No reason to keep them.

Apart from above reason, if you want to highlight the media type of API using file extenstion then you should rely on the media type, as communicated through the Content-Type header, to determine how to process the body’s content.

1. http://api.example.com/device-management/managed-devices.xml /Do not use it/
2. http://api.example.com/device-management/managed-devices /This is correct URI/

Never use CRUD function names in URIs

URIs should not be used to indicate that a CRUD function is performed. URIs should be used to uniquely identify resources and not any action upon them. HTTP request methods should be used to indicate which CRUD function is performed.

//Get all devices
HTTP GET http://api.example.com/device-management/managed-devices  
//Create new Device
HTTP POST http://api.example.com/device-management/managed-devices  
//Get device for given Id
HTTP GET http://api.example.com/device-management/managed-devices/{id}
//Update device for given Id
HTTP PUT http://api.example.com/device-management/managed-devices/{id}
//Delete device for given Id  
HTTP DELETE http://api.example.com/device-management/managed-devices/{id}

Use query component to filter URI collection

Many times, you will come across requirements where you will need a collection of resources sorted, filtered or limited based on some certain resource attribute. For this, do not create new APIs – rather enable sorting, filtering and pagination capabilities in resource collection API and pass the input parameters as query parameters. e.g.

http://api.example.com/device-management/managed-devices
http://api.example.com/device-management/managed-devices?region=USA
http://api.example.com/device-management/managed-devices?region=USA&brand=XYZ
http://api.example.com/device-management/managed-devices?region=USA&brand=XYZ&sort=installation-date

Do not use trailing forward slash (/) in URIs

As the last character within a URI’s path, a forward slash (/) adds no semantic value and may cause confusion. It’s better to drop them completely.

http://api.example.com/device-management/managed-devices/

/*This is much better version*/
http://api.example.com/device-management/managed-devices

Use forward slash (/) to indicate a hierarchical relationships

The forward slash (/) character is used in the path portion of the URI to indicate a hierarchical relationship between resources. e.g.

http://api.example.com/device-management
http://api.example.com/device-management/managed-devices
http://api.example.com/device-management/managed-devices/{id}
http://api.example.com/device-management/managed-devices/{id}/scripts
http://api.example.com/device-management/managed-devices/{id}/scripts/{id}

REST API Versioning

To manage this complexity, version your API. Versioning helps you iterate faster when the needed changes are identified.

When to Version

APIs only need to be up-versioned when a breaking change is made. Breaking changes include:

a change in the format of the response data for one or more calls
a change in the response type (i.e. changing an integer to a float)
removing any part of the API.
Breaking changes should always result in a change to the major version number for an API or content response type.

Non-breaking changes, such as adding new endpoints or new response parameters, do not require a change to the major version number. However, it can be helpful to track the minor versions of APIs when changes are made to support customers who may be receiving cached versions of data or may be experiencing other API issues.

How to Version

There are a a lot of methods to version.

We give a convention here: Versioning using Accept header.

Accept: application/vnd.example.v1+json
Accept: application/vnd.example+json;version=1.0

In the real world, an API is never going to be completely stable. So it’s important how this change is managed. A well documented and gradual deprecation of API can be an acceptable practice for most of the APIs.

Let's have a look at github example:

application/vnd.github[.version].param[+json]

//The most basic media types the API supports are:
application/json
application/vnd.github+json

//You can specify a version like so:
application/vnd.github.v3+json

//If you're specifying a property (such as full/raw/etc defined below), put the version before the property:
application/vnd.github.v3.raw+json
application/vnd.github.VERSION.base64
application/vnd.github.VERSION.html

REST API Security Essentials

Security isn’t an afterthought. It has to be an integral part of any development project and also for REST APIs. There are multiple ways to secure a RESTful API e.g. basic auth, OAuth etc. but one thing is sure that RESTful APIs should be stateless – so request authentication/authorization should not depend on cookies or sessions. Instead, each API request should come with some sort authentication credentials which must be validated on the server for each and every request.

REST Security Design Principles

The paper "The Protection of Information in Computer Systems" by Jerome Saltzer and Michael Schroeder, put forth eight design principles for securing information in computer systems, as described in the following sections:

1. Least Privilege: An entity should only have the required set of permissions to perform the actions for which they are authorized, and no more. Permissions can be added as needed and should be revoked when no longer in use.
2. Fail-Safe Defaults: A user’s default access level to any resource in the system should be "denied" unless they’ve been granted a "permit" explicitly.
3. Economy of Mechanism: The design should be as simple as possible. All the component interfaces and the interactions between them should be simple enough to understand.
4. Complete Mediation: A system should validate access rights to all its resources to ensure that they’re allowed and should not rely on cached permission matrix. If the access level to a given resource is being revoked, but that isn’t reflected in the permission matrix, it would violate the security.
5. Open Design: This principle highlights the importance of building a system in an open manner—with no secret, confidential algorithms.
6. Separation of Privilege: Granting permissions to an entity should not be purely based on a single condition, a combination of conditions based on the type of resource is a better idea.
7. Least Common Mechanism: It concerns the risk of sharing state among different components. If one can corrupt the shared state, it can then corrupt all the other components that depend on it.
8. Psychological Acceptability: It states that security mechanisms should not make the resource more difficult to access than if the security mechanisms were not present. In short, security should not make worse the user experience.

Best Practices to Secure REST APIs

Below given points may serve as a checklist for designing the security mechanism for REST APIs.

1. Keep it Simple, Basic Authentication

Secure an API/System – just how secure it needs to be. Every time you make the solution more complex "unnecessarily", you are also likely to leave a hole.

The API should supports Basic Authentication as defined in RFC2617,
the RFC requires unauthenticated requests to be answered with 401 Unauthorized responses.

Example：
To use Basic Authentication with the GitHub API, simply send the username and password associated with the account.
For example, if you're accessing the API via cURL, the following command would authenticate you if you replace <username> with your GitHub username. (cURL will prompt you to enter the password.)

curl -u username https://api.github.com/user

2. Always Use HTTPS

By always using SSL, the authentication credentials can be simplified to a randomly generated access token that is delivered in the username field of HTTP Basic Auth. It’s relatively simple to use, and you get a lot of security features for free.

3. Use Password Hash

Passwords must always be hashed to protect the system (or minimize the damage) even if it is compromised in some hacking attempt. There are many such hashing algorithms which can prove really effective for password security e.g. MD5, SHA, PBKDF2, bcrypt and scrypt algorithms.

4. Never expose information on URLs

Usernames, passwords, session tokens, and API keys should not appear in the URL, as this can be captured in web server logs, which makes them easily exploitable.

https://api.domain.com/user-management/users/{id}/someAction?apiKey=abcd123456789  //Very BAD !!

Above URL exposes API key So, never use this form of security.

5. Consider OAuth

Though basic auth is good enough for most of the APIs and if implemented correctly, it’s secure as well – yet you may want to consider OAuth as well. The OAuth 2.0 authorization framework enables a third-party application to obtain limited access to an HTTP service, either on behalf of a resource owner by orchestrating an approval interaction between the resource owner and the HTTP service, or by allowing the third-party application to obtain access on its own behalf.

6. Consider Adding Timestamp in Request

Along with other request parameters, you may add a request timestamp as HTTP custom header in API request. The server will compare the current timestamp to the request timestamp, and only accepts the request if it is within a reasonable timeframe (1-2 minutes, perhaps).

This will prevent very basic replay attacks from people who are trying to brute force your system without changing this timestamp.

7. Input Parameter Validation

Validate request parameters on the very first step, before it reaches to application logic. Put strong validation checks and reject the request immediately if validation fails. In API response, send relevant error messages and example of correct input format to improve user experience.

8. API Key

API key authentication requires each request to be signed (enhanced security measure). If your application requires access to other users’ accounts, do not use API Key. To securely access other users’ accounts, use OAuth2.

You should take great care to store your credentials securely. If someone obtains your api_secret, they will be able to control your account.

9. Validating SSL Certificates

It is also very important that your application validates our SSL certificate when it connects over https. This helps prevent a man in the middle attack. If you are using a client library, this may be turned on by default, but you should confirm this. Whenever you see ‘verify SSL’ you should always ensure it is set to true.

10. Whitelist IP addresses

For enhanced security, we should have whitelist IP addresses that are permitted to make requests with a particular service.

11. Two-factor authentication

Basic Authentication requires an extra step to get enhanced security.
The account receives its two-factor authentication codes via SMS or via an application.

12. SSO, SAML, OpenID Connect

With GitHub Business Cloud, you can: Allow or require organization members to authenticate to GitHub using SAML SSO. So if you service support SSO, you can choose SAML or [OpenID Connect]

13. Asymmetric encryption

BlockChain system and Alipay use asymmetric encryption to authenticate between client and server.

14. Rate limit

The Search API has a custom rate limit. For requests using Basic Authentication, OAuth, or client ID and secret, you can make up to 30 requests per minute. For unauthenticated requests, the rate limit allows you to make up to 10 requests per minute.

Caching REST API Response

Caching is the ability to store copies of frequently accessed data in several places along the request-response path. When a consumer requests a resource representation, the request goes through a cache or a series of caches (local cache, proxy cache or reverse proxy) toward the service hosting the resource. If any of the caches along the request path has a fresh copy of the requested representation, it uses that copy to satisfy the request. If none of the caches can satisfy the request, the request travels all the way to the service (or origin server as it is formally known).

Using HTTP headers, an origin server indicates whether a response can be cached and if so, by whom, and for how long. Caches along the response path can take a copy of a response, but only if the caching metadata allows them to do so.

Optimizing the network using caching improves the overall quality-of-service in following ways:

1. Reduce bandwidth
2. Reduce latency
3. Reduce load on servers

Caching in REST APIs

Being cacheable is one of architectural constraints of REST. GET requests should be cachable by default – until special condition arises. Usually, browsers treat all GET requests cacheable. POST requests are not cacheable by default but can be made cacheable if either an Expires header or a Cache-Control header with a directive, to explicitly allows caching, is added to the response. Responses to PUT and DELETE requests are not cacheable at all.

There are two main HTTP response headers that we can use to control caching behavior:

Expires

The Expires HTTP header specifies an absolute expiry time for a cached representation. Beyond that time, a cached representation is considered stale and must be re-validated with the origin server. To indicate that a representation never expires, a service can include a time up to one year in the future.

Expires: Fri, 20 May 2016 19:20:49 IST

Cache-Control

The header value comprises one or more comma-separated directives. These directives determine whether a response is cacheable, and if so, by whom, and for how long e.g. max-age or s-maxage directives.

Cache-Control: max-age=3600

Cacheable responses (whether to a GET or to a POST request) should also include a validator — either an ETag or a Last-Modified header.

ETag

An ETag value is an opaque string token that a server associates with a resource to uniquely identify the state of the resource over its lifetime. When the resource changes, the ETag changes accordingly.

ETag: "abcd1234567n34jv"

Last-Modified

Whereas a response’s Date header indicates when the response was generated, the Last-Modified header indicates when the associated resource last changed. The Last-Modified value cannot be later than the Date value.

Last-Modified: Fri, 10 May 2016 09:17:49 IST

REST Resource Representation Compression

REST APIs can return the resource representations in a number of formats such as XML, JSON, HTML or even plain text. All such forms can be compressed to a lesser number of bytes to save bandwidth over the network. Different protocols use different techniques to enable compression and notify the clients about compression scheme – so that client can decompress it before consuming the representations.

HTTP is most widely used protocol for REST – so I am taking example of HTTP specific response compression.

Accept-Encoding

While requesting resource representations – along with an HTTP request the client sends an Accept-Encoding header that says what kind of compression algorithms the client understands.

The two standard values for Accept-Encoding are compress and gzip.

A sample request with accept-encoding header looks like this :

GET        /employees         HTTP/1.1
Host:     www.domain.com
Accept:     text/html
Accept-Encoding:     gzip,compress

Other possible usage of accept-encoding may be:

Accept-Encoding: compress, gzip
Accept-Encoding:
Accept-Encoding: *
Accept-Encoding: compress;q=0.5, gzip;q=1.0

/*
Quality factors allow the user or user agent to indicate the relative degree of preference for that media-range, using the ‘q’ value scale from 0 to 1. The default value is 1
*/

Accept-Encoding: gzip;q=1.0, identity; q=0.5, *;q=0

If an Accept-Encoding field is present in a request, and if the server cannot send a response which is acceptable according to the Accept-Encoding header, then the server SHOULD send an error response with the 406 (Not Acceptable) status code.

Content-Encoding

If the server understands one of the compression algorithms from Accept-Encoding, it can use that algorithm to compress the representation before serving it. When successfully compressed, server lets know the client of encoding scheme by another HTTP header i.e. Content-Encoding.

200 OK
Content-Type:     text/html
Content-Encoding:     gzip

If the content-coding of an entity in a request message is not acceptable to the origin server, the server SHOULD respond with a status code of 415 (Unsupported Media Type). If multiple encodings have been applied to an entity, the content encodings MUST be listed in the order in which they were applied.

Most web browsers automatically request compressed representations from website host servers – using above headers.

REST – Content Negotiation

Generally, resources can have multiple presentations, mostly because there may be multiple different clients expecting different representations. Asking for a suitable presentation by a client, is referred as content negotiation.

At server side, an incoming request may have an entity attached to it. To determine it’s type, server uses the HTTP request header Content-Type. Some common examples of content types are "text/plain", "application/xml", "text/html", "application/json", "image/gif", and "image/jpeg".

Content-Type: application/json

HATEOAS Driven REST APIs

HATEOAS (Hypermedia as the Engine of Application State) is a constraint of the REST application architecture that keeps the RESTful style architecture unique from most other network application architectures. The term “hypermedia” refers to any content that contains links to other forms of media such as images, movies, and text.

This architectural style lets you use hypermedia links in the response contents so that the client can dynamically navigate to the appropriate resource by traversing the hypermedia links. This is conceptually the same as a web user navigating through web pages by clicking the appropriate hyperlinks in order to achieve a final goal.

For example, below given JSON response may be from an API like HTTP GET from github.

GET /projects/columns/cards/:card_id

{
  "url": "https://api.github.com/projects/columns/cards/1478",
  "id": 1478,
  "node_id": "MDExOlByb2plY3RDYXJkMTQ3OA==",
  "note": "Add payload for delete Project column",
  "creator": {
    "login": "octocat",
    "id": 1,
    "node_id": "MDQ6VXNlcjE=",
    "avatar_url": "https://github.com/images/error/octocat_happy.gif",
    "gravatar_id": "",
    "url": "https://api.github.com/users/octocat",
    "html_url": "https://github.com/octocat",
    "followers_url": "https://api.github.com/users/octocat/followers",
    "following_url": "https://api.github.com/users/octocat/following{/other_user}",
    "gists_url": "https://api.github.com/users/octocat/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/octocat/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/octocat/subscriptions",
    "organizations_url": "https://api.github.com/users/octocat/orgs",
    "repos_url": "https://api.github.com/users/octocat/repos",
    "events_url": "https://api.github.com/users/octocat/events{/privacy}",
    "received_events_url": "https://api.github.com/users/octocat/received_events",
    "type": "User",
    "site_admin": false
  },
  "created_at": "2016-09-05T14:21:06Z",
  "updated_at": "2016-09-05T14:20:22Z",
  "archived": false,
  "column_url": "https://api.github.com/projects/columns/367",
  "content_url": "https://api.github.com/repos/api-playground/projects-test/issues/3",
  "project_url": "https://api.github.com/projects/120"
}

In the preceding example, the response returned by the server contains hypermedia links to related resource, which can be traversed by the client to read resources belonging to card.

The advantage of the above approach is that hypermedia links returned from the server drive the application’s state and not the other way around.

Similar to a human’s interaction with a website, a REST client hits an initial API URI and uses the server-provided links to dynamically discover available actions and access the resources it needs. The client need not have prior knowledge of the service or the different steps involved in a workflow. Additionally, the clients no longer have to hard code the URI structures for different resources. This allows the server to make URI changes as the API evolves without breaking the clients.

Above API interaction is possible using HATEOAS only.

Each REST framework provides it’s own way on creating the HATEOAS links using framework capabilities e.g. in this RESTEasy HATEOAS tutorial, links are part of resource model classes which is transferred as resource state to the client.

HATEOAS References:

1. RFC 5988 (web linking)
2. JSON Hypermedia API Language (HAL)

Idempotent REST APIs

If you follow REST principles in designing API, you will have automatically idempotent REST APIs for GET, PUT, DELETE, HEAD, OPTIONS and TRACE HTTP methods. Only POST APIs will not be idempotent.

POST is NOT idempotent.
GET, PUT, DELETE, HEAD, OPTIONS and TRACE are idempotent.
Let’s analyze how above HTTP methods end up being idempotent – any why POST is not.

HTTP POST

Generally – not necessarily – POST APIs are used to create a new resource on server. So when you invoke the same POST request N times, you will have N new resources on the server. So, POST is not idempotent.

HTTP GET, HEAD, OPTIONS and TRACE

GET, HEAD, OPTIONS and TRACE methods NEVER change the resource state on server. They are purely for retrieving the resource representation or meta data at that point of time. So invoking multiple requests will not have any write operation on server, so GET, HEAD, OPTIONS and TRACE are idempotent.

HTTP PUT

Generally – not necessarily – PUT APIs are used to update the resource state. If you invoke a PUT API N times, the very first request will update the resource; then rest N-1 requests will just overwrite the same resource state again and again – effectively not changing anything. Hence, PUT is idempotent.

HTTP DELETE

When you invoke N similar DELETE requests, first request will delete the resource and response will be 200 (OK) or 204 (No Content). Other N-1 requests will return 404 (Not Found). Clearly, the response is different from first request, but there is no change of state for any resource on server side because original resource is already deleted. So, DELETE is idempotent.

Please keep in mind if some systems may have DELETE APIs like this:

DELETE /item/last

In the above case, calling operation N times will delete N resources – hence DELETE is not idempotent in this case. In this case, a good suggestion might be to change above API to POST – because POST is not idempotent.

POST /item/last

Now, this is closer to HTTP spec – hence more REST compliant.

POST vs PUT

Generally, in practice, always use PUT for UPDATE operations.Always use POST for CREATE operations.

GET     /device-management/devices : Get all devices
POST    /device-management/devices : Create a new device

GET     /device-management/devices/{id} : Get the device information identified by "id"
PUT     /device-management/devices/{id} : Update the device information identified by "id"
DELETE  /device-management/devices/{id} : Delete device by "id"

Statelessness

As per the REST (REpresentational “State” Transfer) architecture, the server does not store any state about the client session on the server side. This restriction is called Statelessness. Each request from the client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is therefore kept entirely on the client. client is responsible for storing and handling all application state related information on client side.

There are some very noticeable advantages for having REST APIs stateless.

1. Statelessness helps in scaling the APIs to millions of concurrent users by deploying it to multiple servers. Any server can handle any request because there is no session related dependency.
2. Being stateless makes REST APIs less complex – by removing all server-side state synchronization logic.
3. A stateless API is also easy to cache as well. A specific software can decide whether or not to cache the result of an HTTP request just by looking at that one request. There’s no nagging uncertainty that state from a previous request might affect the cacheability of this one. It improves the performance of applications.
4. The server never loses track of “where” each client is in the application because the client sends all necessary information with each request.

REST API – N+1 Problem

N+1 problem is mostly talked in context of ORMs. In this problem, the system needs to load N children of a parent entity where only parent entity was requested for. By default, ORMs are configured with lazy-loading enabled, so 1 query issued for the parent entity causes N more queries i.e. one each for N child entities.

This N+1 problem is often considered a major performance bottleneck and so shall be solved at the design level of application.

Though mostly directly associated, yet the N+1 problem is not specific to ORMs only. This can be associated with the context of web APIs as well e.g. REST APIs.
In case of web APIs, N+1 problem is a situation where client applications are required to call the server N+1 times to fetch 1 collection resource + N client resources, mostly because of collection resource not had enough information about child resources to build its user interface completely.

<books uri="/books" size="100">
    <book uri="/books/1" id="1">
        <isbn>3434253561</isbn>
    </book>
    <book uri="/books/2" id="2">
        <isbn>3423423534</isbn>
    </book>
    <book uri="/books/3" id="3">
        <isbn>5352342344</isbn>
    </book>
    ...
    ...
</books>

Here /books resource return list of books with information including only it’s id and isbn. This information is clearly not enough to build a client application UI which will want to typically show the books name in UI rather than ISBN. It may be that they want to show other information such as author and publication year as well.

In above scenario, client application MUST make N more requests for each individual book resource at /books/{id}. So in the total client will end up invoking REST APIs N+1 times.

Above scenario is only for example. Idea is that insufficient information in collection resources may lead to N+1 problem in REST APIs.

The solution is easy, include more information in individual resources inside collection resource. But this method make collection's API return more data. We should adjust our design case by case. By the way, GraphQL is a good choice to solve the problem.

Error Code And Response

HTTP defines forty standard status codes that can be used to convey the results of a client’s request. The status codes are divided into the five categories presented below.

• 1xx: Informational - Request received, continuing process
• 2xx: Success - The action was successfully received,
  understood, and accepted
• 3xx: Redirection - Further action must be taken in order to
  complete the request
• 4xx: Client Error - The request contains bad syntax or cannot
  be fulfilled
• 5xx: Server Error - The server failed to fulfill an apparently
  valid request

HTTP status codes are extensible. HTTP applications are not required to understand the meaning of all registered status codes, though such understanding is obviously desirable. However, applications MUST understand the class of any status code, as indicated by the first digit, and treat any unrecognized response as being equivalent to the x00 status code of that class, with the exception that an unrecognized response MUST NOT be cached. For example, if an unrecognized status code of 431 is received by the client, it can safely assume that there was something wrong with its request and treat the response as if it had received a 400 status code. In such cases, user agents SHOULD present to the user the entity returned with the response, since that entity is likely to include human- readable information which will explain the unusual status.

Let's have a look at coinbase example below to get a brief introduction of the design idea for the response.

Richardson Maturity Model

Reonard Richardson analyzed a hundred different web service designs and divided them into four categories based on how much they are REST compliant. This model of division of REST services to identify their maturity level – is called Richardson Maturity Model.

Richardson used three factors to decide the maturity of a service i.e. URI, HTTP Methods and HATEOAS (Hypermedia). The more a service employs these technologies – more mature it shall be considered.

image

1. Level 1 tackles the question of handling complexity by using divide and conquer, breaking a large service endpoint down into multiple resources.
2. Level 2 introduces a standard set of verbs so that we handle similar situations in the same way, removing unnecessary variation.
3. Level 3 introduces discoverability, providing a way of making a protocol more self-documenting.
   The result is a model that helps us think about the kind of HTTP service we want to provide and frame the expectations of people looking to interact with it.

Framework And Example

There are a lot of framework to help us implement Resful API quickly.
Because we use Java and Node.js, I will list related framework here for reference.

Java - JAX-RS 2.0 Specification

JAX-RS provides portable APIs for developing, exposing and accessing Web applications designed and implemented in compliance with principles of REST architectural style.

The Java EE 6 release took the first step towards standardizing RESTful web service APIs by introducing a Java API for RESTful web services (JAX-RS) JSR 311. JAX-RS ensures portability of REST API code across all Java EE-compliant application servers. The latest version is JAX-RS 2.0 JSR 339, which was released as part of the Java EE 7 platform.

JAX-RS focuses on applying Java annotations to plain Java objects. JAX-RS has annotations to bind specific URI patterns and HTTP operations to individual methods of your Java class. It also has annotations which can help you handle in input/output parameters.

As we already said that JAX-RS is specification; it means we need to have its implementation to run REST API code. Some of the popular JAX-RS implementations available today are:

1. Jersey
2. RESTEasy
3. Apache CXF
4. Restlet

HATEOAS need you add HATEOAS links to existing REST APIs.if you use spring boot, please follow this example: Spring Boot HATEOAS Example

Node.js Framework

KOA is a popular and expressive HTTP middleware framework for node.js to make web applications and APIs more enjoyable to write.

You can extend KOA easily, here is examples:
hal_specification

Relationship and difference between HAL and HATEOAS

HATEOAS is a concept of application architecture. It defines the way in which application clients interact with the server, by navigating hypermedia links they find inside resource models returned by the server.

To implement HATEOAS you need some standard way of representing resources,that will contain hypermedia information (links to related resources), for example, something like this:

{
    "links": {
        "self": { "href": "http://api.com/items" },
        "item": [
            { "href": "http://api.com/items/1" },
            { "href": "http://api.com/items/2" }
        ]
    },
    "data": [
            { "itemName": "a" }, 
            { "itemName": "b" } 
    ] 
}

HAL is one of such standards. It is a specific format of resource presentation, that can be used to implement HATEOAS.

You can fully implement HATEOAS without following HAL at all if you prefer to follow another standard or use your own.

Example from Github

Cross origin resource sharing

The API supports Cross Origin Resource Sharing (CORS) for AJAX requests from any origin. You can read the CORS W3C Recommendation, or this intro from the HTML 5 Security Guide.

This is what the CORS preflight request looks like:

curl -i https://api.github.com -H "Origin: http://example.com" -X OPTIONS
HTTP/1.1 204 No Content
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type, If-Match, If-Modified-Since, If-None-Match, If-Unmodified-Since, X-GitHub-OTP, X-Requested-With
Access-Control-Allow-Methods: GET, POST, PATCH, PUT, DELETE
Access-Control-Expose-Headers: ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
Access-Control-Max-Age: 86400

Rate limiting

The returned HTTP headers of any API request show your current rate limit status:

curl -i https://api.github.com/users/octocat
HTTP/1.1 200 OK
Date: Mon, 01 Jul 2013 17:27:06 GMT
Status: 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 56
X-RateLimit-Reset: 1372700873

Header Name	Description
X-RateLimit-Limit	The maximum number of requests you're permitted to make per hour.
X-RateLimit-Remaining	The number of requests remaining in the current rate limit window.
X-RateLimit-Reset	The time at which the current rate limit window resets in UTC epoch seconds.

If you exceed the rate limit, an error response returns:

HTTP/1.1 403 Forbidden
Date: Tue, 20 Aug 2013 14:50:41 GMT
Status: 403 Forbidden
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1377013266
{
   "message": "API rate limit exceeded for xxx.xxx.xxx.xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)",
   "documentation_url": "https://developer.github.com/v3/#rate-limiting"
}

Conditional requests

Most responses return an ETag header. Many responses also return a Last-Modified header. You can use the values of these headers to make subsequent requests to those resources using the If-None-Match and If-Modified-Since headers, respectively. If the resource has not changed, the server will return a 304 Not Modified.

Making a conditional request and receiving a 304 response does not count against your Rate Limit, so we encourage you to use it whenever possible.

url -i https://api.github.com/user
HTTP/1.1 200 OK
Cache-Control: private, max-age=60
ETag: "644b5b0155e6404a9cc4bd9d8b1ae730"
Last-Modified: Thu, 05 Jul 2012 15:31:30 GMT
Status: 200 OK
Vary: Accept, Authorization, Cookie
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4996
X-RateLimit-Reset: 1372700873

curl -i https://api.github.com/user -H 'If-None-Match: "644b5b0155e6404a9cc4bd9d8b1ae730"'
HTTP/1.1 304 Not Modified
Cache-Control: private, max-age=60
ETag: "644b5b0155e6404a9cc4bd9d8b1ae730"
Last-Modified: Thu, 05 Jul 2012 15:31:30 GMT
Status: 304 Not Modified
Vary: Accept, Authorization, Cookie
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4996
X-RateLimit-Reset: 1372700873

curl -i https://api.github.com/user -H "If-Modified-Since: Thu, 05 Jul 2012 15:31:30 GMT"
HTTP/1.1 304 Not Modified
Cache-Control: private, max-age=60
Last-Modified: Thu, 05 Jul 2012 15:31:30 GMT
Status: 304 Not Modified
Vary: Accept, Authorization, Cookie
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4996
X-RateLimit-Reset: 1372700873

Hypermedia

In github, all resources may have one or more *_url properties linking to other resources. These are meant to provide explicit URLs so that proper API clients don't need to construct URLs on their own. It is highly recommended that API clients use these. Doing so will make future upgrades of the API easier for developers. All URLs are expected to be proper RFC 6570 URI templates.

Feel Less Guilty About Being Non-RESTful

AliPay API just reach L2 of Richardson Maturity Model, but it still work very well, because AliPay have a very good document to understand its APIs.

Please remember RESTful is just a design style to implement your service. You should follow a architecture which fulfill your requirement and keep consistency, not the truly RESTful, will help you build a well-designed system.

Reference

1. Restful API Tutorial
2. Richardson Maturity Model
3. Java REST HATEOAS Example
4. Github API
5. CoinBase API
6. AliPay API
7. Relationship and difference between HAL and HATEOAS