devkuma – HTTP

Introduction to HTTP

kc@example.com (kc kim) — Tue, 06 Nov 2018 01:34:33 +0900

HTTP Overview

HTTP (HyperText Transfer Protocol) is a protocol for sending and receiving data, such as web pages, between web servers and clients (browsers) on the WWW. It basically exchanges text message resources, and it can also exchange various other resources such as .jpeg files.

HTTP is an application-level protocol built on TCP/IP.
HTTP is a stateless protocol that does not maintain state.
It consists of Method, Path, Version, Headers, Body, and similar parts.

The following specifications are published for each HTTP version.

HTTP Messages

There are two types of HTTP messages.

HTTP request message: the client requests a resource from the server.
HTTP response message: the server responds to the client with the result of the HTTP request.

HTTP Request

kc@example.com (kc kim) — Tue, 06 Nov 2018 01:34:33 +0900

HTTP Request Messages

When a web page is opened in a browser, the browser sends a request message like the following to the server.

GET / HTTP/1.1
Accept: image/gif, image/jpeg, */*
Accept-Language: ko
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (Compatible; MSIE 6.0; Windows NT 5.1;)
Host: www.xxx.zzz
Connection: Keep-Alive

A request message consists of the following syntax.

HTTP request line

GET / HTTP/1.1

HTTP request header

Accept: image/gif, image/jpeg, */*
Accept-Language: ko
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (Compatible; MSIE 6.0; Windows NT 5.1;)
Host: www.xxx.zzz
Connection: Keep-Alive

HTTP body

Message body (used with methods such as POST)

Start Line

The start line of an HTTP request has the following three elements.

HTTP method
Request target (path name)
HTTP protocol version

The request line is displayed in the following format.

Request line

[HTTP method] [request target] [HTTP protocol version]

The path name is usually like /aaa/bbb/ccc.html, and either a path name starting with a slash or a URL starting with http:// is specified. Version 1.1 is commonly used.

HTTP Method

The methods supported by HTTP/1.0 and HTTP/1.1 are as follows.

Method	HTTP/1.0	HTTP/1.1	Description
GET	◎	◎	The most commonly used method. The browser asks the server to retrieve a page.
HEAD	◎	◎	Requests header-only information. Servers must support GET and HEAD.
POST	○	○	Used to send data entered in a form with `method="POST"` to the server.
PUT	○	○	Used to upload a file to the server.
DELETE	○	○	Requests that the server delete the specified resource.
CONNECT	×	○	Used for SSL communication through a proxy server.
OPTIONS	×	○	Used to query the methods and options supported by the server.
TRACE	×	○	Used to trace HTTP behavior, such as which proxy servers an HTTP request passes through. The final server that receives this message returns the entity contained in the request message, usually headers plus message body, as-is.
LINK	○	×	Creates a link relationship between the specified URL and resource. No longer used in HTTP/1.1.
UNLINK	○	×	Removes the link relationship between the specified URL and resource. No longer used in HTTP/1.1.

(◎: required, ○: supported, ×: not supported)

Apache also supports methods such as PATCH, PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, LOCK, and UNLOCK.

GET

The GET request method asks the server to retrieve information held by a URI (URL).

Transmission format

GET [request-uri]?query_string  
HTTP/1.1\r\n  
Host: [Hostname] or [IP] \r\n

POST

The POST request method is used when form input is processed by a server-side script such as ASP, PHP, JSP, or a CGI program configured for the request URI (URL). It is sent together with the form action, and the request information is placed in the data part rather than in the header.

Transmission format

POST [request-uri]?query_string  
HTTP/1.1\r\n  
HOST:[Hostname] or [IP] \r\n  
Content-Length:[Length in Bytes] \r\n  
\r\n  
[query-string] or [data]

HEAD

The HEAD request method is similar to GET, but the web server sends no data other than header information.
It can be used to check whether a web server is down (health check) or to obtain web server information such as its version.

Transmission format

HEAD [request-uri] HTTP/1.1\r\n  
Host: [Hostname] or [IP] \r\n

OPTIONS

rfc2616
This method can be used to check which methods are supported by the system.

Transmission format

OPTIONS [request-ri]  
HTTP/1.1\r\n  
Host: [Hostname] or [IP] \r\n

PUT

Because it has a transmission structure similar to POST, a message (data) is sent together with headers.
It is used to store specified content on a remote server and is often abused for website defacement.

Transmission format

PUT [request-uri] HTTP/1.1\r\n  
Host: [Hostname] or [IP] \r\n  
Content-Length:[Length in Bytes] \r\n  
Content-Type:[Content Type] \r\n  
\r\n  
[data]

PATCH

PATCH is used to modify (UPDATE) the requested resource, similarly to PUT.
PUT means updating the whole resource, while PATCH means replacing part of that resource. In other words, it partially changes a resource.

Transmission format

PATCH [request-uri] HTTP/1.1\r\n  
Host: [Hostname] or [IP]\r\n  
Content-Length:[Length in Bytes]\r\n  
Content-Type:[Content Type]\r\n  
\r\n  
[data]

DELETE

DELETE is used to delete a file on a remote web server and is the opposite concept of PUT.

TRACE

TRACE is used to call a loopback message on a remote server.

CONNECT

CONNECT is used to request proxy functionality from a web server.

Request Target

The request target includes URLs, paths, query strings, and similar values.

Request target form	Example	Description
origin form	- POST / HTTP/1.1 - GET /background.png HTTP/1.0 - HEAD /test.html?query=alibaba HTTP/1.1 - OPTIONS /anypage.html HTTP/1.0	Common form
absolute form	GET http://developer.mozilla.org/en-US/docs/Web/HTTP/Messages HTTP/1.1	Used when doing GET through a proxy
authority form	CONNECT developer.mozilla.org:80 HTTP/1.1	Used with CONNECT
asterisk form	OPTIONS * HTTP/1.1	Used with OPTIONS

HTTP Protocol Version

Indicates the HTTP version. Usually HTTP/1.1 or 2.0.

HTTP Header

There are three types of request headers.

Header type	Description	Examples
Request header	Headers not related to the data in the request body	- Host - User-Agent - Accept - - If- - Referer
General header	Resource to retrieve	- Date - Cache-Control
Entity header	Applies to the request body	- Content-Type - Content-Length - Expires

HTTP Body

The HTTP request body is the data sent with the request.
In general, GET, HEAD, DELETE, and OPTIONS do not have request bodies. They are used only when retrieving or deleting resources.
The request body is used when sending data to a resource with POST, PUT, and similar methods.

Checking a request body

% curl --trace-ascii - http://www.devkuma.com/ -XPOST -d 'user=kimkc,password=1234'

=> Send data, 24 bytes (0x18)
0000: user=kimkc,password=1234

HTTP Response

kc@example.com (kc kim) — Tue, 06 Nov 2018 01:34:33 +0900

HTTP Response Messages

When a request arrives, the server returns a response message like the following.

HTTP/1.1 200 OK
Date: Sun, 11 Jan 2014 16:06:23 GMT
Server: Apache/1.3.22 (Unix) (Red-Hat/Linux)
Last-Modified: Sun, 07 Dec 2013 12:34:18 GMT
ETag: "1dba6-131b-3fd31e4a"
Accept-Ranges: bytes
Content-Length: 4891
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html

<!DOCTYPE html>
<html>
  :
</html>

A response message consists of a response line (status line), HTTP response headers, a blank line that marks the end of headers, and an HTTP body.

Line breaks use the Windows-style CR LF("\r\n"). The first line is the request or response line, headers can span multiple lines, the blank line is one line, and the message body can span multiple lines.

Start Line

The start line (status line) of an HTTP response has the following three elements.

HTTP protocol version, the same as an HTTP request message
Response status code
Status text

The response start line is displayed in the following format.

[HTTP protocol version] [response status code] [status text]

The status text returns a message that supplements the meaning and details of a status number, such as OK or Not Found.

Example:

% curl http://google.com/ --head
HTTP/1.1 301 Moved Permanently

HTTP protocol version: HTTP/1.1
Response status code: 301
Status text: Moved Permanently

HTTP Protocol Version

Indicates the HTTP version, the same as an HTTP request message. Usually HTTP/1.1 or 2.0.

Response Status Code and Status Text

HTTP/1.1 status code definitions are specified in RFC 2616. In summary:

Response code	Meaning	Description
1xx	Informational	Information
2xx	Successful	Successful response
3xx	Redirection	Additional action (redirection)
4xx	Client Error	Client-side error
5xx	Server Error	Server-side error

1xx, 2xx, and 3xx codes indicate states between the server and user agent, so general users usually do not see their contents. 4xx and 5xx codes are commonly seen by general users.

If a web page does not appear normally in a browser for any reason, it corresponds to a 3xx or 4xx error. To understand and recover from the cause of an error, it is useful to know roughly what the error code means.

Response code	Meaning	Description
100	Continue	The server has received part of the request and asks the client to continue sending the rest.
101	Switching protocols	Protocol switching. The requester asked the server to switch protocols, and the server is approving it.
200	OK	The request was successfully performed.
201	Created	The request succeeded, and the server created a new resource.
202	Accepted	The web server accepted the request but has not processed it yet.
203	Non-authoritative information	The server successfully processed the request but is providing information received from another source.
204	No content	The server successfully processed the request but provides no content.
205	Reset Content	The server successfully processed the request but provides no content.
206	Partial content	The server successfully processed part of a GET request.
300	Multiple Choices	Several options exist for obtaining the content.
301	Moved permanently	The requested data has moved to another URL.
302	Found temporarily	The requested data was found at a temporary URL.
304	Not modified	The resource was not updated, so local cache information was used.
305	Use Proxy	Uses the proxy specified in the Location header.
306	(Unused)	Unused.
307	Temporary Redirect	Temporarily moving to another location.
400	Bad request	The request is invalid. The user’s invalid request cannot be processed.
401	Unauthorized	Not authenticated. Occurs when requesting a page that requires authentication.
402	Payment required	Reserved. Payment is required.
403	Forbidden	Access is not allowed.
404	Not found	The requested page does not exist.
405	Method not allowed	An unsupported HTTP method was used.
407	Proxy authentication required	Proxy authentication is required.
408	Request timeout	The request timed out.
410	Gone	Permanently unavailable. The requested content has disappeared.
411	Length Required	Add a Content-Length header to the request.
412	Precondition failed	The condition specified in an If-* header was not met.
414	Request-URI too long	The request URL is too long.
415	Unsupported Media Type	The media type is not supported.
416	Requested Range Not Satisfiable	The requested range is invalid.
417	Expectation Failed	The extension request specified in the Expect header failed.
500	Internal server error	An unexpected server error occurred.
501	Not implemented	The web server cannot process the request.
502	Bad Gateway	The gateway is invalid.
503	Service unavailable	Service unavailable.
504	Gateway timeout	The gateway timed out.
505	HTTP version not supported	The HTTP version is not supported.

HTTP Header

Response headers have the following three types.

Header type	Description	Examples
Response header	Header not related to response body data	- Location
General header	Resource to retrieve	- Date - Cache-Control
Entity header	Applies to the response body	- Content-Type - Content-Length - Expires

HTTP Body

Contains the contents of the resource (file).

% curl http://www.devkuma.com
<!DOCTYPE html>
<html lang="ko">
<head>

HTTP Header

kc@example.com (kc kim) — Tue, 06 Nov 2018 01:34:33 +0900

HTTP Header

HTTP headers are metadata exchanged in HTTP requests and responses. They describe how the message should be interpreted, how caching should work, what content types are acceptable, authentication information, connection behavior, and entity information such as content length and encoding.

Category	Request	Response	Headers
General header	○	○	Cache-Control, Connection, Date, Pragma, Trailer, Transfer-Encoding, Upgrade, Via, Warning
Request header	○	×	Accept, Accept-Charset, Accept-Encoding, Accept-Language, Authorization, Expect, From, Host, If-Match, If-Modified-Since, If-None-Match, If-Range, If-Unmodified-Since, Max Forwards, Proxy-Authorization, Range, Referer, TE, User-Agent
Response header	×	○	Accept-Ranges, Age, ETag, Location, Proxy-Authenticate, Retry-After, Server, Vary, WWW-Authenticate
Entity header	○	○	Allow, Content-Encoding, Content-Language, Content-Length, Content-Location, Content-MD5, Content-Range, Content-Type, Expires, Last-Modified, extension-header

Common Request Headers

Accept: tells the server which data formats (MIME types) the browser can receive. * means “all.”
Accept-Charset: tells the server which character sets the browser can receive.
Accept-Encoding: tells the server which encodings the browser can receive. For example, if gzip is supported, the server can gzip-compress the message body and the browser can decompress and display it.
Accept-Language: tells the server which languages the browser can receive.
Authorization: sends authentication information for a resource that requires authentication.
Host: specifies the host name and port number of the server to which the request is sent. It is the only required header in HTTP/1.1.
If-Match, If-None-Match, If-Modified-Since, If-Unmodified-Since, If-Range: conditional request headers used with cache validation and partial requests.
Range: requests only part of an entity from the server.
Referer: passes the URL of the page that was the source of this request.
User-Agent: sends information about the client application, browser, operating system, version, platform, and similar details.

Common Response Headers

Accept-Ranges: tells the client which units are available for Range requests. Currently bytes is defined.
Age: indicates the estimated elapsed time in seconds since the entity was generated.
ETag: indicates an identifier that uniquely identifies an entity and its version.
Location: indicates the redirect target URL.
Proxy-Authenticate: indicates that authentication is required between the proxy server and client.
Retry-After: returned with 503 Service Unavailable or 3xx redirection to indicate when to retry.
Server: returns server information to the browser.
Vary: indicates headers used for server-driven negotiation, such as Accept, Accept-Charset, and Accept-Language.
WWW-Authenticate: indicates that authentication is required.

Common Entity and General Headers

Allow: provides the list of methods that can be used for the resource indicated by the request URL.
Cache-Control: indicates cache directives. In HTTP/1.0, Pragma: no-cache is used.
Connection: indicates persistent connection behavior such as Keep-Alive or close.
Content-Encoding: indicates the content encoding method, such as gzip.
Content-Language: indicates the language of the content, such as en or ko.
Content-Length: indicates the length of the content (message body) in bytes.
Content-Location: indicates another URL where the content is accessible.
Content-MD5: provides check data for verifying that content was not changed during communication.
Content-Range: indicates the range of content being sent.
Content-Type: indicates the MIME type of the resource, and can also include a character set.
Date: indicates the time at which the response is returned.
Expires: indicates the expiration date of the entity.
Last-Modified: indicates the time when the entity was last updated.
Max-Forwards: specifies the maximum number of forwarding or relay facilities to pass through for OPTIONS and TRACE.
Pragma: used for various purposes, such as telling proxies or clients not to use cache.
TE: tells the server which transfer codings or trailer fields the browser can process.
Trailer: lists headers appended after chunked content.
Transfer-Encoding: indicates the encoding format used for transfer.
Upgrade: tells the other party that another protocol is recommended.
Via: indicates the message delivery path through proxies.
Warning: passes warning codes and messages added to the status line.
extension-header: various additional headers can be implemented by servers.

Virtual Hosts

HTTP/1.1 supports virtual hosts. An HTTP/1.1 client must send the host name in the Host header. The server responds with content corresponding to that virtual host. This makes it possible to support multiple websites on one server.

GET / HTTP/1.1
Host: www.devkuma.com

Persistent Connections (Keep-Alive)

Early HTTP opened a new socket connection every time an HTTP request was sent, which was inefficient. Persistent connections solve this inefficiency by sending multiple requests over one connection. HTTP/1.0 introduced Keep-Alive, which became standardized.

Clients can improve communication performance by requesting multiple contents through a single TCP connection. When keeping a persistent connection, the Connection header usually specifies Keep-Alive; the final request specifies close.

Chunked Transfer

When a server does not know the length of content, such as when generating CGI results, it can return chunked data. In chunked data, the byte count of continuous data is represented in hexadecimal. 0 means the end of data.

BASIC Authentication

With HTTP Basic authentication, the server returns the WWW-Authenticate header in response to a client request. When the client receives it, it displays a dialog asking for a login name and password, encodes the entered values, and requests the content again.

HTTP Cookie

kc@example.com (kc kim) — Mon, 27 Dec 2021 18:02:00 +0900

Overview

Cookies are used to implement features such as the following.

Record and display how many times a visitor has visited the page.
Record a visitor’s recently visited page in a web service and show that page on the next visit.
Record the username entered in a bulletin board or chat so the user can skip entering it next time.
Maintain a login session.

The following specifications are published.

RFC2109 (February 1997)
RFC2965 (October 2000)
RFC6265 (April 2011)

This kind of data can be recorded on the server side using CGI and similar mechanisms, but cookies can also record cookie information on the client-side hard disk, meaning the side where the browser runs.

The file where cookie information is stored depends on the OS and browser version.

For example, on Windows it may be recorded in folders or files such as:

- C:\Document and Settings\(UserName)\Cookies
- C:\Program Files\Netscape\Users\(UserName]\cookies.txt
- C:\Program Files\Netscape\Navigator\cookies.txt
- C:\Windows\Cookies\~.txt

For Chrome on macOS, it is stored under:

- ~/Library/Application Support/Google/Chrome/Default/Cookies

Writing Cookies

When setting a cookie with JavaScript:

document.cookie = "~";

To specify it with HTML, use the following. This method is not recommended.

<meta http-equiv="Set-Cookie" content="~">

The ~ part specifies a string in the following format.

NAME=value; expires=value; domain=value; path=value; secure

Everything except NAME=value; is optional.

Parameter	Meaning
`NAME=value`	Specifies the desired value for the desired name. Semicolons, commas, spaces, and Korean text must be encoded in an appropriate format. URL-style encodings such as `%3B`, `%2C`, and `%20` are often used.
`expires=value`	Specifies the expiration date of the cookie recorded on the client side in a format such as `Thu, 1-Jan-2030 00:00:00 GMT`. The time zone is always GMT. If omitted, it expires when the browser closes. If a past value is specified, the cookie is deleted.
`domain=value`	Specifies the name of the web server that publishes the cookie. If omitted, it becomes the web server name, such as `www.devkuma.com`.
`path=value`	When browsing a page matching this path name, the browser sends the stored cookie information to the server.
secure	If this variable is included, cookie information is sent only when the connection to the server is secure.

The simplest writing example is as follows. It is valid until the browser ends and is sent to pages in the same folder or lower layers as the page where it was set.

Set-Cookie: NAME=devkuma;

Specifying an expiration date:

Set-Cookie: NAME=devkuma; Tue, 31-Dec-2030 23:59:59;

Reading Cookies

To read cookie values with JavaScript, refer to document.cookie.

alert(document.cookie);

HTTPS

kc@example.com (kc kim) — Thu, 18 Aug 2022 18:04:25 +0900

HTTP and HTTPS

HTTP

HTTP exchanges text, and HTML pages are also text.
Because it sends and receives plain text rather than binary data, if someone intercepts signals on the network, the contents can be exposed. HTTPS was created to solve this security problem.

HTTPS

HTTPS is a communication protocol in which the client and server exchange data using the SSL (Secure Socket Layer) protocol, which encrypts information on the Internet.
HTTPS encrypts HTTP text.
The S in HTTPS means Secure Socket, a secure communication network.

HTTPS Encryption Principle

The core principle of HTTP encryption is public key encryption.

Certificates are issued so that messages are encrypted with a public key.
Because encrypted messages can only be decrypted with the private key, no one in the middle can obtain the original data.

Characteristics of HTTPS

HTTPS implements HTTP on top of SSL/TLS.

Main functions

Encryption
Authentication
Change detection and similar functions

URI Scheme prefix

https

Port used

If a URL has an https URI scheme prefix instead of http, it uses HTTPS port 443 rather than the usual HTTP port 80.

Standard

RFC 2818

SSL/TLS

By using SSL (Secure Sockets Layer) or TLS (Transport Layer Security), which are encryption methods for HTTP communication, websites can be used safely.

SSL and TLS are essentially the same. SSL was invented by Netscape. As it became widely used and later came under the management of the IETF standards organization, its name changed to TLS. TLS 1.0 succeeds SSL 3.0, but the name SSL is still used much more commonly.

Structure for Securing HTTPS

SSL/TLS secures websites through the following three mechanisms.

Preventing Eavesdropping (Encrypted Communication)

Because website browsing passes through several servers, a third party can relatively easily intercept communication contents.
Even if data is intercepted, encrypting it before transmission prevents third parties from reading it.

Preventing Tampering

Message digests are used as a countermeasure against data tampering, such as rewriting product quantities in online shopping.
A message digest calculates a unique short value (hash value) from specific data. By comparing hash values when sending and receiving data, tampering can be detected.

Preventing Spoofing

By placing an electronic certificate called an SSL server certificate on the web server and verifying it at connection time, the identity of the website operator can be confirmed.
The certificate must be issued after authentication work by an approved certificate authority. If an untrusted certificate is used, a warning screen appears in the web browser.

How HTTPS Security Works

When a client (web browser) receives a URL with an https scheme rather than plain http, it opens a TCP connection to port 443 instead of port 80 on the web server and exchanges several security parameters in binary format (handshake, key exchange). Then the related HTTPS commands are executed.

HTTPS Exchange

To start HTTPS communication, four main phases are exchanged.

Decide the encryption method
There are many encryption methods, so the web browser and web server must decide on an encryption method that both can use. The TLS version and message digest method are also decided at the same time.
Prove the communication partner
The browser checks with the SSL server certificate whether the web server is the correct target. If it cannot verify this, a warning is displayed.
Key exchange
The “key” used for data transmission is exchanged. The key is used for encryption during transmission and decryption during reading.
Confirm the encryption method
The final confirmation of the encryption method actually used is performed. When this step is complete, encrypted communication begins.

These four stages in HTTPS are called the SSL/TLS handshake.

HTTPS Server Certificate

A server certificate is an electronic certificate issued by a certificate authority to perform encrypted communication between a server and client.

It is based on X.509, and website information is added to the certificate.

Main Fields Included in a Server Certificate

Certificate serial number and validity period
Website name
Website DNS host name (FQDN)
Website public key
Name of the signing authority (certificate authority)
Digital signature of the signing authority

Main Certificate Checks for Website Information

Date check
- Checks certificate validity period, expiration, activation, and similar status.
Signer validity check
- Checks whether the certificate was derived from a valid certificate authority (chain of trust).
Signature check
- Checks certificate integrity.
Site identity check
- Checks whether the domain name specified in the certificate matches the actual domain name.
Virtual hosting check
- When multiple sites or hosts are operated on one server, additional management and procedures are required.

References

HTTP 2.0

kc@example.com (kc kim) — Fri, 12 Aug 2022 07:47:00 +0900

HTTP 2.0

HTTP/2, also called HTTP 2.0, stands for Hypertext Transfer Protocol Version 2. It is the next version of HTTP/1.1, the existing standard, and was officially published by the IETF in 2015.

IETF: Internet Engineering Task Force means the international Internet standards organization. It is an Internet standardization working group that discusses Internet operation, management, and development, and analyzes protocols and architectural issues.

HTTP/2 operates on a TCP connection between the server and client. The client initializes this TCP connection. HTTP/2 requests and responses are contained in one or more frames with a defined length (maximum 16,383 bytes). Requests and responses in frames are sent through streams, and one stream handles a pair of request and response. Because multiple streams can be created simultaneously over one connection, multiple requests and responses can be processed at the same time. HTTP/2 also provides flow control and prioritization for streams. If the server thinks a resource is needed by the client, it can actively send it even without an explicit request.

SPDY

SPDY is a non-standard network protocol developed by Google. As the web environment continued to change, with more resources, many domains, dynamic web services, and the growing importance of security, it was designed to speed up HTTP by focusing on solving latency problems on the Internet based on packet compression and multiplexing. It was Google’s own protocol, built into early Chrome browsers, that provided high loading speed.

HTTP/2 is based on SPDY and introduces a new binary layer below the HTTP protocol layer in the TCP communication layer, pursuing better efficiency for the TCP connections that HTTP depends on.

SPDY showed significant performance improvements and efficiency compared with HTTP/1.1 and became a reference specification for the HTTP/2 draft.

SPDY’s characteristics include:

Always operates on TLS
- Applies only to websites written with HTTPS.
HTTP header compression
- The more requests there are, the larger the compression ratio becomes, and the effect is large in mobile environments with low bandwidth.
Binary protocol rather than text
- Parsing is faster and errors are less likely.
Multiplexing
- Processes multiple independent streams simultaneously in one connection.
Full-duplex interleaving and prioritization
- Allows other streams to interleave.
Server Push

Ultimately, SPDY can be seen as modifying HTTP’s data transfer format and connection management so TCP connections are used more efficiently. SPDY became a reference specification for HTTP/2, and most of these characteristics also exist in HTTP/2.

Main Characteristics

The main characteristics aim to improve performance.

Packet Capsulation

As HTTP/2 packets are encapsulated into smaller units, the concepts of Frame, Message, and Stream are introduced.

Frame
- The smallest communication unit in HTTP/2. Every packet includes one frame header.
- This frame header at least identifies the stream to which the frame belongs.
- HEADERS type frames and DATA type frames exist.
Message
- The full sequence of frame data mapped to a logical request or response message.
Stream
- The flow of a connection. It is a bidirectional flow of bytes delivered within an established connection, and one or more messages can be delivered.

All HTTP/2 connections are TCP-based streams and communicate messages with frame headers in both directions.

Data is binary encoded, and multiplexing plus performance optimization algorithms are applied.

Request Multiplexing

HTTP/2 can send and receive multiple data requests in parallel through one TCP connection.

A stream is an independent bidirectional sequence of frames exchanged between client and server through an HTTP/2 connection. A pair of HTTP request and response is made through one stream. The client creates a new stream and sends an HTTP request; when the server responds on the same stream, that stream is closed.

In HTTP/1.x, after sending a request over one TCP connection, another request could not be sent over the same TCP connection until the response arrived. In HTTP/2, multiple streams can be opened simultaneously in one connection, so multiple requests can be sent at the same time.

This reduces additional RTT (round-trip time), loads websites faster without extra optimization, and makes domain sharding unnecessary.

Header Compression

In HTTP/1.1 and earlier, HTTP headers were sent without compression. Older web pages did not send as many requests as today, but modern web pages send countless requests, so header size affects both round-trip delay and bandwidth.

HTTP/2 compresses HTTP message headers and avoids retransmitting duplicate fields.

HTTP headers were plain text before, but HTTP/2 uses a header compression method called HPACK, which uses Huffman coding, to improve data transfer efficiency.
Headers are split into “header block fragments,” sent, rejoined by the receiver, decompressed, and restored to the original header set.

Huffman coding is an algorithm that uses codes of different lengths depending on the frequency of data characters.

Binary Protocol

The latest HTTP version greatly improved functionality and properties by changing from a text protocol to a binary protocol. HTTP/1.x processed text commands to complete request-response cycles, while HTTP/2 uses binary commands (1 and 0) for the same work. This reduces frame-related complexity and simplifies implementation.

Benefits of the Binary Protocol

Data parsing is faster and less error-prone.
Network resources can be used more effectively.
Network latency is reduced and throughput improves.
Security issues related to text characteristics are addressed.
Other HTTP/2 features such as compression, multiplexing, prioritization, flow control, and efficient TLS processing are enabled.

Server Push

HTTP/2 allows the server to send files that the client will likely need, such as JavaScript, CSS, fonts, and image files, together with a single HTTP response even if the client did not explicitly request them.

This is useful when the server can predict which resources the client will require. When the server receives a request for an HTML document, it can push resources linked by that document, such as images and CSS files, to the client. This reduces traffic and round-trip delay caused by the client parsing the HTML and requesting required resources again.

Benefits of Server Push

The client stores pushed resources in cache.
Cached resources can be reused across multiple pages.
The server can send pushed resources together with requested information using multiplexing.
The server can prioritize pushed resources.
The client can manage optional resources, reject pushed resources, or disable server push.
The client can limit the number of multiplexed push streams.

Stream Prioritization

Streams can have priorities. In other words, the client can specify its preferred way of receiving responses.

For example, if a document contains one CSS file and two image files, receiving the CSS file later than the images may cause rendering problems. HTTP/2 can solve resource loading problems by setting priorities based on dependencies between resources.

Every stream also has a unique identifier. A stream identifier once used in a connection cannot be reused.

References

CORS

kc@example.com (kc kim) — Thu, 27 May 2021 10:59:45 +0900

Overview

Modern web browsers apply the Same-Origin Policy to prevent information held by one website from being abused by another malicious website.

For example, when a frontend communicates with a backend API on a different domain and requests resources, an error occurs because the Origin, meaning domain, protocol, and port number, is different. If a page at https://api.devkuma.com/ tries to read data over HTTP(S) from another website, https://www.devkuma.com/, using XMLHttpRequest (XHR) or the Fetch API, an error occurs.

However, if even trusted websites used for data integration are blocked, it becomes inconvenient. CORS (Cross-Origin Resource Sharing) is needed so that websites allowed to access data can do so even when the Origin is different.

What Is CORS?

CORS stands for Cross-Origin Resource Sharing and means resource sharing between origins.
Browsers generally prohibit communication between different origins, but CORS settings allow communication between different origins.
It is a method that permits data to be received from a domain different from the page being viewed in the browser.
It is used by browsers to prevent cross-site scripting for security.
There is a restriction that communication can only occur with the origin domain’s server.

What Is Origin?

To understand CORS more precisely, you need to understand origin. The origin of web content is defined by the URL scheme (protocol), host (domain), and port used to access that content. Two objects are considered to have the same origin only when all three match.

On the web, work is limited to same-origin content by the Same-Origin Policy, and this restriction can be relaxed using CORS.

Same Origin Examples

The scheme (http) and host (www.devkuma.com) are the same, so they are the same origin.

http://www.devkuma.com/app1/index.html
http://www.devkuma.com/app2/index.html

The server uses the default port 80 and serves HTTP content, so it is the same origin.

http://www.devkuma.com:80
http://www.Devkuma.com

Different Origin Examples

Different scheme:

http://devkuma.com/app1
https://devkuma.com/app2

Different host:

http://devkuma.com
http://www.devkuma.com
http://blog.example.com

Different port:

http://www.devkuma.com
http://www.devkuma.com:8080

Why Is CORS Necessary?

Browsers adopt the same-origin policy for security reasons. It is used to prevent other origins from freely accessing your resources.

If a site you do not operate can obtain session requests, that site may hijack your session and perform malicious actions. Therefore, browsers block these requests.

Phishing sites are a representative attack example, and CORS is needed to stop such attacks and allow requests only from origins that you have permitted.

How CORS Works

When the browser requests a resource, it includes additional information in headers: what the origin is, which method will be used, and which headers will be included. The server sends back the origins it can respond to in response headers. The browser checks these headers and allows the resource transfer if the request is allowed from that origin; otherwise, it raises an error.

CORS Preflight Request

This is a system composed of HTTP header exchanges. The browser determines whether frontend JavaScript code may access the response to a cross-origin request.

Request Headers

Origin: shows which origin is accessing.
Access-Control-Request-Method: tells the server which method will be used in the actual request during a preflight request.
Access-Control-Request-Headers: tells the server which headers will be used in the actual request during a preflight request.

Response Headers

Access-Control-Allow-Origin: allows the browser to access the resource from that origin. * allows access from all origins only for requests without credentials.
Access-Control-Expose-Headers: lists which headers may be exposed as part of the response.
Access-Control-Max-Age: indicates how long preflight request results can be cached.
Access-Control-Allow-Credentials: indicates whether the response to a request can be exposed when credentials are true.
Access-Control-Allow-Methods: indicates methods allowed in response to a preflight request.
Access-Control-Allow-Headers: indicates HTTP headers that can be used in the actual request.

CORS Examples

The examples describe allowing HTTP(S) access from https://www.devkuma.com to https://api.devkuma.com.

Allowing Simple Data Loading

To allow GET and POST from XHR or Fetch API, the client declares CORS for Fetch, and the server adds appropriate information to the HTTP response headers.

var xhr = new XMLHttpRequest();
xhr.open('GET', 'https://api.devkuma.com');
xhr.addEventListener('load', onLoadFunc, false);
xhr.send(null);

fetch('https://api.devkuma.com', {
  mode: 'cors'
}).then(onLoadFunc);

GET /api HTTP/1.1
Origin: https://www.devkuma.com

HTTP/1.1 200 OK
Access-Control-Allow-Origin: https://www.devkuma.com

For simple cases, a wildcard can be used.

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *

Allowing Cookies

To allow cookies to be sent and received during HTTP(S) communication, both the browser and server need configuration. In this case, wildcard values are not allowed for Access-Control-Allow-Origin.

HTTP/1.1 200 OK
Access-Control-Allow-Origin: https://www.devkuma.com
Access-Control-Allow-Credentials: true

Using Detailed HTTP Communication

If the request method is not GET, POST, or HEAD, or if special request headers or content types are used, CORS performs an OPTIONS preflight request before the actual request.

OPTIONS /api HTTP/1.1
Access-Control-Request-Method: {request HTTP method}

HTTP/1.1 200 OK
Access-Control-Allow-Origin: https://www.devkuma.com
Access-Control-Allow-Methods: GET,POST,HEAD,OPTIONS

If custom request headers such as X-MyRequest and X-MyOption are used, the server must allow those headers using Access-Control-Allow-Headers. If custom response headers should be readable by the browser, the server must specify them with Access-Control-Expose-Headers.

Preflight request results can be cached using Access-Control-Max-Age.

devkuma – HTTP

Introduction to HTTP

HTTP Overview

HTTP Messages

HTTP Request

HTTP Request Messages

Start Line

HTTP Method

GET

POST

HEAD

OPTIONS

PUT

PATCH

DELETE

TRACE

CONNECT

Request Target

HTTP Protocol Version

HTTP Header

HTTP Body

HTTP Response

HTTP Response Messages

Start Line

HTTP Protocol Version

Response Status Code and Status Text

HTTP Header

HTTP Body

HTTP Header

HTTP Header

Common Request Headers

Common Response Headers

Common Entity and General Headers

Virtual Hosts

Persistent Connections (Keep-Alive)

Chunked Transfer

BASIC Authentication

HTTP Cookie

Overview

Folder and File Where Cookie Information Is Stored

Writing Cookies

Cookie Writing Example

Reading Cookies

HTTPS

HTTP and HTTPS

HTTP

HTTPS

HTTPS Encryption Principle

Characteristics of HTTPS

SSL/TLS

Structure for Securing HTTPS

Preventing Eavesdropping (Encrypted Communication)

Preventing Tampering

Preventing Spoofing

How HTTPS Security Works

HTTPS Exchange

HTTPS Server Certificate

Main Fields Included in a Server Certificate

Main Certificate Checks for Website Information

References

HTTP 2.0

HTTP 2.0

SPDY

Main Characteristics

Packet Capsulation

Request Multiplexing

Header Compression

Binary Protocol

Benefits of the Binary Protocol

Server Push

Benefits of Server Push

Stream Prioritization

References

CORS

Overview

What Is CORS?

What Is Origin?

Same Origin Examples

Different Origin Examples

Why Is CORS Necessary?